32

I'm doing some research on my own as a side-project. In fact, I want no one from my research center to be involved in this research as I feel that they will take credit for my idea, like in past times. Nevertheless, if their involvement is completely necessary, I would consider it.

Anyways, this project requires to upload completely nonsense papers to Arxiv or similar. They are completely nonsense as they are artificially generated. The idea is to see if this is possible or how easy it is.

On the one hand, I don't want to pollute the scientific community with rubbish but in the other hand, I see that this step is necessary to continue further with this research.

My question is, how should I proceed? Should I do it and once I finish my research tell Arxiv that they can delete the files? Should I ask for permission first? Should I not do it at all?

EDIT: Just to make sure that everyone understands, this is not something that I have done yet. In fact, after seeing the responses, it is more than likely that I try to avoid publishing in arXiv. The objective is not to upload a paper to arXiv, is something else, but I was considering uploading it to arXiv as a medium for reaching another goal. Nevertheless, it is not my objective to pollute arXiv or the community.

  • 35
    Here's a short note from Ginsparg about how arXiv screens (well, screened back in 2014) for fake papers. If you're interested in seeing "how easy it is" to get a paper past the filter (as opposed to more social aspects) arXiv might be interested in collaborating, in order to strengthen their filters. – Anyon Apr 11 at 11:05
  • 12
    What are you hoping to learn from this research? What is your hypothesis? – Buffy Apr 11 at 12:10
  • 18
    This paper might be of interest. It was reviewed and rated as excellent by this journal. – Eric Duminil Apr 11 at 14:57
  • 17
    @EricDuminil It is an excellent paper. It has a clear abstract and message, crisp and compact, it really manages to get the story across. Its graphics is pertinent, and it makes also clear what the way forward in the future is and it spawned insights beyond its own initial scope. It is a paradigmatic example of clear and unambiguous communication. It is also not significantly compressible by gzip, which means that there is little redundancy. Every line has something new to say. I am sure this is one of the best papers that this journal has ever published. – Captain Emacs Apr 11 at 15:26
  • 7
    For the matter, something like this has already been done. How does your idea to waste their time differ from it? – lucasgcb Apr 11 at 15:31
75

"Shall I poison a well to see how many drink out of it?"

Frankly, the idea of debunking review processes is not that new, and the latest wave started with the Sokal hoax. There were a couple of follow-ups, with computer-generated papers.

The result was a scandal, a journalistic anecdote. It created attention. Been there, done that. It's no longer an original idea. We got it. You can fool badly screened journals and predatory conferences.

But ok, now, you talk about actual "research" with fake paper dissemination. That may or may not be a legitimate research question, but the fact that you talk about "poisoning" arXiv, which is a serious and expressly not peer-reviewed site makes it already very questionable what you are trying to achieve.

It is one thing to embarrass badly screened journals or conferences (possibly even suspected predatory ones - namely the conferences in question) which Sokal and the SciGen people did, but quite another to abuse a perfectly legitimate repository for non-peer-reviewed papers. In other words, it relies on good will and honesty of the submitters trying to submit good quality work. There are minor safeguards in place to avoid obvious scientific spam, but not against antagonistic submitters.

I do not know (and I do not want to know) what you are trying to achieve with this experiment, but the damage that you are likely to do with this can be so substantial in terms of loss of trust of arXiv and preprint services etc, that you would seriously need to pass it through an ethics committee who can evaluate what legitimate experimental parameters for such a large scale social experiment would be.

Without such a permission and clear delineation of your duties and limitations (which your institution's ethical committee has to evaluate), this research will be unpublishable in a serious journal. And frankly, your academic reputation will be finished. Damaging one of the most valuable resources of openly accessibly publications will make you persona non grata in academia.

Let me add one more thing. There are other preprint servers which have a less strong reputation. There, your experiment may do less damage, though also be less expressive. However, even there, I should believe that they rely on good will and that wilful tainting of wells for scientific knowledge is simply maximally antagonistic to the endeavour of science.

TL;DR: I believe the experiment is highly unethical and extremely damaging. However, you may want to see whether your institution's ethics committee can identify conditions under which such a social experiment might be feasible.

  • 7
    First of all, thank you for your answer. The idea is not to see "how many drink from it", it is something that goes further but I'm checking how to do it without damaging Arxiv or the community. In fact, Arxiv was thought to be the tool, no the end result. I will consider how to do this in a different way, if it is going to affect the community so much. Thank you again – AnonimousPhD Apr 11 at 10:48
  • 9
    @AnonimousPhD Ok, I understand. Perhaps better tests can be done in communities where there is an active evaluation model, such as slashdot, reddit, etc. – Captain Emacs Apr 11 at 11:02
  • 1
    Sokal nothing. An auto-generated content-free paper got into a peer-reviewed journal and only got pulled because their attempt to get funding to present the associated talk hit the news. – Joshua Apr 12 at 16:35
17

Any research with human subjects must go through an IRB approval process. I'd expect in this case that the people moderating the arxiv and reading the arxiv would be considered human subjects. Typically a key part of ethical research on human subjects is obtaining informed consent, which is not done in your proposed experiment. I expect it would be difficult to IRB approval for this experiment, but maybe with the right design it is possible. More importantly though, it's unethical and a serious violation of university rules (and rules of most funding agencies) to go ahead with an experiment on human subjects without prior IRB approval.

So ask your IRB, their answer is much more important than the opinion of a bunch of random people on the internet.

  • 5
    IRB can waive informed consent if necessary to achieve the goals of the work and participants are sufficiently protected in other ways. (I don't think this proposed work meets that bar.) – Reid Apr 12 at 16:14
11

Without more information on your goals this is highly suspect. As has been mentioned many times above, the Sokal affair and other similar efforts were designed specifically to show that it is possible to create scientific sounding papers that are accepted, either by legitimate experts in their field (Sokal) or by predatory journals who are in it for the cash. I'm pretty sure a huge number of researchers have, for fun, submitted a BS article to a predatory journal just to see how far they could take it. I know I have, but I've never bothered to pay the fee to get it actually published because, as has been pointed out above, it's been done.

So, to arXiv. arXiv is designed specifically for rapid dissemination of results with the explicit understanding that the papers are not peer reviewed. It's up to the readers to make judgments about the validity and contribution of the papers. From that perspective, you're probably going to end up getting one of three types of responses, depending on the person reading it:

  1. An expert (or even someone mildly knowledgeable) will see it for what it is and probably ignore it.
  2. A student who is not an expert but is trying to be will try very hard to read the paper and think they can't understand it because it's too advanced. Most likely this student will give up, or embarrass themselves by asking their advisors or someone else for help. That would be mean on your part but, it's the Internet and not really unexpected.
  3. Someone from an unrelated field will come upon it, shrug at how different the other field is (thinking they don't understand because it's not their field), and move on.

The real problem I see is the damage done to arXiv, along the lines of "they'll publish anything". However, I'm not convinced that's a bad thing. There are many news articles that reference papers in arXiv as though they are peer reviewed. Researchers using the service get it, but journalists and the general public may not. The big question is whether or not polluting arXiv like this would lead to researchers not using it as much anymore. If you are reasonably sure that won't happen, then your harm is minimized (except for the poor student in point 2 above). On the other hand, if this leads to researchers having a suspicion that arXiv is not useful and they stop disseminating their research on it, that would be a terrible blow.

I think your next step is to perform a comprehensive study of researchers and their opinions on the service, what they think they would do in this situation, and then base your idea on how to proceed off of that.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.