8

arXiv, by itself, and not as a preprint server, is already moderated by top computer science researchers at Cornell.

I understand if it's not the top journal, but isn't at least getting into arXiv actually more reputable than submitting directly to a lower-tier journal? Especially if 90% will just publish it anyway.

There is also evidence that arXiv papers are cited more, although this isn't conclusive.

2
38

First, arXiv is hosted at Cornell but this does not mean it’s moderated by Cornell people (that may be true in CS but it’s not true in physics, where the pool of moderators is quite wide).

Next, different disciplines have different traditions, so beware of self-selection. In my field of physics, there is now a growing trend where authors upload on arXiv only after they have received comments from a journal referee. This avoids major updates that may result from the refereeing process, minimizes the number of arXiv versions, and also prevents the material from going stale before the next submission if it has to be resubmitted (nothing worse than a paper that has been “sitting” in arXiv for 18 month). Thus, some fraction of authors only submit to arXiv after third-party validation through usual refereeing channels.

ArXiv is fantastic to get pre-publication feedback - someone will provide additional references, will point to possible improvement in a proof, will ask for additional clarifications. However, be reminded that moderators keep out the obviously bad and crackpotty stuff, but that’s about it. Since arXiv is easily searchable and open access, it is no surprise that submissions there will have greater visibility than if only published in a journal behind a paywall.

Very few active researchers will referee more than one paper per month because refereeing is quite time-consuming. A fortiriori no moderator has time to do any refereeing-level checks on their share of daily submissions to check: one would spend 24hrs of everyday checking papers.

6
  • 10
    Not the original question, but I'm puzzled by this trend of only uploading to arXiv after reviewing. In my field (theoretical CS) this looks like the opposite of what we'd like to encourage. It just wastes all the benefits of pre-publication feedback that you mention in your third paragraph, and the arguments for not doing it (avoiding an arXiv version) really don't seem convincing to me. If you think your work is ready to be refereed by expert researchers, taking them several hours/days/weeks, then why wouldn't you spend 10 minutes making it available to everyone on arXiv?
    – a3nm
    Jul 28 at 18:36
  • @a3nm It is a trend. I suppose in my personal case it’s 50/50 depending on preferences of the co-authors. Methinks there are too many submissions hastily uploaded to arXiv; my personal preference is to submit a later version, wait a few days for independent comments, and then submit full corrections. This means the arXiv version is closer to the final published product and has fewer bugs. Some high profile journals explicitly ask not to submit to arXiv until the paper is published (which I will agree will negate the value of feedbacks), others are fine with uploading rough work. Jul 28 at 19:00
  • @a3nm Some also feel the rise of satellite discussion boards (v.g. scirate) means the referee might be contaminated by comments from casual readers. Again… I am ambivalent myself. I prefer to wait but in some cases co-authors need the arXiv submission for an annual report or a grant application. I can’t recall ever objecting. Jul 28 at 19:04
  • 1
    Given that arXiv is not moderated, I don't see the value in self-moderating. You can write in the arXiv Comments field that the work is still a draft and not peer-reviewed yet. In my area, the one reason I've heard to not submit to arXiv is the risk of being "scooped", i.e., publish paper X on arXiv, other people notice it and improve the result in a paper X' (citing the preprint X), the paper X' gets published in a conference/journal faster than X, and now getting X in a conference/journal becomes difficult. But again I think that this is a bogus reason, in terms of making science advance.
    – a3nm
    Jul 29 at 6:29
  • 2
    … Others presumably have their own reasons, and some people I know basically use arXiv not as a pre-print server but more as a post-print service to fulfill the open-access mandate of their grant. (I have one specific colleague in mind.) Jul 29 at 8:05
38

The standard for the arXiv is "looks like an actual paper on the first glance". So the fact that a preprint appeared on the arXiv isn't really adding any respectability to it that a researcher in the field couldn't spot themselves. The point of the filtering the arXiv does is instead about avoiding spam.

2
  • "The point of the filtering the arXiv does is instead about avoiding spam." Wouldn't it also filter out a lot of cranks, since they need an actual scientist to vouch for them, or has there been enough incautious invitations that cranks have begun inviting other cranks and their presence has metastasised?
    – nick012000
    Jul 27 at 21:15
  • 10
    @nick012000 I'm counting cranks as a type of spam here. The point is, it helps the knowlegable reader to not waste a short moment reading the crank stuff; rather than providing a non-expert reader with justified confidence that what they read is true.
    – Arno
    Jul 27 at 21:20
22

Moderation on arXiv is extremely quick - papers typically go out within 24 hrs of submission, except for weekend submissions. The moderation process is described in this arXiv blog post:

The team performs checks with software support to confirm that the submission is classified to the appropriate subject category, that it is in compliance with our technical requirements, including formatting and metadata information, and that the content is appropriate, which, generally, describes content that would be refereeable by a conventional journal and in the format of a research article.

In particular they are mainly checking that the author has submitted it correctly, including proper classification, as well as filtering out anything obviously inappropriate, e.g. if it isn't even an attempt at writing a research paper.

The moderation process is not about whether the paper is correct or not, it's whether it's a well-formed paper. Saying "this paper got on arXiv so it must be decent" would be like saying "Microsoft Word didn't find any errors in this paragraph, so it must be true".

0
16

I strongly encourage you not to trust anything on arXiv (unless it has a gazillion citations).

I had the chance, many times, to (officially) review papers that I "noticed" beforehand on arXiv. As someone else mentioned "at first glance they looked OK". Then, after reading into them, I started to notice a lot of issues. And I'm not talking about small inconsistencies; I am talking about significant problems that completely invalidated the papers' main claims.

Just to give you two examples.

  • A paper once stated in the abstract "Our method is superior than the state of the art method X by y%". Then, after reaching the experimental section, I discovered that the authors used a completely different setup than what was used to evaluate the original method X.

  • Another time the abstract stated "our attack breaks the state of the art technique X". Then, later, the authors wrote "we did not implement X as it should be done".

Any reviewer would spot such issues right away and reject these papers even in low-tier journals because they are flawed.

Note that I'm not saying that any arXiv publication is trash. There may very well be (actually, there is) good source material on arXiv. However, while you can take any peer-reviewed publication (almost) at face value, when it comes to arXiv you need to thoroughly question its validity first - and use not just a grain, but an entire bag of salt.

My bottom line: "publishing" on arXiv is not devoid of meaning, but the scientific value of arXiv papers is either negligible, null, or negative. I would take a low-tier journal paper over an arXiv publication any day, if given the choice.

7
  • 6
    To be clear, this answer addresses submissions on arxiv that have not been published elsewhere. Jul 28 at 6:04
  • 3
    @MateenUlhaq yes, but an article that has been published elsewhere then it is not an article "published on arXiv". The question specifically asked if "publishing on arXiv is more reputable than a low tier journal" - which is not.
    – P. Shark
    Jul 28 at 9:00
  • 6
    "Any reviewer would spot such issues right away and reject these papers even in low-tier journals because they are flawed." Stated without evidence. Indeed, there is significant evidence to the contrary.
    – Xerxes
    Jul 28 at 14:39
  • 4
    Citations are not an accurate indicator of quality. Jul 28 at 14:45
  • 1
    @AnonymousPhysicist: Of course citations don’t infallibly imply a paper can be trusted, but they’re a very good first-pass metric, especially if you look a little closer and filter on “citations from reputable authors, and/or in reputable journals”.
    – PLL
    Jul 29 at 14:06
2

arXiv itself isn't intended to be treated as a publisher. Submitting a paper to a low-tier journal helps its credibility at the very least because at least it receives some peer review. Posting a paper on arXiv just means you satisfy the requirements of their site and you have some credential (i.e. a university email address) to post papers. A good rule of thumb is to verify a paper has undergone peer review and publication in some trustworthy journal, and then read its preprint on arXiv.

Regarding your point on citations: Many documents have been cited from arXiv because authors post their preprints there before formal publication in journal or conference proceedings and the formal publication is not available. Or, it's just because people don't want to pay to read a document on a site like IEEEXplore and just cite the preprint on arXiv.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy