Academia Stack Exchange is a question and answer site for academics and those enrolled in higher education. It only takes a minute to sign up.
Sign up to join this community
arXiv, by itself, and not as a preprint server, is already moderated by top computer science researchers at Cornell.
I understand if it's not the top journal, but isn't at least getting into arXiv actually more reputable than submitting directly to a lower-tier journal? Especially if 90% will just publish it anyway.
There is also evidence that arXiv papers are cited more, although this isn't conclusive.
First, arXiv is hosted at Cornell but this does not mean it’s moderated by Cornell people (that may be true in CS but it’s not true in physics, where the pool of moderators is quite wide).
Next, different disciplines have different traditions, so beware of self-selection. In my field of physics, there is now a growing trend where authors upload on arXiv only after they have received comments from a journal referee. This avoids major updates that may result from the refereeing process, minimizes the number of arXiv versions, and also prevents the material from going stale before the next submission if it has to be resubmitted (nothing worse than a paper that has been “sitting” in arXiv for 18 month). Thus, some fraction of authors only submit to arXiv after third-party validation through usual refereeing channels.
ArXiv is fantastic to get pre-publication feedback - someone will provide additional references, will point to possible improvement in a proof, will ask for additional clarifications. However, be reminded that moderators keep out the obviously bad and crackpotty stuff, but that’s about it. Since arXiv is easily searchable and open access, it is no surprise that submissions there will have greater visibility than if only published in a journal behind a paywall.
Very few active researchers will referee more than one paper per month because refereeing is quite time-consuming. A fortiriori no moderator has time to do any refereeing-level checks on their share of daily submissions to check: one would spend 24hrs of everyday checking papers.
The standard for the arXiv is "looks like an actual paper on the first glance". So the fact that a preprint appeared on the arXiv isn't really adding any respectability to it that a researcher in the field couldn't spot themselves. The point of the filtering the arXiv does is instead about avoiding spam.
Moderation on arXiv is extremely quick - papers typically go out within 24 hrs of submission, except for weekend submissions. The moderation process is described in this arXiv blog post:
The team performs checks with software support to confirm that the submission is classified to the appropriate subject category, that it is in compliance with our technical requirements, including formatting and metadata information, and that the content is appropriate, which, generally, describes content that would be refereeable by a conventional journal and in the format of a research article.
In particular they are mainly checking that the author has submitted it correctly, including proper classification, as well as filtering out anything obviously inappropriate, e.g. if it isn't even an attempt at writing a research paper.
The moderation process is not about whether the paper is correct or not, it's whether it's a well-formed paper. Saying "this paper got on arXiv so it must be decent" would be like saying "Microsoft Word didn't find any errors in this paragraph, so it must be true".
I strongly encourage you not to trust anything on arXiv (unless it has a gazillion citations).
I had the chance, many times, to (officially) review papers that I "noticed" beforehand on arXiv. As someone else mentioned "at first glance they looked OK". Then, after reading into them, I started to notice a lot of issues. And I'm not talking about small inconsistencies; I am talking about significant problems that completely invalidated the papers' main claims.
Just to give you two examples.
A paper once stated in the abstract "Our method is superior than the state of the art method X by y%". Then, after reaching the experimental section, I discovered that the authors used a completely different setup than what was used to evaluate the original method X.
Another time the abstract stated "our attack breaks the state of the art technique X". Then, later, the authors wrote "we did not implement X as it should be done".
Any reviewer would spot such issues right away and reject these papers even in low-tier journals because they are flawed.
Note that I'm not saying that any arXiv publication is trash. There may very well be (actually, there is) good source material on arXiv. However, while you can take any peer-reviewed publication (almost) at face value, when it comes to arXiv you need to thoroughly question its validity first - and use not just a grain, but an entire bag of salt.
My bottom line: "publishing" on arXiv is not devoid of meaning, but the scientific value of arXiv papers is either negligible, null, or negative. I would take a low-tier journal paper over an arXiv publication any day, if given the choice.
arXiv itself isn't intended to be treated as a publisher. Submitting a paper to a low-tier journal helps its credibility at the very least because at least it receives some peer review. Posting a paper on arXiv just means you satisfy the requirements of their site and you have some credential (i.e. a university email address) to post papers. A good rule of thumb is to verify a paper has undergone peer review and publication in some trustworthy journal, and then read its preprint on arXiv.
Regarding your point on citations: Many documents have been cited from arXiv because authors post their preprints there before formal publication in journal or conference proceedings and the formal publication is not available. Or, it's just because people don't want to pay to read a document on a site like IEEEXplore and just cite the preprint on arXiv.
