15

I have recently noticed a failed case of peer review at The Astrophysical Journal (ApJ). It is a solid journal in which researchers of astrophysics can present good science without worrying about the topic being fashionable (as opposed to, e.g., Nature and Science). The peer-reviewed paper in question was published in ApJ but retracted by the journal within a week after it was reported as a case of plagiarism.

However, the issue I'd like to raise is not about plagiarism but about the review process of the journal. In the aforementioned paper, there are obvious technical errors that do not even pass a basic sanity check. I believe that had it been properly reviewed, the paper could in no way have been accepted in the first place, plagiarized or not.

(For more information, see my comment on Jeffrey Beall's blog post about the incidence of plagiarism. Moreover, my Astronomy SE post contains some technical details. The latter was posed as a question but I'm now sure that my suspiscion---that there are blatant errors in the paper---is well-founded. I had at first shied away from the obvious conclusion just because I have no expertise in astrophysics.)

It is disturbing that the acceptance rate of the journal is very high (between 85--90%) while only a single referee participates in the review process. (ApJ seeks an additional opinion only if the acceptance decision cannot be made without one.) As the low quality of the retracted paper demonstrates, any such a single-reviewer model falls apart when a referee hastily signs off the paper.

This incident seems to show the poor review process of ApJ. Could this be justified simply on the grounds that no better practice is available in the field? Or is this worrysome even after acknowledging that academic customs greatly vary across different fields?

Improve this question
25
  • 8
    I find some of the statistics here eye-opening. An acceptance rate of 85-90% is not inherently problematic, but it is surprisingly high. Also publishing 20,000 papers per year boggles my mind: my Fermi-type calculation puts this at the same order of magnitude as all papers published in astrophysics in a given year. So this journal sounds culturally exceptional to me. Perhaps it functions more like a preprint server than a conventional journal: i.e., you expect to find a large portion of papers in your field there, and you expect the stuff there not to be nonsense, but that's about it.
    – Pete L. Clark
    Commented Nov 29, 2015 at 4:37
  • 3
    What makes this journal a respectable one? It seems either the standards of the scientific community are very low or exceptionally high ( they send out papers so good that no reviewer can improve it).
    – Greg
    Commented Nov 29, 2015 at 4:53
  • 2
    @TheDarkSide: Well, I hope you know that a "Fermi-type calculation" is fancy language for "a quick/lazy mixture of calculation and guesswork", but e.g. see arxiv.org/year/astro-ph/14.
    – Pete L. Clark
    Commented Nov 29, 2015 at 5:34
  • 6
    '@David: Ah, that makes a lot more sense. :) "Only" 4,000 papers then.
    – Pete L. Clark
    Commented Nov 29, 2015 at 6:34
  • 2
    If there was a respectable journal with even just 50% acceptance rate, it would be flooded with "spam" submissions. Hey, this is unfinished or did not work, but let's try submitting it there anyway?
    – Has QUIT--Anony-Mousse
    Commented Nov 29, 2015 at 10:25

2 Answers 2

Reset to default
21

By saying

This incidence demonstrates the poor review process of ApJ

you are already asserting an answer to your question. What you can say based on the facts you have presented is that the peer review process failed in one instance. The document you have linked indicates that the ApJ publishes roughly 4,000 papers per year, so one failure per year would represent just twenty-five thousandths of a percent. Extrapolating based on that seems wrong (and in any case, you haven't mentioned any failures of the system in previous years).

The term acceptable refers to agreement of a group of people. The refereeing process of the ApJ is clearly not only "acceptable", but in fact accepted, as it is a leading journal in its field.

More generally, should a high acceptance rate be a cause for concern? Independent of other indicators, I think the answer is no. A high acceptance rate for a leading journal may be surprising to researchers from other fields. But it seems the ApJ is mainly interested in the correctness of a paper (rather than, as you say, whether it is fashionable). So the acceptance rate should depend on the ratio of correct and incorrect papers submitted. Here of course "incorrect" means fundamentally incorrect and unfixable. I don't know of any fundamental reason that there must be a large fraction of incorrect papers.

Should the use of a single referee be a cause for concern? For many (including myself) this is very surprising and goes counter to familiar practices. But the ApJ is being up-front about this, so any group that wants to change the practice could start a new journal or lobby the editors of the ApJ. I think this question is one the astrophysics community should handle for itself. So far, their answer seems to be "no".

Improve this answer
8
  • 6
    @arendellean: For what it's worth, the majority of journals in my field (mathematics) use a single referee, including some very prestigious ones.
    – Pete L. Clark
    Commented Nov 29, 2015 at 9:08
  • 4
    I disagree with the acceptance rate discussion. If a journal has a high reputation, it will receive a substantial amount of junk. If you had the reputation of a high acceptance rate, you would become the one place where everybody dumps their junk. I read that e.g. Science rejects 80% of articles prior to peer review. It's not as if every author would only submit "correct" papers - in particular not if you know a venue has a high acceptance rate. Because high reputation and acceptance rate attract low quality submissions, and you will have to either drop your quality, or your acceptance rate.
    – Has QUIT--Anony-Mousse
    Commented Nov 29, 2015 at 10:47
  • 8
    @Anony-Mousse What you don't understand about astrophysics is that it is a small community. Submit junk to the journal, the journal will ignore you, and then your career is over. I'm willing to bet the same phenomenon occurs in other niche fields, and that the fraction of junk submissions scales with the size of the community.
    – user4512
    Commented Nov 29, 2015 at 10:54
  • 5
    Simple correctness doesn't make a paper significant or any kind of quality. Stating that the only requirement to be published is to be formally correct is a very low scientific standard in most fields.
    – Greg
    Commented Nov 29, 2015 at 13:02
  • 5
    I do not know of a single astronomy-related journal employing more than one referee per paper. I think the reason is not exactly that it's a small community, but rather that it is a very fragmented one, and for a lot of topics of current research interest, I cannot imagine there to be more than a half-handful of possible referees. This ties into what @ChrisWhite said: publish garbage and it sticks to your name.
    – user42177
    Commented Nov 29, 2015 at 23:01
2

Just to give an example: Advances in Colloid and Interface Science also has an unusually high acceptance rate (I've seen numbers from ~65% to ~85%), and is very reputable / high impact. It has a much lower volume, though.

You can find acceptance rates for all Elsevier journals, but it's a very round-and-about process. I really wish they would give you the same list searchable by journal names. (I'm all ears if someone knows how to find this). Here's how I've been able to find them:

  • Go to the Elsevier Journal Finder: http://journalfinder.elsevier.com/
  • Put words in "Title" and "Abstract" that will give you the journal you're interested in (this is tricky)

  • The result is the 10 journals "most relevant for the title/abstract", with the following statistics:

    • Name and Scope (click "Scope and Information")
    • Impact factor
    • Acceptance rate
    • Average time to first editorial decision
    • Time from accept to published
    • Open access status (none/optional/mandatory)
    • Open access fee
    • Embargo time (time from published until access is free (as in, no subscription needed) )
    • License (Creative Commons etc.)
Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .