12

Sometimes the review process can take a long time, sometimes it can be very short. I suppose that one of the important factors determining the speed of review process can be the quality (i.e., the arguments) of the article itself but I am not certain whether the quality affects the speed positively or negatively.

Therefore, are good papers easier to review than bad ones or the other way around (on average)?

  • On one hand, if you are reading a good paper, you need to make sure you understand it well.
  • On the other hand, if you are reading a bad paper, you need to find why is it so poor and sell this to editors.

What I consider to be a "good" and "bad" paper:

  • Good: Paper which presents an argument that is correct or sound.
  • Bad: Paper with incorect argument (which can be refuted).
2
  • 3
    In addition to your first two bullets, if you are reading a middling paper, it is harder to decide and explain your opinion of it.
    – toby544
    Commented Jan 27 at 15:40
  • 6
    Another thing that can make a difference: quality of exposition. If a paper is clear to understand, and clear about its significance, then it is much easier to evaluate, and the review can be completed faster.
    – Kimball
    Commented Jan 27 at 22:51

5 Answers 5

19

Very bad papers are easiest to review. The reviewer needs to point out one fatal flaw (e.g. see this paper which got the same results ten years ago; or Proposition x is wrong and here is why) and the reviewer is done.

Very good papers are still hard to review, since a good reviewer will vouch for the accuracy of the paper and needs to understand the paper thoroughly.

Bad papers are still time consuming because the faults and the reasons for not publishing are more subtle. Also, if the authors are inexperienced, the reviewer has a moral obligation (to be weighted against all other obligations) to make helpful comments.

2
  • What if "Proposition x is wrong and here is why" is just one part of the article (i.e., fatal for certain part of the argument but not for the argument as a whole)? Can you still end the review, then? Regarding the obligation to make helpful comments, why is this not associated with any review but just review of inexperienced authors?
    – Athaeneus
    Commented Jan 27 at 14:43
  • 14
    @Athaeneus If that proposition is central to what comes next the rest doesn't matter. Thomas isn't talking about some side-issue in a reference. It's not up to a reviewer to exhaustively edit the author's manuscript, only to determine it's suitability for publication. If one flaw breaks the whole thing, why go into the details?
    – Bryan Krause
    Commented Jan 27 at 15:06
29

For the vast majority of peer reviews, the deciding factor is when the reviewer finds the time for it amongst all of their other duties. Quality doesn’t factor into this.

My peer-review workflow is that I start reading the manuscript when I have a gap in my other duties (that is too small to reasonably fill it with something else). Since pausing the review makes it more laborious and worse, I then try finish it as soon as possible. In the vast majority of cases, I submit my review within a day after I started reading – irrespective of the quality of the manuscript. Rather, the key factor is when I start reading, and until then I don’t really know the quality of the manuscript.

There are some factors which correlate both with review time and quality. For example, I know somebody who punishes the authors of particularly annoying manuscripts by delaying the submission of the review as long as possible (and no, I don’t consider this ethical). Other reviewers might accelerate peer reviews of manuscripts that are doomed from the abstract to save other reviewers from having to read them. However, I would find it unlikely that such effects suffice to make quality an important factor, not only because of their magnitude but also because I don’t see a clear tendency as to whether bad manuscript take longer or shorter.

1
  • There's a fine line between punishing the authors by submitting as late as possible, and finding it too much trouble to wrap up a review of an annoying paper so causing a long delay - but I assume your acquaintance is clear about which side of the line they fall.
    – Chris H
    Commented Jan 29 at 11:48
10

The quality of a paper can affect the amount of time that it takes to perform a good review in a number of ways, as noted by other answers.

However (and I realise this isn't something you asked, but it's something that others will take from reading it), the time that it takes for you, as the author, to hear back - i.e. the time from submission to decision - is not correlated with the quality of the paper in any way that you can have confidence in. So you should not draw conclusions about the quality of a paper from the length of time you have waited.

This is because, as one of the other answers has mentioned, most of the time between submission and decision is taken by finding reviewers, and then the reviewers finding time to read the paper. The actual time spent reviewing (hours) is much less than the total turnaround time (weeks to months).

5

There are too many variables so it can't be predicted. Some "well written" papers have little that is new in them and can be easily (quickly) rejected. Some, on the other hand, have deep ideas that need to be carefully evaluated, which takes time. I'm thinking of theoretical stuff here, such as in math and CS for example. This would be especially true for something rather radically new.

On the other end of the scale, some poorly written papers are easy to reject as they don't make a lot of sense to either the editor or the reviewers leading, perhaps, to desk-reject. Others, again, can have deep ideas and deep analysis and, probably many re-writes to reach the publishable stage. A reviewer might recognize that early on and struggle with the analysis.

But if you have the option (yes, yes you do) to write a good well-written paper with clear explanations, then do that.


A paper showing how to Square the Circle will be quickly rejected whether written elegantly or not.

A paper presenting the Theory of Everything will take a long time to review, even if elegantly written.


I'm making a bit of a guess here since I no longer read German and don't know the "quality" of the writing itself, but Einstein's 1905 Special Relativity paper was reviewed quite quickly though it contained deep concepts. But I think he had already discussed those ideas fairly widely with those who would be the most obvious reviewers, so reviewing it wasn't a particularly time consuming task (guessing).

4
  • So deep ideas can balance out poor ideas, to some extent? If the reviewer recognizes one very deep idea among plethora of ballast, do they usually stop the review there without acknowledging it?
    – Athaeneus
    Commented Jan 27 at 14:47
  • 2
    Actually, I was referring to the writing itself. A paper full of garbage with one good idea will probably be rejected with a note to follow up on the hidden gem if it is seen. Rejection probably quick. Some people with good ideas find it hard to express them to others.
    – Buffy
    Commented Jan 27 at 14:49
  • 3
    IIRC, peer review as we know it today was not a thing in 1905. Einstein had some notably negative reaction when he was first subjected to peer review thirty years later.
    – Wrzlprmft
    Commented Jan 27 at 15:19
  • 1
    @Wrzlprmft Yes, in those days papers submitted to Annalen der Physik were decided on directly by the editors. The editor for theoretical physics happened to be Max Planck. He certainly was very capable of judging the papers and quickly realized their importance, but it's noteworthy that the journal's overall rejection rate at the time was very low (about 5-10% from memory).
    – Anyon
    Commented Jan 27 at 15:36
4

Assuming the editorial process runs how I know it, the fastest decisions are made when an editor decides to desk reject a paper. These are (normally) bad papers, and they are dealt with fast, because no review process has to be started (of course there can always be factors that delay the process that don't have to do with the paper quality such as an editor having to put loads of time into other things).

The next factor is how easy it is to find reviewers. This depends on many things apart from quality (for example whether the editor knows good experts for the topic of the paper who are generally keen on reviewing; as editor by the way I know one or two topics where hardly anyone good who writes about it does reviews - very annoying). Quality can play a role insofar as reviewers may decline to review badly written papers, or they will take quite a while, or even not do a promised review, in case the paper gives the impression that something good and worthwhile is there, but it isn't well elaborated and it is hard to get to the bottom of it. So the papers that are most problematic are those that potentially have some really good content that is badly written or surrounded by things that need criticising such as unfair representation of related/competing work, inappropriate "marketing" of an idea, issues with mathematical formalism or application/interpretation of results from other fields etc., bad and over-convoluted writing, or excessive length caused for example by many side remarks that don't add to the main point of the paper. If the paper is bad enough to deserve a rejection, it often isn't worthwhile to point out all issues in detail, but if there is a potential for acceptance, effort needs to go into improving it, and this can cost time.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .