39

When you review papers submitted for publication, is there an “optimal” length for reviews? In my experience as an author and referee, I have seen a large range of review lengths (for reference, a paper in my field is typically between 3 and 8 printed pages):

  • Zero length: for some of the papers I have authored, the reviewer just clicked the “accept as is” or “reject” checkmark on the review sheet, without adding any comment (at least, not any comment visible to me). It's not so common, but it has happened.
  • Short length: a lot of time, I received reviews who consisted of a single paragraph.
  • Medium: one full page, maybe two pages.

Although I have never received any such long reviews, I have myself written on a few occasions reviews that exceed two pages, including once or twice a four or five-page review. These were cases where the paper was good, but could be (in my view) much improved and some of the aspects/consequences had escaped the authors' consideration.

I am wondering how useful reviews of various length are to editors. I often consider that “unmotivated” reviews are useless, as they do not give any real insight about the paper to the editor. For example, if the editor gets two conflicting non motivated reviews, how is she to decide?

On the other hand, although I write some from time to time, I have never received long reviews, so… is this something frowned upon?

  • 7
    I'm not an editor but as an author, I certainly prefer a long review full of detailed suggestions to a one-line "your paper does not meet quality standards, reject". Sometimes a reviewer ends up contributing more to a paper, than one of the "minor" co-authors. – gerrit Oct 30 '13 at 10:38
  • I know one of the journal reviews went on for a year and finally the author had to push the reviewers a lot to get the final results. This is the longest I have seen. – user2915398 Oct 30 '13 at 12:24
  • 1
    @user2915398 I was thinking in term of length of the written review, not time-wise – F'x Oct 30 '13 at 12:50
  • 1
    This is awesome! I have been wondering about that as well. And may I suggest the answerer to also mention if you are speaking as an editor or not? I value all inputs and tips but would also love to know some norm from the people in the industry. – Penguin_Knight Oct 30 '13 at 13:08
21

Reviews can be of quite varying length but obviously the extremes indicate some problems.

A review consisting of "Accept as is" would be highly suspicious in my mind (as an editor). It usually means the reviewer has not done any work, essentially no manuscript is that close to perfect (although it may of course happen). A review of "Reject" without additional comment is equally pointless (I am then assuming the journal has some form of quality check before accepting for review). An absence of comments is just a big warning sign since there is no perspective on why the MS is either perfect or perfectly worthless.

Considering the length of a review, it is governed by two factors: the quality of the manuscript and the personality of the reviewer. To some extent longer reviews indicate more questions to be resolved. At the same time some reviewers may be more nit-picking than others so that also influences the length. Based on my experience as an editor, I would say, as a rule of thumb, that at least a page of (single spaced) comments would be a basis for a descent review for a normal manuscript (15-20 pages double spaced excluding references, tables, figures) in the field experiment/observation based science where I work. A review of more than three or four pages of (single spaced) comments would be unusual and probably involve comments down to spelling issues. "A decent review" involves providing clear and constructive comments that will allow the editor to value the manuscript and the author to improve the manuscript.

So I would not say that a long review would necessarily be frowned upon, it clearly depends on how constructive it is. If someone spends a lot of effort improving language and grammar (which does not necessarily constitute the expectations from a review) that could be very useful. Normally such comments may be made as revisions in a file rather than a written report. So length is not a major issue, constructiveness is.

  • 2
    As an editor, how do you handle zero length reviews? If the review just says "accept" or "reject", do you grudgingly follow the reviewer's advice, or do you look for another reviewer? – Nate Eldredge Oct 30 '13 at 13:36
  • 6
    Most likely add another reviewer. The least one can do as a reviewer is to justify why one makes the recommendation. It does not necessarily have to be a long justification, but should provide a perspective on the paper and why it should be accepted/rejected. – Peter Jansson Oct 30 '13 at 13:45
24

Speaking from the point of view of an editor: One of the best reviews I ever got was longer than the paper. The author, a young researcher, had proved three theorems, one of which I recognized as a known result. So I asked the original discoverer of that known result to referee the paper. In my cover letter, I mentioned that I recognized one of the theorems as his, and asked whether the other two theorems had enough novelty for a publication. It turned out that the other two theorems weren't new either. The referee could easily have just given citations for those two theorems and recommended rejection. Instead, he gave me (or, really, gave the author) a long, clear explanation of the state of the art in that subject, and he suggested some open problems that the author could try working on.

  • 3
    I at least once wrote a review longer than the manuscript itself. The paper pushed some buttons of mine, and I couldn't stop writing. (Similarities with answers of mine here at Academia.SE are coincidental.) I think I recommended rejection and explained in considerable detail the non-trivial error the authors had fallen into. The editor appears to have appreciated this review; he still has me review regularly and introduces me as "the guy that writes reviews longer than the original paper." – S. Kolassa - Reinstate Monica Dec 2 '15 at 20:00
10

An "accept as is" option is useful after resubmissions; it signifies that no more work needs to be done. However, it is unusual to see that happen in an article on the first round of submissions. (I've had that happen precisely once in my career.)

Otherwise, I would say that the more detailed a review can be, and the more precise the suggestions for improving the paper are, the better it will be.

One to two pages is typically the norm; however, I have submitted a few three- to four-page reviews when I thought an article was already quite good, but could be better.

On the other hand, if a paper is already of relatively poor quality, I will explain the methodological or other significant flaws, but skip over an analysis of minute points; (it's simply not worth the time to rearrange the furniture when the roof is going to collapse any minute.)

2

As an author, reviewer, and key reader of a respected engineering journal, I can offer some perspective. The shortest review I received was one I solicited from a highly-respected professor at a prestigious university. His review was basically “This manuscript is not written well enough to be reviewed.”

The longest review I have received as an author was about five bulleted comments some of which were optional revisions and some minor but necessary clarifications; the shortest was one minor comment approving the manuscript.

As a reviewer I have on several occasions completely rewritten a non-English language author’s manuscript as a gratis professional service. To my surprise, I received thank you letters from the professional society publications chair and the editor thanking me for my ‘laudatory’ service. I took that to indicate my effort was unusual.

My shortest key reader review summary was to a VERY famous author who after a 22 page derivation, which he summarized as ‘simple’. With feigned seriousness, I ‘required him to remove the word ‘simple’ since he was on this uncustmary occasion communicating with mortals.

Not the answer you're looking for? Browse other questions tagged or ask your own question.