Take the 2-minute tour ×
Academia Stack Exchange is a question and answer site for academics and those enrolled in higher education. It's 100% free, no registration required.

Sometimes, when I get the comments of the reviewers of a submitted paper (mine or some else paper), I feel like the reviewers just ignore the paper. More precisely, I feel that reviewers did not read the whole paper. Even more, I feel like they do not even complete reading the abstract (it is a conclusion from their comments of course).

Is it possible that reviewers ignore papers? Maybe because the authors are not known or maybe they did not write the paper well?

My only argument is that authors believe that their paper is a breakthrough paper and it has a very good contribution. Imagine reviewers start reading a paper in a high journal rank and they found (just examples, do not take them as serious ones): "This paper shows that 1+1=3" or "This paper demonstrates that if the determinant of a matrix is 0, the matrix still has an inverse."

Although, I do not think that in a high quality journals, reviewers can do some kind of ignorance. How can one be sure that their paper is well treated? How to write papers that make the reviewers do not throw the paper away.

share|improve this question
10  
Do reviewers sometimes do a bad job? Yes. Do reviewers sometimes focus on minutiae? Yes. Do authors sometimes think their papers are better than they actually are? Yes. –  Thomas Jun 17 at 20:40
    
    
Thank you very much. –  det Jun 17 at 20:46
1  
Also related-ish: Paper rejected. Should I appeal against biased reviews? and I believe I have solved a famous open problem. How do I convince people in the field that I am not a crank? (both examples where authors feel their paper was not reviewed properly) –  ff524 Jun 17 at 21:10

3 Answers 3

up vote 12 down vote accepted

Let me expand a bit on Thomas comment.

By its very nature, peer review is a guaranteed item of discussion in all academic discussions. Every author has stories to tell about unfair reviews, reviewers that missed the entire point of the article, reviewers that rejected a paper only to publish something similar themselves, and so on. What is much rarer talked about is that the vast, vast majority of rejected papers are simply not good enough! Indeed, this surely also includes a large percentage of those papers that the authors are sure were handled unfairly.

An important point is the following:

My only argument is that authors believe that their paper is a breakthrough paper and it has a very good contribution.

Clearly, all (ethical) authors believe their paper has merit. Otherwise, one would likely not submit it. It has to be said that the author's own opinion is not a good indicator of the actual qualities of a paper.

Is it possible that reviewers ignore papers? Maybe because the authors are not known or maybe they did not write the paper well?

Somewhere on this site, I have previously listed reasons why a paper can get rejected, but to be honest I think "the authors are not well-known enough" is very low on this list. The second reason ("the paper is not well-written") is more common, but it should be noted that rejecting a badly-written paper is not unfair. It is the responsibility of the authors to make sure that their paper is graspable. However, again, I suspect the majority of papers are well-understood by the reviewers, and just rejected because the reviewers were of the opinion that the contribution was not substantial enough.

So why do papers sometimes get very short, high-level reviews, even if the reviewers read and understood the papers? One possible explanation is that many reviewers follow the golden rule "spend your time on manuscripts that deserve your attention". In practice, a reviewer has finite time to invest into reviews. When receiving (say) 5 papers for review, two being borderline and three being clear-cut rejects, it is an understandable decision to spend more time scrutinising the papers that still have a chance, and only provide high-level feedback on the ones that the reviewer sees as clearly too weak.

Now on to your questions:

How can one be sure that their paper is well treated?

The philosophy I try to communicate to my students is to stop worrying about optimizing manuscripts for peer review and start doing the best research they can. I am convinced that the best way to ensure that your papers are treated well is to write good research papers.

How to write papers that make the reviewers do not throw the paper away.

Make sure that your papers (1) have a clear contribution, which is evident from the abstract and the introduction, (2) are written as clearly as you can, (3) are correct in terms of grammar and spelling (try to have a native-speaker spell-check your manuscripts), and (4) have a compelling and relevant contribution. If Your paper is technically sound but boring as watching grass grow, you are not entitled to complaining that the reviewers did not read the manuscript carefully.

share|improve this answer
4  
The best way to ensure that your papers are treated well is to write good research papers. This. As a reviewer, I love reading (and usually recommending acceptance of) really good papers. –  ff524 Jun 17 at 21:17
    
@ff524 Indeed. Unfortunately, it is a pleasure that I can enjoy way too rarely. –  xLeitix Jun 17 at 21:19

I doubt that it is common for peers to disregard the content of a manuscript under review to that extent. However, it is quite usual for authors to understand more on their paper than the reviewers do. The following simple things may help:

  1. accurate title, that really absorbs the content of a paper
  2. very well written abstract, that says precisely what is done in a paper
  3. well written introduction

  4. a clear cover letter, explaining what is done and why this is important and new;

  5. recommendation of at least a couple of referees for your paper who should understand it.
share|improve this answer
    
I'd like to emphasize 5. In fact, if you can name many potential referees (say on the order of 10), it doesn't look like you are cherry picking and you are likely to get people directly from your list. If you only name 3, they are likely to take people who aren't on the list, for several reasons. –  MHH Jun 17 at 21:11

Your question is highlighting one of the major problems of the current peer-review process. Numerous studies have been conducted on various parts of peer review, and one of the most consistent findings across these studies is the absence of reliable reviews. That is, reviewers often disagree with each other, and editors often disagree with reviewers. At the same time, these findings are not at all surprising.

I do believe that there are very conscientious reviewers, and those numbers will vary considerably over time and by discipline. I believe there are many persons who are simply unqualified to review for journals. And, I believe that some reviews are not completely objective. Some people may be particularly critical on a given day -- perhaps their grande mocha latte arrived with no-whip when in fact they wanted whip. Or, they just had one of their own papers rejected, which makes them think that the system of reviews is becoming increasingly tighter, so the subsequent reviews are made under a more critical lens.

The bottom line is there are many places where there are faults in the peer review process, and this question certainly highlights one of them. Proposals for improving the system regularly enter the debate, but the current peer review system is not likely to change much over time.

If the review is particularly egregious, you can certainly talk with the editor. But, as a submitting author, you will always have less power and influence relative to the reviewer.

share|improve this answer
1  
I am not convinced that the "state of peer review" is even close to as bad as people make it out to be. –  xLeitix Jun 17 at 21:15
    
@xLeitix Is this statement based on empirical findings of the peer review process, or your general appraisal? Do you disagree with the empirical findings? –  Brian P Jun 17 at 21:18
    
I have to say I am only aware of one study into this direction in computer science, and the results therein were interesting but methodically rather unconvincing. Mostly, my statement is based on personal impression. From a practical point of view, if you have conferences with an acceptance rate of <20%, it seems unsurprising to me that you find many more people that are pissed than satisfied ones. That does not necessarily mean that the selection system is indeed inherently broken. –  xLeitix Jun 17 at 21:24
    
Anyway, I guess it depends largely on what you expect. If you expect a system with a number of false positives/negatives close to 0, peer review certainly does not deliver, but I am not sure if you will be able to find an alternative that works as you hope. –  xLeitix Jun 17 at 21:26
1  
Numerous studies have been conducted on various parts of peer review - perhaps you could link to a couple of these and quote their specific findings. Otherwise those w/o database access have to just take your word for it. –  ff524 Jun 17 at 21:42

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.