Can I recommend rejection of a paper after simply reading its Abstract and Introduction?

Question

I am reviewing a paper of a top-tier computer science conference. After reading the abstract and introduction of the paper, I think that the paper does not have that standard to make it up to the conference reputation. In other words, the problem that they have solved is a minor one and perhaps belongs to some low-tier conference.

Can I simply recommend to reject the paper without reading it further? Is it right to do so? I am feeling that it is a waste of time to read it further. Or should I must read the full paper and verify their claims and suggest the improvements (if any)?

---

Note: I am somewhat new to reviewing; this is the 6th paper that I am reviewing this year.

Answer 1

It is of course possible that a single portion of a paper will be sufficient for rejection, so you can recommend rejection on the basis of a single part alone. However, you should remember that the best practice for a peer-reviewer is not just to reach a recommendation but to offer useful advice to the author on how they might improve the paper. As a reviewer, you have an opportunity to help the authors of a paper make it better for the next submission.

As a secondary matter, except in extreme cases, failing to read the entire paper may be viewed as a bit of a dereliction of duty by some people --- I would certainly view it that way. Even desk-rejections will almost always say that the editor read your paper, so even here, the author is given the courtesy of having their paper read in full.

Answer 2

must read the full paper and verify their claims and suggest the improvements

Read again the letter editor sent you. Why did they want your input on the paper?

In my field, the review is not about "rejecting" or "accepting" but about finding errors and flaws, as well as good points, and assessing the value of the artefact. The editor will decide whether these issues are suitable for rejection, acceptance, or acceptance with modifications.

For example, here are the top-level questions that editors ask reviewers in PNAS:

• Suitable Quality?
• Sufficient General Interest?
• Conclusions Justified?
• Clearly Written?
• Procedures Described?
• Supplemental Material Warranted?

As you can see, only one question is about suitability for the outlet. The others are about the quality of the research paper as it stands.

Answer 3

I have two points, one objective and one subjective, why you should not do this.

Objectively, it is quite common to form an opinion of a paper only by reading the abstract and introduction. I have heard that most reviewers decide by the end of reading the introduction whether they want to recommend revision or rejection; reading the rest of the article does not usually change this initial impression significantly. However, sometimes a promising introduction might fail to deliver and so an initial inclination towards recommending revision might change to a recommendation of rejection. On the other side, sometimes a lackluster introduction might be followed by an article with real potential, in which case the reviewer who initially felt inclined to recommend rejection might realize that the introduction needs to be rewritten to properly highlight the value of the article. So, objectively, you cannot properly evaluate an article until you have completely read it. Bad beginnings might end well, just as good beginnings might end badly.

Subjectively, it is irresponsible and even morally wrong to claim to give a scholarly review of an article that you have not read completely. Would you write in your evaluation to the article, "My recommendation for rejection is based only on reading the abstract and introduction of the article; I saw no need to read further"? If you would not openly write that, then that should tell you that there is something wrong with doing that. I am sure that you would feel rather upset if you found out that a reviewer of one of your articles did that to you. So, subjectively, it is wrong to recommend rejection of an article you are asked to review if you have not read the entire article.

Answer 4

As a small addition to the other answers:

Whatever you choose to do, please be honest and open about it towards the editor, the other reviewers and the authors. If you really made your recommendations after reading the abstract and the introduction, (and think that was quite enough), then say so in your review. This helps the others in putting your review in perspective.

Answer 5

Look at this through the author's perspective. They spent time working on this research and writing the paper - probably because they believe it is important enough to be worth their time and the readers' time. Now, as a reviewer, you may think that this is wrong and the research is not so good to merit publication, because

the problem that they have solved is a minor one and perhaps belongs to some low-tier conference.

Perhaps you are right and the author's are wrong. But how helpful would it be for the authors to receive a rejection stating only this? Surely, they would like to know how to become a better researcher and how to increase the chances for their next paper to be accepted. Can you help them with it? Can you offer constructive critique and helpful suggestions for improvement? This will make the whole experience more pleasant and rewarding and ultimately worth everyone's time.

Answer 6

As others have pointed out, your responsibility is to help the editor decide what to do.

I don't think you can fairly recommend rejection without actually reading the paper. You could write the editor declining to review because, based on the abstract and introduction you don't think the paper is significant enough to warrant the time it would take you to read the manuscript carefully.

Then the editor can find other reviewers, or agree with your assessment and desk reject.

Answer 7

In theoretical CS I wouldn't recommend this. If mathematics I would definitely suggest it would be improper. You don't have enough information on what the future value of the paper might be.

It is possible, though unlikely, perhaps, that the techniques used to prove a "minor result" are actually more valuable than the result itself. Those techniques might, in theory, be used to establish other, much more important, things. How something is done can be much more important than what is done; certainly in mathematics and possibly in other theoretical fields.

I'll agree that the authors should have pointed this out, of course, if they thought it was important.

If you told me, as an editor or conference program chair, that you'd done this, I'd likely feel like I needed additional advice before rejecting the paper. You are admitting to providing little advice of value.

Answer 8

There are two scenarios both consistent with the data that you have described.

1. The paper makes only a minor contribution to the field and you should recommend rejection from this journal.
2. The authors have found an important result but failed to effectively describe/"sell" its importance in the abstract and introduction. This is not uncommon when researchers are wrapped up in their own field, they see the importance and feel it should be obvious to the reader. Good scientists are not always good writers. Here you might need to recommend revisions in order to better highlight the contribution of the paper.

If you stop reading after the introduction, you don't know which scenario is the case, and you are doing the authors a disservice by assuming (1).

If you were submitting work for feedback, would you feel satisfied if someone read only the first section and returned it to you saying it wasn't good enough?

Answer 9

If a math journal is asking for a quick opinion, then yes it's ok to recommend rejection based on just the introduction but you must actually read the introduction carefully! If the introduction clearly lays out the main results of the paper and you think those results aren't interesting enough to merit acceptance at the level of the journal where it's submitted then there's no need to read the rest of the paper carefully.

By contrast, if they've already gotten quick opinions and are asking for a more thorough refereeing job then I think it's inappropriate to not read the paper more thoroughly. Remember that other people may have different opinions about the merits of the paper.

Answer 10

Based on the other answers, I guess this will probably be a contrarian opinion, although to be honest I suspect this is probably how many people behave even if they don't admit it. I don't think you have an obligation to thoroughly read the paper, although I think you should spend, say, an hour or so going through it. For a top tier journal / conference, "results are not sufficiently impactful for this venue" is a valid reason to reject the paper. I think you should read the discussion, main figures/results, and read through the text (not necessarily checking every step) to make sure that you understand what is there and can summarize it. You should at least be able to confirm that the content of the paper matches what is in the abstract and introduction, or else find if there is something in the paper that you need to look into in more detail because it could change your opinion.

In an ideal world, every referee would thoroughly read every paper they were sent and send detailed feedback on how to improve the paper. In the real world, people are busy, and if you already think the headline results of the paper are not sufficiently interesting for the top tier journal/conference you are reviewing for, I don't think you are obligated to spend days checking every claim.

It would be different if the journal/conference were mid or lower tier... then I would tend to say that something correct and novel could be publishable even if the impact of the main results was not very large. Judging correctness requires more work to actually read the paper and check things, compared to judging the importance of the headline results, assuming you have a good sense of the quality standards in the field.

Answer 11

I’ll sacrifice politeness for the sake of clarity: If you have time to write here - you have time to read the paper. You may feel that you can’t understand it (I’ve seen such badly written) but at least you should give a try.

Answer 12

(copy-pasting directly from the OP comments, to enable room for discussions)

In some (most?) top-tier CS conferences, papers are screened before they are assigned to reviewers. Usually, this involves checking for obvious plagiarism, but it can also extend to verifying whether the manuscript is (just from its looks) acceptable. The idea is to maximize the gain from reviewers: assigning 5 papers that do not even "need" reviewing to be rejected would waste the reviewer's time; and would also waste the reviewer potential contribution to the proceedings.

If the paper passed such screening it's likely that the abstract was deemed "somewhat" appropriate for the venue.

You should read it thoroughly and - moreover - question whether YOU are the problem (ie, lack of expertise, which is common and acceptable): if this is the case, then avoid clear statements and do mention that your recommendation is just an educated guess.

If the abstract is really, really bad, then you can still recommend a rejection. But - for the sake of science - try to provide at least some additional motivations.

Answer 13

I wouldn't base a decision to accept or reject a peer-reviewed paper based solely on the abstract and introduction sections and an opinion "the problem that they have solved is a minor one." Perhaps there is more in the paper that authors haven't explored and perhaps their focus is not correct.

There's a plugin in chrome called Read Shit Faster (ugly name but what a powerful tool it is) that allows you to read the paper in a super-easy manner allowing you to detect major flaws almost instantly and know what to look for in a more thorough reading.

The order in which I read carefully these contributions goes like this:

1. Abstract: What am I suppose to learn from this piece? What is it that the authors claim to have found that has not been addressed before?
2. Conclusions: Yes, I go to that section right away to see if it is consistent with the abstract and if they are delivering what was promised.
3. Methods: If this section has problems from the experimental plan, setup, tools, etc., I will start considering rejection.

By now, you should have seen if the language is correct or if there are major problems in this department. I collect those right away to either help the authors write a better paper or simply to justify a rejection in case there is no substantial contribution.

If at this time I consider that the authors have something worth reading, I'll read their Results and Discussion section to check for consistency with the sections that I have already read and then I will move to he references section to see how up to date is the bibliography. By the way, I would dig some extra references that are more current to suggest some improvements for the authors and perhaps to see if there's something remotely similar that has been published... Sometimes, if you know the authors, you may see if the work has been published before and point those works to the editor. Turn it in can also detect plagiarism (intentional or not) and you may point that to the conference organizers.

Typically, by the end of a Read Shit Faster session I would have made my mind in regards to reject, major editing needed, or accept (rarely) a paper. The remaining process is to document everything.