As a reviewer, when would it be appropriate to reject a paper (rather than recommend "major revision") for publication? I am interested in views from seasoned reviewers/editors who served for journals with moderately high importance factor (IF > 4.0) - primarily in engineering/science fields. Thank you,
First, I'll take the position that if a paper can be salvaged, then the author should have the opportunity to try to salvage it. However, some situations are beyond help.
Situations that call for rejection
The paper is out of scope for the journal/conference. But usually an editor will recognize this and send it back.
The results of the paper follow trivially from well known facts.
The result itself is trivial. That is, not worth the paper it is written on. Obvious. Sometimes the question, itself, is trivial.
The author seems unaware of recent work that impacts on the results. Here, it may be a judgement call about how much was missed. Naive author.
The methodology is unethical.
The methodology is inadequate to answer the question posed.
The results don't follow from the methodology.
The results can be shown to be wrong. Or dangerous.
If revision has been offered and the author is overly argumentative about revisions.
Plagiarism or other ethical lapses.
My personal view is that poor expression/writing should not be grounds for rejection as long as the author can find a way to make the presentation clear enough for a professional reader. However, writing can be so poor that the reader can't make a connection between the methodology and the result. If you can't understand the paper as a reviewer, you may not be able to make any helpful suggestions. Then, you may be forced to reject.
I might have missed a few cases, of course, and it depends on the field. But generally, leave it to the editor to make fine-grained decisions.
Note that I don't think the answer to this depends, really, on the level of the journal. I think the "scope" qualification covers that aspect pretty well.
I don't think that there are a lot of "questionable" cases here.
The implication is that one prefers to suggest revision unless one cannot.
Actually, I think a much harder situation is one in which a paper seems "obvious" for rejection but has some gem of insight in it that requires tweaking out. This might imply a truly huge revision and acceptance of something in the future. These are hard to recognize, of course, due to the limits of the review process.
As a referee of a scientific or engineering paper, you should recommend rejection if you do not believe that (1) the paper makes a contribution that is worthy of publication and/or (2) the paper is not a good fit with the quality or mission of the specific journal.
Sometimes referees want to be encouraging and therefore suggest major revisions for papers that may not be likely to ever meet the bar for publication. When a major revision recommendation is not focused on how the existing results are presented but instead on developing new results (or proposing significant new experiments, or new models, or new hypotheses, etc), then it would have likely been better to simply recommend rejection.
In the end, the decision is made by the Editor and you can only give your opinion. Unlike the Editor, who has to chose at some point, you are allowed to be in doubt. In your review you can say "I am struggling to give a specific recommendation because on the one hand -positive comments about the paper-, but on the other hand -negative comments -". Just be careful because in most journals I have reviewed for, these specific recommendations are supposed to go to the Editor only, not to the authors.
The editor will average your comments with the ones from the other reviewers. For high impact factor journals, the difference between recommendations for rejection and major revision might not even make much of a difference: they receive so many articles that they can just reject anything beyond "minor revision".