I feel bad about rejecting a paper during review

Question

I'm must declare at the very beginning that I'm a PhD student that suffers from major impostor syndrome, and very much feel very unconfident about myself or my abilities.

That being said, I recently received a paper to review for a conference, and when I read the paper I felt it wasn't well-motivated, and the introduction was shaped badly. Even though I felt that the rest of the paper was okay, it was generally hard to follow for me. I'm someone with related experience in the field of the paper, I understood much of the paper after a thorough look, and evaluated based on my opinions. After a deep battle with myself, I decided to 'weak reject' the paper and sent in my reasons to do so.

However, the other submitted reviews were quite positive, and now I feel like maybe I didn't do justice to the authors' work by rejecting it. As I looked further into the draft, I saw some basic things that I didn't consider before. So maybe I didn't understand it well enough because I'm just stupid, and not because the work was actually bad. Now I feel extremely guilty and feel like I'm not capable of sending confident reviews anymore.

Perhaps this is a question for my therapist rather than you all, but being experienced in academia, I want to know if these feelings are normal, how to cope with them, and is it okay to reject a paper the way I did?

Answer 1

I decided to 'weak reject' the paper

No you didn't. It is the editor that rejects the paper, or not. You merely provided input based on your technical expertise, which it sounds like you did in good faith and with considerable care.

and sent in my reasons to do so.

This is the key point. Editors do not make decisions simply by tallying the votes; rather, they will consider the technical points you raised, and those of the other reviewers.

I want to know if these feelings are normal, how to cope with them, and is it okay to reject a paper the way I did?

As you move up in your career, you will find yourself doing this more and more. Eventually, you will be regularly giving students bad grades, rejecting candidates for jobs, and reporting students for misconduct (or making similarly difficult decisions outside of the university). The feeling of second-guessing your decisions never really goes away, but at some point, you get used to it.

I find it helpful to trust that it's pretty rare to be the single point of failure in someone's life. For example, even if you were the sole decider for this paper, the authors would just submit somewhere else; it's not like you torpedoed the paper forever. Similarly, if you fire someone, there is normally a long series of events (validated by other decision makers) before it reaches the point where you have to make the final decision. And it's also helpful once you have more experience on the other side of bad decision making -- for example, if you accept a bad paper and it ends up being publicly criticized or retracted, then you'll feel less bad the next time you recommend rejection.

Answer 2

First and foremost, you don't make the final determination, and thus have a somewhat limited responsibility. For a conference, some section chair or committee makes the final determination. It is their job to read all the reviews, and make a determination as to suitability for a conference. They had other reviews, and could certainly have looked at them and accepted the work.

In my field, my experience is that conference standards are lower than those of a good journal by design -- the research is a bit more cutting edge and raw. Referees may or may not know the standards, and again, it is the responsibility of the section chair to make the determination. The best you can do is be specific about the problems you see, so the responsible party has the tools they need to make the right determination.

Whether you're too tough or not, that's a different story. We all get our experience some way, and reviews for conferences are a great place to start.

My personal policy for reviews of anything short of a grant is that I have no reason to write down anything that I would hesitate saying to the author's face (whether the review is blind, double blind, or signed). Whether this is the right policy for you, that's a personal choice. It's different for reviewing grants, where a single review can have a profound effect on a career.

Answer 3

As a counterpoint to the world-view expressed in other answers: I've never understood claims that grad students or recent PhDs could be good referees for serious papers. The lack of experience, if nothing else, diminishes perspective...

In particular, if/when a novice doesn't understand something in a paper, especially larger issues, it's not surprising. Yes, we might imagine a standard for writing which makes things intelligible to people not-quite expert... but that is (in math) not the standard for writing. Intelligibility to experts (plus meeting certain implicit stylistic expectations) is the goal.

In mathematics, anyway, it is not so easy for even very talented beginners to be able to appreciate/appraise serious papers. Despite some mythologies, getting a PhD doesn't really make anyone an expert in anything. It's a beginning. People equally capable and with decades' head start on a beginner operate in a much different context. No moral virtue there, but definitely scientific/intellectual virtue.

But, in your particular case, you did the best you could. Good to learn something about your own limitations, such as they may be. Don't worry, except to try to be a better scholar for future episodes.

Answer 4

Don't feel bad and don't assume you are less qualified than other, more senior reviewers. Often, reviewers at the beginning of their "reviewer careers" are much more thorough in their work.

My first review was what I percieved to be a very bad paper with many errors and shortcomings. It was within my specific area of expertise, though, and I was very much able to judge the correctness of what they were writing. I wrote a very detailed review and voted to "decline". A few months later, I got the paper to review for a second time (the editor had decided for "major revision"), along with the comments of my fellow reviewers: most of the points that I raised had not been mentioned or noticed by the other reviewers.

Because of my comments, the authors revised several sections of the paper, redid some experiments and corrected wrong references. Whithout me bringing up all these points and my harsh judgement, the paper might have sailed through with only minor revisions and a bad paper would have been published. Because I was harsh, a better paper was published.

I also initially felt bad after I handed in my first review, and even wrote a private letter to the editor, explaining that I am not 100% sure about my judgement because this was my first ever review. But afterwards, it felt right and I have learned to trust my judgement.

Being able to honestly judge the work of other people (including saying it is bad when it really is bad) is an important skill in academia, but it takes time to acquire. Keep up the good work.

Answer 5

As others have said, you did your best as a reviewer, given your level of expertise and perspective on the field.

Other responses also point out that graduate students may not be the best to review research papers due to lack of experience. But, I don't fully agree.

Due to this impostor syndrome, I've declined to review many papers, though I am sure I would have done better, as a student, than 30% of the reviewers that reviewed my papers.

In your case, I would suggest to keep reviewing papers and getting better, even if it is at the expense of a few authors who see delays in their paper publication dates due to you.

There are plenty benefits to it. First, you learn to read papers better, frame them better in the big picture of the field. Second, you learn to write your own papers better. Third, if you are a consistently honest reviewer, you will start adding much more value to the field than you subtracted when you were just learning to review papers. Fourth, and most important, it helps you become a better scientist, by teaching you what is important and what is less important in your field and who the strong players are. All these will help you in your future career, even on the short term. For example, it's one thing to give an interview talk stating what you did and how, and one other thing to be able to tell exactly how your work is advancing the science and what it is built upon and what can be built upon it.

Related to the impostor syndrome, I don't think it's a psychological problem that should require years of therapy. When you do serious research, you are constantly humbled by your apparent powerlessness in the face of the infinite unknown. Whatever knowledge you may have, there is always need for more, and whatever skills you may have, they are rarely sharp enough. Faced with this constant humiliation, you are bound to experience the delusion that you're not worthy of being called a scientist.

But, so are the others that do science, regardless of their status in the scientific community. So, the only way to cure this ailment is to look at what your peers are doing. You'll probably notice that you are not worse than most, even at reviewing papers. There are outliers, some people that are exceedingly good, and, at the same time come from very strong places, but, even those people suffer or suffered from this disease.

In short, stop worrying and keep doing your best, and improve your work with every iteration.

Answer 6

At least, in my field, when an editor gives a paper to a Ph.D. student for review, they have probably already decided that they are going to reject it. Your job is to find reasons to reject it, and to give the authors helpful feedback. As a Ph.D. student, you should probably only recommend publication of papers that are unexpectedly good. You should feel uncomfortable letting it through, not rejecting it.

Give the authors useful feedback, positive and negative, and then feel very comfortable rejecting it as that was probably what the editor wants/expects you to do. You are just doing your job.

Talk with someone senior in your field or maybe at the journal to make sure as some practices do vary by discipline. I think you will find that this is the case.

Answer 7

Something you should learn during your doctoral studies is to have confidence in your own judgement. This comes with time, but you can speed up the process by remembering that it is YOUR opinion and nobody else's. As long as you can justify it with facts, quotes etcetera, it is not a personal attack on the author[s] of the manuscript that you are reviewing.

Practise being more confident !

Answer 8

It is true that the editor makes the final decision, but this decision is highly based/relied on the reviewer's comments. I know a Ph.D. student who couldn't sleep well for a few days because the paper that took months of research got rejected. In a case like this, you can always adjust your decision when you think it is the right thing to do.

Answer 9

If this worries you, there's a fortunately easy solution: never recommend rejection at the first review; always recommend revision. The line between revision and rejection is a fuzzy one, and no one will blame you if you recommend revision for a paper which you have serious doubts about. If, when you get the paper back after revision, you think the authors have not made a real effort to address your remarks, then you can still recommend rejection.

Based on your description I would also guess that the paper was sent for revisions and you're now looking at the revision. You can still recommend acceptance. It's not common for a reviewer to flip from recommending "reject" to recommending "accept", but it's not rare either. Of course, only do so if you think all concerns have been addressed.