40

I'm must declare at the very beginning that I'm a PhD student that suffers from major impostor syndrome, and very much feel very unconfident about myself or my abilities.

That being said, I recently received a paper to review for a conference, and when I read the paper I felt it wasn't well-motivated, and the introduction was shaped badly. Even though I felt that the rest of the paper was okay, it was generally hard to follow for me. I'm someone with related experience in the field of the paper, I understood much of the paper after a thorough look, and evaluated based on my opinions. After a deep battle with myself, I decided to 'weak reject' the paper and sent in my reasons to do so.

However, the other submitted reviews were quite positive, and now I feel like maybe I didn't do justice to the authors' work by rejecting it. As I looked further into the draft, I saw some basic things that I didn't consider before. So maybe I didn't understand it well enough because I'm just stupid, and not because the work was actually bad. Now I feel extremely guilty and feel like I'm not capable of sending confident reviews anymore.

Perhaps this is a question for my therapist rather than you all, but being experienced in academia, I want to know if these feelings are normal, how to cope with them, and is it okay to reject a paper the way I did?

1
  • Comments are not for extended discussion; this conversation has been moved to chat.
    – eykanal
    Nov 21 '21 at 4:24
93

I decided to 'weak reject' the paper

No you didn't. It is the editor that rejects the paper, or not. You merely provided input based on your technical expertise, which it sounds like you did in good faith and with considerable care.

and sent in my reasons to do so.

This is the key point. Editors do not make decisions simply by tallying the votes; rather, they will consider the technical points you raised, and those of the other reviewers.

I want to know if these feelings are normal, how to cope with them, and is it okay to reject a paper the way I did?

As you move up in your career, you will find yourself doing this more and more. Eventually, you will be regularly giving students bad grades, rejecting candidates for jobs, and reporting students for misconduct (or making similarly difficult decisions outside of the university). The feeling of second-guessing your decisions never really goes away, but at some point, you get used to it.

I find it helpful to trust that it's pretty rare to be the single point of failure in someone's life. For example, even if you were the sole decider for this paper, the authors would just submit somewhere else; it's not like you torpedoed the paper forever. Similarly, if you fire someone, there is normally a long series of events (validated by other decision makers) before it reaches the point where you have to make the final decision. And it's also helpful once you have more experience on the other side of bad decision making -- for example, if you accept a bad paper and it ends up being publicly criticized or retracted, then you'll feel less bad the next time you recommend rejection.

9
  • 2
    "No you didn't. It is the editor that rejects the paper, or not. " --- There was no editor in play here. It's a conference hence the PC has seen the OP's review and based on it and other considerations decided to reject. But it's reasonable to assume that the OP's review definitely played a big role here.
    – Dilworth
    Nov 17 '21 at 23:17
  • 36
    Yes, I'm using "editor" here loosely to refer to whoever the primary decision maker is
    – cag51
    Nov 17 '21 at 23:27
  • 9
    This is good advice. Relax.
    – Buffy
    Nov 17 '21 at 23:34
  • 2
    @Dilworth I'm not sure how you get from "it's a conference" to "there was no editor". Putting aside the bigger point that there will be someone collating reviews and making decisions, in many conferences that person (or the stack of people involved) will have "editor" in the title.
    – RLH
    Nov 19 '21 at 3:03
  • 3
    I find your formulation of Eventually, you will be regularly giving students bad grades, rejecting candidates for jobs, and reporting students for misconduct (or analogous functions in non-university jobs) slightly problematic. Maybe the OP will never reach a position where they do so, and that's OK. I would suggest replacing "will" by "may" to avoid the impression that not reaching such a position would be somehow a failure.
    – gerrit
    Nov 19 '21 at 9:48
17

First and foremost, you don't make the final determination, and thus have a somewhat limited responsibility. For a conference, some section chair or committee makes the final determination. It is their job to read all the reviews, and make a determination as to suitability for a conference. They had other reviews, and could certainly have looked at them and accepted the work.

In my field, my experience is that conference standards are lower than those of a good journal by design -- the research is a bit more cutting edge and raw. Referees may or may not know the standards, and again, it is the responsibility of the section chair to make the determination. The best you can do is be specific about the problems you see, so the responsible party has the tools they need to make the right determination.

Whether you're too tough or not, that's a different story. We all get our experience some way, and reviews for conferences are a great place to start.

My personal policy for reviews of anything short of a grant is that I have no reason to write down anything that I would hesitate saying to the author's face (whether the review is blind, double blind, or signed). Whether this is the right policy for you, that's a personal choice. It's different for reviewing grants, where a single review can have a profound effect on a career.

4
  • 4
    This is also good advice. We tend to question lots of the decisions we make if they seem important to us. All you can do is your best.
    – Buffy
    Nov 17 '21 at 23:34
  • Could you clarify what you mean with your distinction between grant reviews and other types of review? What kind of criticism are you thinking of, that you wouldn't write in a review of a paper, or say to someone's face, but would write in a grant review? Or do you just mean that you apply higher standards when reviewing grants?
    – twestley
    Nov 19 '21 at 16:34
  • @twesley The level of importance and impact, where you know that there is a real chance that a grant can have such a profound impact on a career, dictates a slightly different code of conduct for me. Oddly, in most processes, it's fairly likely that the applicant can figure out who reviewed their grant, as they know the ID of everyone on the study section -- but I don't think worrying about how an applicant will feel when you tell them what they need to hear should be a high priority. Nov 19 '21 at 18:34
  • Also, you can get real nitpicky when reviewing a paper, but getting nitpicky on a grant can really keep good research from getting funded. Grant reviewers need to learn to keep their mouths shut unless something is truly important. Nov 19 '21 at 18:36
11

As a counterpoint to the world-view expressed in other answers: I've never understood claims that grad students or recent PhDs could be good referees for serious papers. The lack of experience, if nothing else, diminishes perspective...

In particular, if/when a novice doesn't understand something in a paper, especially larger issues, it's not surprising. Yes, we might imagine a standard for writing which makes things intelligible to people not-quite expert... but that is (in math) not the standard for writing. Intelligibility to experts (plus meeting certain implicit stylistic expectations) is the goal.

In mathematics, anyway, it is not so easy for even very talented beginners to be able to appreciate/appraise serious papers. Despite some mythologies, getting a PhD doesn't really make anyone an expert in anything. It's a beginning. People equally capable and with decades' head start on a beginner operate in a much different context. No moral virtue there, but definitely scientific/intellectual virtue.

But, in your particular case, you did the best you could. Good to learn something about your own limitations, such as they may be. Don't worry, except to try to be a better scholar for future episodes.

3
  • 2
    Presumably, the decision makers know who the reviewers are, so they can appropriately weight the reviews Nov 18 '21 at 2:33
  • 2
    When I was mentored, my boss kept an eye out for reviews up my alley to cross his desk. I think this is part of a mentor's responsibility. Nov 18 '21 at 14:18
  • 2
    @ScottSeidman, true, in a structure where higher-ups receive review requests and (are expected to) delegate them to juniors, things could be more sensible. If only ... :) Nov 18 '21 at 19:51
8

Don't feel bad and don't assume you are less qualified than other, more senior reviewers. Often, reviewers at the beginning of their "reviewer careers" are much more thorough in their work.

My first review was what I percieved to be a very bad paper with many errors and shortcomings. It was withing my specific area of expertise, though, and I was very much able to judge the correctness of what they were writing. I wrote a very detailed review and voted to "decline". A few months later, I got the paper to review for a second time (the editor had decided for "major revision"), along with the comments of my fellow reviewers: most of the points that I raised had not been mentioned or noticed by the other reviewers.

Because of my comments, the authors revised several sections of the paper, redid some experiments and corrected wrong references. Whithout me bringing up all these points and my harsh judgement, the paper might have sailed through with only minor revisions and a bad paper would have been published. Because I was harsh, a better paper was published.

I also initially felt bad after I handed in my first review, and even wrote a private letter to the editor, explaining that I am not 100% sure about my judgement because this was my first ever review. But afterwards, it felt right and I have learned to trust my judgement.

Being able to honestly judge the work of other people (including saying it is bad when it really is bad) is an important skill in academia, but it make time to acquire. Keep up the good work.

3

As others have said, you did your best as a reviewer, given your level of expertise and perspective on the field.

Other responses also point out that graduate students may not be the best to review research papers due to lack of experience. But, I don't fully agree.

Due to this impostor syndrome, I've declined to review many papers, though I am sure I would have done better, as a student, than 30% of the reviewers that reviewed my papers.

In your case, I would suggest to keep reviewing papers and getting better, even if it is at the expense of a few authors who see delays in their paper publication dates due to you.

There are plenty benefits to it. First, you learn to read papers better, frame them better in the big picture of the field. Second, you learn to write your own papers better. Third, if you are a consistently honest reviewer, you will start adding much more value to the field than you subtracted when you were just learning to review papers. Fourth, and most important, it helps you become a better scientist, by teaching you what is important and what is less important in your field and who the strong players are. All these will help you in your future career, even on the short term. For example, it's one thing to give an interview talk stating what you did and how, and one other thing to be able to tell exactly how your work is advancing the science and what it is built upon and what can be built upon it.

Related to the impostor syndrome, I don't think it's a psychological problem that should require years of therapy. When you do serious research, you are constantly humbled by your apparent powerlessness in the face of the infinite unknown. Whatever knowledge you may have, there is always need for more, and whatever skills you may have, they are rarely sharp enough. Faced with this constant humiliation, you are bound to experience the delusion that you're not worthy of being called a scientist.

But, so are the others that do science, regardless of their status in the scientific community. So, the only way to cure this ailment is to look at what your peers are doing. You'll probably notice that you are not worse than most, even at reviewing papers. There are outliers, some people that are exceedingly good, and, at the same time come from very strong places, but, even those people suffer or suffered from this disease.

In short, stop worrying and keep doing your best, and improve your work with every iteration.

1

At least, in my field, when an editor gives a paper to a Ph.D. student for review, they have probably already decided that they are going to reject it. Your job is to find reasons to reject it, and to give the authors helpful feedback. As a Ph.D. student, you should probably only recommend publication of papers that are unexpectedly good. You should feel uncomfortable letting it through, not rejecting it.

Give the authors useful feedback, positive and negative, and then feel very comfortable rejecting it as that was probably what the editor wants/expects you to do. You are just doing your job.

Talk with someone senior in your field or maybe at the journal to make sure as some practices do vary by discipline. I think you will find that this is the case.

1
  • Heh. I just left a comment about this above. I was told when I was in grad school that journal editors sometimes intentionally send papers they know are bad to PhD students to see if they will reject them. This answer sounds like it corroborates that (at least in some field).
    – Andrew
    Nov 20 '21 at 3:53
0

Something you should learn during your doctoral studies is to have confidence in your own judgement. This comes with time, but you can speed up the process by remembering that it is YOUR opinion and nobody else's. As long as you can justify it with facts, quotes etcetera, it is not a personal attack on the author[s] of the manuscript that you are reviewing.

Practise being more confident !

1
  • As it’s currently written, your answer is unclear. Please edit to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers in the help center.
    – Community Bot
    Nov 18 '21 at 18:50
-3

It is true that the editor makes the final decision, but this decision is highly based/relied on the reviewer's comments. I know a Ph.D. student who couldn't sleep well for a few days because the paper that took months of research got rejected. In a case like this, you can always adjust your decision when you think it is the right thing to do.

5
  • 6
    Back in the days my paper were rejected, too. I invested months, but I learned from the reviews. You cannot pass everybody, just because they worked hard. You have to keep the quality of conference contributions about some threshold.
    – usr1234567
    Nov 18 '21 at 7:10
  • 1
    We are talking about a paper rejected due to some misunderstanding by a reviewer. The point that I am trying to make here is that if you (as a reviewer) found that all the other reviewers have provided positive feedbacks and you found that the paper deserves a second thought then why not just adjust your decision accordingly.
    – Adam Ra
    Nov 18 '21 at 7:39
  • 4
    "you can always adjust your decision when you think it is the right thing to do." Can you? This seems very unusual to me in a peer review process. Have you ever done that? Nov 18 '21 at 7:40
  • 1
    @Snijderfrey At least at easychair, reviewers can edit the scores and add more clarifications if they want to, and of course they can always send a message to PC.
    – Adam Ra
    Nov 18 '21 at 9:56
  • 4
    "I know a Ph.D. student who couldn't sleep well for a few days because the paper that took months of research got rejected." Failure and how to cope with failure is an important lesson to learn. Plus, the paper was rejected, most likely with some explanations, so the PhD had to put some more work in it, with some guidance. Maybe the issue for that PhD was "PhD student will miss their defense, loose their job and their visa and will be deported to Alabamistan because of that rejection" ... the issue then is not the rejection, but the non-sensical PhD-contracts/expectations/etcetc ...
    – EarlGrey
    Nov 18 '21 at 12:50
-4

If this worries you, there's a fortunately easy solution: never recommend rejection at the first review; always recommend revision. The line between revision and rejection is a fuzzy one, and no one will blame you if you recommend revision for a paper which you have serious doubts about. If, when you get the paper back after revision, you think the authors have not made a real effort to address your remarks, then you can still recommend rejection.

Based on your description I would also guess that the paper was sent for revisions and you're now looking at the revision. You can still recommend acceptance. It's not common for a reviewer to flip from recommending "reject" to recommending "accept", but it's not rare either. Of course, only do so if you think all concerns have been addressed.

7
  • 10
    Having a major revision first and then a rejection is a bad move. You make false hopes and waste the time of the authors. If it does not suite the journal / conference, give the authors early indication. Then they can withdraw and apply to a better place for publication.
    – usr1234567
    Nov 18 '21 at 7:12
  • 2
    Often enough, papers never come back for a second round of reviews. I think following this advice would lead to papers published without reviewers actually agreeing. Nov 18 '21 at 7:49
  • 2
    @usr1234567 the OP makes it clear though that this was not a "your paper is not suited to our conference" kind of reject.
    – Allure
    Nov 18 '21 at 7:53
  • 3
    @Snijderfrey you really think it's often that papers never go for a second round of review? In my experience most papers go for a second round of review, because even papers that are eventually accepted go through a 'revise' decision.
    – Allure
    Nov 18 '21 at 7:55
  • 1
    I don't have data to do statistics on this, it is just my experience. There are quite a few published papers that I reviewed where I recommended crucial revisions, that were however not taken seriously by the authors, and the editor decided that that was enough. That's fine with me because I value other peoples' judegement, I just would be careful to expect that you get back the manuscript for a second round, even if you recommended major revisions. Therefore, if you really think a paper should be rejected, I think it makes sense to recommend this in the first round. Nov 18 '21 at 8:40

Not the answer you're looking for? Browse other questions tagged or ask your own question.