21

I am now serving as a reviewer at a computer science conference. I could identify that one of the papers I am reviewing comes from a prestigious lab based on

  • The type of research problem the paper is trying to address is in line with what they have always been doing (I am working in a relatively niche field), and the research methodology looks quite familiar.
  • The list of papers they cite bias towards the publications from their lab.
  • The style of graphics and presentation of the contents look pretty familiar.

However, as much as this lab previously generated many good papers (in fact, they opened up the area I am working on), this particular paper is quite borderline; I am leaning towards rejecting it.

What concerns me is there may be some way they could know who gave this rejection decision, given the fact that they may know the PC member (the person responsible for matching papers with reviewers), who could incidentally and subconsciously give them a few names. I hope the academia is better than this, but I could not help thinking of the bad consequences like my own paper and even myself will receive unfair treatment in the future; even if these things happen, I do not have any solid evidence to prove it.

I am not sure what I should do at this point. Should I go against my initial judgement and accept this paper (there is a "weakly accept" option for my conference).

4
  • 10
    Paper tanks can generate manuscripts that have a distribution of quality. The fact that some are excellent does not mean they all are. The challenge as a reviewer is to give direct feedback on what would make a particular paper excellent even if it's not. I saw a paper with such horrid writing, I wanted to just recommend rejection, but I said the writing needs significant improvement. By the third round of R&R it was clear they did as much, and ultimately it got published.
    – AdamO
    Jul 26, 2023 at 16:56
  • 2
    You've seen the good papers... Who's to say other reviewers have not rejected the poor papers?
    – IronEagle
    Jul 28, 2023 at 2:24
  • 1
    You mention PC members, so I assume you are a subreviewer for a CS conference. In this case, the PC member relies on the subreviewer to carefully read and assess the paper, but they are ultimately the person responsible for the review (or at least co-responsible with the subreviewer). So it's highly unlikely that they would even disclose their identity (and consequently any subreviewer's) to the authors.
    – Andrea
    Jul 28, 2023 at 8:33
  • Prestigious labs are prestigious because they do good work, not because people blindly accept their papers at good journals / conferences. Be critical and recommend rejection if you believe it is warranted.
    – Kev C
    Jul 30, 2023 at 1:51

4 Answers 4

49

If the journal’s editor is even somewhat ethical- you should be fine. This is the point of anonymous review.

I don’t think anyone can 100% absolutely guarantee that your identity will not be exposed, but again this is really unheard of. If the journal is reputable, exposing reviewer identity is grounds for removal from the editorship position most likely, and other consequences might arise.

On another note, if the lab is truly prestigious they should welcome critical, constructive feedback on their work. You learn a lot from rejections, they can often make the paper better. Even if I knew the identity of my reviewers I would not take negative reviews personally, as long as their tone is constructive and polite.

Finally, an editor can theoretically ignore the reviews and just accept the paper. This kind of bad behavior has far fewer repercussions towards them than revealing your identity. If authors have so much leverage over the editor that they can get reviewer names, they also have enough to just get their paper accepted despite bad reviews. Again this last point is irrelevant if the journal is halfway reasonable.

Edit:

When reviewing for a conference there are usually several more checks and balances in place (at least for CS conferences). There are reviewers, meta-reviewers, area chairs and program chairs. All of them are expected to report and handle inappropriate behavior on the side if authors/reviewers.

7
  • 2
    Thank you so much! What you said is really assuring! I am gonna give my honest review anyway.
    – Mr.Robot
    Jul 25, 2023 at 20:12
  • 1
    The answer is for a journal, not a conference.
    – VitaminE
    Jul 25, 2023 at 20:13
  • 3
    Same idea in both
    – Spark
    Jul 25, 2023 at 20:57
  • 11
    "if the lab is truly prestigious they should welcome critical, constructive feedback on their work": agree! I've gotten lots of rejections in my career; my first reaction is to be defensive, sure, but after I sleep on it I use their comments to improve the manuscript as much as I can. Jul 26, 2023 at 6:58
  • 2
    "an editor can theoretically ignore the reviews and just accept the paper" -- a nuance of this is that the editor might consider the reviewer's reason for recommending rejection, but not agree, and still accept the paper (especially if the other reviewers are more positive, and the reasons are somewhat subjective). Jul 27, 2023 at 13:02
19

Ethically, you should deal with the paper as it is and ignore the other aspects. If you aren't comfortable with this, then you should quit reviewing, as such decisions affect what is published and what is not.

If the paper is worthy recommend acceptance. If it is unworthy, recommend rejection. The lab from which it emerged, and the authors, themselves, should be irrelevant in any decision about the paper.

You can, of course, ask the Program Chair to take special precautions about your identity and that you don't want it known. This should be standard for a reputable single blind conference in any case, but it is ok to remind them.

If you can't do the job, quit. Sorry to be blunt.

6
  • 1
    Thank you! I will ask PC to take precautions and give honest reviews.
    – Mr.Robot
    Jul 25, 2023 at 20:14
  • 3
    I don't think it's very responsible to advise an early career researcher to quit reviewing. They may take you seriously!
    – jakebeal
    Jul 26, 2023 at 10:29
  • @jakebeal, actually, I think the OP got the intended message, which is to do the job properly.
    – Buffy
    Jul 26, 2023 at 11:46
  • 1
    @jakebeal If the alternative is for them to "peer-review" by focusing on the name of the lab rather than the math and experiments, that's for the best. Either way, they aren't writing a real review.
    – Ray
    Jul 27, 2023 at 14:44
  • 1
    @Ray The alternative is to coach them on how to review well, as most of the answers here are suggesting.
    – jakebeal
    Jul 27, 2023 at 16:16
5

What concerns me is there may be some way they could know who gave this rejection ... I could not help thinking of the bad consequences like my own paper and even myself will receive unfair treatment in the future

Let's assume for the sake of discussion that they will know it was you. If you do not feel you can stand behind your review in this situation, and live with any potential consequences - then, frankly, you are ethically compromised, and should have a long hard look in the mirror.

Peer reviewing should be considered a sacred public duty. Something that you do for humanity, for science, for posterity - not in the sense that it has to be extensive (because people can always read the paper for details), but because of the weight your reviewer opinion will carry.

Now, you might say I'm being too strict and idealistic; but inappropriate reviewing is part of what makes many conferences so full of fluff and junk which should have been rejected - often because it comes from the lab or research group of somebody important.

0

This may be received as a stupid idea, and it possibly is.

However, it could be a test of the peer review process: If they are so prestigious and they acknowledge this, they could be afraid that peer reviewers are disinclined to reject their papers, just because they are who they are.

So, by sending in a bad paper, if accepted they know the peer reviewer is biased because of their reputation. Rejecting it as you should, sends the signal back: We don't care who you are, send in garbage and we reject it!

As I said, maybe this is a bit far out, but it could be a possibility...

1
  • 7
    Usually prestigious labs have a lot of people in them, not all of whom are up to the perceived standards of the lab. If a PI isn't micromanaging all the members of their lab, it's easy for a substandard submission or two to get out the door.
    – jakebeal
    Jul 26, 2023 at 10:31

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .