79

Suppose I review someone's paper anonymously, the paper gets accepted, and a year or two later we meet e.g. in a social event and he/she asks me "did you review my paper?". What should I answer? There are several sub-questions here:

  1. Suppose the review was a good one, and the paper eventualy got accepted, so I do not mind telling that I was the reviewer. Is there any rule/norm prohibiting me from telling the truth?

  2. Suppose the review was not so good, so I do not want to reveal. What can I answer? If I just say "I am not allowed to tell you", this immediately reveals me... On the other hand, I do not want to lie. What options do I have?

  • 15
    I would love to see a good answer to this question, especially since I can also see benefits of disclosing the identity: The author might have particular questions about the review that could improve their future work on the topic. – henning Mar 31 '17 at 12:21
  • 41
    I'd love to tell you, but then I'd have to reject you ;-) – Captain Emacs Mar 31 '17 at 12:58
  • 50
    "I can neither confirm nor deny that I reviewed your paper." – Jeff Mar 31 '17 at 14:44
  • 9
    "I'll keep you in suspense, OK?" – Dirk Mar 31 '17 at 18:45
  • 8
    "I don't know, i read tons of crap… so propably yes." – Jan Ivan Apr 3 '17 at 7:40

13 Answers 13

108

You could say something like

"I am not in the habit of telling people whether I did or did not review their papers, sorry"

or

"I don't feel comfortable answering this question."

Or you could defuse the question with a humorous answer, e.g.

"I would tell you but then I'd have to kill you"

or

"I don't remember, I always take an amnesia pill immediately after reviewing a paper"

or some similar kind of obviously nonserious smart-assery. None of these answers provide any useful information to the asker, and all of them convey some level of disapproval on your part at being asked, making it unlikely that the asker would press the case any further.

  • 78
    + 1 for humour. It's the only solution that leaves some ambiguity. – henning Mar 31 '17 at 13:31
  • 14
    Be very careful with humor, especially the first one. While I hope that most people would recognize it for what it is, there's always the exception who will take offense. And if it's someone in your field... – tonysdg Mar 31 '17 at 18:47
  • 21
    @tonysdg be very careful with humor: also be very careful with a lack of humor, since I for one find that (especially a lack of humor resulting from people stripping away any semblance of humor from what they're saying to avoid any chance that they will ever offend anyone) somewhat offensive as well. – Dan Romik Mar 31 '17 at 21:13
  • 10
    I agree very much with the idea of diffusing with humor. Bravo! I disagree very much that "I would tell you but then I'd have to kill you" is humorous. – Randall Stewart Mar 31 '17 at 21:50
  • 15
    @RandallStewart thanks for cleverly hinting that you liked my second joke :) – Dan Romik Mar 31 '17 at 22:52
32

I think it is inappropriate of someone to ask you if you reviewed their paper. As you point out, if you are in a position where you would only confirm if you were positive, and declining to respond implies a negative review, you are essentially forced to confirm unless you explicitly deny being a reviewer. In fact just having your jaw drop as the question is asked is probably confirmation enough. So I think the right answer to the question is an immediate Whether I reviewed that or not is confidential, and that's an inappropriate question.

That being said, I'm not aware of rules that would prevent you from acknowledging the review. As I mentioned in a comment, for conferences I've seen people explicitly unblind themselves in reviews by choice. If you wanted to acknowledge it, I think you could. I've occasionally admitted that to someone in the past, but not because the author asked.

  • 8
    Somebody might not be comfortable telling the author that they're behaving inappropriately, especially if the author is more senior. It's perfectly possible to decline to answer the question without criticizing the person for asking it. – David Richerby Mar 31 '17 at 14:35
  • Yeah, @dan_romik had a better answer in that regard. – Fred Douglis Mar 31 '17 at 15:03
  • 1
    It may be inappropriate but it’s fairly frequent. – Konrad Rudolph Mar 31 '17 at 15:18
  • 1
    @KonradRudolph, must depend on your circle of contacts. Once in 30 years for me. – Fred Douglis Mar 31 '17 at 16:55
  • This approach leaks information. If the author asked me (a non reviewer), I would probably answer confidently and tersely no - not being aware that the review was anonymous. Who besides reviewers would answer "confidential"? – emory Apr 1 '17 at 16:54
22

There's a principle I go by now that took me a long time to learn, mainly because it involved a lot of unlearning things I'd internalized since childhood:

It's always okay to lie when someone asks you an inappropriate question. And usually it's the best answer, especially if you have it prepared, since the whole point of the inappropriate question is usually to extract information from you involuntarily through your reaction.

As such the best answer is just "no", regardless of whether you did. If you feel comfortable with the power dynamics, you can follow up with something like "You know, really, you shouldn't go around asking people that. If they did happen to be the reviewer, it would put them in a really bad position."

  • 16
    -1. I agree it's okay to lie in this instance (though that wouldn't be the ideal way to handle it in my opinion), but the blanket statement "It's always okay to lie when someone asks you an inappropriate question" feels much too strong and general to be correct. E.g., if someone asks how much money you make, it's okay to state a figure that is double your actual income? Doesn't feel right to me... – Dan Romik Mar 31 '17 at 17:21
  • 3
    I would agree that the best answer to the question asked by the OP is to just say 'No' and move on with life. Nothing good can come of saying 'yes', or any other answer that just invites discussion and likely indicates to the one asking the question that you did indeed review their paper and reject it. – Jon Custer Mar 31 '17 at 17:46
  • 13
    @DanRomik: A couple things: 1) Appropriateness there depends on the context. If it's a coworker trying to determine if they're being cheated, the question is rather reasonable. Likewise if it's a credit application. But if it's just a random person you meet, you don't owe them any answer. 2) Just because it's okay to lie in situations where the question is inappropriate doesn't mean you're obligated to. My point is that someone asking a question does not automatically obligate you to answer it truthfully or in any particular way at all... – R.. Mar 31 '17 at 19:27
  • 4
    ...for yes/no type questions and other situations where a refusal to answer often implies one of the possible answers, lying is often the optimal solution for not giving them information. But for more open-ended things a good NOYB might suffice. – R.. Mar 31 '17 at 19:28
  • 3
    @R Let's assume that you are ethically right and that is okay to lie in my situation. But it is not enough to lie - I will also have to lie effectively, i.e, without blinking or showing signs of nervousness. To do this, I will have to practice lying. And I believe that such a practice is not good for the mental health. – Erel Segal-Halevi Apr 1 '17 at 17:54
16
  1. Suppose the review was a good one, and the paper eventualy got accepted, so I do not mind telling that I was the reviewer. Is there any rule/norm prohibiting me from telling the truth?

I'm not aware of any such rule. You can tell them if you want but I would advise against it. It just feels weird: is the person trying to set up some kind of coterie of friends who write positive reviews for each other?

  1. Suppose the review was not so good, so I do not want to reveal. What can I answer? If I just say "I am not allowed to tell you", this immediately reveals me... On the other hand, I do not want to lie. What options do I have?

The best policy for answering questions where the possible answers are "No" and "Yes but I'm not allowed to tell you that, so I'm going to refuse to answer, but then you'll know I mean yes" is to just refuse to answer in all cases. Security agencies call this policy neither confirm nor deny (NCND). You don't need to explain why you're refusing to answer; just be polite and say that you never answer questions about what papers you have and have not reviewed. If the person doesn't accept this, they are being rude and you don't have to keep talking to them. If you feel you have to keep talking to them (e.g., they're a senior professor in your field and you don't feel comfortable just walking away), you can explain about NCND but, honestly, they already know that and they're being a jerk.

  • 1
    Regarding the friendly reviews... yes, this is a real risk. Sometimes people are allowed to recommend reviewers, who then shill off the author. A practice that should be banned universally! – Fred Douglis Mar 31 '17 at 16:56
  • 2
    @FredDouglis "is the person trying to set up some kind of coterie of friends who write positive reviews." Doubtful. There are many innocent reasons why a person may ask this [e.g. 1. genuinely curious, 2. thankful for a high effort review, 3. interested in a specific comment, 4. want to start a collaboration based on a reviewer suggestion, 5. asking for clarification on an tool suggested by the reviewer for future work etc.]. Obviously this doesn't make it an appropriate question, but it is best not to assume any bad intentions when they could easily be explained by neutral or positive ones. – WetlabStudent Apr 2 '17 at 12:08
  • @WetLabStudent, I was simply saying it happens, and I've seen it. Not that it is the most likely reason. – Fred Douglis Apr 2 '17 at 13:55
  • I agree with 2. If you never wish to reveal which papers you review, then having a personal NCND policy is a good idea, because then you can automatically answer any unwanted question by stating the policy. If you do this immediately when someone asks you such a question, without even stopping to think about what their paper was about, then I think you can indeed manage to avoid giving any information to the requester. – a3nm Apr 4 '17 at 15:19
7

There's a Warren Buffett quote (in The Essays of Warren Buffett) that I think is a good answer:

If we deny those reports but say "no comment" on other occasions, the no-comments become confirmation.

In other words, tell them that your policy is to neither confirm nor deny.

  • 2
    Good quote, but it's worth pointing out that the same idea was already expressed quite well in @DavidRicherby's answer. – Dan Romik Mar 31 '17 at 19:54
4

How awkward! I once had someone tell me that he was the reviewer of a grant I submitted, which was rejected the year that he reviewed it and it was accepted the following year. Although he told me how happy he was that I resubmitted it the following year, it was very awkward.

Is it possible that the person asking the question is junior and doesn't know that this is inappropriate? I think that a polite response about peer review is appropriate, such as: "In the spirit of peer review, I don't like to discuss whether or not I have reviewed specific papers or grants. However, if there is something about your study that you would like to discuss? If so, I'd be happy to talk with you about it!"

This way you can avoid the question, but demonstrate interest in their work. If the person does not know that their question is inappropriate, you also politely inform them that they should ask people about who reviewed their work.

  • 3
    I'm a bit surprised you're so taken aback. I think it's terribly inappropriate to ask, but I don't find it so horrible to volunteer that someone liked your paper or proposal despite its bad fate. Especially if it later was accepted. – Fred Douglis Mar 31 '17 at 23:43
  • 1
    @FredDouglis one of my levelheaded managers once gave me a great tip: he said "at the end of the day, it's just work" implying that it's important not to take the ups-and-downs personally. – Neil G Apr 1 '17 at 6:55
  • Yes, Fred Douglis, in looking at my response, it comes off a little exaggerated. However, I will say that the way that the person approached me and the larger conversation was awkward (and not provided in detail here), even though the comments I included in my response were benign. – Nicole Ruggiano Apr 1 '17 at 17:50
4

"I believe the referee process works best when referees remain anonymous; as a result I neither ask nor answer this type of question."

4

Perhaps you should follow the example of Howard Percy Robertson (known as the 'R' in the famous FLRW, or Friedmann-Lemaître-Robertson-Walker metric used in physical cosmology.)

He was the referee of the famous Einstein-Rosen paper, which was rejected by Physical Review, prompting Einstein never to publish in Physical Review again.

Einstein ignored the referee report, but months later, it seems, Robertson had a chance to talk to Einstein and may have helped convince him of the error of his ways. However, as far as we know, he never revealed to Einstein that he was the anonymous referee for Physical Review. It was not until 2005 I believe, long after the death of all participants, that Physical Review chose to disclose the referee's identity (http://physicstoday.scitation.org/doi/full/10.1063/1.2117822).

  • 2
    An interesting bit of trivia, but I don't see how it answers the question, even indirectly. We don't know whether Einstein asked if R was a reviewer, and if so, we don't know what R's response was. – user2390246 Apr 3 '17 at 7:55
  • We don't know for certain, but we have a pretty good idea that Robertson talked to Einstein and helped convince him of the error of his ways, without revealing his identity as the referee. Even if it is not known with certainty, I think it still serves as a useful example to follow. – Viktor Toth Apr 3 '17 at 12:22
2

This is a good question...

In general terms, you should not tell him/her, as peer review, unless open (eg in the BMJ), it is by definition confidential.

If you are willing to, but wish to remain on the safe side, check the peer review policy of the journal for which you reviewed the manuscript. If there is an explicit embargo (e.g. 6 months after publication), then you can probably tell him/her.

Consider also that in the era of Kudos and Publons it will become easier and easier to identify peer reviewers.

  • 2
    Isn't the embargo about copyright matters? – henning Mar 31 '17 at 12:17
  • @henning Yes, but they might occasionally have a time frame for peer reviewer identity as well. – Joe_74 Mar 31 '17 at 12:18
  • 4
    I am not aware of policies that require the reviewer identity to be confidential and I know reviewers who choose to put their names in their reviews under the rationale that they won't say anything they wouldn't mind the authors knowing they say. Can you cite a specific example of this? – Fred Douglis Mar 31 '17 at 12:47
  • Embargoes are about posting free-to-access versions of the paper, which has nothing at all to do with revealing who the reviewers were. – David Richerby Mar 31 '17 at 15:12
  • Unfortunately, one cannot check the peer review policy when put on the spot by an awkward question. – user2390246 Apr 3 '17 at 7:56
2

I am aware of at least one paper where a referee went out of cover (after the review process of course) and was explicitly mentioned in a later paper:

X and Y thank Z, who as the anonymous referee was kind enough to point out the error (and later became non-anonymous).

so it is sure fine to answer truthfully that yes you did review, but only if you wish of course (and most likely if you have been helpful and the authors of the paper responsive).

  • In CS systems conferences it is increasingly common to have shepherds, who are known to be a reviewer, though not which one. They are acked by name as well. – Fred Douglis Mar 31 '17 at 23:45
  • I'd really love to be in the position to write this sentence in an article: "The authors would like to thank the anonymous reviewers, John Smith and Jane Doe." – Federico Poloni Apr 2 '17 at 7:55
  • To clarify, I meant they don't know which review the unmasked reviewer authored, though it's often easy to guess. – Fred Douglis Apr 2 '17 at 13:57
1

I suppose that in a year or two you review more papers than just one. Also I suppose that it must have been really good / really bad paper so you can remember it for a year or two.

Honestly, I don't remember what I reviewed year ago.

If you want to discuss the article, you can as what article they are talking about and phrase your answers as "If I were the reviewer I'd suggest..."

  • This is the answer I would use. If the conversation takes place at the conference where the paper in question was presented or if I had read the paper in one of the journals I read, then I might comment upon the paper as a member for the audience or as a journal reader regardless of whether I reviewed it or not. – CyberFonic Apr 4 '17 at 8:38
0

Why don't you say something like "a lot of people ask me that, and I don't tell anyone whether or not I did". And if you read their paper and liked it or had questions about it, you might as well tell them that, since it's an easy segue into a more productive conversation.

0

Edit after comments....

I suggest that in reply you would use the method of deflection - not answering directly.

You could say reply to the question with something like....

I tend to find that if a reviewer wants to be identified they request one or more of their papers to be cited, which makes their identity known. If your reviewer didn't do this or come right out and tell you then they probably want to remain anonymous and I'm not sure I want to help you figure out who it was. Did you get a particularly nasty review?

So I am suggesting

  • first, that you tell the person that if their reviewer did not reveal their identity then they probably want to remain anonymous.

  • secondly, that you say you don't want to help them figure out who it was because if you say 'no, not me' then they will have a smaller group of people to hunt through

  • thirdly, you ask if they got a really nasty review and if that is why they are asking.... this may lead on to a more general discussion about reviewing

Please note

I do not reveal my identity in reviews by asking for my papers to be cited--- but on several occasions i have seen reviews were people have asked for their papers to be cited.

Original brief reply

Deflection....

You could saysomething like....

I tend to find that if a reviewer wants to be identified they request one or more of their papers to be cited, which makes their identity known. If your reviewer didn't do this or come right out and tell you then they probably want to remain anonymous and I'm not sure I want to help you figure out who it was. Did you get a particularly nasty review?

  • I disagree with this comment. If a reviewer likes to make her identity known she should just do that. The reviewer should only ask for a citation of their papers, if this really is helping to improve the paper. Quite often as an anonymous reviewer I recommend other people's papers because they did some highly relevant work, and this would have been misinterpreted following your logic. Admittedly, if you suggest citations to your paper, you will often be suspected of being the reviewer. – frederik Mar 31 '17 at 21:18
  • Reviewers should never reveal their identities unless it is the policy of the journal to have named referees. – user67075 Apr 1 '17 at 1:24
  • I agree with @frederik, you ask for papers to be cited because they are relevant, and not for self-promotion, if you are an ethical reviewer. I hate to recommend my own papers and set a much higher bar to mention my own than those of others. – Fred Douglis Apr 2 '17 at 14:00
  • 1
    @frederik --- I agree with you, and I did not want to suggest that people should reveal their identities in this way - it is just something that I have seen happen - I have editted my answer to make it clearer - I hope this makes sense now - sorry for any confusion – tom Apr 2 '17 at 15:54
  • @ZeroTheHero -- I agree totally, but sometimes people have revealed to me when they have been referees for mine or other people's papers. I do not endorse this, --- I have editted my answer and I hope this makes clearer what I meant – tom Apr 2 '17 at 15:55

protected by eykanal Apr 2 '17 at 2:19

Thank you for your interest in this question. Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged or ask your own question.