33

I agreed to review a paper that falls within my area of expertise. However, I am not particularly familiar with one of the techniques used by the authors. On detailed reading, their use of this technique feels wrong to me, but I am by no means certain. It is not a method that I have seen applied before in similar contexts. I can't put my finger on a definite flaw, but it just doesn't quite seem to "add up".

Given my uncertainty, how should I comment on this in my review? I have the option of simply stating “I cannot comment on the validity of [one particular aspect of the methodology] as it is outside my area of expertise.” Alternatively, I could raise my concerns, while making my uncertainty clear, but would this undermine confidence in my review overall? Is it unfair on the authors to raise an objection on the basis of my incomplete understanding?

Improve this question
1
  • 8
    Even if the method is correctly applied, the text has failed to convince you, a member of the paper's audience with an incentive to evaluate the paper with fairness and care, that the science is fully correct. At the very least, that implies that the paper needs to be rewritten to prevent that from happening with a wider audience.
    – E.P.
    May 26, 2017 at 23:14

3 Answers 3

Reset to default
32

Yes, it is fair to raise concerns when you don't understand a paper. It is the authors' job to explain in enough detail what they did. Especially if the method used is uncommon, then it needs to be justified. That justification needs to be done in a way that it is understood by the intended audience, not just the specialist within that audience.

Tell the authors in as much detail as possible why you think their method does not feel right. Maybe the method is problematic, and they will appreciate that feedback. Maybe they did not explain their method well enough, and they can use that feedback to pinpoint what was not clear.

Improve this answer
3
  • 1
    Yes for this answer! If the reviewer had doubts, the readers will have doubts. Even if the method is indeed correct, further information is necessary, and pointing that out is part of the review process.
    – Fábio Dias
    May 27, 2017 at 0:32
  • I'm not sure enough to say this very authoritatively but this feels wrong to me. Techniques for solving problems can get more and more sophisticated and complicated as time goes on, requiring more and more specialty. People can't be expected to re-explain everything coming before your contribution in your paper indefinitely; at some point they need to be assumed to have the necessary background. So I'm not sure it's always fair to make this criticism necessarily. But maybe I'm wrong.
    – user541686
    May 27, 2017 at 9:58
  • The method used is not some irrelevant technicality, it is an integral part of your argument. You either explain it in such a way that your audience understands it or your entire argument falls apart. On a more practical note: not being able to explain your method clearly is strong signal that you don't understand your own method.
    – Maarten Buis
    May 28, 2017 at 19:17
10

If possible, ask someone you know who is an expert on the technique for advice. Being able to give a more informed response, or drop any objection if there is no basis, would be preferable. Of course you would need to get input while maintaining confidentiality, but normally I wouldn't expect this to be a problem. If you can describe it in the hypothetical, without reference to the specifics of the research, it should be fine.

There is no harm in admitting limits to your knowledge, or giving criticism that you are not 100% confident in. However, I don't think saying "I have a vague sense this doesn't seem right" is likely to be very useful. If you can't at least communicate a concrete reason for your concern, I don't think an objection is very fair to the authors; nor is it likely to be productive. So if you aren't able to further clarify what you find wrong, I would not raise this as a criticism of what they did. You don't really have enough to object to that.

However, it's fair game to raise the issue that they failed to explain/justify what they did (I 100% agree with Maarten Buis's answer on this point).

Improve this answer
1
  • I've accepted @MaartenBuis' answer, but the suggestion to try consulting with someone who is more knowledgeable about the technique is also a very helpful one - thank you.
    – user2390246
    May 28, 2017 at 9:04
6

This happens more frequently than you think. Editors are sometimes under a lot of pressure to secure reviewers and sometimes they settle for non-experts.

In such cases it is more than fair to state your concerns of the paper, while writing to the editor (private comment) explaining the situation by stating that you are not expert, but you tried to understand the paper but you feel that something is wrong for the reasons outlined in the review. This, will force the authors answer one way or another

It is not possible to undermine like that your review. Editors want honest opinions. In any case, such techniques should be clear to a wider context, not only on the few experts on their fields, so if you feel something is unclear, please make service to the authors and the editors and state it!

Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .