19

I recently reviewed a paper that made some very strong claims that were not supported by the data. However, it was much more than that. These claims were so extreme that that were actively misleading: e.g., "The present results represent the most important findings in personality psychology to date." (Not as bad as this example, but not too far off..)

I wrestled with two options in my review.

A.) I could respond in a scientifically disinterested manner: e.g., "These strong claims are not supported by the data. Please revise them."

OR

B.) I could go beyond that and politely "critique" the authors for their questionable practice: e.g., "These strong claims are not supported by the data. In fact, they represent such an "overselling" of the manuscript's findings that they are actively misleading. Please revise them and we urge the authors to avoid such hyperbolic claims in the future."

In the end, I went with something like option B.) because I think that there is inherent value in stating firmly that these kinds of practices are not okay. As a scientific peer, I did not at all appreciate these obvious efforts to oversell the results for the sake of "a top publication." As reviewers, we are the guardians of scientific literatures, and when appropriate, perhaps we should scold our fellow researchers for questionable scientific practices to deter them against such behaviors in the future.

This begs the intriguing question: Do academic reviewers bear a responsibility to critique authors for engaging in questionable practices that undermine the validity of scientific literatures?

Very interested to hear the views of others!

Edit: "Reviewer" is intended to refer to one's role as a reviewer of publication submissions for journals.

  • 1
    By "reviewer" you mean what? "A paper in a top journal" suggests that the paper is already published? – GEdgar May 14 '16 at 18:13
  • 7
    As I see it, your only duty is to give the editor your honest and well-reasoned opinion as to whether or not the manuscript should be published. Anything else is your own preferences. – Nate Eldredge May 14 '16 at 18:19
  • 6
    Strictly rejecting and calling out bad substandard manuscripts that contain for example overreaching scientifically unfounded claims is important to prevent junk science from taking over ... Or at least from becoming too widespread. – Dilaton May 14 '16 at 18:52
  • 4
    @StephanKolassa no, this is an important issue for academics and it therefore should discussed here.Leave open. – Dilaton May 14 '16 at 19:44
  • 3
    @Dilaton: many important topics are nevertheless opinion-based. StackExchange aims at providing correct answers. It is not well suited to discussions. Forums or chat are a better venue for this. I cast one vote for closure, five are required - we will see whether four others agree with me, and then the question can still be re-opened. Or we could discuss this specific case on Meta. – S. Kolassa - Reinstate Monica May 14 '16 at 19:54
24

My feeling is that rather than explicitly telling the authors what to do in a context other than the present submission, it would be better to explain your objection to their present submission in a way which would make clear to any moderately receptive party that the advice would apply more generally. E.g. something like

In my opinion, the authors overestimate the significance of their work to a degree that becomes actively misleading: virtually no work in the field has the level of significance claimed by the authors here. In general, work which is described in such hyperbolic terms can be hard to take seriously. Those who do make major breakthroughs tend to describe the advances in more sober, factual ways.

This is plenty heavy-handed. Anyone who is listening will get the message not only to tone down the language in this submission but that such language is never appropriate. Still, you aren't telling them what to do, but only giving your (expert) opinion on the consequences of their actions.

-2

Be a professional, and be helpful. What's the point of chastising the authors? will they think you are 'smarter' than them? No, to them, you'll sound clueless or/and arrogant. Hence, it is a waste of your time and frankly, doesn't help the editor because he/she will think you have some axe to grind. Your ONLY job is to help the editor make a decision, and if you are kind, provide constructive comments to the authors.

  • 7
    I respectfully disagree that one's "ONLY job is to help the editor make a decision." It goes beyond that. One is a representative of the field at large and every review shapes its practices. And no, they certainly won't think the reviewer is "smarter than them." But that's not at all the point. The point, articulated by the commenter above, is to "prevent junk science from taking over." Which, to a disconcerting extent, it has. – ATJ May 14 '16 at 19:26
  • 3
    No -1: the purpose of the review process is keeping up the level and scientific standards. – Dilaton May 14 '16 at 19:46
  • @ATJ you assume the authors will listen to reviewers. I doubt that very much if you are not professional in your comments -- the operative work is chastise, which the questioner is advocating. Ask yourself, are your more incline to listen if your parents berate you or when they talk to you kindly? – Prof. Santa Claus May 14 '16 at 19:50
  • 2
    @Dilaton Yes, to review and help the editor reject poor papers. In other words, ensure papers with questionable claims do not see the light of day. – Prof. Santa Claus May 14 '16 at 19:51
  • 2
    Perhaps "chastize" was the wrong word. I feel like that carries with it a negativity I did not intend, and I've changed it. And authors do listen to reviewers--whether consciously or implicitly in future behavior. That is certain. – ATJ May 14 '16 at 21:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.