0

We received a decision letter of major revision recently. One of the four reviewers raised many comments like "Fig. XX, magnitude of effect too small." or "Fig. XX, scale condensed, magnitude of effect is very small", etc.

I have to say that the P-values are highly significant (P < 0.001 in many cases) and our statistical analyses are solid as recognized by another reviewer in his or her comment. Although the absolute differences of measurement between two groups are quite small. The variation in each group is also very small. Therefore, if we quantify the magnitude of effect with effect size (a stanardized version of magnitude of effect), the effect size for the observed differences is medium of large based on the criteria here.

The reviewer has no specific comment about what we should do. How to respond to such comment properly?

  • 2
    Your comments read to me like you are possibly forgetting the distinction between statistical significance and practical significance. I can't tell from what you've written whether the effect you've found is big enough for anyone to actually care. – Jessica B May 10 '18 at 5:59
  • @JessicaB, I am afraid the reviewer is thinking this way. He or she may thought the differences are too small and of little practical use although it is significant. – mt1022 May 10 '18 at 6:07
7

I do not believe the reviewer is impugning the significance of your results. Instead, it sounds like the reviewer is suggesting that the scaling used on the graph makes it too hard to see the magnitude of the effects you're talking about. Perhaps he wants you to show a blown-up subset of the graph?

  • I would be happy if it is the case you are talking about. However, we have only displayed range of measurement (min to max, like the "blown-up subset" in your answer) in order to highlight the differences rather than a range from 0 to max. Maybe the word "condensed" is quite confusing. – mt1022 May 10 '18 at 5:24
  • 1
    Or perhaps it's the reverse, but the word "scale" definitely implies that the problem is with the representation, not the significance. – aeismail May 10 '18 at 5:54
  • That sounds likely - the might consider the blown-up subset to be misleading because it exaggerates the magnitude compared to what it actually is (which is what they might mean with "too small")? – Vincent May 14 '18 at 7:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.