4

I've submitted a paper to a reasonably good journal in my field. One of the reviewers questioned the statistical methods and recommended major revisions. Without getting into the nitty-gritty of the analysis, I sought help from a statistics expert and whilst they said that my analysis wasn't strictly "wrong" they suggested that I correct my p-values for multiple comparisons in order to pacify the reviewers. The issue is, when I applied the correction none of my findings came out to be significant. I am fine with that, for most of the paper we're describing general trends anyway, so I think that removing mention of significance isn't a huge deal, and everyone talks about the need to publish negative findings, so why would this be so bad?

My advisor however disagrees, they think that the adjustment is too stringent and we're penalizing the findings unnecessarily (I am using false-discovery rate, so the most lenient you could get). They also think that if we remove the mentions of significance the paper is likely to be rejected because then we don't have much to back up the trends we're presenting. My advisor wants me to go ahead and resubmit the paper by making slight adjustments. This would be the first paper I've published as first author, and I am struggling to get onboard with blatantly ignoring a the statistical error I would be committing for the sake of publishing it. My advisor has agreed for the correction of the p-values to be applied and discussed in my thesis.

My dilemma lies in whether I try to fight my advisor on this (I don't think I will win), or do as they ask and hope the paper gets rejected, or maybe I'm blowing this out of proportion and my advisor is right. I am concerned that I will regret publishing something I know to be questionable for the sake of publishing.

Improve this question
7
  • 2
    Fighting with your advisor is seldom a win. But the editor may not be happy to publish without "major revisions".
    – Buffy
    Commented Jan 28, 2021 at 21:38
  • 3
    It sounds like this isn't a negative result, it's a poorly designed and underpowered study that shows nothing. Those things aren't synonyms.
    – user133933
    Commented Jan 28, 2021 at 21:44
  • 2
    Being sloppy with statistics is incredibly common. Everybody uses statistics, hardly anybody properly understands statistics. Be proud to have taken the step to talk to an expert. Take their advice seriously and learn. Could you improve your study by acquiring more data for a proper major revision? You might be able to estimate how much more data you will need in order to be significant again.
    – Snijderfrey
    Commented Jan 28, 2021 at 22:46
  • 4
    @Snijderfrey Collecting more data to chase a significant result is one of those forms of sloppiness. Probably not the worse of them, of course.
    – Bryan Krause ♦
    Commented Jan 28, 2021 at 23:17
  • 5
    Looks like you’ve learnt independently what Doug Altman and others have been saying for years - if you go to the statistician after your experiment is done, he can do a post mortem and tell you what went wrong but it’s probably too late to fix it properly. You’ll know what to do next time. Hopefully your “advisor” will learn too.
    – rhialto
    Commented Jan 30, 2021 at 10:44

1 Answer 1

Reset to default
2

Well, if you're having second thoughts is because deep down you have a solid reason for it. And, you do, as you said in your text, the statistical data is lacking additional background support. If possible add more data to the existing data, and make changes to the text accordingly.

Is frequently found on research papers, some shortcuts when presenting experimental data that is lacking some explanation/support. For instance, changing the axis scales, present data in a normalized axis scale, among many others. These data presentation techniques will force the reviewers to spend more time analyzing your data looking for clues on lack of validity. In such cases, even if it gets published, later on, it will be reflected on the number of citations (for instance). To simply put, I don't recommend doing it nowadays.

Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .