24

I am a med school student who is writing her master's thesis. I finished compiling my thesis with all the statistical analyses and wrote them as well. My problem is that today, when talking to the professor who is my advisor, he thought that the statistical part was wrong and asked me to "correct" it in a way that I am sure it is absolutely wrong. Now he wants me to write things his way, even though the deadline is tomorrow. I tried to calmly explain the theory behind it, but I quickly realized that being a doctor, he understands virtually no math or statistics.

My problem is that aside from the tight deadline, I don't want my name to appear on an analysis that is that wrong. I don't even know what I would do during the dissertation since I know for sure that those things are wrong, and I wouldn't be able to defend my thesis. What would you do in my shoes? Should I just write the bulls**t and be over with it? I'm exhausted from all the work I have done, and can't find the force or motivation to work on something I know has 0 value.

About the analysis, if you think I might be the one wrong:

  • It's a retrospective study where I pooled around 80 patients. Each of them had done an examination 2 times within a certain time period, but with two different machines (once with machine A, once with machine B). We wanted to see if machine B was better when considering around 20 parameters that describe radiation dose and image quality in various ways. So the goal was to see if the same patient had more favourable parameter values when using machine B, when compared to machine A. The professor was very clear in saying that he didn't want to see the overall difference between the two machines: he wants to see how much the average patient benefits when switching from machine A to machine B.
  • what I did: since we wanted to see the difference within the same patient, I first computed the difference of each parameter when using machine A and machine B, for each patient (so I had all the differences for the 20 parameters). Since patient number was limited, I first assessed if each difference was normally distributed across the patients (using a test called saphiro-wilk). When it was normal, I used the a paired t-test. For parameters that were not normally distributed, I used the Wilcoxon's ranked sign test. Additionally, for all significant results I computed the Cohen's d to quantify the effect size.
  • what he wants me to do: compute the mean and standard deviation of each parameter as it is (parameter 1 in machine A, and parameter 1 in machine B). Then run a normal t-test. According to him, the tests that I used are too complicated, no one ever heard of them and no one would understand them, and anyway they make no sense. According to him, a normal t-test preserves the notion that it was the same patient who repeated the test twice. Also, he changed his mind and out of all the 20 parameters he made me look during the study, he suggested that we delete some, and for some others, I should add them together or take the mean (which makes absolutely no sense?!?!?!)
  • I'm no statistician, I'm just a regular med student that is passionate about science and math. I spent the last month reading books I wasn't familiar with, and writing code on python to implement my analysis. So of course I might be wrong. But the professor's version doesn't sound right to me. Moreover, I feel desperate because I put so much work in it. I just don't feel like going back to coding to implement his wrong ideas (I know it's a simple iteration and I know I can use scipy.stats and that it wouldn't be so long. But I just can't take it anymore)

Sorry for the rant. If you can share your mind I will be grateful. For now, I'm just crying my eyes out on my bed, out of frustration, but maybe I'm not being objective, since I'm running on almost no sleep.

Edit: I finished everything in time and the result is somewhat decent! There still are some parts that refer to the older analysis, but I included it in the appendix (as some of you suggested) and mentioned it, so it should be fine. I can't thank you enough for your kind and constructive support! Having the chance to talk it out and hear some feedback really helped in getting started again.

The tight timeline is not so unusual at my university - at least for my department. But this one was definitely extreme. I addressed the issue this morning when talking to the advisor, hopefully he will take my feedback into account when dealing with future students.

16
  • 7
    I suppose it is probably too late to do this if the thesis is due tomorrow (or is this the due date of an initial draft of the thesis?), but would it be possible to visit the statistics or mathematics department of your university (or of a nearby university, if your medical schooling takes place outside of a comprehensive university) and talk about this with an expert in the "no one ever heard of them and no one would understand them" stuff? Jun 19, 2023 at 10:18
  • 10
    I agree with you that a paired t-test is better (although your two-stage procedure is potentially problematic: (see Rochon et al 2012)). I agree that a pooled t-test is suboptimal, but I don't think it's horribly wrong - if anything I would guess it would be conservative. The other stuff (deleting/combining parameters) feels a little like snooping though. Rochon et al. 2012. “To Test or Not to Test: Preliminary Assessment of Normality When Comparing Two Independent Samples.” BMC Medical Research Methodology 12 (1): 81. doi.org/10.1186/1471-2288-12-81.
    – Ben Bolker
    Jun 19, 2023 at 13:34
  • 11
    Based on @BenBolker's comment, maybe another option may be to take your analysis and put it in an appendix. If the pooled t-test is probably right and specifically if it will be conservative (and if it agrees with your more sophisticated tests) then including it as a handwavy argument that people will feel comfortable in the main text, and then supporting it with what you've done might form a nice balance between the two cases you've considered. Keep in mind to explain what you've done, you need to be understandable by your target audience, so you can convince one way and prove with the other. Jun 19, 2023 at 16:23
  • 8
    @Elek If your thesis can still be described as a “written draft” and your advisor has only just seen it now, one day before you have to submit, then there's something very seriously wrong with your process at a much deeper level. Your advisor should have seen your drafts months ago; the final 24 hours of thesis writing should be for last-minute polishing and proofreading, not turning a draft into a thesis. I wish you the very best of luck getting your thesis handed in, but I also hope you take something away from this so you don’t end up in a similar problem again later on. Jun 19, 2023 at 23:54
  • 5
    By the way I'm a statistician but I can't really tell from your overall description who's right and who's wrong. It's a complicated issue and there are a number of aspects I'd need to know that would be to complicated to explain. I'd assume you made a passable effort there and this should not be a reason to fail the thesis even if your supervisor has a point, which may well be. But of course your supervisor may have other ideas - we can't know. Jun 20, 2023 at 10:59

4 Answers 4

29

I’m not in a position to comment on the specifics of the problem, so will answer the abstract question: what to do if my advisor tells me to do something that I know is incorrect, as a condition for my graduation?

This is tough. I’d say that if you have time, get the opinion of additional experts on the topic (statistics) who may explain why your method makes sense. You should graduate on time, so I do suggest changing things in accordance to what your advisor said, and then follow it up with advice from a domain expert who will hopefully clarify.

As an aside, the more serious issue is that your advisor only saw your results a day before they were due. Research and mentoring are all about good and regular communication, which is where things broke down in my opinion.

Good luck!

3
  • 2
    Thank you for your advice, I think I will do that! I was also surprised of the timing because in the previous meetings, he always seemed to agree with me. The problem arose when he saw the written draft. Now I know that he was probably not very invested in our conversations. I'm now back to work.
    – Elek
    Jun 19, 2023 at 16:41
  • 3
    Another piece of retrospective advice: This is a big reason why thesis and dissertations are normally evaluated by a committee rather than individual advisors. Ideally, your committee should have a few different people with enough knowledge to round out your advisor's expertise.
    – David
    Jun 20, 2023 at 17:59
  • 1
    Thank you for really answering the question and not berating OP "prof is right, you must be wrong".
    – user111388
    Jun 21, 2023 at 20:44
4

I have to agree his arguments you've listed don't convince me either. Also, it appears you chose the tools mindfully and, if asked, can explain the decision. In a pinch, that should suffice.

I understand it has already been 24 hours since the question was asked and the deadline has come. You have made your honest best effort and followed scientific integrity. Whatever follows, you will learn from this experience and become a better academic, medic, scientist for it. This one paper, one day of revisions doesn't define you as an academic. As many joke, your thesis hopefully will be your worst paper (because there are many more better and better papers to be produced by you).

-1

Academia is a system which rewards popularity (citations) over truth. It's a system which hasn't fundamentally changed even though it's known that most published findings are false (Ioannidis, 2005). Instead, most of science is still occupied with providing intricate theories from within ivory towers which do not need to work in the real-world at all. The only requirement is that the peers in the field like what you are saying, so you can just echo some important-sounding words around and get many citations. For example, "psychology’s near-total focus on explaining the causes of behavior has led much of the field to be populated by research programs that provide intricate theories of psychological mechanism but that have little (or unknown) ability to predict future behaviors with any appreciable accuracy." (Yarkoni & Westfall, 2017). Similarly, software effort estimation research is mostly based on a few tiny datasets from the 1980-1990s, which are then analyses and re-analysed with trendy data science techniques even though the data itself doesn't reflect the changes in software development since then (Jones, 2021).

As someone who is also close to finishing my PhD, you can be sure that I and many other academics will respect your decision in both cases. Yes, I would prefer that you choose the "right" way. However, I fully respect the fact that you have almost no say in the matter. I wouldn't blame you if you decide to follow your supervisor's orders. I wouldn't even blame your supervisor since he probably has many responsibilities and he probably tries to do the best he can within the constraints that he works with.

6
  • 4
    Nice diatribe but it doesn't answer the question.
    – user438383
    Jun 20, 2023 at 11:52
  • 4
    Yes it does answer the underlying question with which I also struggle on a near daily basis. The question is "Should I just write the bulls**t and be over with it?" I answered with: whatever you decide, you can be sure that this is normal and that you will not be disrespected for the decision that you make.
    – RikH
    Jun 20, 2023 at 12:02
  • 1
    In the end, only the person who asked this question can answer the nuanced question of whether it is possible to push back on the professor.
    – RikH
    Jun 20, 2023 at 12:03
  • 4
    If someone asks you 'what shall I do?' and you answer with (a) a mostly irrelevant rant and then b) 'I wouldn't blame you if you did either thing', then you haven't answered the question, you've just given a platitude
    – user438383
    Jun 20, 2023 at 18:05
  • 2
    Could you please give substantive counterarguments instead of the poisoning the well fallacy? You use the words "diatribe", "rant", and "platitude", but offer little substantive counterarguments.
    – RikH
    Jun 21, 2023 at 10:16
-4

You seem to be omitting quite a lot of what is/was going on, but let's ignore that for now and focus on the question. Tests have the main purpose of showing that something is correct/incorrect (and by how much). You are writing your thesis for doctors - this is the department that will award your degree. Many/most will understand a simple t-test, but fewer have even heard of some of the fancier stuff you describe - let alone understand it.

Putting all of those arguments in support of machine B, while lacking basic ones, is therefore NOT convincing. Such people will read the thesis, see fancy analysis and conclude that if you need all of this fancy analysis to show some benefits, there is actually no benefit to be had in practice and all of this is just smoke and mirrors.

Is it still valuable to have these better tests? Of course, so I believe that you should start the "Analysis" section or whatever it is named with the basic test. IF it shows improvements then great but if not then too bad.

In any case you should continue with results of fancier statistical analysis in the same section. Results of those methods should be in the main part of the thesis, while code could be in appendix; perhaps detailed description/derivation too.

Now, about grouping of those indicators, you can always have a single "goodness" indicator - which would likely be "health" in your case. It is an aggregate of all other indicators you can think of and there can be millions. Obtaining that single number is prone to fudging even if unintentionally. On the other hand, measuring all the separate indicators often leads to stuff like "we get 1% reduction in number of floaters at the cost of 1% higher neck muscle pain" (caricature of the idealized case. Too often only improvements are given). Besides, it is easy to fall in the trap of thinking you got some improvement, only for it to happen by chance (think https://xkcd.com/882/).

I believe you should trust advisor here as he likely knows which indicators matter. Obviously you should NOT simply sweep discarded parameters under the rug, you should mention all the parameters you measured during study, together with what you combined or discarded and most importantly - why. Even if others do not agree with your decisions, at least they will know the potential issues and limitations of your study.

4
  • 5
    Hmmm looks to me like the conversation was the day before submission so I guess that the draft was not handed in the day before of the conversation, it was probably sent earlier and I expect the advisor had it for some time - you may want to modify your reply
    – tom
    Jun 20, 2023 at 10:53
  • @tom Given the question it doesn't seem advisor had it for a while - it is possible but not plausible imo. If he did, despite prompts to give some feedback, this indeed belongs in the "not you" group in the second sentence of my answer. In any case, getting significant feedback a day before deadline means someone screwed up really bad. Jun 20, 2023 at 11:11
  • 2
    Funny that this answer is sitting at -5 even though it is suggesting OP to do exactly what OP ended up doing. Perfect example of how a bad opening paragraph can destroy an otherwise fine answer!
    – walen
    Jun 21, 2023 at 14:37
  • 2
    Yea, the accusatory language in the first para, even though it may be in slight jest, doesn't do the rest of the answer justice, especially since the question itself doesn't mention anything about the draft being given to the advisor the day before deadline, only that they discussed it the day before deadline. Also, it's not an unusual phenomenon that professors don't really read things sent by their students until near a deadline, so the reaction in the first para (with those multiple question marks) feels off.
    – justhalf
    Jun 21, 2023 at 15:15

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .