How to deal with an adviser that wants to force you to get his desired results?

Question

I'm doing research for my PhD thesis based on a sensitive subject related to biomedical applications. In fact, our workflow is:

1. Build a computational model
2. Verify the developed computational model based on tests and data available in the literature
3. Apply this developed and verified model to some other data to measure an important parameter for clinicians that people's life will depend on that.
4. Investigate the result of this application on that data as well as its outcome and relevance for clinicians.

My problem is the fourth stage in this workflow. First of all, nobody ever did the fourth stage in this workflow for that particular application before. There are some similar models in the literature that tried to investigate the outcome of applying a similar model for that application but their conclusions are so general and vague where a definite conclusion cannot be drawn. When we apply that developed and verified model to that data, it produces some results which may look counter-intuitive at the first place, but there are a few papers in the literature that actually confirm similar observations. These results are not bad but kinda look like a negative result. We are confident in our results because this model is validated and verified based on several independent cases.

Unfortunately, in my PhD adviser's eyes, these results are worthless cause they are not desirable and he thinks nobody will buy this results if our conclusion is something counter-intuitive (well counter-intuitive based on his thoughts at least...). Every week in our group meetings, he reminds me that these results are worthless and I should change the developed model in a certain way. He doesn't give me direction regarding what way I should change the model, but it is important that we get intuitive results right now.

I'm feeling like he is forcing me to search for his desired results. It is possible for me to do that but I believe that's cheating or could be called hiding the truth. My question: Should I change my model to get his desirable results? if no, what's the proper way to convince him that this counter-intuitive results maybe is the truth and we should live with them?

Answer 1

The meaning of unexpected results

It is important to be skeptical about your computational approach. However, at the same time, computational approaches are (almost) completely worthless if we just ignore them when results are unexpected (unless you already have dominant evidence that the result is not only unexpected but also simply wrong). An exception would be if your approach is in the area of generative models, where a parsimonious model is suggestive of an underlying mechanism, which is not the area of your model: you are trying to do prediction for an unknown case (extrapolation).

The art is in determining whether your initial model of the world (i.e., expectation) is wrong or if your computational model is wrong.

In a long discussion in chat, I think we came to a conclusion that in your specific case, it may be that this is an issue of extrapolation to a condition where you do not have truly comparable training data.

How to stop worrying and learn to love the model

If you want to convince your advisor, colleagues, peer reviewers, or yourself that your model should be trusted, your next steps are to test the conditions that lead to your result.

Do all the appropriate tests for model convergence in the original training. Check for input parameters that are outside the range in the training set. Use graphical representations of your model to show how input are mapping to outputs. Remove or scale variables to test the sensitivity of your model to those changes. And additionally, as your advisor suggests, figure out what it takes to make your model fit the expected result. All of these approaches will help you find if something is wrong in the model or support you if something is wrong in the prior expectations.

Answer 2

Any computational model which tries to model a disease scenario has flaws, since all models try to reduce an extremely complex problem to a simple one as Buffy pointed out in their answer.

Furthermore, your question makes me think that you are working in/with a computational/bioinformatics group. If the results that you present are counter-intuitive, I must side with your advisor on the statement that a study which presents counter-intuitive results will not be well received. Any counter-intuitive results derived via computational models will need to undergo vigorous hypothesis testing via experimental methods to be well accepted by the community.

If you still want to present such findings, you can

1. Avoid any mention of causal links.

2. You can present the results as a secondary finding while comparing your model to other such models described in literature.

3. You can also break up the bigger finding into smaller parts which may be well received by themselves, but not together (Present them independently).

Coming to the part about

measure an important parameter for clinicians that people's life will depend on that

Investigate the result of this application on that data as well as its outcome and relevance for clinicians.

Results from single academic studies are rarely used as a backdrop for larger clinical applications. Any academic finding however grand they may be, will undergo control analysis in multiple rounds of replication studies, and then it will be presented as part of a larger landmark review article. Results presented in such a context may end up reaching the desk of a clinician. Even then, they will think twice before applying those results towards their patients.

Although it is great to think about the ethical context of studies in basic research, I would strongly advice you to think about yourself in the grander schema academic research before you associate a high weightage towards such concerns.

Answer 3

There are many good answers that discuss the topic from "What is the right thing to do in research?" standpoints. Let me give a bad one from practical standpoints.

My question: Should I change my model to get his desirable results?

Yes, so you can finish your degree on time. Since you are talking about thesis, I assume you are in the very late stage of Ph.D. It's too risky to not finish it. If you were in an earlier stage, I'd recommend you to find another advisor quickly.

I am not in your field so I cannot judge. You might be the one that is right but it's irrelevant. From all stories I've heard and experienced myself, very rarely do I see an example of a graduate student successfully change advisor's mind. More often than not, these arguments go badly and things fall apart, the only one that gets hurt is the graduate student.

I was in a R1 university doing computational chemistry and faced a very similar but worse dilemma as the one you described. People in that field regularly publish overfitted computational model that has no other use except for fitting a few well known numbers from experimental data. I argued that these models cannot produce useful predictions and provided my own simulation evidence.

Then a few professors from that department including my advisor at that time decided to kick me out, called me "not suitable for doing a Ph.D." (OK a lot more detail here but that's not important for this answer if you are really curious search my other posts)

It was 5 years ago and now I am in the late stage of another Ph.D. program in a different field. People I know that used to do much worse than I did are mostly postdocs./professors/in senior positions in industry now.

Now, go back and change your model to get his desirable results so you don't have to follow my footsteps. If you feel this is wrong, just quit and do something else you are happy about. Yes the real world, even in academia, is this unfair.

Answer 4

I guess I'm going to have to support your advisor here. Your characterization seems a bit odd for a situation that wants to make clinical recommendations where people's health or safety may be endangered by bad advice.

You seem to want to imply that your model is a substitute for reality, and even is superior to reality since it has been "validated". But in reality, no model is perfect. No model perfectly captures reality. It is an abstraction from reality of course and makes some simplifying assumptions. Models are simple and reality is extremely messy.

A model that "seems" to predict reality is useful, of course, though it may not do a good job in some cases - edge cases in particular. But perhaps "his desirable" results, simply means "sufficiently matches reality".

The validation of your model is not proof that it gives good advice in all necessary situations. If you have any evidence that it sometimes fails, then it would be extremely unwise, even unethical, to apply it as is in clinical situations - at least without other evidence for the suggested regimen.

A model with little consequence for human health and safety can be somewhat flawed and still useful. But I'm worried about this situation. Perhaps all your advisor is saying is that you can, and should, do better. It seems to me, at least, to be wise advice.

Again, a model that shows any evidence of failure in some situations is suspect. If the application of the model is critical, then even minor evidence can be dangerous.