My advisor has been asking me to generate plots I am uncomfortable with, since they overestimate the success of our approach, or make the data seem less noisy than it is. He says this is okay because the numbers in the paper don't matter as much as the high level message. What should I do?
We haven't fabricated data per se but some of the analyses are sketchy. For example in one plot we are comparing our approach to a baseline, but due to lack of data, any point that our approach fails on will also register as a failure when we apply the baseline method to it, meaning our approach will always look better.
Due to lack of data, there's no obvious "alternative plot"; the alternative would be not generating any plot, or generating a plot that underestimates the success of our approach (which he says is just as bad as overestimating its success). In some cases, our plot and another plot are equally defensible, but our plot looks a lot better so we kept it.