Is it ok to fail with analysis?

Question

I'm a graduate student and I am really worried about the quality of my master thesis. It's my second year in graduate school and I've already done some pilot studies. But, entering the school, I was absolutely new in the real science (cognitive psychology) and now I'm afraid that data I collected is too dirty and statistical analysis I provided just incorrect. I have to present and publish my results anyway if I want to end graduate school and become a phd, but I feel ashamed of them. So, I wonder is it ok to have problems with data when I'm just studying or I should refuse to publish or present these results? My supervisor thinks everything's great and local journals will accept papers, but it is just bad, isn't it?

Answer 1

Since your supervisor thinks that everything is fine, you should not be ashamed of publishing your results.

If you are still unsure: You should talk about your concerns to your supervisor and request an honest opinion. Keep in mind, that he/she is also (partly) responsible for the quality of your results!

Answer 2

Imposter syndrome

I am really worried about the quality of my master thesis

This is normal. You are still being educated in your field, but working alongside professionals, reading professional papers; this means you're judging yourself against an unduly high standard: of course your pilot studies are not "as good" as a "finished" piece of work from professor or a postdoc.

This can cause, and be compounded by, imposter syndrome. In imposter syndrome, you fail to correctly judge the standard of your own work (or set your standards unduly high) and, as such, underestimate your own ability leading to the feeling of being an "imposter". This is incredibly common - most (non-sociopathic) academics will experience it at some point in their careers!

To overcome this (and to identify when perceived concerns are actually valid) it is important that you take on board feedback from your colleagues. Your peers (the master's and PhD students you directly work/collaborate with, attend class with etc.) and supervisors/lecturers will be able to help you identify your strengths and weaknesses. Most importantly, they will be able help you overcome the imposter syndrome by providing a realistic judgement of your skills. In your case, the feedback mechanisms are suggesting that everything's fine:

My supervisor thinks everything's great and local journals will accept papers

---

How to judge/assess your own work

I'm afraid that data I collected is too dirty and statistical analysis I provided just incorrect

The best way to deal with this situation, is as follows

• Make a list of specific problems you perceive in your analysis (e.g. should have used this kind of hypothesis test, should have compared these variables).
• Make any easy fixes (running a T test, for instance, can often be done quite easily).
• Now consider whether the remaining "problems" seriously undermine your conclusions.
  • Do not use perfection as your standard for this: nothing in research is ever finished or complete, especially "pilot studies"; just consider whether the data supports your conclusions.
  • Consider if you can resolve any undermining by hedging or changing your conclusions (e.g. "x causes y" --> "x show strong positive correlation with y")
• Look at published papers - is your analysis of comparable methods/standards? Have you performed a similar level of rigour?
• Listen to your supervisor's advice. They're the expert. They've published before. If they think the paper's fine, it probably is.

When you follow this, you'll probably find that your work is in far better shape than you initially thought.

---

On a related note:

I have to present and publish my results anyway if I want to end graduate school and become a phd

Where does this come from? Unless publication is a requirement for graduation (unusual for a master's?), this simply isn't the case. You do not require publications to successfully apply for a PhD. So (unless they're a requirement for graduation) don't stress over publications, treat that possibility as a stretch-goal instead.

---

Some light relief

Answer 3

I think the beautiful thing about a statistical analysis is that, when done right, it includes the odds that it is incorrect.

The honest thing to do is to include as part of your paper possible confounding factors that may make your conclusions invalid. A good reader is thinking of these anyway, so it only makes you look better to state them in an open way.

You don't have to (and shouldn't) put this in a way that insults your work. But perhaps something like, "A future study where [the data in question] is gathered in [suggested "less dirty" method] would be useful in further understanding this issue."

Frankly, I think your impulse to want a high quality standard for your work is an admirable one.

Answer 4

It is really difficult to answer this question without knowing your work (and knowing your field to some extent). Maybe there really is a problem with your work, maybe it is just impostor syndrome, which in my personal experience seems to be really common among young scientists. I don't think anyone here is in a position to tell you which one it is.

Personally, I believe that the reason scientists do low-quality work often is that they fail to recognize the problems with their methodology. Having doubts is normal, and I would even go as far as to argue that it might be a good sign because it shows that you have the right attitude, whether your doubts are justified or not.

I agree with the general sentiment of the other answers that it is probably best to ask for feedback from your peers. You have already convinced your supervisor, which is obviously quite important. If you need more positive reinforcement and are worried about particular problems in your work such as the statistical analysis part, consider asking someone to review just that part for you. I have often asked fellow graduate students who I believed to have more expertise than me in a specific field for their opinion on specific parts of my work. Of course, their opinion doesn't carry as much weight as the opinion of an experienced scientist, but it could provide you with some orientation. If several others think your work is fine upon closer inspection, I wouldn't worry about it.