Have you all ever had different results when repeating your own experiment? I did some work for my dissertation and am now redoing it for better images and hopefully for publication. The results are not replicating at all which leads me to think either an error was made before or is being made now. Would you think to repeat with more replicates and document those results instead of the initial results? It's just so draining, but I don't want to publish inaccuracies.
1
-
3It depends on what type of experiment you are performing. If the experiment is rolling a die and recording what face lands up, then you would expect different results. If the experiment is counting how many sides the die has, then you would expect the same results.– JRNMay 27, 2020 at 1:30
Add a comment
|
2 Answers
That certainly means that this material is not ready for publication.
No matter when the error (is it even an error?) was made (in the initial or subsequent experiment), one first has to figure out why the results are the way they are. From some angle, it is even a blessing that the results are mismatched: it forced to "bake the research" a little longer until it is ready.
I strongly suggest analyzing the experiments setup, experiment procedure, raw data, post-processed results in order to explain what is going on. Maybe, it's worth conducting a third experiment (if it is a viable approach).
While triple modular redundancy adage does not fit here perfectly as an analog:
Never go to sea with two chronometers; take one or three.
the third (fourth/fifth/...) experiment might give you some insight into what is going on. And taking only one chronometer—just because it leads to a potentially faster publication—would be a wrong approach.
answered May 27, 2020 at 1:33
-
1I certainly do not plan to publish as is, I was thinking of repeating a few more times and being as methodical as I can be with the approach. I also plan on going over my methods with a fine tooth comb to see I can understand where the discrepancy lies. Thanks! Repeating a few more times is sound advice May 27, 2020 at 1:36
It is difficult to tell without knowing the nature of the experiment. If you are using simulated data, discrepancies are normal. Perhaps both runs are correct but some simulated values caused the results to diverge greatly. Also, there may be a mistake or inadvertent change the second time. Perhaps both runs are correct or perhaps both are false and you need to fill in a practical or theoretical gap. You should double-check everything but in a meaningful, consistent way. "Why the results are wrong" might be the wrong starting point - try something like "what is different in the setup/ layout/ data/ process/ interpretation?". Don't focus on matching the results, focus on spotting the differences and then try to assess their effect.