11

Several years ago, I did a summer project as an undergrad with a PI not at my undergrad institution (I'm in grad school now). Now, the project is finished and he wants to publish it with data that I've gathered.

The problem is, after checking my data, that the spreadsheets that I used to calculate the data are very inaccurate, to say the least. I can't remember exactly what I did, but there do seem to be some egregious inaccuracies (as in, values from the wrong dataset) in my spreadsheet compared to the raw data from the instrument either that I was colossally bad at copying data over correctly (all of the data in my spreadsheet do appear in the raw data someplace, but not necessarily in the right location, the right dataset, etc.), or worse, I'm terrified that I may have done it deliberately at the time. (How would someone be able to tell?) I did analysis with the faulty data back then and produced graphs.

Obviously, I will have to supply the correct data analysis before it gets published. But is it ok to explain it as "I majorly screwed up the analysis and made a lot of data entry errors, here's the corrected version"? I'm hoping that my PI will just use my corrected data and analysis and just focus on that instead trying to pore over my wrong spreadsheets to individually compare all the changes, many of which are embarrassing at best and look suspicious at worst. I am not under any suspicion of wrongdoing thus far.

I redid my data analysis with correct data entry and the results look messier on most of the things I messed up, though they do not affect the underlying trends. Not sure what this means - how likely is it for honest error to have the effect of making data look cleaner than it actually is? Also, my god, I was awful - mishmashing data from one trial with another, copying data from the wrong wavelengths and/or from the wrong trial, wrong normalization numbers, etc. etc. etc.

If you were a professor and you were approached by a student who says that made egregious errors on past data analysis and the corrected data looks not nearly as good, what would you think? Would you think they were honestly trying to fix a mistake, or trying to save face and cover for past misconduct?

15
  • 31
    How is it that your own data dwell in some Schrödinger-esque contraption in which they are either falsified or merely erroneous? Do you mean to say you've actually falsified these data but are trying to be coy and test whether it can go unnoticed as you correct them? Or looking for a reason to keep it in the altered form? – Bryan Krause Feb 23 at 0:56
  • 47
    A bit of a tangential point, but this is also why you should not be doing data work in spreadsheets. Data work should be done through code, without altering the originals, so that it can be reproduced and audited. – Jeff Feb 23 at 3:54
  • 35
    I find it hard to believe your implication that you do not know whether you deliberately mishandled the data in the past. Do you have any reason for not knowing for sure? – user21820 Feb 23 at 9:25
  • 22
    @8263xiao: Hmm. I don't have a problem remembering whether I was deliberately dishonest or not, even a decade ago. From your last comment, it sounds like you do not have a habit of being dishonest, and you do not remember intentionally being dishonest. If that is so, take it at face value, because carelessness can cause a huge amount of errors. In particular, a mouse-slip might cause an entire chunk of text to be dragged from one point to another, and if middle-click does a paste operation then an accidental middle-click can overwrite data with data from somewhere else without you knowing. – user21820 Feb 23 at 9:47
  • 23
    People who has perfect memory cannot get this. I totally can. Many times in my life including when I was young, I had situations where I could not remember some pieces of information, other people would deem "impossible to forget". One of the situation was very similar, I was accused of a misconduct, and I could not remember the exact chain of events, that would clear my name, I could not confidently say I did not do it. I suffered the punishment, and then a witness that was unwilling to come forward during the process, approached me and informed me that I in fact did not do it. It does happen – Andrew Savinykh Feb 23 at 21:32
55

I infer from the question that you still have the raw data. In that case, I think you need to re-do all the steps in which errors could be introduced (such as data entry, spread-sheet calculations, analysis of trends). Then see what effect the corrections have on the conclusions of your paper, and rewrite as necessary. I realize that this sounds like a lot of work, essentially repeating most of what you had done earlier, but since you apparently did it wrong the first time, you now need to do it right.

Meanwhile, you should tell the PI about the problem and your plans for correcting it. If I correctly understand the situation, the raw data are the only reliable information currently available. You should make sure he knows that, so that he (and others working with him) won't build further research on a foundation of wrong results.

2
  • 9
    I do still have the raw data, and I did redo the analyses prior to asking my question. Most of the errors were in the data entry - copy/pasting numbers from the wrong places, and there was quite a lot of that such that it's almost inexplicable. The effect was that the trends stayed the same but the error got bigger. There were other things I had to do with the spreadsheets, such as re-organizing and making a new graph, so I just sent him the spreadsheets/graphs I corrected. Hopefully he takes it well and doesn't react with anger or suspicion of previous wrongdoing. – 8263xiao Feb 23 at 2:23
  • 42
    And, most importantly, stop doing science with spreadsheets – Azor Ahai -him- Feb 23 at 19:52
17

Everyone will have their own reaction, of course, but the more important thing is that you point out the errors and the need to correct them. Going forward with bad data is the worst outcome.

Work up the best data you can and tell the prof that the older data was seriously flawed.

I think most people will accept your honesty, even if it means some delay. And most will forgive at least some of it due to your lack of experience at the time.

Hopefully the original data is still available making corrections possible.

But, you also need to be honest with yourself about how this all happened. I can think of a few very bad scenarios that I hope weren't contributing factors. There are actual medical conditions that might have such an effect. You don't want a recurrence.

7
  • I did. Hopefully that will satisfy him. My previous work was admittedly atrocious (I was definitely a very sloppy undergrad), and my fear of suspicion is because the corrected data looks messier and has much larger error bars. – 8263xiao Feb 23 at 2:27
  • 2
    @Buffy If the internship was relevant to a grade or a reference, the discovery of fabrication of data may unravel the conditions for OP being in grad school. It is very much to be hoped that the data were just misgenerated in a fit of sleep deprivation rather than in an act of intentional fabrication. I know that I would wash my hands off a student that I suspect of previously fabricating data. This is the one capital scientific misconduct that a reputation can not recover from, in my opinion. – Captain Emacs Feb 23 at 5:22
  • 9
    "And most will forgive at least some of it due to your lack of experience at the time." I think so, too, but publishing data generated by an undergraduate student without double checking is also a mistake, maybe also caused by lack of experience. Both student and advisor will have learned something from this. (Not saying anything about the intentional vs. accidental discussion which can only be decided by looking at the actual data.) – Snijderfrey Feb 23 at 8:08
  • 1
    A mistake is unfortunate, fabrication is evil. Intentions matter. If it was a mistake, it's still not nice (it is a nuisance to have sloppy collaborators, but if they learn to become more careful, one may accept that they improved). Bad intentions will however leave an ever-lingering suspiciion. – Captain Emacs Feb 23 at 18:13
  • 1
    @Buffy It's the OP's own words. I must confess I find it hard to put myself in the shoes of someone who does not even remember how their data came about. I remember the circumstances about the development of practically every paper I did as a first/main author. That's why I suspected that the situation might be the result of lack of sleep (or otherwise under extreme time pressure). – Captain Emacs Feb 24 at 0:04
9

I'll take your questions one at a time.

I'm terrified that I may have done it deliberately at the time. (How would someone be able to tell?)

I'm not sure if by "someone" you mean you, or the person you send this to. I can definitely imagine looking back at my own work from undergrad and doubting the integrity of it if it looked fishy. If I were trying to figure out whether I had done it intentionally, I would look for patterns in the errors (e.g. maybe treatment group is only ever misrecorded for data points that pattern against the tested hypothesis), and see if they jog your memory. I suppose that's probably what I'd do if I were trying to figure out whether someone else had done something intentional, but I seriously doubt most people would bother.

But is it ok to explain it as "I majorly fucked up the analysis and made a lot of data entry errors, here's the corrected version"?

Absolutely. You were an undergrad and it's the prof's responsibility to check your work and to ensure you know what you're doing. Also, the unfortunate reality is that lots of profs intentionally teach students in their labs to mishandle data to "improve" results, so that's a potential explanation here too. Because of that, I'd be more likely to say something like "Looking back at the data now, I'm finding a lot of errors, so many that I don't think correcting them is feasible, so unfortunately I believe that this data would have to be reanalysed from scratch to proceed with the project." That is, tread lightly if you can't recall the origin of the discrepancies, but make your own position clear.

Also, it is absolutely not your responsibility to provide corrected data if you don't want to continue with the project. I assume as an undergrad you were either paid or volunteering in exchange for the training/experience, and I assume neither of those is true of your relationship with this PI/lab now. I recommend asking yourself whether you honestly want to continue with this project before putting any more work into this.

I redid my data analysis with correct data entry and the results look messier on most of the things I messed up, though they do not affect the underlying trends. Not sure what this means - how likely is it for honest error to have the effect of making data look cleaner than it actually is?

More likely than not. Unfortunately, we're a lot more likely to double-check steps in our work if we don't like the look of the data, no matter how thorough/neutral we might intend to be.

If you were a professor and you were approached by a student who says that made egregious errors on past data analysis and the corrected data looks not nearly as good, what would you think? Would you think they were honestly trying to fix a mistake, or trying to save face and cover for past misconduct?

Depends on the student and the professor. Personally I trust the students that I work with or I wouldn't be working with them, so I would believe them. And I promise you, every researcher has encountered egregious errors in their past work at one point or another, often at this very stage of preparing/doublechecking for publication.

10
  • 1
    Now looking back again, the most suspicious thing I saw in the data was that I had copied only some of the results of different trial into a spreadsheet for a particular trial. While certainly that is possible with error (I may have had both trials' raw data open at the time and thus it was possible to mix up), there were more data points in the trial I erroneously copied from, which resulted in the original trial having more points than it actually did, matching the other trial. 1/ – 8263xiao Feb 23 at 20:00
  • 1
    It's possible that when I was analyzing the data I misremembered the setups for each trial and thought they were both the same, and I may have set up my spreadsheet that way and the mixups gave me the wrong impression that I had the additional points for both. But that does look suspicious. For my corrected version, I just reported fewer data points, consistent with what I saw in the correct raw data. It didn't really change the overall results but did make them a bit more noisy. 2/ – 8263xiao Feb 23 at 20:11
  • 2
    My former PI never delved into the spreadsheets or compared them to raw data; I only showed him graphs. I don't think he ever pressured me to "improve" results per se, though as an undergrad I was definitely very insecure, eager to please, but not particularly professional/competent. I do want to get this published, but certainly not with bad data. I do have the original raw data that I now analyzed correctly, and I told him I found a number of errors in my past analysis. Hopefully he believes me and looks mainly at the updated/new analysis without delving into the past too much. /3 – 8263xiao Feb 23 at 20:27
  • 4
    If you can't remember what happened, I'd give yourself the benefit of the doubt and assume it was an honest mistake unless and until you discover evidence to the contrary. Basically treat your work as you would someone else's. It's really no different at this point, since you can't remember what you did. Your work is as much a black box to you as someone else's would be. – bob Feb 24 at 18:38
  • 3
    I'm not sure honestly. Notes in the Excel workbook that indicate dishonest intent ("round down to minimize appearance of errors")? Maybe someone else could comment here? But basically if it doesn't scream "this looks like academic dishonesty", I'd take a deep breathe and relax. It probably wasn't. This of course is taking at face value an inability to remember what happened. – bob Feb 24 at 19:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.