8

After a four year break, I'm finally getting around to submitting my MSc. research to a journal. My research involved time series data - analyzing a food sample over a series of days using a number of methods. For one of the methods, HPLC, the equipment was malfunctioning on the first sampling date, so I only have results from the following two sampling dates. I realized this significantly impacts my work, but I do have a statistical difference in between the two dates that the equipment was working, as well as 15 other attributes that were successfully analyzed on every sampling dates and tons of other takeaways. Any advice on how I can professionally say "the damn equipment didn't work and I was a lone grad student in the lab on fall break" without throwing out the HPLC data that I did collect?

1
  • 17
    "I realized this significantly impacts my work, but I do have a statistical difference" - Not related to your question asked, but be careful with this type of thinking, as it results in publication bias and negatively impacts scientific progress across the whole literature. Your research results have value whether or not a test for statistical significance finds sufficient grounds to reject the null hypothesis. If people only report significant results, then the literature will be saturated with false positives without contrary evidence.
    – Bryan Krause
    Oct 3, 2022 at 7:25

4 Answers 4

18

Maybe:

"The first data point was lost due to a malfunction of the equipment."

"Due to problems with the instrumentation, we could only obtain two HPLC values."

You might need to give an explanation why the experiment could not be completed or why the data is not necessary.

Something a bit stronger than "The HPLC values obtained conform to the general tendency."

3
  • 6
    Data was not lost. Data were recorded or attempted to be recorded. For the experimental reporting it's best to spell these things out rather than paraphrasing. Oct 3, 2022 at 12:32
  • 3
    This approach just raises the questions of why we should believe any of the HPLC results: so if it was broken on day one, maybe on days 2 and 3 it just spat out random numbers which just happened to match expectations based on the other techniques? OP needs to demonstrate (to themselves, at least) that the kit was working and the output is worth bothering with in the paper...
    – Lou Knee
    Oct 3, 2022 at 20:34
  • That is not the question that was asked. I assume that posters are ethical. That implies that it was a clear, diagnosable, and diagnosed malfunction. OP cannot repeat the experiments because OP has been out of the lab for four years. Oct 4, 2022 at 18:09
10

the damn equipment didn't work

... which is nothing particularly unusual. You mention it and that's it.

You may want to spell out that & why your samples could not be stored without degradation or were used up by the HPLC, which later on was found to be unreliable. Or whatever actually happened.

and I was a lone grad student in the lab on fall break

My professor would have said: that may be a practical reason, but it is not a scientific one. I.e., this isn't anything that goes into the paper
A more precise formulation of this catchy quote would maybe be: this is a practical (lab organization) reason that however isn't needed here, because it doesn't add anything of scientific relevance over the more direct (also practical) reason of "detector not working" - and the scientific impact, i.e. that the loss of data needs to be taken into account by limiting the conclusions, isn't changed.

The milk is spilt, and that's it for the paper. For the paper, the only remaining question is whether it still has sufficient content without these data, and whether you can justify not doing an additional run of the experiment (you may want to discuss this).

For your professional experience, it will be good to consider planning experiments in a more fault-tolerant manner in future.
E.g., to plan experiments in a way that does not hamper finding help if needed. Or adapting the analytical method to retain aliquots (if proper storage is possible) in case something goes wrong, planning more samples from the beginning so the results of a few missing chromatograms are not devastating. Or to be more redundant in your measurement planning in general.

 without throwing out the HPLC data that I did collect?

You only throw away the unreliable (or non-existing) part of the data. That may make the data analysis more complex, but in general it is no excuse to exclude the remaining perfectly fine measurements.

Typically the way to go in the paper is to include what you have, and limit your conclusions accordingly if some data is missing.

If the other measurements provide sufficient information, the paper is still fine.
If the loss of one (set of) chromatograms means there's not sufficient information left, that is not really different from not having obtained sufficient information for any other reason, including bad planning of what analyses are needed.


update wrt. @Snijderfrey's comment:

(I'm analytical chemist, though specialized on spectroscopy rather than chromatography; have working experience also on food samples, and have collaborated with food chemists)

And yes, science is to some extent subject to practial considerations. So a more precise distinction (and less black/white than the catchy quote) would be is the scientific impact of the practical decision justifiable?

The loss of a complete day of chromatograms out of 3 days in total is an order of magnitude where as a reviewer I'd like to see some explanation.

Part of this is that there are (bad) scientific reasons that could have lead to the outcome, e.g. "we never thought of HPLC until the first experiments were over" - which would have me think what else they did not think about...

The practical reasons @Snijderfrey lists are justifiable to me as they fall in the category of things that inevitably happen - though one can and should work towards low probability (as in: working efficiently):

  • Lab equipment breaking down: detector lamp do burn out, capillaries or fibers or cuvettes break, columns go bad, you name it. (wet-lab work unfortunately doesn't have a Crtl-Z key, and in particular not if it's destructive analysis like HPLC)
    Again the line is not sharp between the need to replace consumables, regular maintenance, and repairs.
    In OP's case, it seems that the proper person not being on holiday may have saved the day's experiment. Still, of all possibilities how such instruments can and do break, only a fraction can be repaired in short time. Or the alignment and/or calibration after the repair takes so long that the sample is anyways degraded. Yes, it is possible to guard against this failure mode, but it is maybe (likely) not worth while for OP's lab. (As opposed to e.g. a clinical lab that may have a policy that people have to plan their holidays so that certain skills are always available - because the consequence is patient safety rather than some bothersome explanations in the paper)

  • a sample being spilt/preparation error.
    This often doesn't lead to 1/3 of the data being lost: one may be able to take an additional one, or only one replicate/aliquot/portion is lost (this failure mode is one of the reasons why we do replicates).

  • Science operates at the boundaries of the known. We deliberately do experiments where boundary conditions dictate work flows that are not optimal from some point of view but that are still justifiable or necessary trade-offs: I've had samples where portioning had to be suboptimal because we could not grind (seeds that were potentially to be sown later on).
    I've also had far too few patients in a data set to really draw conclusions - but they were all we could get during 8 years of collecting - which we considered justifiable. Whereas I've also had too few patients in a data set where more should have been much easier to collect, and that IMHO amounted to bad science...

The flexibility to adapt analysis to slightly unbalanced data and limiting conclusions is to me just another tool to adapt to the risk of things going wrong - like planning replicates in advance so the loss of some data doesn't endanger the whole study/paper.


BTW: unless you have concentration data from other methods (with better time resolution), you may have significant difference between days 2 + 3, but you don't know whether that's day-to-day variation, field sampling error (unless you can rule that out since you took a sufficient number replicate samples each day), or time-dependent change of the original specimen.
BUT: this is no very substantial change whether you have day 1 or not - also with day 1 it would be extremely unlikely that you could separate these sources of variance.

5
  • Good answer, one remark to "that may be a practical reason, but it is not a scientific one": Isn't that also true if there is a malfunction of the HPLC device, mistake during sample preparation, or similar? Who cares for such reasons if a data point is missing? Including such an explanation into the paper inevitably results in a bigger chance of unfavorable reviews. The key point for me is "include what you have, and limit your conclusions accordingly if some data is missing". Oct 3, 2022 at 18:15
  • 2
    @Snijderfrey By giving the reason of one missing HPLC set, it looks less like you are suppressing data that does not match your hypothesis. And sure, the malfunction could be made up, but it still leaves a better impression and proactively hints the reader to a potential weakness of the data set.
    – usr1234567
    Oct 3, 2022 at 23:27
  • 2
    @usr1234567, I agree that it is a good idea to "proactively" make the missing data points known to the reader. This can be easily achieved by properly describing and discussing the data on a scientific level. There is no need to say anything about equipment not working. You can do it, of course, but it does not bring you any closer to draw conclusions about your hypothesis. Oct 4, 2022 at 6:52
  • @Snijderfrey: you're right, this is also practical - I'll clarify my answer. Oct 4, 2022 at 16:26
  • There may be field-specific "cultural differences" here. To me, describing how the data were lost is properly discussing the data on a scientific level. I was once in a cross-disciplinary workshop with physicists and one interesting outcome was that the level of detail considered appropriate in a publiation differed: we chemists thought of the physics papers that they didn't give sufficient experimental details, and the physicists said of the chemistry papers that they'd be afraid to be scooped revealing so many details. Oct 4, 2022 at 17:19
4

The question you need to settle first, is why you believe that your HPLC data from days 2 & 3 is meaningful. "the damn equipment didn't work and I was a lone grad student in the lab on fall break" doesn't come across as though you understand what the problem was, and so how can we trust that you can then be certain that on days 2 and 3 you got meaningful data? If you're not sure, throw the data out.

If, and only if, you are sure that the data is actually valid, then a trivial formulation would be "We analysed the data using techniques A, B, and C, and on days 2 and 3 we were also able to use HPLC to..."

Whether the HPLC kit was unavailable on Day 1 because it was broken or because someone else was using it isn't the readers' business. That the HPLC data presented is meaningful, very definitely is.

4
  • 1
    It could be as simple as the recorder did not record, the laptop crashed prior to having the data saved, or OP forgetting to plug in HPLC device the first day.
    – usr1234567
    Oct 3, 2022 at 23:35
  • @usr1234567: I doubt that - I'd be surprised by OP being allowed to operate an HPLC on their own without having sufficient proficiency and experience with the instrument to operate it well under usual circumstances. Not recording or computer crash would typically mean the loss of one run/replicate, not a whole day's data. Not being plugged in is obvious and would mean that OP noticed that before the first aliquot is used up. Oct 4, 2022 at 17:44
  • @LouKnee: There is no risk of the HPLC producing "random numbers" and OP not realizing that. Plus: HPLC typcially requires that you run calibration samples - and for sure you need to do that after repairs. There are certainly more subtle ways things can go wrong (also in HPLC), but I don't see any reason to mistrust OP's day 2 + 3 data more than usual for published HPLC data. Oct 4, 2022 at 17:49
  • @cbeleites unhappy with SX That's not really my point: it's for the OP to know how reliable the day 2 & 3 data are, but they probably don't actually have to justify missing day 1 at all (unless they pre-registered the protocol or something like that).
    – Lou Knee
    Oct 4, 2022 at 18:14
0

How about something simple like

Despite having problems with occasionally malfunctioning equipment, we were able to obtain valid results except for one set.

I'm not sure I've captured exactly what happened, but something simple like that. Add the name of the manufacturer if you must, or give them an acknowledgment elsewhere in the document. It might be best to separate the two, putting the ack in a footnote or acknowledgement section.

You can, of course, strengthen the wording if you like (...serious problems...), but don't rant in a scientific paper.

9
  • 3
    I would avoid the word "tinker" since it can suggest making changes without understanding of the problem. In British English the word also describes a mischievous child.
    – Rob
    Oct 3, 2022 at 12:02
  • 1
    This is too indirect. Oct 3, 2022 at 12:30
  • 1
    @user2705196, OTOH, it is a mistake to vent rage in a scientific paper. It isn't a social media post.
    – Buffy
    Oct 3, 2022 at 12:54
  • 3
    Not really :( ... if the equipment can't be relied on then you can't trust the results, and part of the experimentalist's skill set is understanding the kit to the extent that you know when it's working properly, and when it isn't and why. If you don't know it's working properly then it's spouting garbage...
    – Lou Knee
    Oct 3, 2022 at 21:22
  • 2
    @usr1234567: name of manufacturer anyways (always) goes into the materials & methods section in (analytical or food) chemistry. Regardless of how well the equipment did work. Oct 4, 2022 at 17:25

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .