3

The worst thing I am worried about is losing my legitimate data and then maybe being accused of falsifying it.

how does the university differentiate the data fabrication and honest error? (I have lost all the data used for my deep learning training, and I cannot use the raw data to defend myself). Did the reproducibility problems of deep learning contribute to academic misconduct?

The dataset collected and made by myself has been lost. However, I can repeat the experiment based on my code (I have backed up all my code but lost the private dataset, the method in the thesis is feasible). The experimental results will be different from my thesis results if I repeat the experiment (because I have to collect the dataset again from websites, and the image dataset will change). In addition, the reproducibility flaw of deep learning will impede the reproduction of the previous results (there are many random parameters in GPU or other settings).

I did my thesis by myself without any data fabrication thoughts. But the lost dataset and the reproducibility flaw of deep learning made me worried about my thesis if someone accused me of data fabrication. Readers can get similar results and conclusions by repeating my experiment (they can use their own dataset to get something). But readers cannot get the same results in my thesis again. Can I defend myself by the similar results and replacing the previous experimental results in the thesis?

1
  • 1
    There is no requirement for the undergraduate students to back up the final year report data after graduation. But I am worried about this now. I would be happy if you could answer my worries!!! Best regards!!
    – GabiY0
    Commented Jun 17, 2022 at 11:23

1 Answer 1

3

To answer your title question, no, fabrication of data is not the same as losing data. The latter is certainly still undesirable, but nowhere near as bad as the former. As you point out, losing data means that analysis is not reproducible, which damages the scientific basis for conclusions.

As far as how a university might differentiate between data loss and fabrication, one thing they would do is to look for corroboration from others that the data existed at some point. This might include asking your supervisor or other relevant people whether or not they can corroborate your sourcing of the data at any point. Another thing they might do is to look to see if repetitions of your experiment get results that are anywhere near your previous results. Fabricated data generally differs systematically from reality, which means that when others repeat an experiment that used fabricated data they may get wildly different results. Contrarily, when data is merely lost, the underlying analysis is still reality-based, so when people repeat the experiment they will generally get results that are somewhere near the previous results. Finally, fabricated data sometimes has some telltale statistical properties that are used in fraud detection, and these can be used to differentiate data fabrication from data loss.

In the present case, the data has been lost to you due to an overwrite by the owner(s) on the external source. In such a case the best thing to do would be to write to the owner(s) of the data and ask if they have a copy of the previous overwritten version that you used, and get them to send this to you if they have it. Lastly, take this experience as a lesson about reproducible data practices in the future. In future, when you work from data on an external website, first download and keep a local copy of that data in its raw form, with appropriate metadata specifying when and where you downloaded it. (If you want to be able to demonstrate that the external data was in that form at the time, you can also use the wayback machine to archive the data, though this is also not foolproof.)

5
  • I cited the data source website in the thesis but did not show the specific data (3000 images). I found that the website has updated the data this year, but most data are the same. I cannot recover all the raw data, but I can find most of them (these data still exist on this website, I only forget which image I downloaded). However, the results will not be identical to the previous ones (one different image will lead to slightly different results) if I cannot recover all the raw data. I have demo photos and videos showing how I process some images. It is very kind of you! Thank you so much
    – GabiY0
    Commented Jun 17, 2022 at 12:58
  • I got my thesis Code deposited in the Github repository during the thesis writing-up. Moreover, I can easily repeat similar results from a similar dataset. I can use the evidence above to prove that the experiments exist, and I have everything except the identical dataset. Can the evidence above prove that there is no data fabrication?
    – GabiY0
    Commented Jun 17, 2022 at 13:28
  • 1
    If I get it from your comment: you did not lose the data but the data set that you used was updated at some point in time, and you cannot reconstruct the original data set as it stood when you did your experiment. You would thus have two items indicating absence of any data frabrication: you cited the original data source, and their could be traces that it has since been updated, furthermore, as Ben states in his answer, the underlying basis has merely been updated and repeating the processing on the updated dataset should lead to similar conclusions.
    – PLD
    Commented Jun 17, 2022 at 14:05
  • @PLD You are right. The data exists on the data source website. I only lost the dataset constructed by myself (I labelled and classified the dataset), and the website shuffled the orders of images. Therefore, I cannot reconstruct the identical dataset for machine learning training. Thank you so much!
    – GabiY0
    Commented Jun 18, 2022 at 12:54
  • Dear Ben, the owner(s) have no copies of the previous version. But I think a similar result and conclusion by running a similar dataset is enough to prove that my data is not fabricated. Thank you so much!
    – GabiY0
    Commented Jul 1, 2022 at 0:36

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .