11

A statement from this answer has caught my eye:

certainly you can't be expected to hold onto old datasets in perpetuity

So far, this is exactly what I have been striving to do, to the best of my ability. It is often impractical, it consumes a lot of resources (time being the most costly one), and pragmatically, I am near-certain I would never ever have any use for much of this data.

However, ethically, I consider it to be a part of the burden associated with the job: no one likes chasing apocryphal datasets and results, and sometimes new ideas require revisiting decades old work. Assuming the lack of convenient options for archival, what is the reasonable retention time for the experimental datasets? 10 years? 15? 30?

What would be the guiding principles to determine it?

Improve this question
11
  • 4
    I think this rather depends on the type of the data set, but in the current age where many data repositories are freely available, why not publish the data if feasible?
    – Anyon
    Commented Oct 13 at 17:40
  • 3
    @Anyon: biological, medical or psychological data often has strong privacy implications. (Fun fact: maybe you collected data ten years ago under then-legal privacy and data protection rules, which nowadays are not OK any more... but you may need to hold on to the raw data ten years beyond the last publication based on them, which might only have happened recently.)
    – Stephan Kolassa
    Commented Oct 13 at 20:02
  • 1
    Is it really that impractical? In my experience, storage prices have been dropping faster than how fast the rate at which I produce research data grows. This means that the older the data, the cheaper it is to hold on to it, to the point where it's cheaper to just keep it indefinitely than to think whether and what could be deleted. A year's worth of my raw research data from 15 years ago is just a couple dozen gigabytes, and that's what I produce in just a few days nowadays.
    – TooTea
    Commented Oct 14 at 8:36
  • 4
    I have several boxes of 5" floppies from way back. They are available. They have been retained. Getting the data off them might be a slight problem...
    – Jon Custer
    Commented Oct 14 at 12:26
  • 2
    @Lodinn As far as archival goes, I'd just store the raw files and forget about the rest of the stack. That's probably the most cost-efficient option, because with a very slim chance of the data ever being needed again, it makes more sense to potentially have to pay thousands to rebuild the stack at that point if needed than to maintain it all the time. 1T of archival storage (e.g. S3 Glacier archival tiers, no endorsement) can be had for just a few €/$/£ per year, so some 100T costs a tiny fraction of the yearly budget of a typical lab.
    – TooTea
    Commented Oct 14 at 13:51

2 Answers 2

Reset to default
6

The job of archiving stuff is best done by professional archivists. There are various archives that focus on preserving data and making them available in a legal and ethical way, e.g. ICPSR in the USA, Gesis in Germany, DANS in the Netherlands, etc. Go to the website of the archive that is relevant to you, and you will find instructions on how to submit your datasets.

That way there is a legal department that takes care of all legal problems that may arise. If 40 years from now nobody can read the dataformat of your dataset, then the archive has people specialized in reading ancient file formats. If you are no longer alive 40 years from now, the archive still exists and your data is still available. So this is not something you should try to do yourself.

Improve this answer
12

A good starting point would be your funder's requirements. In Germany for example the German Research Foundation (DFG) funds a lot of university research. Its guidelines for handling of research data state that data should be stored for at least 10 years (see their guidelines).

In fact, I have had a lot of projects funded by the DFG, but they never checked on our research data management so far. Also, although you have to include a statement about data handling in proposals, my project partners rather often do not take this very seriously, and also the reviewer reports never mentioned anything critical about it. So for all practical purposes, my take is that we are still working towards a standard best practice.

Improve this answer
1
  • 1
    UKRI on the UK also specifies 10 years.
    – Ian Sudbery
    Commented Oct 15 at 8:09

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .