A lot of Hype goes in reproducible, open and transparency for data analysis theses days in some fields mores that other. The code is to be share as well as the dataset. How to achieve this in a context of patient data protection that all ethics' committees will require ? What are the best practice to overcome theses antagonistics goals ? My team seems to think patient's data protection means the data (already anonymized) shouldn't leak and thus we have to work on the database's server without ever getting the anon db on our stations. This feels a tad too much / an overkill ?
4
-
1I assume there are laws and that the laws vary by country.– BuffyCommented Oct 31, 2019 at 16:09
-
2I think "reproducible" doesn't really mean running exactly the same code on exactly the same data. You get much stronger evidence if an independent research design is developed for the same question and the results agree (or not).– BuffyCommented Oct 31, 2019 at 16:11
-
@Buffy please see a previous answer about the difference between reproducible and replication academia.stackexchange.com/a/118511/33210– Richard EricksonCommented Oct 31, 2019 at 16:26
-
Yes I wasn't thinking about replication, more about transparency regarding the database : 1. there is a huge amount of data where the initial analysis answers only a few questions (for example by lack of power for the others) or 2. where the data would show the analysis done although(theoricaly sound by itself), isnt appropriate ...– SamuszCommented Oct 31, 2019 at 17:36
Add a comment
|
2 Answers
Its totally normal practice for patient data to be allowed to be stored only on properly secured, regulated and monitored servers.
In my field (genomics), databases exist where researchers can deposit anonymized patient data, that is not "publically" available, but can be accessed by users that can demonstrate that they have good reason to use the data, that their use falls within the consent agreement and have the IT systems in place to keep the data safe (see for e.g. dbGaP and EGA) by making an application to a data-protection committee.
Code can be written in such a way that no patient data is in the code (loaded from secure locations, etc), so that the code can be shared.
answered Oct 31, 2019 at 16:20
-
Never said the practice to regulate access to sensitive data isn't sound, (actually it is of our ethical responsibility to do so and enforce it). But to me this practice seems to impeds real analysis transparency. And thus real openness regarding the justifications for some of the data manipulations or analysis done. (not showing the data, how can you really discuss those in any other form than vague ? I was desperate for (dreaming for?) a middle ground between ethics & reproducibility, guess it's a fool's errand if secure means closed. (what about reviewers be given same access than PIs?)– SamuszCommented Oct 31, 2019 at 18:06
-
You said "This feels a tad too much / an overkill ?" In the perfect reproducible world, you'd make decisions about data manipulations independently of the data. In face in the ideal world, you'd have written your analysis scripts before the data was even generates (yes, we all know that never happens). But you can still share you code, which shows what manipulates were done on the data, and provide some reasoning for why. You are still allowed to show summaries of the data (shapes of distributions etc) to justify things. Commented Nov 1, 2019 at 9:29
-
In my field, reviewers would generally be granted access if they asked for it and when through the appropriate vetting process, but to be honest, given that the processing steps generally take many years of CPU time in total, the chances of a reviewer rerunning them are pretty slim. Commented Nov 1, 2019 at 9:30
People have already though about the this. The American Statistical Association has guideline for reproducible results (here).
- Reproducibility is enhanced by following best current practices, including:
a. Ideally, exclusive use of publicly available data. However if the research domain does not allow for publically available data for widely accepted reasons (e.g., medical data with high confidentiality concerns), the principles outlined in items (b) - (e) should still be followed;
b. Use of version control for all (collaborative or individual) code development;
c. Exclusive use of open-source software freely available to anyone in the world
d. End-to-end scripting of research, including data processing and cleaning, statistical analyses, visualizations, and report and/or manuscript generation, with the full workflow made available to others;
e. Use of container/virtual machine tools to capture software versions, dependencies, and platform specifics;
f. Publication of code in public repositories as with data; and
g. For projects that develop algorithms, implementing algorithms on standard computational platforms (e.g., R packages, Python packages, source code packages installable via standard methods, etc.).
Reproducibility shouldn't be thought of as a binary state: either reproducible or not-reproducible. It's more useful to think about it as continuum from hard to reproduce to easy to reproduce. The goal of any reproducibility effort should broadly be to move as many people as possible further towards easy-to-reproduce. This involves some technological (to make the right thing easier than the wrong thing) and some social (to give people the activation energy to learn a better process even though it's harder in the short-term) components.
It’s perhaps worth noting that in this era of “replication crisis”, reproducibility is the only thing that can be effectively guaranteed in a published study. Whether any claimed findings are indeed true or false can only be confirmed via additional studies, but reproducibility can be confirmed immediately.
Also, the medical literautre has discussed this. For example, Tucker et al. describe how to protect patient privacy when sharing data. Their two key recommendations were:
Data anonymisation/de-identification: Data holders are responsible for generating de-identified datasets which are intended to offer increased protection for patient privacy through masking or generalisation of direct and some indirect identifiers.
Controlled access to data, including use of a data sharing agreement: A legally binding data sharing agreement should be in place, including agreements not to download or further share data and not to attempt to seek to identify patients. Appropriate levels of security should be used for transferring data or providing access; one solution is use of a secure ‘locked box’ system which provides additional safeguards.
Searching through the medical and statistical literature will give you more ideas we well.
answered Oct 31, 2019 at 16:24
