4

Quite many (if not most of) scientific articles, related to computational sciences, which I have seen, do not share both data and code leading to the figures.

This may have of course good reasons due to other than scientific, still substantial aspects involving intellectual property and data protection laws.

Not widely known (less than 1000 Google results) is though this quote on reproducible science:

An article about computational science in a scientific publication is not the scholarship itself, it is merely advertising of the scholarship. The actual scholarship is the complete ... set of instructions [and data] which generated the figures. (David Donoho, 1998)

Obviously, scientists trying to climb on the giants' shoulders should experience more and more difficulties with growing science complexity.

Then, there is also a set of "TOP" (Transparency and Openness Promotion) guidelines to establish science reproducibility.

According to the CoS, Center for Open Science:

Over 5,000 journals and organizations have already become signatories of the TOP Guidelines.

There are of course open access journals, but this is about free distribution of articles (very good but not necessarily reproducible).

Now my question is, is this TOP model elsewhere binding or is it sort of idealistic: specifically, are there journals/universities really inforcing the highest Level 3?

Level 3: Data [and code] must be posted to a trusted repository, and reported analyses will be reproduced independently prior to publication.

Further sources:

  • "Building Research Integrity Through Reproducibility University of Utah" Victoria Stodden, 2018 slides
Improve this question
2
  • I'm trying to figure out how Level 3 would actually work - who would do the independent analysis before publication? I don't think running the provided code on the provided data to be an independent analysis (and if the TOP guidelines do, shame on them). So, I'm going to go with a no, it isn't done since there isn't really a way to do it properly.
    – Jon Custer
    Sep 12, 2018 at 13:23
  • @JonCuster regarding independent, I think they refer here just to reproducibility, or? i.e. me, you, anybody, could get the code and the data and produce same results without even much understanding what is it all about. - following Donoho, you have so to say the complete set of instruction to reproduce the experiment. This of course does not setup an "independent" scientific review.
    – J. Doe
    Sep 13, 2018 at 8:30

1 Answer 1

Reset to default
2

There is at least one peer-reviewed journal, reScience, that explicitly publishes replication of existing work.

ReScience is a peer-reviewed journal that targets computational research and encourages the explicit replication of already published research, promoting new and open-source implementations in order to ensure that the original research is reproducible.

So as such it does require high TOP levels.

Overlay journals such as those in EpiSciences play well with reproducible science, since their model is that you put everything online first, and the "journal label" is applied by a review committee that agrees that the results are correct and worthy of inclusion in the journal

Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .