2

My idea is learning new methods/fields by reproducing someone else results, and I would like to pick nice/good papers to learn from. If we define reproducibility as starting from the same samples/data, use the same methods, get the same results I would like to know papers which has already been reproduced.

Is there any project/web-service/method that tracks reproducibility?

Of course, some people citing an article might have (partially) replicate it, but that wouldn't help me as I don't have access to new samples/data.

I am interested in bioinformatics field.

  • 1
    I am not aware of any such service, and I doubt it would exist. The main reason behind my skepticism is as follows: "why would anyone go through that trouble? how would it pay back?" Considering that failure to replicate is very rarely published in biomedical journals, it would be considerable amount of effort with little or no return of investment, unfortunately. – posdef Jun 1 '17 at 9:31
  • 1
    Are you looking for something like this (accept that it is focused on economics): replication.uni-goettingen.de/wiki/index.php/Main_Page – Maarten Buis Jun 1 '17 at 13:15
  • @MaartenBuis yes, specially the first table 339 that links to articles replicating. Replicating would be perfect but reproducing with be also fine. – llrs Jun 1 '17 at 13:21
  • 1
    Have you seen rescience.github.io ? – Alexander Konovalov Jun 2 '17 at 22:19
2

I would suggest graduate level textbooks for that. They often leave proofs or part of it to the reader and some of them even have exercises, allowing you to learn the new topic by actually working on it. Furthermore, they made sure that what they left open is solvable in an appropriate amount of time. Another nice point is that they often discuss the most important theories and concepts in a field, where on the other hand a paper might be focused on a very tiny topic.

There are textbooks for many different levels (starting at undergraduate and going as far as to being interesting to a professor who wants to extend his knowledge to a new field), depending on how much you already know.

  • I have some of these books but they usually explain the common, standard workflows. They don't keep up with the latest methods published which is my point of doing this. – llrs Jun 1 '17 at 12:28
  • The latest methods aren't used that often (that is a logical consequence of being new), so there is very little published material to choose from. That could also be an advantage, as choosing between the two or three papers out there is not as hard as choosing one out of two or three million potential papers... As soon as a method is used more often, such that it becomes a problem to choose a paper, you can bet there there will be a textbook discussing it. – Maarten Buis Jun 1 '17 at 13:11
  • On the other hand, a really recent method is not always the best one. It might still have some flaws, might not be efficient or in the worst case might even be wrong (as it is knew, no one noticed yet). So if you really want to learn a new topic, it might not always be the best idea to start with the recent work. Once you are an expert on a field, you can of course study new methods - but then again you should be able to reproduce them once you are an expert (otherwise I'm wondering how they passed peer review...). – Dirk Jun 1 '17 at 13:14
  • I am not interested in learning the best method for X, but the newest method for X. (I am not sure about the goal of peer review or the outcomes of it yet, but that would be another question) – llrs Jun 1 '17 at 13:27
  • Well, then you should simply read all papers. You asked for a way to select certain papers in your question - but this will most often give you not the latest method but only some rather recent method. – Dirk Jun 1 '17 at 13:31
0

With publications like doi: 10.1126/science.1213847 and here, cross-disciplinary surveys like here and policy changes like here, your mentioned interest in informatics / computation may lead you to runmycode.org. It is a repository for both data and code. Other inspirations regularly appear in PLOS, for example, both in publication (e.g. doi 10.1371/journal.pbio.1002333), or its dedicated collection.

While eventually directed towards deploying an editor for reproducible research, Evan Misshula's presentation at NYC Emacs user group about the topic (https://www.youtube.com/watch?v=CGnt_PWoM5Y) provides some perspective, too.

0

Check "pre-registration" of studies, or "registered reports". This is a relatively movement (started, I think, by the Psychology community), all about reproducibility. New large reserch projects about the effectiveness of this are in the works. However it's not my field so I cannot give any more information.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.