16

The majority of papers don't share their code. However, I sometimes want to reproduce the results, and therefore re-code the experiments to try to reproduce the papers.

Where shall I share the code, findings, comments, and potentially some data sets, put aside from GitHub? Ideally I am looking for a common place where people share the work they have done when reproducing papers.

I am mostly interested in computer science papers.

  • 3
    Your home page would be an option. – Roboticist Oct 30 '15 at 17:28
  • @Matinking True. Is there any more centralized option? – Franck Dernoncourt Oct 30 '15 at 17:29
  • 1
    Seems like arxiv.org would be a good fit, assuming that you write up your findings into something resembling a paper. – Nate Eldredge Oct 30 '15 at 17:33
  • 2
    @FranckDernoncourt: Actually, if you are affiliated with a research group, you would better to upload the code there, instead of your own webpage. My own experience had proved that utilization of such private repositories are better than the public ones, for such kind of aims; Because the minor and major edits regarding the code would be, considerably, easier. – Roboticist Oct 30 '15 at 17:34
  • 3
    Look also at the ReScience Journal: rescience.github.io – Alexander Konovalov Oct 30 '15 at 19:11
7

In the field of Computer Networks, there is the Reproducing Networking Research blog, which allows anybody to contribute replications of networking experiments, or original research in a form that allows it to be replicated:

This blog is a collection of network research stories, each of which includes full instructions to replicate experiments. Our goal is to kickstart a discussion of repeatable research in the network systems community, by showing that “runnable papers” are indeed possible, today. If every result in every figure of a paper could be replicated easily (by anyone with a local or EC2 VM), it would be much easier to build on prior code, results, and scripts, understand the concepts behind them, and most importantly: put them to practice in the real world.

If there is no such equivalent in your field, consider starting one. You can do what Nick McKeown did with the networking blog - seed the blog by teaching a class in which students are required to reproduce a published result as an assignment, and invite them to contribute it to the blog.

4

Sometimes you may even be able to submit a replication to the original journal or conference. For instance, the International Journal of Forecasting explicitly encourages short correspondences on replications.

If you are already writing up your findings anyway, it wouldn't hurt to submit them just to see what happens. The journal itself certainly does have a vested interest in replications of their articles. Of course, replications could be judged "too boring" and be rejected. However, the interest in replications seems to have been growing recently.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.