6

There are at least four benefits of releasing the code and data produced during a thesis or research project (citing textually this laboratory):

  • Allows to reproduce figures in the revisions of a paper
  • Other people who want to do research in the field can start from the current state of the art, instead of spending months trying to figure out what was exactly done in a certain paper
  • Makes easier to compare the method to existing ones
  • Increases the impact of the research

This is really cool for someone trying to develop its own study as he/she can have access to world-quality material for free. However, I have observed that this is a practice followed by well-known universities, with federal financial support, and that usually host international students. Is this practice also recommended for universities with little or no federal financial support and no grants for students?

I am curious if a student from a university like the latter would improve his/her chances to work in academia by releasing his/her material in this reproducible research modality.

|improve this question
  • 1
    Yes, you should release all the data and code. And university doesn't necessarely correleate with quality of the work done. – DSVA Nov 20 '18 at 5:33
4

An article about a computational result is advertising, not scholarship. The actual scholarship is the full software environment, code and data, that produced the result. (Buckheit & Donoho, 1995)

Yes, you should publish your code and data. Reproducibility is part of the definition of science: if the results of your experiments or computations cannot be replicated by different people in a different location, then you're not doing science. Far from being a mere philosophic concern, reproducible research has been a key issue in prominent controversies like climategate and cancer research clinical trials.

The person most likely to benefit from your efforts to clean up and publish your code and data is your future self. Why?

Error is ubiquitous in scientific computing...I find that researchers quite generally forget what they have done and misrepresent their computations. (Donoho)

The first step toward working reproducibly is simply to put the code and data that is used in your published research out in the open.

You may adopt reproducible research practices for philosophical reasons, but you will soon find that they bring more direct benefits. Because you write code and prepare data with the expectation that it will be seen by others, you'll find it much easier for yourself and your colleagues to build on past work. New collaborations may form when others discover your work through openly released code and data. And the code itself may be the main subject of publications in journals that have come to recognize the importance of scientific software.

More resources:

|improve this answer
  • Thanks David. Could you please include in your answer some possibilities of PI protection? Patents can be expensive and there is a risk that they would never be used. – JFonseca Nov 20 '18 at 19:40
  • @JFonseca I don't know what "PI protection" refers to. I'm also not familiar with patent law in any country; if you're planning to patent your work then you should probably get advice from someone who knows your country's patent laws. – David Ketcheson Nov 21 '18 at 6:09
  • Two selfish reasons to publish that relate to this answer: 1. It may increase how well known you and your code are; 2. If others use it, they may find bugs. That will initially be embarassing, but is better than continuing to use code and getting the wrong answers from it. – Flyto Nov 21 '18 at 8:03
  • Well, my country follows the international standards defined by the WIPO, so patent's law should be similar to the ones stablished in the US. Now, I was curious about using reproducible research as a way to boost your chances to get funded, which seems something improbable unless you are already affiliated to an institution, like you seem to be. Afterall, there is no issues if government pays. – JFonseca Nov 21 '18 at 19:25
  • Well, my country follows the international standards defined by the WIPO, so patent's law should be similar to the ones stablished in the US. Now, I was curious about using reproducible research as a way to boost your chances to get funded, which seems something improbable if you are not affiliated to a public institution. I was hoping to see some evidence of the private sector or personally financed researchers releasing their code. – JFonseca Nov 21 '18 at 19:25
4

The core of scientific research is that we are open on how we came to a certain conclusion. That way we can move beyond "I am wearing a white lab-coat, so you must believe me", to "this is how I got my results, you can replicate them and see if they are robust, or you can try different approaches and we can discuss and learn how that affected the outcome". So making sure others can understand in detail how you got to your results is central to doing good research. Releasing code is an important step in that, regardless of the university you belong to.

So don't think of this as a service to others (it is nice if that also happens, but that is not the goal). It is about doing good research by documenting how you got to your results, and making that documentation available.

|improve this answer
  • Thanks, @MarteenBuis, is there any standard way to determine by beforehand if a journal is fine with you if you decide to release your code on the Internet? I am asking because it may exist some conflict of interest if your code is also used to generate the figures included in the paper. – JFonseca Nov 23 '18 at 14:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.