18

What is policy are you following about publishing data analysis code on GitHub? Do you do it after publishing or as a work-in-progress?

I developed a number of Python algorithms to analyse a large dataset, and I would like to make my work visible.

  • What's required by: your funding body? your department / university? your PI (if that's not you)? your publisher? – EnergyNumbers Apr 15 '14 at 12:23
  • 4
    Definitely share it. I don't know of any policies. See also: academia.stackexchange.com/q/16785/386 – Memming Apr 15 '14 at 15:04
  • 2
    Version control is about work in progress, by definition. If you are worried about people downloading incomplete or buggy code, make releases. – Faheem Mitha Apr 15 '14 at 15:53
  • 7
    Git is a computer program for managing code. When you say "git", do you perhaps mean "GitHub"? (GitHub is one of many many websites used for sharing code.) – TRiG Apr 28 '14 at 20:59
  • 1
    @albus_c You might consider putting it in a private repository until you're ready to publish. – Jonathan Landrum May 23 '14 at 16:21
19

There is a movement gathering strength lately to encourage publishing the code:

Nature-Publish your computer code: it is good enough

Or, more vehement: If there is no code, there is no paper

The reasons outlied on the article are very reasonable. If you are expected to publish detailed derivations, experimental methods, and proofs of theorems, why would you be allowed to keep the code? No one will accept a theorem if you claim: "the proof is too messy to show, but hey, here are three cases where it works".

I think the best way is to publish the code used as supplementary material, and include a link to the repository, so people can get the improved versions. If you are concerned about people using too bleeding edge versions, make releases, but leave the development public. This will also help you get bugfixes and contributions.

Thank you for wanting to release your code. I really believe this attitude will help make research better.

Edit:

After some time, I have something to add. Most of the code in an application is there for "administrative purposes": load and write data, massage, check conditions... For publishing, that part can be as hackish as one needs it to be. The real "research" is usually in a small part. That is where one should dedicate one or two hours of adding a few comments and clearing the code.

For the rest, a docstring in the functions or a paragraph explaining the aim, should be fine.

  • 1
    To touch on "when", I generally do it when I submit the paper - your odds of being "scooped" by someone once you've hit the submission process is pretty low. – Fomite Apr 29 '14 at 6:18
  • 4
    No question code can be night or day, I've read countless papers where code would have been extremely valuable. Even if the academic used less common language, it can be of immense value. – J. M. Becker Jun 14 '14 at 5:39
  • > the proof is too messy to show, but hey, here are three cases where it works. Actually, this happens regularly in mathematics. If the examples are representative of the proof, then it can be considered enough to give a sketch and not fill in the messy details. – JeremyKun May 5 '15 at 16:19
2

Styles and technologies come and go. Git and Github are the flavor of the moment. Tomorrow it will be something else.

What is more fundamental is that scientific results are normally expected to be reproducible. If the code has secret details in it that are crucial for producing the result, then the result is not reproducible. If the code is simply the embodiment of the methods described in the paper, then there is no problem with not publishing the code.

As an example, there is a device called the Bodybugg that people buy and strap onto their arms in order to measure (or attempt to measure) how many calories they're burning each day. There is publicly available information on what sensors the device has built into it, but the algorithms used for putting the sensor readings together to get an estimate of energy consumption are proprietary. That means that any scientific research that uses a Bodybugg is basically worthless.

On the other hand, there can be perfectly legitimate reasons for not wanting to release one's code. For example, there could be a concern that people who lack the relevant expertise will play with it and use it to publish their own half-baked "gee-whiz" papers that turn out to be wrong. That then harms the reputation of the original author. The author may also not want to be in the business of answering questions from lots of people using their code, or they may want the freedom to make major changes that would upset users who were counting on the code to remain stable and backward-compatible. Science is not software development. Scientists want to focus on doing science, not on software distribution, licensing, making regular releases, and supporting a user community.

It may not even be legally possible to release a working version of the code. E.g., it may have some old FORTRAN routines in it that calculate Clebsch–Gordan coefficients, and if the author of those routines is dead, then it may not be legally possible to publish them.

I'm also skeptical about the long-term value of releasing code in most cases. Github will be gone in five or ten years, and the vast majority of the software it hosts will then cease to exist, since the vast majority of coders will not bother to migrate their code.

  • 4
    The Nature letter in my answer addresses your concerns about questions. The software is provided "as is", no necessity to support it. And there is no shame in other people doing bad research with your code, as there is no shame on Schrödinger every time someone claims homeopathy works because of quantum stuff. – Davidmh Apr 28 '14 at 21:19
  • 6
    Github disappearing in ten years means there will be no code available then; not publishing means there will be no code ever. This is why the code should be published as complementary material, then the journal will be responsible for keeping it for the record. And if I am interested in some code on Github, a git clone and I have on my hard drive exactly what Github has: I no longer depend on them, they can disappear. Keeping mirrors in git (or any other DVC) is trivial. – Davidmh Apr 28 '14 at 21:22
  • 6
    "Github can disapear" is no argument at all. Chemical or material suppliers can disappear too, but it is still unacceptable to not describe in detail how you obtained your materials. – Superbest May 22 '14 at 2:09
  • 2
    Sf.net (sourceforge), which was arguably never as popular as github, has been around for 15 years. Though it has certainly lost some "market share" to github and others, it doesn't seem to be showing any signs of disappearing. – bbarker May 22 '14 at 3:53
  • 3
    "Scientists want to focus on doing science, not on software distribution, licensing, making regular releases, and supporting a user community." This is not a legitimate argument. Code on Github doesn't have a support contract, it just makes it easier for people to figure out what you did. – Shep Jan 28 '15 at 23:54

Not the answer you're looking for? Browse other questions tagged or ask your own question.