6

Is common practice to release an academic software before the publication of the accompanying paper in a scientific journal?

What are the pros and cons of a prior release?

Edit: I am not concerned about patents, software will have an open source license.

Improve this question

4 Answers 4

Reset to default
7

My experience in computer science has been that papers are rarely about software per se, but rather about the science that is either enabled by or instantiated in the software. Computer science also typically seems to have a fairly open culture where people don't worry much about being scooped. As such, research software is often available on open repositories like GitHub or Bitbucket long before it is published---it's just that nobody notices or pays attention to it before there is a significant paper to motivate them to do so.

So, to finish answering the question:

  • Pros: demonstrate commitment to openness, people might use it before publication
  • Cons: you can't do this with something you want to patent
Improve this answer
5

This may be field-specific, but within neuroscience, the typical path seems to be as follows:

  1. Publish papers detailing new signal processing/statistical analysis/etc techniques
  2. Write software that makes use of these techniques
  3. Continuously add techniques, refine the code, and improve the UI (well, sometimes)
  4. Write a paper on the software itself

The main benefit of this approach is that the algorithms that power the program are vetted via peer review and the general community even before you attempt to write the first line of software code. The only novelty to the software is that it makes the techniques available to a larger audience, specifically, those who can't write their own analysis code. Even better, as you write and publish new algorithms and analysis techniques, you can eventually add those to the software.

Some software that follows this approach is SPM and MNE, among many others.

Improve this answer
4
  • 1
    I can't believe that people do #1 in your list without having written "the first line of software code". In my business (CFD, finite-elements, high-performance computing in general). The biggest open source codes are in continuous release, more or less, and the authors publish papers about the science or new software improvements as they happen. This usually means the papers are delayed 6 months to a year after the code is out there due to publishing cycles.
    – Bill Barth
    Dec 31, 2014 at 15:17
  • 1
    @BillBarth - I guess in my mind there's a distinction between writing the algorithm in MATLAB to test whether it works and re-coding the algorithm in c/java/whatever the language will be used to write the actual software package. I'm referring to the second, not the first.
    – eykanal
    Dec 31, 2014 at 15:18
  • Even in that respect, I don't think your method is common except for people who are just starting out. I don't think most people start over for every new approach. They develop a baseline framework or infrastructure and add new methods to that. That being said, my experience is that software engineering sophistication varies widely across disciplines (not just across science disciplines).
    – Bill Barth
    Dec 31, 2014 at 15:20
  • @BillBarth - Fair enough. FWIW, in neuroscience, people who are just starting out virtually never make software like this; the software is almost all from very well respected labs, developed by people in the field for years.
    – eykanal
    Dec 31, 2014 at 15:27
4

This varies very widely by field, but also by PI preferences; in my experience, younger faculty (and newer software packages) are more likely to be open early. My experience in Chemistry has been similar to what @eykanal describes (especially when patentable methods or results are involved)...

BUT: software often contains bugs, which may change the results you get in ways that are not obvious from a paragraph description of the algorithm. In some cases, this has lead to huge, sweeping errors (like a retraction of papers cited 729 times), and some journals or funding agencies are adopting policies that encourage release of code alongside the first paper that uses it. If it's not already part of your peer review process, releasing the code early might become a requirement in the future. (It is difficult to properly review findings when they depend on subtle, unstated decisions in potentially buggy code)

So, just be sure to separate what you are expected to do from what best accomplishes your goals. Norms involving software are still evolving, and it's a good idea to evaluate your strategy based on time, resources, and expected audience.

Improve this answer
2

Note: In the US, the time window for submitting patents starts when the invention is first disclosed "publicly" (which can include publicly demonstrating it even if you don't discuss how it works) and ends a year from that date. Depending on your plans for this work, you may or may not want to release earlier than you must.

Improve this answer
2
  • Two important cautions: 1) The US system is currently in flux, and 2) this strategy still precludes obtaining patent protection in any other country, even with a valid US patent.
    – jakebeal
    Jan 1, 2015 at 0:37
  • I wasn't so much suggesting a strategy as pointing out that failing to consider this could rob you of options.
    – keshlam
    Jan 1, 2015 at 6:20

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .