35

My colleague and I have developed a software tool and intend to release it open-source.

This tool is specifically for tasks in my field but we think it would be helpful for the wider community. Our institution will permit us to release it provided we get appropriate credit.

Thus, we wish to publish it in peer-review. Is peer-review publication of domain-specific software available? If so, what is required to publish it?

In this case we intend to publish the method and tool on it's own merits without supporting data or an application.

8

5 Answers 5

45

Yes, Open Source software can be published. What's required varies depending on the venue.

There are general journals that focus on the software process. The idea is to encourage better software development gets the credit it deserves. Examples of journals with this approach are the Journal of Open Research Software and the Journal of Open Source Software.

Then there are domain specific journals that have specific software paper policies. An example would be the AAS Journals which state

AAS Journals welcome articles which describe the design and function of software of relevance to research in astronomy and astrophysics. Such articles should contain a description of the software, its novel features and its intended use. Such articles need not include research results produced using the software, although including examples of applications can be helpful.

(Emphasis mine to link to a point in the original question)

1
  • 1
    Nice answer. Feel free to copy over my answer's list of journals as well if you'd like to expand your answer. Commented Feb 6, 2020 at 19:52
29

Yes. Software can be published as an open source tool with a peer review process. Several tradition-styled academic journals exist. Given the OP's profile, here are some journals that publish R packages, genetics tools, or environmental software:

Also, some government agencies have formal peer review processes for software and code for public release (e.g., the US Geological Survey, the US Department of Energy).

2
  • 1
    I came to this post specifically to mention Journal of Statistical Software. Commented Feb 7, 2020 at 21:01
  • 1
    @MichaelLugo I’ve heard good things about it but I’ve been desk rejected for being out of scope of JSS before for other work. I can’t discuss details for unpublished work but this tool is not statistical in nature so wouldn’t fit there. Commented Feb 8, 2020 at 11:19
8

Sure it's publishable. You write up a paper detailing what is in the code, how to use it, examples, potential problems, and so on.

Here's an example, and here's the Github link to the source code.

2

SoftwareX:

SoftwareX aims to acknowledge the impact of software on today's research practice, and on new scientific discoveries in almost all research domains. SoftwareX also aims to stress the importance of the software developers who are, in part, responsible for this impact.

To this end, SoftwareX aims to support publication of research software in such a way that:

  • The software is given a stamp of scientific relevance, and provided with a peer-reviewed recognition of scientific impact;
    • The software developers are given the credits they deserve;
    • The software is citable, allowing traditional metrics of scientific excellence to apply;
    • The academic career paths of software developers are supported rather than hindered; The software is publicly available for inspection, validation, and re-use.
0

The answer to your question in the narrow sense is "yes". However, I'd like to answer what I perceive as a gist of your question. Because the short answer to that is yes, but.

Software or method?

Basically, the situation as I infer it, is: You have some problem. Typically it emerges from some applications, such as biomedicine, material sciences, chemistry, etc. Solving the problem might involve some lab work, but it does not suffice. To actually solve the problem, you need some software. Now, you wrote the software and ask how is it publishable.

The major point is that classical computer science and close fields (mathematical software, for example) has been for very long focused on methods. It's not about why, it's not about how technically, it's about the theoretical way to solve the problem. Now, this does not mean that there is no implementation, backing up the theory. In an overwhelming majority of cases there is one. But publishing the code, especially as a separate entity is a relatively new (but welcome!) development.

There were times, when reproducibility in computer science meant: take a BSc student, give them a paper, let them implement it for months, now you have an implementation that you can compare with your own approach.

The languages die, but ideas don't.

I can name two understandable reasons for such a strange (for outsiders) mindset of a computer scientist. Firstly, for a long time the actual idea what to do, the thing we can concisely formulate as an algorithm or describe in a paper, was much shorter than the actual low-level code implementing the idea. There is a lot of bookkeeping, technical overhead, and maybe even some ingenuous tricks – interesting on their own merit, but not contributing to the general high-level idea. Computer science was and in part still is focused on such bird-view ideas, though ingenious hacks are also publishable nowadays.

The second reason is that the practical details of the implementation age ungracefully. This includes some technical solutions and also the programming language the implementation is written in. Stretching a bit, it's both easier and more eternal to describe a way to compute a singular value decomposition in terms of linear algebra calculations than an ancient Fortran implementation of DBDSQR.

The trend described above is changing. I see more and more papers that refer to GitHub repos with the accompanying code. This is good. It serves the reproducibility. Less poor BSc students having to implement other's papers. But what people still publish in CS are more high-level descriptions, theoretical considerations, and results of practical evaluation. But not the code as is.


Notice, that your fellow biologists, geologists, chemists, and so on might very well appreciate the working product. "Clone this github repo and plug in your data" works as a charm.

Still, if there is a high degree of scientific novelty in your software and if you want to publish it in a computer science venue in a broad sense (there are some journals for publishing code, as other answers state), you might be much better off, if you publish the method and accompany the method description with a link to GitHub, where the actual software is deposited.

Oh, and there is the third component: the data. Again, there are some journals where you can publish scientific datasets. But the general development is to put the data into a repository (such as Dryad or Zenodo, it's separate question, really) and link it in the paper.

1
  • In this case it’s a narrow but useful tool to reformat data for certain applications. It’s not a novel method by any stretch of the imagination. Hence my hesitation as of course novel methods and data can be published. I’m not a computer scientist and I’d expect it be a lot of work to satisfy reviewers of a CS journal that have very different requirements to my field (nor would readers in my field be familiar with the journal). Commented Feb 8, 2020 at 11:23

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .