7

This question is related to Physics research papers, particularly one that will be coming up for my ongoing PhD research (4th paper in all).

A core part of the research is developing, testing and validating relevant self-written code. So, my question is how much of the code should be included in the upcoming paper? Should it be the whole thing, or a written summary of the main components/subroutines used?

  • Did you ask your advisor ? – Suresh Jun 15 '13 at 16:50
  • 1
    Yes, and we're both wondering, as we've seen papers that include some code, some that included all code and some that had details of the main subroutines used. – user7130 Jun 15 '13 at 21:34
  • 3
    I think that if you are discussing the code or using it to illustrate a point, it can go into the paper, but preferably only snippets. If not, it should be in a repository somewhere where people can download it, eg github bitbucket or similar. If you put a lot of code in the paper it adds clutter, making the paper harder to read, and is also harder for people to run the code. I mean, cutting and pasting code from a document is no fun. – Faheem Mitha Jun 18 '13 at 9:28
  • You might also try asking on scicomp.SE. – David Ketcheson Jun 18 '13 at 14:14
  • 2
    Regardless of the content of the paper, it would be great if you uploaded your whole working code to GitHub (with some full examples of usage). – Piotr Migdal Jun 23 '13 at 17:44
12

If the goal of your paper is to show that your code solves a particular problem or performs a specific task, then it is your responsibility to prove to the reviewers that your code does what it claims to do. The easiest way to do this is to include the code as part of the supporting information for your paper.

However, reproducibility—which is what you're asking about, to a certain extent—is a not-so-simple question when it comes to showing codes do what they're supposed to. For instance, a code that runs one way under one configuration may return a slightly (or perhaps _very) different result when run under another configuration. This doesn't mean that one result or the other is wrong—it just means that this behavior needs to be taken into account when evaluating the correctness of software.

One way to help this is to provide as much information as possible on your testing environment, so that any differences between the system on wich you were working and the conditions the reviewers and future potential users of your code will have can quickly be identified. This would be included, perhaps, as a text file that accompanies the code in the supporting information.

  • 1
    very good points, particularly on the testing environments - which would not only be the 'external' environment (field testing etc), but also the 'internal' environment - each subroutine of the program itself. – user7130 Jun 16 '13 at 6:33
  • As long as there is input and output should be easy to debug the code. Of course, THERE ARE EXCEPTIONS... – NPcompleteUser Jun 16 '13 at 14:01
  • @NPcompleteUser: The issue here is that there might not be a bug in the code, but simply how the code was compiled and executed could make a substantial difference. In my field, for instance, you will likely get very different outputs if you run on different numbers of processors—although the long-term averages you get should still be consistent. So you can't simply say "my run does not agree with yours, therefore the code doesn't work!" – aeismail Jun 16 '13 at 14:53
8

I would second @NPcompleteUser's answer about having the code and some sort of walkthrough for reproducing your results online. However your question about how much to include in the paper is very journal specific. There are a number of publications specifically for computational science with formats where large chunks of code are expected, but if it's a journal whose focus is more on the science than the techniques, I would just describe in words the tricky parts of your algorithm and give a footnote reference to the website where the code lives.

7

I would put the current version [of the code] on gist. Providing inputs and gnuplot (matplotlib/whatever) scripts for graph reproduction.

  • Would that also work if the inputs are photos/pixel data? – user7130 Jun 15 '13 at 14:26
  • 1
    as long as it is tar-gzipped - should be fine. Usually I also describe each file in the archive, so that GH stuff doesn't have to guess if I break any copyrights. – NPcompleteUser Jun 15 '13 at 14:45
1

Papers on physics which utilise code should primarily describe the algorithms and their relevance to the physical problem at hand, and not focus on nuts and bolts of the code. There are several reasons for this: the printed article is the wrong place to include code, as page space is generally limited and not suited to formatting of code; there are issues with copyright as in most cases the ownership of the code is transferred to the journal, or if open access then put in the public domain; generally the audience of a paper should be broad enough to be useful to a wider range of researchers than your specialist area.

As others have suggested, you should release your code on GitHub or similar and apply a suitable licence, and simply refer to it in the paper. There are some exceptions to including code in a paper, generally when understanding the algorithm is essential for the reader or complicated, in which case pseudo code is acceptable since the essential information is included without any language specific distractions.

1

Another option, in the field of astrophysics, is to submit your code to a repository, and register it in the Astrophysics Source Code Library (ASCL) at https://ascl.net, which is indexed by the SAO/NASA's Astrophysics Database System (ADS), the official compilation of literature (and now source code and datasets) in astrophysics.

If that's your case, you look at this link to find out how to submit your code to it: https://ascl.net/submissions

The page has additional resources telling you how to cite said codes, and this is what a citation might look like:

Garrido, J. et al., 2013, AstroTaverna, Astrophysics Source Code Library, record ascl:1307.007

With the record being held at https://ascl.net/1307.007, and the ADS entry at http://adsabs.harvard.edu/abs/2013ascl.soft07007G

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy