Advantages/disadvantages of bundling source code from separate, related papers?

Question

Say that you have several papers, each with a set of experiments, in which the source code overlaps appreciably. If you wish to release the code for each paper, which is better:

1. isolate the relevant code and release each (possibly redundant) version independently; or
2. keep the code bundled with clear instructions of how to reproduce the experiments from each paper (say by indicating which revision number corresponds to the code used in each paper)?

This scenario can arise, say, because each paper includes the same benchmark algorithms or the code was initially developed to be extensible (with, e.g., good memory management). It may be faster to deploy new code by extending the existing code base. One can imagine a public github repository where the software suite is continually expanded.

Some advantages for isolating the code:

• Easier to understand in the context of the single paper
• Clearly reproducible

Some advantages for bundling the code:

• Greater visibility for all the papers
• More likely that software from older papers is up-to-date and easy to run
• Builds towards a comprehensive software suite
• Less redundancy and less overhead (assuming this is the manner in which the code is developed)

Is there clearly a preferable choice among these alternatives? Is there another alternative entirely? Have I enumerated the most important advantages in deciding between these alternatives?

Answer 1

I face a similar situation. I'll tell you what I'm planning to do, although I don't claim this is the best approach. I think it's a reasonable compromise between making things easy for the reader, and making things easy for me.

First, I wrote several libraries that I keep on GitHub. One or more of these is needed to reproduce the research I'll discuss in each paper. I'll create a release for each library, with a citable DOI using Zenodo. A set of instructions is available.

Likewise, I'll have a configuration file, plus some scripts, and maybe some data, that are unique to each experiment. This is where I'll put the README file describing how to build/run everything. I'll create a GitHub repo for those as well, and create a release using Zenodo.

So each paper I write will cite all the library/other releases needed to reproduce the software.

EDIT: One advantage of this is that while the DOI will point to the exact version of everything I used for my published results, it will also be clear where people can go for the latest version of everything.