Say that you have several papers, each with a set of experiments, in which the source code overlaps appreciably. If you wish to release the code for each paper, which is better:
- isolate the relevant code and release each (possibly redundant) version independently; or
- keep the code bundled with clear instructions of how to reproduce the experiments from each paper (say by indicating which revision number corresponds to the code used in each paper)?
This scenario can arise, say, because each paper includes the same benchmark algorithms or the code was initially developed to be extensible (with, e.g., good memory management). It may be faster to deploy new code by extending the existing code base. One can imagine a public github repository where the software suite is continually expanded.
Some advantages for isolating the code:
- Easier to understand in the context of the single paper
- Clearly reproducible
Some advantages for bundling the code:
- Greater visibility for all the papers
- More likely that software from older papers is up-to-date and easy to run
- Builds towards a comprehensive software suite
- Less redundancy and less overhead (assuming this is the manner in which the code is developed)
Is there clearly a preferable choice among these alternatives? Is there another alternative entirely? Have I enumerated the most important advantages in deciding between these alternatives?