3

Our research project is ready for being shared as a preprint and submitted for peer review. What are the best practices for transforming the private research repository (which includes several pilot experiments and paths not taken) into a codebase that can be shared as supporting material?

In principle, we can create a new repository with a single cleaned-up version. The downside of this would be a split between the actual research repository and the published one, and the omission of the commit history, which provides specific credit to the different students who contributed to different parts of the code. (to make this clear, these students are also listed as authors.)

EDIT: This question originally mentioned GitHub. I edited it to remove the mention of this service. As some of the comments suggested, there are alternative platforms that can be used for the task. This is a general question on how one should transform a private codebase into something that can be shared as an open scientific code.

11
  • 2
    have a look at Zenodo.org , instead of being dependent on commercial endeavours that in theory may disappear tomorrow after a market crash.
    – EarlGrey
    Mar 30, 2022 at 14:21
  • Might be a better question for support at github (or dedicated forums). This issue won't be unique to academia. Mar 30, 2022 at 19:27
  • 1
    I am not convinced that commit history is a useful way to give credit. Mar 31, 2022 at 13:10
  • @Ian I am not underestimating GitHub, if you scratch the fancy layout of that site, they are simply associating with some other institutions to provide storage, institutions that are more stable than a for-profit company. In fact, one of the best way to preserve your GitHub is to do so through zenodo.org , but I do not see it as "look how nice they are the GitHub guys, they allow to preserve your repository somewhere else for free", rather I see it as "people at GitHub know that they may fold down any moment, so they found a way for free to provide safety to their content provider".
    – EarlGrey
    Mar 31, 2022 at 15:55
  • 1
    "Do we worry that our Excel spreadsheets will stop working?" yes, in fact while the average Joe employed in a financial company does "finance" with Excel, COBOL is used for critical applications ( fingfx.thomsonreuters.com/gfx/rngs/USA-BANKS-COBOL/010040KH18J ). A part from being a theoretical risk, there is the exploitation factor: if a for-profit company is providing you something for free, you are the product. Isn't the peer reviewing provided to for-profit editors enough as a gift from academics to companies?
    – EarlGrey
    Mar 31, 2022 at 16:04

2 Answers 2

2

What are the best practices for transforming the private research GitHub repository (which includes several pilot experiments and paths not taken) into a codebase that can be shared as supporting material?

For best practices, first determine where you want to host your code. You have two options, each with tradeoffs. Here are the two I could consider:

  1. Make your GitHub repo public and create an immutable tag with the version.
    • Downsides: You depend upon GitHub being public forever, Some countries block GitHub, GitHub might fall out of favor with researchers.
    • Upsides: GitHub is the lingua franca for open science right now, having your code public would increase it's visibility, your complete workflow is open source.
  2. Put a snapshot of the code somewhere else either as a supplement with the journal or on a long term digital archive such as Zenodo.org (thanks to EarlGrey for suggesting in a comment).
    • Downsides: Need to make sure you archived version matches final version, less visible, slightly more work.
    • Upside: Should be around for a really long time, you get a DOI minted for your code.

Personally, I would combine options 1 and 2. Specifically, if I wanted the code to be public, I would clean up the repo and then make it public. I would create a tag for the final version of code used for the paper and then put an archive on Zenodo.org. I recently did this for a paper I had published in the Journal of Open Source Softwware

Second, make sure your repository is clean. My agency has guidance for scientists who develop code, some of which would be helpful for you. The full page is here. I would also look for domain specific best practices. For example, the article, Ten Simple Rules for Taking Advantage of Git and GitHub, has been published in PLOS Computational Biology.

3

Based on my experience working in a simulation-oriented computational science, I'd recommend simply setting the repository to visibility to public.

The crucial steps in making this work are:

  1. Making sure that documentation accompanies the repository on a per-publication basis
  2. Making sure that the state of the codebase used to produce each publication is captured alongside the analytical code and documentation. This includes package versions.
  3. Adding a license

Organizationally -- particularly for projects with multiple papers -- splitting the repository into separate archival branches has been critical. Paper 1 gets a branch, with a readme explaining the layout of the project and the specifics needed to replicate the results. Links to the paper itself and other supporting documentation also belong there along with the versions of the software packages used. Leaving out the packages can destroy replicability.

Paper 2, …, Paper n get the same treatment.

We keep working branches in active development for further work when relevant.

Judging your success:

If you can take a paper, strip out all the tables and figures, and reconstruct it using only that paper’s github branch then you have a foothold (streamlining this can yield great improvements in your analytical workflow for the next project).

You'll know that you hit actual reproducibility when you can send a link to someone who was not on the project and they can do the same.

2
  • All good points. Aren't you concerned about including the entire development history of the project in the repository? files that are no longer relevant? analyses not described in the papers? Mar 31, 2022 at 13:21
  • 1
    You can cull or organize files in the archival branches if they are a particularly onerous issue, but as long as the documentation is very clear it should be obvious which parts are actually used in replicating the paper. If they are not mentioned in a procedure, they can simply be ignored by the reader. IMO these presentation concerns are secondary to simply having code publicly available and in a usable state. Papers and docs are the parts that give the code clarity. Tidying code is vital, but I would urge you to build that into your workflow rather than as a publication step.
    – Brickman
    Mar 31, 2022 at 14:49

Not the answer you're looking for? Browse other questions tagged .