Similar to the following question, I have developed a code to produce results in a manuscript that I've written which I wish to now share with the community. But this manuscript, which is currently on arXiv, is still under journal review (which has taken several months and still sits with the referees). Therefore, I was wondering if I should put the code publicly on Github while the manuscript remains under review or only after the manuscript has been fully accepted/published by a journal? Does the right answer depend on the journal? Is it better to wait for any reason(s)? Any and all insights are welcome.

  • How did you make code available to the reviewers? Is this a bespoke analysis of something or is the code the product?
    – user133933
    Mar 22, 2021 at 5:32
  • 1
    @Libor The journal/reviewers have not requested the code directly (at least thus far). The manuscript is essentially applying and benchmarking the code which is a data analysis technique that can be easily adapted to different data sets.
    – Mathews24
    Mar 22, 2021 at 5:48
  • 3
    @Mathews24 Speaking as a reviewer: The fact that the reviewer agreed to go ahead without insisting on the code first does not mean that they were not annoyed, and does not mean that you didn't make their job harder by withholding the code. The importance of the code depends on the specific paper and can range from irrelevant to critical. However, purposefully withholding it only until after the review is counterproductive and might be wasting the reviewer's time.
    – Szabolcs
    Mar 22, 2021 at 13:39
  • 5
    Follow-up question: do you need to anonymize a published code base for a double-blind review?
    – moooeeeep
    Mar 22, 2021 at 15:57
  • @moooeeeep Nope, there is no requirement to anonymize a published code base for a double-blind review
    – Mathews24
    Mar 23, 2021 at 15:12

4 Answers 4


Particular to your situation, I think the exact answer is highly dependent on some missing information:

  • How do you plan to license the code once released?
  • Do you have some sort of agreement with the journal that the code is part of the manuscript that they will own the copyright to?

In General: When should one release the code relative to publication? Before! (Allow me to explain)

My knee-jerk response was "why isn't the code already on GitHub (or another hosting site)"? By publicly posting the code, publicly affiliating yourselves to it, and having a time-stamped "paper" trail of commits. I would expect this provides a good hedge against being "scooped", similar to, but more robust than, the old signed-and-dated lab books for maintaining development records in the case of IP/patent disputes. I work, directly or indirectly (and often in a very minor way) on computational chemistry code, but we write the code and store it in a GitHub repo where it's public facing and has open source licenses. The code either gets cited in papers that use it (not necessarily well - see this question) and/or there's an article written up detailing the software that provide details specific to it (and maybe even later publications of new major versions with added features, e.g. RMG and RMG 3 with actively-developed source here)

  • 1
    I have published few github repos related to papers and I've done both, before and after. I fully agree with this answer: publish your code in github early, particularly more if there is already a preprint or submitted paper (you can always reference an article submited for publication in a repo) Mar 22, 2021 at 15:03
  • 1
    I plan to use an MIT License. There is no such formal copyright agreement with the journal regarding the code that I am aware of.
    – Mathews24
    Mar 23, 2021 at 15:24
  • 2
    MIT open-source license / the journal doesn't have rights to it = I would post it now
    – DoctorSoup
    Mar 23, 2021 at 16:20
  • Furthermore, if the code is not part of the journal submission, if you publish the code prior to submission, then the reviewers have more detailled information to review. That is however strongly field-dependent.
    – Dohn Joe
    Mar 24, 2021 at 15:16

You should post it before the paper is sent for review. Your paper is already on arXiv, and you do plan to make the code public after publication, so I see little reason to be secretive with it just for the duration of the review.

With certain types of papers, the review can be much more productive (and also easier for the reviewer) if the reviewer has access to the code. In some cases, having access to the code is critical for the review.


The concerns to weigh are:

  • Scooping. Your manuscript is already out on Arxiv, so there is no risk that someone will publish your same result earlier than you. By publishing code, you make it easier for other researchers to publish follow-up works. Is this something you want? Generally yes: more science, more reproducibility, more common world knowledge, and, more selfishly, more reputation in the community and more citations for you! But some prefer to keep their cards close to their chest, especially if they are already working on a natural follow-up.
  • Anonymity. Is this a double-blind submission where you wish to hide your identity? I suppose not, in your case, since there is a preprint already out on Arxiv, so you are already non-anonymous.
  • Replication. The referees may want to try your code and replicate your results. In some journals/areas, the code is considered a part of your paper, so if that is egregiously badly written that would reflect badly on your manuscript.
  • License. Some journals forbid you to publish post-prints, because they already incorporate their 'work' (that referees and editors did for free for them). In theory, this could extend to code, so it is better to get it out early, when they can make no claim. I am not sure if it is even legal (and I am not sure if publishers ever sued anyone to enforce that clause or sent out DMCA requests to Github), but if you want the publishers to keep their greedy hands off your code it's better to publish it before submission (and the same applies to preprints).

The standards on all these points vary in different fields, and individual researchers have different opinions, so it is difficult to give a unique catch-all answer. At least in my field, though, sharing code is considered a good thing and I encourage you to do it.


I frequently end up reviewing papers where code is a portion of the results, and in that case if the code isn't provided I'll ask for it while reviewing the other parts of the paper that I can review. If the paper is entirely about a computational method and the code isn't included I turn it right back around to the editor with "Unreviewable without the code". I think this is necessary, because I would say that 90% of my review for these types of papers can be summarized as "Write a better methods section and write tests for your code".

I'm going to add one thing to the other answers (which are all quite good). It's very helpful to me to have a versioned release on GitHub (or PyPi or CRAN or conda or Zenodo, or really whatever release channel you want) that's the exact version that you used for your analysis. I do occasionally have the desire to see how a specific thing was implemented, and if it's buried in 700 commits because you're still working on your software (which is totally fine), it's really hard to figure out where your results came from.

  • 1
    > It's very helpful to me to have a versioned release... I'd call that an understatement: it should be mandatory good practice to be referring to a specific defined version; if the peer reviewers find it really hard to recreate the actual code used then what chance do the rest of us have?
    – Lou Knee
    Mar 23, 2021 at 23:17
  • Why would a paper about a computational methods be unreviewable without code? I rarely see code provided in JCP. Mar 24, 2021 at 8:02

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .