Making your code publicly available: Does it have to undergo review?

Question

In cases when the authors would like to make their code available online after it is published, is it recommended for authors to make their code available for review when they submit technical papers for review and possible publication?

Side Note: The author may not necessarily choose to include his code for review because he is "hiding" something. The author may have questions like: (1) Is it the responsibility of the reviewer to review code? (2) What if one of the reviewers proves treacherous with the code? Even if he doesn't replicate the idea, he could add a few things to it and publish it without approving the initial submission for acceptance. For instance it had happened that a reviewer rejected a paper and now wants to publish an extension of such work. The rights of the author has to be respected to the letter especially for example a PhD contribution. The author must be confident that all these are addressed. Also, it is usually assumed that a good research is reproducible even when the code is not made available. Besides, several papers have earned the "respect" of researchers far and wide - and their source codes were not published.

Answer 1

For computer science conferences recently ACM introduced Artifact Review and Badging which incorporates code review in the usual publication review process. As far as I know currently it is not mandatory to submit your code for review but it is recommended and their future direction is to make code review mandatory.

From the ACM website:

... there remain many circumstances in which such enhanced review will be either infeasible or not possible. As a result, such review processes are encouraged, but remain completely optional for ACM journals and conferences, and when they are made available, it is recommended that participation by authors also be made optional.

If the venue you are interested in publishing at has the option of code review I would highly recommend it. Having your code reviewed by the conference reviewers will give your work more credibility if it passes the review process as your experiments are reproducible. Also the code review will potentially prevent any missed logical errors if any in your code, its always a good idea to have someone revise your work.

From the ACM website:

A variety of recent studies, primarily in the biomedical field, have revealed that an uncomfortably large number of research results found in the literature fail this test, because of sloppy experimental methods, flawed statistical analyses, or in rare cases, fraud.

---

In response to your addendum side note: the best you can do is posting your code to your website or a public code repository such as GitHub which gives you the option of choosing from several copyright licenses you can publicize your code under, the list includes but is not limited to (GNU v3.0, MIT, BSD). These licenses will not prevent someone malicious from stealing your code or work and claiming that it is theirs, but it will give you the bases to claim that someone tried to steal your work. Also a common practice I have seen is adding a summary or a paragraph in each file of your source code saying that the following code is protected by xyz license.

Answer 2

This is in part a field-dependent issue, but if you’re planning to release it anyways, why not make it available for inspection during the review process? There’s no particularly valid argument against releasing the code at that stage, especially if it’s a significant part of your new reported work.

Answer 3

My view is that all code should be made available at submission. There are no arguments against this that wouldn't also apply to the main manuscript as well.

So should you make code available for review at submission time? Yes. Will reviewers review the code you submit? Depends, but often they won't. Will many places require this? No.