14

Quite too many times I have read in a research article claiming that the source code will be made available, and, when I look for it, it turns out that the source code still hasn't been released.

Are there any journals or conferences that take into account the availability of the source code when selecting the papers to publish? By availability I mean present availability, not some vague promise of code release sometime in the future somewhere on Internet.

Now code availability is one thing, clarity is another. I have seen a lot of emphasis on the papers' clarity in the paper selection criteria, do some publication venues pay attention to code clarity during the paper selection process?

Obviously, I have the same issue with datasets, so I am wondering the same for them, i.e. are there any journals or conferences that take into account the availability of the dataset(s) when selecting the papers to publish?

Improve this question
3
  • (If the question contains too many subquestions, let me know and I'll split it.)
    – Franck Dernoncourt
    Jul 7, 2014 at 20:36
  • 4
    What is your field? And to answer your question, yes, there are journals that to my understanding ask you to submit the source code as supplementary material. These often have "Computational" in their title. A quick Google search returned Computer Physics Communications (published by Elsevier). I am not personally very familiar with the journal but they write that "Manuscripts that describe computer programs and NVAs must be accompanied by: the program source code ...". PLoS One recently, as I recall, made headlines by making it an editorial policy that access to datasets be guaranteed.
    – alarge
    Jul 7, 2014 at 20:49
  • @amirg: I'd put that as an answer if I were you!
    – Flyto
    Jul 7, 2014 at 21:01

2 Answers 2

Reset to default
8

I can confirm the answer given by amlrg in comments. I have recently published a paper in Computer Physics Communications and the referees pay significant attention to the code. There was no requirement, however, to upload the code as Supplementary Material with the paper — the referees were happy to consider our public github repository. It was probably the first time in my practice when referees have actually bother themselves to reproduce the results claimed. We had a number of comments regarding the clarity of the run files in the code (but not in the rest of it), and even a suggestion on design.

Improve this answer
7

There exist at least one journal, Ipol, where it is mandatory to provide the source-code related to the paper. In fact, they raise the bar much higher: the authors must implement the code in a given language, so that the algorithm is available for experimentation on line. You can actually test and try the algorithms of all published papers right now, in a few clicks.

It is a specialized journal in image processing, but it could (should) inspire other fields to do the same.

Improve this answer
1
  • 2
    That's phenomenal... it would be great for computing conferences to have online experimentation like that.
    – Ehtesh Choudhury
    Aug 23, 2014 at 15:49

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .