Academia Stack Exchange is a question and answer site for academics and those enrolled in higher education. It only takes a minute to sign up.
Sign up to join this community
When publishing a paper, some researchers publish the source code used for the paper.
Is there any research/study/survey/... that looked at how much effort do researchers take to publish their source code? I.e. how many hours do researchers take to publish the source code for a given paper?
This isn't a peer-reviewed article, but nonetheless it's worth linking to because it specifically addresses your question, albeit as an n=1 case:
Bruna, E. 2014 THE OPPORTUNITY COST OF MY #OPENSCIENCE WAS 36 HOURS + $690
In this blog post, a biologist Emilio Bruna states it took about 25 hours of his time to appropriately document his code (associated with a research paper) to make it good enough for open source release on github.
This is a study that analyzes whether computer science papers include source code that makes it easy to reproduce their results.
http://reproducibility.cs.arizona.edu/
The study found that out of 601 papers analyzed, 139 included source code that could be obtained without contacting the authors, and the study's researchers were able to email authors to get the source code for an additional 87 papers.
Of the 226 papers the authors obtained source code for, they were able to configure and run the source code within half an hour on 130 papers, without contacting the authors on an additional 64 papers, and after contacting the authors on a further 23 papers. For 9 papers, the study's researchers could not run the source code at all.
These results don't show how much time researchers spend on making their source code available, but it does show how frequently papers are published with accompanying source code and what quality that source code tends to be.
PLL made an excellent comment. I'd like to add it to my answer in case it disappears later:
Just to summarise: the overall success rate should be seen as 217 out of 402. Of the full sample of 608, 206 were excluded for some reason or another --- e.g. their results weren't based on code in the first place. 402 were left that should have contained code.
I have a significant number of papers where we make the (nontrivial) code available through various means. Examples are here:
In all of these cases, documenting the code adequately to make it suitable for publication along with the paper was part of writing the code (like for all significant code, documenting should be part of writing it) and would likely have taken at least 2 days in the case of the tutorials, and maybe 4-8 hours in the case of some of the specific codes and algorithms. It's something one should do anyway, but even if one doesn't, doing it is not usually an overwhelming effort.
Of course, this would not apply to a code like ASPECT for which writing the documentation (such as the 230 page manual) is an effort that likely represents month of work.
I disagree with your premise that it actually takes any effort at all.
Version control is needed anyways for collaboration with your coauthors, as backup and for version history, which GitHub, BitBucket and SourceForge offer for free.
Writing good code is necessary so hat your colleagues can understand what you are doing and even if it is a one-person project you need to understand it half a year later.
There are even additional benefits:
So you only hurt yourself if you don't publish your source code which means there is actually negative effort, all things considered.
