Are there any proposals to make sharing code and data with publications a requirement?

Question

Depending on the project, many researchers have to rewrite code and reproduce datasets from previous works in order to perform experimental evaluations. Although some dataset are understood to contain private data, and thus cannot be shared, it would be very nice to have access to the source code and results produced along with the papers.

Most of the big data works present results from experiments run with very large databases, and the code developed, which is usually non-trivial to rewrite, is rarely available to the public. I even heard once that some conferences would start to request the source code of the project to be submitted along with the paper.

My question is: is there any global regulation that states what a conference must request while accepting a paper? And, if so, are there any proposals at all to make source code sharing a must?

Answer 1

The academic 'process' is an unregulated mess of random, contradictory habits

To directly answer your question, NO, there aren't any global regulations on what conferences or publishers should require or how they should do anything else.

It's a key point of academic independence - anybody is free to run their academic conferences or publications as they like. There is an unwritten consensus on what constitutes good practice, but it's not regulated, it's not mandatory, it varies across academic fields, and it varies across countries.

Change happens by convincing lots and lots of unrelated people and organizations

Any proposals to change something (say, make source code sharing a must) only become real when lots of separate organizers (most of them who never ever hear about each other) in different fields agree that it's a good idea; that it benefits them without making it too hard for them; and take the initiative to implement it. It helps if some academic subfield implements the practice and it's widely seen as working well.

The only force is funding

Large funding agencies have the only practical power, as if they make funding conditional on X, then people will try to get X - or at least something that on paper looks similar to X. Note that if they don't think that X benefits them, then it will be the latter option; doing the very minimum possible to tick a checkbox "yes we do have X". And it's by definition not a global regulation, but a country-specific one.

Answer 2

I don't know any global regulations, but scientific community understand the problem that you described and that is why github recently made it possible to get a Digital Object Identifier (DOI) for any GitHub repository archive (blog post) making the code citable.

As far as I remember any DOI should be maintained for at least 10 years.

Answer 3

There is a Coursera course of the Data Science Specialization track which talks about this topic. The course is:

• Reproducible Research
• website: https://www.coursera.org/course/repdata
• Institution: Johns Hopkins University
• Instructors: Roger D. Peng, Jeff Leek, Brian Caffo
• Note: the course can be done for free.

Some of the topics of the course are:

• Explanation of what is the replication a research work
• Explanation of what makes a research reproducible (from your question, you are basically asking whether reproducible research is a standard in the scientific world)
• Description, tutorials and exercise on how to use Rmarkdown which is a package of the R language developed to create code that can be both: converted to a human readable format (Sweave the code) and executed to perform a data analysis of some sort (Tangle the code).
• The last lectures are quite interesting, because they talk about real examples that have occurred in the past where reproducible research has been useful, and cases where the lack of reproducibility has been a problem.

My question is: is there any global regulation that states what a conference must request while accepting a paper? And, if so, are there any proposals at all to make source code sharing a must?

I don't think so. My personal hope is that reproducible research will tend to have more citations and that it will be more valued by peers.

Answer 4

This problem has been recognized, but there is only slow progress on the sort of institutional innovation necessary to address it. Many technological components of the solution are in place, but their are socio-cultural forces of resistance in nearly all academic disciplines and academic journals. NSF and other funding agencies are looking for ways to overcome the resistance.

For a thorough analysis and prognosis, you could listen to this talk: THE CREDIBILITY CRISIS IN COMPUTATIONAL SCIENCE: AN INFORMATION ISSUE (includes slides).

EDIT: Here's a recent blog post about this in the field of bio-medicine: Can you show us that again please?

Answer 5

There is no particular "universal" regulation, and attempts to do so, even for things like the PLoS data sharing policy go somewhat pear-shaped. This is because, as @Peteris mentions, academia and research is a rather unregulated bunch. There's no guiding force, and there isn't really the backing for there to be one.

Even things that are firmly enforced, like the protection of human subjects, have standards that vary from place to place.

Personally, I also think that those advancing these policies often forget that different fields have different problems. For some fields, "Make your data open" is committing them to a rather intensive hosting and software support problem with very little money to back it. For others, "make your data open" may be exceeding the informed consent their patients gave.