Incentives for independent researchers to work on reproducible research

Question

For the sake of high-quality science diffusion, there is no doubt that reproducible research is a good practice as it helps to demonstrate the robustness of the proposed methods and boosts the possibilities of getting more citations for a published paper. Needless is to say that it also allows new researchers to continue the work of a previous mentor. For students wondering if they should or should not release their code in a public repository, this was a topic already discussed here: Is it beneficial for a student to publish their research code and data?.

However, my understanding, from this material, is that reproducible research is a practice followed by institutions in which the government takes charge of the salary earned by researchers, their equipment, and travel expenses. Given that public funds are used in this context, there is no point in publishing results in places that would keep the knowledge inaccessible to the people who need it (and pay for it with their taxes). In fact, many journals offer now the possibility of publishing papers as Open Access. Obviously, this exempts the case of national security affairs that need to be kept in secret.

For independent researchers or non-granted students, the things change a little bit, as they don't have to follow the reproducible research practices because they pay for most (if not all) of the expenses during their studies. My question is, what are the incentives (if there is any) for this group of people to release their code on the internet if they are not looking to exploit it commercially? Is reproducible research something that they can use to improve their chances to get a job in academia or an R&D department in the industry?

Note: by reproducible research, I am taking the definition stated here: all the results from a paper can be reproduced with the code and data available online.

Answer 1

I hope I understand the intent here well enough to offer guidance. I don't think there is any inherent incentive for most researchers to publish more than they do. The current system works well enough for them, so the added effort makes little sense.

The reason that many US government agencies publish more than the minimum is that much (most?) government funded research here has, traditionally, been considered as owned by the American public. Public Domain, if you like. On an orthogonal dimension, the fact that a lot of science has become politicized, implies that many want to be as transparent as possible so that the inevitable conflicts and charges can be openly refuted. But an additional reason is that "public servant" researchers are also generally interested in others being able to leverage and extend the work they do - for the greater good. They don't work for a profit motive and so have little incentive to hold things close.

If individuals want to publish code and data, it is, for the most part, permitted, as long as journals don't impose restrictions for reasons of copyright, etc.

However, a beginning researcher should carefully consider what openness implies. In particular, while you retain copyright (which you can license), you complicate the possibility of obtaining patent on your ideas. It may even be that others, with whose policies you disagree, can use and monetize your ideas against your wishes. Openness in general is good, and being willing to "reveal all" is commendable, but be aware of the implications when you do.

Answer 2

My question is, what are the incentives (if there is any) for this group of people to release their code on the internet if they are not looking to exploit it commercially? Is reproducible research something that they can use to improve their chances to get a job in academia or an R&D department in the industry?

Whether independent or not, the incentives for publishing your software code and/or data resources are:

• Some conferences/workshops encourage the submission of papers devoted to new code/resources (i.e. a quite easy publication).
• Assuming your code/resources can be found, gain popularity and are useful to the community, you can get a lot of citations: every time somebody uses them, they will refer any paper you ask them to cite (this must be clearly stated in the license info).
• Mentions of your open-source software / open-access datasets in your CV are seen positively in the academic community (for the reasons that you mention), so it increases your chances to get an academic job. It can also be seen as a showcase of your skills when applying in the industry.

So overall yes, it does strengthen your research profile.

However it can take a lot of time: polishing the code, writing some clear documentation, maintaining the software, etc. So there is a trade-off between devoting time to this or to other research activities. I assume that this question is especially important if you are an independent researcher.

Edit following OP's comment: for independent researchers it can be a good way to keep in touch with the research community, especially if you don't have the time or financial resources to go to conferences.

Disclaimer: my field is Natural Language Processing, which involves Machine Learning with unstructured data. Results often vary depending on the dataset and the exact experimental setup, and reproducing an experiment can take a lot of time. Thus reproducibility is a major issue in the field and open access resources are strongly encouraged.