2

For example see reproduced papers repository. I am restricting the scope to theory papers that have some results obtained through numerical techniques. The fields may be as diverse as physics, mathematics and the things in between. How can one make this process of reproducing papers more frictionless? Is there a way one could incentivize independent reproduction?

  • 2
    What is the linked repository an example of? – user2768 Nov 28 '19 at 14:28
  • 1
    How can one make...reproducing [results] more frictionless? By writing good papers, especially in terms of describing methodology. – user2768 Nov 28 '19 at 14:29
  • @user2768 the linked repository is an example of papers would be reproduced independently of the authors. – Tejas Shetty Nov 28 '19 at 14:43
  • 1
    What about the differing abilities of the readers? Some won’t have the maths or software skills to create the solutions or debug them... – Solar Mike Nov 28 '19 at 14:46
  • 1
    Thanks for the clarification: Perhaps edit to clarify – user2768 Nov 28 '19 at 15:57
3

Given that creators have some rights to the things they create, you need to provide incentives for transparency to creators/authors. Hopefully they are positive incentives.

But many authors are hesitant to reveal all of their computer code or the actual data that they use in research. There are many reasons for this, some valid, even essential, and some less so.

Some (many) don't reveal all because they fear, reasonably or not, that their ideas will be stolen and that others will capitalize on them before they, themselves, have a chance to follow up.

Some don't reveal their computer code, realizing that it, in final analysis, isn't very good code and is a bit (or a lot) fragile. Or, again, they want to use it in future and don't want to enable others to "scoop" them using the revealed resources.

Another reason, especially for data, is that in many cases the data cannot be revealed for reasons of privacy. This is always an issue in any research with human subjects, even questionnaires on technical topics. It is difficult to anonymize data properly as statistical methods are pretty good at extracting information about individuals from many data sets. So, recognizing the difficulty, it often seems best to just keep it private.

However, I question the value of calling research reproduced if it relies on the same code (or other method) and the same data. If you want to properly reproduce research I suggest that you take the hypotheses of a paper and devise your own experiment independently and run it on independent data. If you get the same results then you have additional information about the value of the hypotheses. The simpler way (same code, same data) guards a bit against some sorts of academic dishonesty, but doesn't otherwise advance the state of knowledge very much.

But if you want to see the code and the data, you have to find ways to overcome the reluctance of the authors to make it public. I think that is easy in some cases and impossible in others, with everything in between pretty common. Moreover, the "easy" cases don't get you very far.


Let me note an important and disturbing current situation in the US. Since the current administration is hostile to science in too many ways, it has declared that scientific studies that are not completely transparent will not be considered in matters related to human health (air and water quality, etc) or climate change.

The effect is to make almost all human studies off limits for consideration, since the law also requires privacy of human subjects and much of the research on which, say, air quality standards depends was done under earlier (confidentially required) rules. Thus, only phony "science" provided by stakeholders (coal burning industries) will now be considered to set air quality regulations. In effect, the Environmental Protection Agency is now misnamed. The administrators are actually hostile to a safe and clean environment.

This is a strong negative incentive for transparency. Reveal all or be punished/ignored for your work. I hope this sort of thing ends soon.

|improve this answer|||||
  • Sorry! @Buffy I slightly changed my question to better reflect what I had in mind. Maybe you would want to revise your answer. – Tejas Shetty Nov 28 '19 at 14:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.