I am working on my research paper. I read a paper that is related to my works. But I cannot reproduce the results. The information provided in the paper is not sufficient for me. How can I ask him to share his simulation works with me. Is there any specific email format for that?
12
-
1You are unlikely to get the author's code. Most of the time, I have students asking me for my simulation code so that they may use it to complete their programming assignment. Hence, I never give my code. I suggest that you ask the authors for their simulation parameters, and how they achieve step-X. You also can double check your understanding with the authors, but never ask for the source code.– Prof. Santa ClausCommented Aug 28, 2019 at 9:04
-
1@user2768 you can ask, however, politely. However, it seems like an attempt at not doing the work or poor background. In both cases, I don't entertain such requests.– Prof. Santa ClausCommented Aug 28, 2019 at 10:32
-
4@Prof.SantaClaus it seems like an attempt at not doing the work, I don't see it that way. If I can't run your code, then why should I blindly trust any of your claims?– user2768Commented Aug 28, 2019 at 12:14
-
4Agree with @user2768, seems very uncharitable to automatically assume malicious intent from a request for resources to reproduce your result. If the paper depends on the source code and the author is too obstinate to release the code, then the paper ought not be published.– Adam WilliamsCommented Aug 28, 2019 at 13:31
-
4If the paper depends on the source code and the author is too obstinate to release the code, then the paper ought not be published, I agree: I believe peer-review should require code to be made public along with the manuscript, on the publisher's website, for instance, possibly with a pointer to any current version that the author intends to maintain.– user2768Commented Aug 28, 2019 at 14:19
|
Show 7 more comments
3 Answers
As a general rule, you can ask anyone for anything. But they may not agree to give it, depending on quite a few factors.
I think the only rule is to ask politely and not make claims that you dispute their results. But asking for the code to replicate the results, and explore them more deeply, is appropriate. But there isn't any specific format.
If the other authors are honest then it is a bit more likely they will agree, but it isn't a given. There may be some aspects of their code that they want to keep private to avoid getting scooped.
Simply send the author(s) a non-confrontational email explaining why you would like access to their code, raw data, or what have you. The email should explain who you are, what specifically you are working on, and why you are interested in reproducing or building upon on their results. Most researchers would be happy to have someone interested in their work and willing to provide external validation.
I should also note that, in my opinion, you contacting the authors and them providing the requested materials creates a mutual (unspoken) agreement: if you find some issue, you will notify the author(s) before calling attention to the issue in a public way. This allows the authors the chance to issue a correction or retraction, which they are owed given their honest participation in the scientific process.
Also, this is a great way to make new contacts in your research area. I have met many friends and collaborators through email discussions started in this manner.
By order of what I consider "best practice", here are a few possible cases.
- In the article, the authors point to a specific version of an open-source software or to an archive of their own code. They also provide all the parameters and the complete simulation procedure, ideally scripted for reproducibility. This is apparently not the case here.
- In the article, the authors use a well documented method, with publicly available data and standard protocols. While they might not reply to you, other experts in the field should be able to reproduce their results with some effort.
- The authors mention vaguely their methods or do not provide the necessary parameters. In this case, you are out of luck as only guess work can get you there. It happens often but you should, in principle, be able to work it out at a possibly large investment of time (and possibly of CPU time, etc).
There are several reasons for authors to keep their code private: publishing code takes time, they might think that their code is not "beautiful" enough, the code might provide them with a competitive advantage, etc.
Whatever the reasons, it is legitimate to request the code, data files and parameter files but the authors have no obligation to provide the files or even to reply to you. Sometimes, to motivate the authors it helps to provide a bit of context on your work.
Now, let us assume that the authors reply and attach a source file to their email. This is fine for study purposes but be aware that unless a proper license is provided with the program you cannot do much with the code.
The absence of code is frequent enough that there is a journal dedicated to replication studies in computational fields, ReScience C. Disclaimer: I am part of their editorial team.
