33

I am in the field of computer science. It is often the case when I am reading a paper I start to wonder, "Wow stunning results, however, I would like to prove that." or "How exactly did he get these amazing results? He/she just wrote a rough overview of the real methodology used in this paper.". As you can see, a lot of times the small things have a huge impact on the overall performance of the underlying methodology. Often they are not part of the paper or not revealed at all.

My idea would be to contact the paper writer, to ask for his research programs to recreate them and understand them. Can/should I do that?

I personally think that there are probably not a lot of researchers, who would let somebody else "look at their cards". What's your experience with that?

22

Ask for the code. Please do, however, explain why you want it and what you intend to use it for.

Personally, I would be quite happy if someone contacted me about my research, and would try to give all the necessary tools to recreate my data. Especially so if I am not currently working on a follow up piece.

That said, I've asked for parameters, codes, procedures etc. several times from the authors when the description in a paper has been vague. I've had a lot of different responses: Some have plain ignored me, others have given me everything I asked for, and yet others have only obliged when I've suggested that I might be willing to put them as co-authors if I built upon their code in a way that leads to a publication. Finally, some have refused my request. This has happened for a number of reasons, for example those discussed in this question.

If you ask for code and say that you just want to verify some of the conclusions of the authors or use it as a reference for your own implementation, you can most typically expect a refusal or no reply. This, at least, is the experience I've had.

  • 3
    Is it possible that your inference about the motives of those who've refused your request is mistaken? Could it possibly be that they are acting in good faith, and have given you an honest reply? After all, telepathy is a very tricky business to get right (AIUI there isn't a single proven case of it ever working), so perhaps yours isn't infallible. – EnergyNumbers Jul 20 '14 at 18:51
  • 1
    @EnergyNumbers I believe that here it is you who is employing telepathy: Of course I am not writing out the specifics of a private matter on a public forum, which you seem to be assuming as you make your rather bold judgements. Some of the cases I am referring to, upon further communication, were rather clearly situations where the codes were not trusted by their authors. Naturally, your first assumption should always be that people act in good faith and mean what they say, which is why I in my answer felt that writing out a possible ulterior motive explicitly would bring more value to my text – alarge Jul 20 '14 at 19:13
  • 3
    I'm afraid you are mistaken: rather than bringing more value to your text, its churlishness detracts from it. – EnergyNumbers Jul 20 '14 at 20:07
  • 1
    @EnergyNumbers Seeing your point that my text can be understood in a way it was not meant to, I have edited it. In the end, though, I suppose we'll have to agree to disagree: I feel that my answer now is more PC, but contains a tiny bit less information. – alarge Jul 20 '14 at 20:50
  • 1
    @amlrg Thx for your encouraging answer! Any suggestion how do you contact the author? Would appreciate it if you could show me one of your mails... – Kare Jul 21 '14 at 6:12
6

Yes, it is fine to ask. From my experience, not many researchers share their code. Sometimes the reason is that the authors are afraid that you will write a better algorithm or that you will extend their work before them. Some other reasons may be that the software has no documentation and they don't have time to prepare some documention, is not written in english, or has dependency to commercial software or software that belong to other authors. However, there is a lot of reasons for sharing the code. It gives more visibility to your work, helps other researchers so that they don't need to reimplement your work, allows for a more fair comparison when comparing algorithms, etc.

Personnally, I have asked the source code or datasets of other authors several times. If they don't want to share, they usually just don't reply to the e-mail, or they may say no. But there is no consequence to asking. So just ask. Besides, sometimes the authors will not provide the source code or datasets but often they can still give you the binary files. In that case, it can still be useful.

Personally, I share the source code and datasets of all my research papers as first author as part of an open-source data mining library ( http://www.philippe-fournier-viger.com/spmf/ )and I believe that all authors should also share source code and datasets. Here is a blog post where I explain in more details why it is important to do so: http://data-mining.philippe-fournier-viger.com/why-it-is-important-to-publish-source-code-and-datasets-for-researchers/

  • what do you mean by sharing the binary files ? could you please clarify these point ? – roni Apr 24 '17 at 7:28
  • 2
    Ok I will explain. To write a program, a programmer has to write some code using some programming language. Then, generally, the programmer "compiles" the program to convert it into an executable (a software that you can run on your computer, such as an .exe file on Windows). That executable file is what we also call a "binary file". Thus, if a researcher give you its source code (the code), you can modify his software. But if a researcher give you its executable files (binary files), then you can only run the program but you cannot modify it. Hope it is clear. – Phil Apr 25 '17 at 8:26
5

I think it is perfectly fine, and I encourage asking for the code. I actually support the position that any published results based on software are worth next to nothing if the authors are not willing to publish the source code along with them. If they have not already done so when publishing their paper, I find it their moral obligation to do so as soon as possible or at least upon request. If the reason for authors not publishing their source code is that they do not fully trust their own software, then the results should not have been published at all. Why should anyone trust your results if you are not willing to defend the way you produced them?

4

It's absolutely acceptable to contact an author and ask for their code - and in my experience, a good portion of researchers are more than happy to share their code. While yes, there are some who won't want to let someone get a glimpse into their work, this isn't a universal stance.

A few things to note:

  • You are asking for a favor, please do recognize it as such. Preparing code is not a zero-effort activity. For example, if you're talking to researchers doing human subjects work, there's potentially data that needs to be stripped out. Even if there isn't, it's possible that code for ongoing, as-yet unpublished research needs to get stripped out, comments cleaned up, things like that.
  • When asking for someone's code, do make it clear what you're planning on using it for. Beyond making it more likely for your request to be met favorably, it'll be useful for helping the person providing the code know what it is you need. For example, do you want to build on their stuff? Simply reproduce their results? Something else? There might be things you need to know that are purpose-specific.
  • If you intend to spin things into a paper, you might want to get them involved in that as well.

protected by Alexandros Jul 20 at 10:40

Thank you for your interest in this question. Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged or ask your own question.