37

I am in the field of computer science. It is often the case when I am reading a paper I start to wonder, "Wow stunning results, however, I would like to prove that." or "How exactly did he get these amazing results? He/she just wrote a rough overview of the real methodology used in this paper.". As you can see, a lot of times the small things have a huge impact on the overall performance of the underlying methodology. Often they are not part of the paper or not revealed at all.

My idea would be to contact the paper writer, to ask for his research programs to recreate them and understand them. Can/should I do that?

I personally think that there are probably not a lot of researchers, who would let somebody else "look at their cards". What's your experience with that?

Improve this question
7

4 Answers 4

Reset to default
27

Ask for the code. Please do, however, explain why you want it and what you intend to use it for.

Personally, I would be quite happy if someone contacted me about my research, and would try to give all the necessary tools to recreate my data. Especially so if I am not currently working on a follow up piece.

That said, I've asked for parameters, codes, procedures etc. several times from the authors when the description in a paper has been vague. I've had a lot of different responses: Some have plain ignored me, others have given me everything I asked for, and yet others have only obliged when I've suggested that I might be willing to put them as co-authors if I built upon their code in a way that leads to a publication. Finally, some have refused my request. This has happened for a number of reasons, for example those discussed in this question.

If you ask for code and say that you just want to verify some of the conclusions of the authors or use it as a reference for your own implementation, you can most typically expect a refusal or no reply. This, at least, is the experience I've had.

Improve this answer
5
  • 3
    Is it possible that your inference about the motives of those who've refused your request is mistaken? Could it possibly be that they are acting in good faith, and have given you an honest reply? After all, telepathy is a very tricky business to get right (AIUI there isn't a single proven case of it ever working), so perhaps yours isn't infallible.
    – 410 gone
    Jul 20, 2014 at 18:51
  • 1
    @EnergyNumbers I believe that here it is you who is employing telepathy: Of course I am not writing out the specifics of a private matter on a public forum, which you seem to be assuming as you make your rather bold judgements. Some of the cases I am referring to, upon further communication, were rather clearly situations where the codes were not trusted by their authors. Naturally, your first assumption should always be that people act in good faith and mean what they say, which is why I in my answer felt that writing out a possible ulterior motive explicitly would bring more value to my text
    – alarge
    Jul 20, 2014 at 19:13
  • 3
    I'm afraid you are mistaken: rather than bringing more value to your text, its churlishness detracts from it.
    – 410 gone
    Jul 20, 2014 at 20:07
  • 1
    @EnergyNumbers Seeing your point that my text can be understood in a way it was not meant to, I have edited it. In the end, though, I suppose we'll have to agree to disagree: I feel that my answer now is more PC, but contains a tiny bit less information.
    – alarge
    Jul 20, 2014 at 20:50
  • 1
    @amlrg Thx for your encouraging answer! Any suggestion how do you contact the author? Would appreciate it if you could show me one of your mails...
    – Carol.Kar
    Jul 21, 2014 at 6:12
8

Yes, it is fine to ask. From my experience, not many researchers share their code. Sometimes the reason is that the authors are afraid that you will write a better algorithm or that you will extend their work before them. Some other reasons may be that the software has no documentation and they don't have time to prepare some documention, is not written in english, or has dependency to commercial software or software that belong to other authors. However, there is a lot of reasons for sharing the code. It gives more visibility to your work, helps other researchers so that they don't need to reimplement your work, allows for a more fair comparison when comparing algorithms, etc.

Personnally, I have asked the source code or datasets of other authors several times. If they don't want to share, they usually just don't reply to the e-mail, or they may say no. But there is no consequence to asking. So just ask. Besides, sometimes the authors will not provide the source code or datasets but often they can still give you the binary files. In that case, it can still be useful.

Personally, I share the source code and datasets of all my research papers as first author as part of an open-source data mining library ( http://www.philippe-fournier-viger.com/spmf/ )and I believe that all authors should also share source code and datasets. Here is a blog post where I explain in more details why it is important to do so: http://data-mining.philippe-fournier-viger.com/why-it-is-important-to-publish-source-code-and-datasets-for-researchers/

Improve this answer
2
  • what do you mean by sharing the binary files ? could you please clarify these point ?
    – roni
    Apr 24, 2017 at 7:28
  • 2
    Ok I will explain. To write a program, a programmer has to write some code using some programming language. Then, generally, the programmer "compiles" the program to convert it into an executable (a software that you can run on your computer, such as an .exe file on Windows). That executable file is what we also call a "binary file". Thus, if a researcher give you its source code (the code), you can modify his software. But if a researcher give you its executable files (binary files), then you can only run the program but you cannot modify it. Hope it is clear.
    – Phil
    Apr 25, 2017 at 8:26
8

I think it is perfectly fine, and I encourage asking for the code. I actually support the position that any published results based on software are worth next to nothing if the authors are not willing to publish the source code along with them. If they have not already done so when publishing their paper, I find it their moral obligation to do so as soon as possible or at least upon request. If the reason for authors not publishing their source code is that they do not fully trust their own software, then the results should not have been published at all. Why should anyone trust your results if you are not willing to defend the way you produced them?

Improve this answer
4

It's absolutely acceptable to contact an author and ask for their code - and in my experience, a good portion of researchers are more than happy to share their code. While yes, there are some who won't want to let someone get a glimpse into their work, this isn't a universal stance.

A few things to note:

  • You are asking for a favor, please do recognize it as such. Preparing code is not a zero-effort activity. For example, if you're talking to researchers doing human subjects work, there's potentially data that needs to be stripped out. Even if there isn't, it's possible that code for ongoing, as-yet unpublished research needs to get stripped out, comments cleaned up, things like that.
  • When asking for someone's code, do make it clear what you're planning on using it for. Beyond making it more likely for your request to be met favorably, it'll be useful for helping the person providing the code know what it is you need. For example, do you want to build on their stuff? Simply reproduce their results? Something else? There might be things you need to know that are purpose-specific.
  • If you intend to spin things into a paper, you might want to get them involved in that as well.
Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .