Take the 2-minute tour ×
Academia Stack Exchange is a question and answer site for academics and those enrolled in higher education. It's 100% free, no registration required.

Recently, our group is trying to reproduce the result reported by a paper whose authors are from an Ivy league university. We are unable to reproduce the results because there are several implementation details are not mentioned in the paper. Hence, we decide to write the first author, who is now already a faculty member in another university, an email.

At first, we asked for the code, which we consider is perfectly fine, as the paper has been published. We feel the code in a published paper is no longer a secret in this transparent research era where reproducibility is highly valued. However, he simply ignored our email (3+ weeks, no response).

Then, we thought, OK, seems that he is reluctant to share the code, so let's just ask him to clarify several implementation details so that we can implement the thing ourselves and hopefully, we can reproduce the result. So, we sent a second email which very clearly asks for clarifications. Again, he ignored the email (1.5 weeks+, no response).

We now feel angry and start suspecting the authenticity of their reported results. However, we cannot accuse them of anything, since we are not able to prove that they cheated, which would be a felony if they really did.

We always feel that upon the publication of a paper, its authors, or at least the correspondence author, hold responsible for any inquiries regarding the paper, especially when the authenticity is being doubted. What they chose to do - ignoring our email - is really irresponsible.

What can we do?

Disclaimer

Thanks for the answers and comments! It is interesting that many start besieging me on my "bad" attitude in the email.

Just to clarify,

  1. I wrote perfectly polite emails to the correspondence author;
  2. I have NOT questioned his results or whatsoever.
share|improve this question
22  
... which would be a felony. — Say what? –  Mad Jack Jul 9 at 18:30
5  
It might take a new faculty member much more than a month to respond, especially right at the end of term. –  StrongBad Jul 9 at 18:45
8  
Two things: if you asked in the past three months you have to realize that May is extremely busy with the end if the semester, then many academics go on break for at least part of the summer. Second, if you have a bad attitude (judging from your rush to judgment about felonious behavior), people are apt to ignore you. –  RoboKaren Jul 9 at 18:51
18  
The authors have no obligation whatsoever to respond to your emails, nor to share their code. –  Cape Code Jul 9 at 19:15
6  
@FarticlePilter they are definitely not obliged to email-clarify the unclear parts in the paper - most gladly do so, but it would also be perfectly acceptable and ethical to ignore your requests if they want to do so for whatever reasons. Corresponding author is responsible for the correspondence with the journal editor during the acceptance and review process; but there is no duty or obligation to correspond with random readers, unless they want to. –  Peteris Jul 10 at 2:38

3 Answers 3

How do you know he ignored the email?

Maybe he never received it because it was filtered before he had a chance to see it. Maybe he hasn't read his email this month because he's on vacation. Maybe that email address was good when the paper is published but not good now, but it's also not bouncing.

If one channel of communication doesn't work, try a different one. Call his office phone or send a letter to him. Write to a different author saying you've been trying to reach the corresponding author without success and you want to check the email address. Don't assume anything when you have no information.

share|improve this answer
5  
+1, especially for "write to a different author". –  Nate Eldredge Jul 10 at 4:09
3  
+1 for "call his office phone". A phone calls is a very effective way to make a first contact. Then you can follow up with an e-mail summarizing what you understood from the conversation and asking for clarification of anything you didn't understand. –  Christopher Bottoms Jul 10 at 13:41

We always feel that upon the publication of a paper, its authors, or at least the correspondence author, hold responsible for any inquiries regarding the paper, especially when the authenticity is being doubted.

Did you actually state that you do not believe in the results of the paper? If yes, this is disrespectful and plain rude. And most people ignore rude remarks from strangers in the internet. When requesting the help of any other human being, you should be polite and cautious. Also, in Academia you must be very careful when you refer to someone's work. Especially his PUBLISHED work. Because that means that the scientific community has already accepted his claims and you are the one who must prove that his results are wrong and not the other way around (if you ever get published on this subject which be hard to do without his help). So, acknowledge the fact that he has nothing to prove and he will be doing you a favor if he accepts to share his code.

Also, think of the possibility that he does not want to share his code. It is his code after all (and not public domain) and he still has the right to keep it for his personal use. He may also plan to expand on his work and sharing the code prematurely deprives him of the 3 months - 1 year time-advantage over you, since you still have to implement it yourself before expanding on the current state-of-the-art, i.e., his work. In this case, provide him with an alternative. Say that you are willing to send him your datasets and if he agrees to do the experiments for you and report to you his results, it should be good enough for 90% of the cases and everyone is happy. You have the necessary data to compare against your method and he did not have to share his code, which is a logical compromise.

Also, academia is a place that you need to use your social skills. You need collaborators and not enemies / antagonists. In that sense, ask for help politely and expect that NO is a very possible answer on the other party. Also, if he is an established researcher and you are not (perhaps you are famous too - I do not know) there is the case that he ignored the email, because he simply does not even know who you are, what you do and how you will use his code. Usually, telling little things about yourself in the introductory email, sending a link to your personal homepage and google scholar profile, suffices not to consider you a crank and reply to you.

Also, sometimes the first author is a graduate student and the student might not want to share his code because he feels threatened. So, check all the authors profiles. See who is the most senior in the paper and CC him as well in your emails. In that case, the senior professor might encourage the student to share his code despite his objections. Either way, it cannot do you any harm.

As you see, there are multiple reasons why he did not reply to your email. Also a little flattery works on most of the cases. Note that in a sense you find this work fascinating because otherwise you would not struggle to improve it. It is not bad to say so and usually this kind of politeness opens more doors and is more useful in the long term.

share|improve this answer
14  
Brilliant. The only quibble I have is that journal acceptance doesn't mean that the scientific community has accepted the data and conclusions as accepted, merely that they have passed the initial scrutiny of peer review (thus likely to be true). Replication and the test of time is still necessary. –  RoboKaren Jul 9 at 20:05
2  
Thanks @RoboKaren. You are also right for "replication and the test of time is still necessary". –  Alexandros Jul 9 at 20:09
    
+1 and thanks a lot for the suggestions. "CCing the senior professor" is an action worth trying. –  Sibbs Gambling Jul 9 at 20:19

I just want to address one minor point:

At first, we asked for the code, which we consider is perfectly fine, as the paper has been published. We feel the code in a published paper is no longer a secret in this transparent research era where reproducibility is highly valued.

It does not matter what your feelings are but rather what is the policy of the journal the article was published. Some journals require disclosure of data or source code but others do not. Also if the research was funded by the NSF there maybe mandatory disclosures policy. I suggest you to see if this is the case.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.