What action to take when questions regarding a published paper are ignored by its author?

Question

Recently, our group is trying to reproduce the result reported by a paper whose authors are from an Ivy league university. We are unable to reproduce the results because there are several implementation details are not mentioned in the paper. Hence, we decide to write the first author, who is now already a faculty member in another university, an email.

At first, we asked for the code, which we consider is perfectly fine, as the paper has been published. We feel the code in a published paper is no longer a secret in this transparent research era where reproducibility is highly valued. However, he simply ignored our email (3+ weeks, no response).

Then, we thought, OK, seems that he is reluctant to share the code, so let's just ask him to clarify several implementation details so that we can implement the thing ourselves and hopefully, we can reproduce the result. So, we sent a second email which very clearly asks for clarifications. Again, he ignored the email (1.5 weeks+, no response).

We now feel angry and start suspecting the authenticity of their reported results. However, we cannot accuse them of anything, since we are not able to prove that they cheated, which would be a felony if they really did.

We always feel that upon the publication of a paper, its authors, or at least the correspondence author, hold responsible for any inquiries regarding the paper, especially when the authenticity is being doubted. What they chose to do - ignoring our email - is really irresponsible.

What can we do?

Disclaimer

Thanks for the answers and comments! It is interesting that many start besieging me on my "bad" attitude in the email.

Just to clarify,

1. I wrote perfectly polite emails to the correspondence author;
2. I have NOT questioned his results or whatsoever.

Answer 1

How do you know he ignored the email?

Maybe he never received it because it was filtered before he had a chance to see it. Maybe he hasn't read his email this month because he's on vacation. Maybe that email address was good when the paper is published but not good now, but it's also not bouncing.

If one channel of communication doesn't work, try a different one. Call his office phone or send a letter to him. Write to a different author saying you've been trying to reach the corresponding author without success and you want to check the email address. Don't assume anything when you have no information.

Answer 2

We always feel that upon the publication of a paper, its authors, or at least the correspondence author, hold responsible for any inquiries regarding the paper, especially when the authenticity is being doubted.

Did you actually state that you do not believe in the results of the paper? If yes, this is disrespectful and plain rude. And most people ignore rude remarks from strangers in the internet. When requesting the help of any other human being, you should be polite and cautious. Also, in Academia you must be very careful when you refer to someone's work. Especially his PUBLISHED work. Because that means that the scientific community has already accepted his claims and you are the one who must prove that his results are wrong and not the other way around (if you ever get published on this subject which be hard to do without his help). So, acknowledge the fact that he has nothing to prove and he will be doing you a favor if he accepts to share his code.

Also, think of the possibility that he does not want to share his code. It is his code after all (and not public domain) and he still has the right to keep it for his personal use. He may also plan to expand on his work and sharing the code prematurely deprives him of the 3 months - 1 year time-advantage over you, since you still have to implement it yourself before expanding on the current state-of-the-art, i.e., his work. In this case, provide him with an alternative. Say that you are willing to send him your datasets and if he agrees to do the experiments for you and report to you his results, it should be good enough for 90% of the cases and everyone is happy. You have the necessary data to compare against your method and he did not have to share his code, which is a logical compromise.

Also, academia is a place that you need to use your social skills. You need collaborators and not enemies / antagonists. In that sense, ask for help politely and expect that NO is a very possible answer on the other party. Also, if he is an established researcher and you are not (perhaps you are famous too - I do not know) there is the case that he ignored the email, because he simply does not even know who you are, what you do and how you will use his code. Usually, telling little things about yourself in the introductory email, sending a link to your personal homepage and google scholar profile, suffices not to consider you a crank and reply to you.

Also, sometimes the first author is a graduate student and the student might not want to share his code because he feels threatened. So, check all the authors profiles. See who is the most senior in the paper and CC him as well in your emails. In that case, the senior professor might encourage the student to share his code despite his objections. Either way, it cannot do you any harm.

As you see, there are multiple reasons why he did not reply to your email. Also a little flattery works on most of the cases. Note that in a sense you find this work fascinating because otherwise you would not struggle to improve it. It is not bad to say so and usually this kind of politeness opens more doors and is more useful in the long term.

Answer 3

I just want to address one minor point:

At first, we asked for the code, which we consider is perfectly fine, as the paper has been published. We feel the code in a published paper is no longer a secret in this transparent research era where reproducibility is highly valued.

It does not matter what your feelings are but rather what is the policy of the journal the article was published. Some journals require disclosure of data or source code but others do not. Also if the research was funded by the NSF there maybe mandatory disclosures policy. I suggest you to see if this is the case.