10

Associate Prof. X is committed to reproducible research and makes his code available. He authors a paper with Dr. Y and others.

Dr. Y is the first author of the paper and the code is shared on Ass. Prof. X's website. It is clearly stated in the paper that the code produces the graphs in the paper.

I downloaded the code, ran the code and found it works well i.e produces the same graphs in the paper. However, the code was not descriptive and easy to understand (Apparently, the variables which makes it easy to read has been changed to something difficult to understand except with much effort and time). Anyway, I studied the code and was able to clearly understand the meanings of various parts of the code. I gave the variables their proper definitions while generating the same graphs

Now, I find the graphs for some figures are not provided, then mailed Ass Prof. X, who after a while informed me that first author Dr. Y will send the code. The code was sent and I was able to understand this paper.

Currently, I am about making an extension to this work. However, I observe there are some errors in several equations in the paper which adversely affects the behavior of some graphs! Furthermore, on a deeper investigation into the paper, some inferences and propositions were made based on some of the mistakes within the paper! At a time I tried contacting Dr. Y about something I wanted to know better about the paper but I didn't get a response.

The challenge I have now is, I want to present my own paper and as well make corrections to these mistakes since it directly extends this work. How do I handle this? Do I say for example: "We corrected equation (y) in [ref] and thus, the graphs behave like this?". Also, I am thinking that since I got the code for some of the graphs (via email), I should thank the authors for sharing the code (for those graphs). However, I am not sure this is appropriate as the public gets to know implicitly that the code for some of the graphs are not provided in the paper.

Note that, I appreciate and consider it as an obligation to be straightforward in my research!

  • What do you mean by "gibberish"? Did they send you intentionally obfuscated code, or is it simply code that was written for their own internal usage and so nobody bothered to add comments to it or make it easy to use for an outsider? – Federico Poloni Apr 8 '18 at 7:52
  • Thanks Federico! I have edited that in the question. It appeared a deliberate attempt was made so that the code is uninteresting and the variables "undescriptive". – Abdulhameed Apr 8 '18 at 8:04
  • 6
    I suspect it was not deliberate so much as the researcher who wrote the code being a typical researcher when it comes to code. Many scientists do not write code for reusability but simply to solve an immediate problem, as unfortunate at it is. – JAB Apr 8 '18 at 15:36
  • 1
    I want to second @JAB's comment -- never ascribe to malice that which can be adequately explained by incompetence. – Thomas Apr 9 '18 at 3:45
9

I would think several times before using the word 'corrected', especially if I don't fully understand the code, as it seems from your question ("...I studied the code and was able to decipher the meanings of various parts of the code."). Understanding various parts is different from understanding it entirely, the latter being a prerequisite to improving upon it or correcting it.

However, if you are sure that mistakes exist in the code, you have two options:

(a) Point out those mistakes and share them with X and Y before trying to publish it. Once you publish, the credibility of their code goes down - if some parts are wrong others could be too. Therefore it is only fair that you convey the errors to them first. You aren't obligated to, but I think it's the right thing to do.

(b) If you're significantly extending the code, write your own- it could be based on their code, and you can acknowledge that, but by writing your own code you take full responsibility for it, and are not propogating any errors inherent to the original code. In terms of quantum of work, I think this is more significant, and gives you more room to develop it for your particular purpose.

  • I was able to properly understand the code, since I have given the variables their proper definitions while generating the same graphs! However, there are issues with equations in the paper. The word decipher refers to the undescriptive nature of the code. Then do you mean share my own code publicly? Definitely I have made corrections to the code based on the framework in the paper (after I have identified the areas of concern in their code). Also, pointing the mistakes to them sounds good but, how do researchers handle such things if the mistakes are pointed to them. – Abdulhameed Apr 8 '18 at 8:27
  • 1
    Well if you are confident about it, then that's great! :) As I understand, there are two things - (a) the issues with equations and (b) the new functionality you have added. The first is all that you should share with X and Y. If the new functionality is significant, you may like to rewrite a fresh code, it gives you more control and you can test it more thoroughly. You should still mention in the paper that X and Y tackled this problem first. As to sharing the fresh code publicly- that is a different question and entirely up to you. – user153812 Apr 8 '18 at 8:38
  • 4
    About how researchers would handle it- that could really vary from profusely grateful to irate, but in any case I think they would appreciate you going to them before going to a publisher. Reason is simple- in case there is an explanation, they can make it, rather than having to write to the journal, which can become acrimonious. – user153812 Apr 8 '18 at 8:40
  • 1
    Also, leverage the wisdom of crowd to make sure the codes (their and yours) are correct. A good venue is Code Review – Borhan Kazimipour Apr 9 '18 at 2:50
  • Yes! I just checked code review, but the code is an analytical interpretation of the equations in the paper. I am thinking I could provide the code during the peer review process. My extension would include their work as a special case. I am thinking of removing the special case from my research. – Abdulhameed Apr 9 '18 at 5:01
3

Firstly, yes, you should definitely thank them for providing their code, and making their research reproducible. This is extremely valuable, and it is precisely the ability to spot and correct errors in research that makes this a good practice. If it were me, I would be quite profuse in my thanks for this, and make sure that any criticisms of errors are contextualised by the excellent work done in preserving their data and code.

The other thing you should be careful of is to make absolutely sure you're right about the errors. I would suggest that prior to submission of your article you first contact the authors of the other work and show them the errors you have found in their work. This will give you feedback to confirm/deny the error. (You might even find it is something simple like they sent you the wrong version of the code.) Most authors are quite reasonable in responding to things like this, and I'm sure they will want to check their work to see if there are errors. If you find them responsive to your concerns, you might even consider suggesting these authors as referees for your paper.

  • Please what do you mean by "make sure that any criticisms of errors are contextualised by the excellent work done in preserving their data and code." – Abdulhameed Apr 9 '18 at 4:18
  • 2
    What I mean is, the reason you are able to identify an error is because they have done the right thing and made their data and code available, so if you make a criticism of an error, it is also a sign that they have done good work by making their data and code available for that purpose - that is what I mean by "context". – Reinstate Monica Apr 9 '18 at 8:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.