In computer science, when submitting a paper, it is common to submit some code to reproduce the experiments performed in the paper.

Is it common for the reviewers to review that code?

  • 8
    No. Almost Never.
    – user35129
    Mar 4 '17 at 15:25
  • More likely, they will ask the author to revise the code.
    – GEdgar
    Mar 4 '17 at 16:18
  • @GEdgar: Even more likely, they won't even do that. Mar 4 '17 at 19:26
  • 1
    Do you mean 'revise' or 'review'? Mar 5 '17 at 19:21

Personally, I always check the code (and I always make the code available as well).

Usually it is just a glance over, unless I want to double check how something was done (and it wasn't clear on the paper). People usually don't do that because "scientific" code isn't always readable. Often is written hastily, as a one-off that can't really be reused only the author can make it work. Even that may fail a couple of years later.

And considering the amount of views I get in my repositories, even a glance over is not common. For my last paper, I had code, live examples, and supplementary material online, and only one of the 3 reviewers checked it (I got only one visit on the url).

TLDR: People usually don't check it, but it is still very important that you make them available.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.