4

When reviewing a manuscript describing a new software tool, reviewers are often asked to assess its utility, quality, novelty, ...

I was wondering if, in addition to the classic comments on the manuscript, it was appropriate to make suggestion to improve the software functionality in itself (if the code is provided with the manuscript)?

This would be in the case of a software that is not good enough for publication (basically it does not do enough), and for which small additions will make it better. The comment on the code would therefore not be related to the coding "grammar" but ratter to improve the software tool usability for the final user.

Improve this question
2
  • 2
    I'd say that of course it's appropriate to offer suggestions if something strikes you about the code. I'd rather ask whether reviewing the code is expected from the reviewer or whether he will only be expected to look at the manuscript.
    – Stephan Kolassa
    Jul 8, 2014 at 12:42
  • 1
    @StephanKolassa - Why don't you post that as an answer instead of a comment and get points for it?
    – eykanal
    Jul 8, 2014 at 13:46

2 Answers 2

Reset to default
7

In my opinion, if the authors emphasise the developed code as one of their research outputs, and make it public, the reviewers not only can, but should consider it as a material for review, and comment on it. There is a delicate balance, though, to strike.

It is important to check that the code does what is promised in a paper (sanity check). More importantly, it is essential to check if a typical reader of the journal can do the same, and can benefit from the code made public. This includes basic documentation, compile instructions (if any), well-written run-files for each of the examples from the paper, and clear guidelines how to adjust them for other problems.

On the other side, remember that not all academic researchers are necessarily as brilliant in code-dev, as the guys who read Stack Overflow. It may be not appropriate to request that the code is developed and maintained to the highest standards of the modern IT community.

Improve this answer
3
  • I agree that all academics are not computer scientist and that "academic software" should be expected to meet the highest IT standards. However, I also believe it is not a reason to publish poorly designed software.
    – Wiliam
    Jul 8, 2014 at 13:15
  • The definition of "poor design" is subjective. Early Blas/Lapack source codes, for example, may be considered ugly and poorly written from the perspective of OOP community.
    – Dmitry Savostyanov
    Jul 8, 2014 at 13:25
  • @dmitry_savostyanov Let say a "little added value" for the scientific community then :)
    – Wiliam
    Jul 8, 2014 at 13:29
0

I would suggest not to comment on code, unless:

  • The code is broken in some way, such that it does not perform what it is supposed to do (e.g., a bug in code changes a critical algorithm)
  • The program itself is unstable due to deep-rooted code smell

Commenting on code in a review seems to me akin to commenting on someone's mastery of a given language in a review. If their usage of the language is so poor as to make the paper unreadable, then you can suggest they use a proofreader or translator. However, in most cases, it doesn't affect the paper, it just makes it harder to understand. With code, it may not get them past a <insert hi-tech company here> interview, but if it works as advertised, it's good enough.

Improve this answer
6
  • 1
    And what if the proposed code is not good enough for publication (it does not do enough), but some minor improvement make it publishable? (I edited my question to make it clearer)
    – Wiliam
    Jul 8, 2014 at 12:54
  • @Wiliam - I would suggest that what you describe is not a code issue, that's functionality (or "utility", to use your term) issue.
    – eykanal
    Jul 8, 2014 at 13:16
  • I edited the question.
    – Wiliam
    Jul 8, 2014 at 13:21
  • @Wiliam - Now I don't understand the question. Reviewing code is one thing, reviewing the functionality of the software is completely different. It's a back-end vs. front-end review. Which as you asking about?
    – eykanal
    Jul 8, 2014 at 13:47
  • Well, I mean proposing modifications in the code to improve the functionality. So, changing the code, but not because it is poorly written, but because it lack functionalities.
    – Wiliam
    Jul 8, 2014 at 17:11

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .