Could a journal reject an article due to the style or quality of accompanying code?

Question

I'm a student working on a doctoral thesis. We're working on a paper for which I've done most of the coding for the statistical analysis (the field is microbio/bioinformatics). I'm new to coding and the whole thing is more of an add on to my "real" career, which is medicine. Therefore, my style of coding is more than basic I'd say. Now I fear if we're handing in the manuscript plus the code to a journal for peer review it will be rejected or ridiculed. Is this possible?

edit: I use R to conduct a large statistical analysis. If someone would like to point out online learning resources, esp. for the fields of medicine/biology, I'd be very grateful.

Answer 1

You will not have a paper rejected for poor coding style. Particularly not in bioinformatics. Unless the paper is a tool development paper, the chance that a reviewer even looks at the code is not that high, and if they do, it will just be to check that its reasonable. In the outside case a reviewer might spot a bug, or poor assumption that actaully makes the result wrong, but they will not reject you for poor style.

However, having the code associated with the paper massively increases the chances that it will be useful to someone else.

Answer 2

Let me bring up a more serious issue that you don't ask about. Poor coding "style" can also mean poor "coding". And poor coding can hide errors, affecting the results. If your research results don't depend on the coding then it may not be an issue and there is probably no need to publish the code itself, beyond some description of it.

But if the research results depend in any fundamental way on the code, as it does in some cases, then you are at risk and your paper may be rejected, properly, for having insupportable results. But that won't be for the "style."

I don't know which is the case here, but you need to assure yourself that your results are sound. You might need to collaborate with a programmer to provide suitable testing of your code, and perhaps improve it.

Answer 3

It could happen in principle, but only in extreme cases, where the code is so bad that it impacts the reliability of the results.

The code is a part of the research you are reporting on. In some fields (e.g. parts of computer science) it is often an key aspect of the research itself; in other fields (e.g. most natural sciences) it’s a more auxiliary part of the setup, about analogous to the lab equipment.

Either way, it’s a part of the research, and so in principle, a paper could be rejected because if it was too bad. However, “too bad” would have to be very bad indeed — typically if it had bugs that affected the results, or could have affected them. (Just as a paper might be rejected if a referee found errors due to contaminated lab equipment.) No referee will care about stylistic issues — the usual standard of most academic code is very variable indeed, and usually not very stylistically clean.

Answer 4

I would probably say that releasing the code is better than not.

In all likelihood the reviewer won't look at the code even if you include it, they may in-fact give you a little tick for including it even if they don't look at it since it would help with the reproducibility of your results by other people who do bother to look at it.

If they do look at it and don't like it there will be one of three outcomes:

1. They request more comments in the code
2. They say your code is not incorrect but it should be re-written for efficiency or style
3. They find a mistake in your code I have ignored the case of them only running your code and not looking through it since it shouldn't give a different result when they run it compared to when you run it.

1) If they request more comments, fair's fair. Comments are important for readability, and in-fact you should make sure there are enough comments there now to make sure you'll quickly remember what you were trying to do with the code when you look at it in a few months time.

2) If they say you should improve the efficiency or style of your code then in all likelihood you can argue to the editor that only the code's results are relevant for the peer review, not the code style for your field. After that the editor will just ignore any similar comments from said reviewer. The exception is if they complain about your choice of variable/function names, which really falls more under 1) rather than 2). However if that happens then all that is involved is ctrl+f to find the bad variable names and give them a better, more explanatory, one.

3) The worst case scenario is they find a mistake in your code, in which case you can correct and check if/how this changes your results and modify your paper accordingly. The people who do this would probably include any reviewers who might try to reproduce your results on their own if they didn't have the code. In which case they may make different assumptions to you and therefore get different results which leads to a headache arguing over who's code is correct. Since they have your code you can now challenge them to point out the mistake if they just say your results don't match with my results.

Now it is possible that you will find nothing of interest after making this correction which makes it seem like all your work in the project was for nothing, but its important to remember you would have done all this work and written/verified your code before reaching this point, so in fact all you've lost is the time taken to write the paper, which while not good is still a learning experience for you so the next paper you write is done quicker/better. Furthermore it means you don't have a paper that is incorrect and leading to confusion when people try to replicate/extend the research that is tied to your name.

While this case sounds very bad it will only occur if your code is wrong. If you are confident in your code then its fairly safe to exclude this case, if you aren't confident in your code then you should check it until you are.

One last thing that you should keep in mind is that if you release your code you will want to provide a license with your code without the license nobody has a legal right to use it/extend it/write code based off reading your code. Or at the very least this falls into a gray area. Normally scientific code is released with a very open license (I think MIT license is the standard) but you can google to find out what types of licenses there are.

Answer 5

It sounds harsh but I sometimes used to cringe at the rats-nest of code some scientists would ask me for advice on.

If your development of the code has been linear rather than cyclical then I would bet my boots it contains significant errors.

A problem with R is that a lot of it is created by amateurs.

Compare this with the various libraries of cast-iron copper-bottomed (professional) routines that have stood the tests of time - the problem is of course licensing fees.

If you wrote your own statistical code then I suggest two ways forward:

1. Put aside your original code and rewrite it again from scratch with the benefit of hindsight. You will write it much quicker and much better the second time. When finished, run it on the data and see if you get the same results.

or

2. Get hold of proper tested routines and write a simple program around them. For example NAG routines. https://www.nag.com/content/using-r-nag-library

If the professional packages cost too much then find out if someone else has a license they will let you use as a temporary measure.

Or pay a keen undergrad to write the new code for you. Test theirs against yours.

Answer 6

Bad coding style can lead to reviewers not reading the code. They usually are not obliged to read it and especially not to review it, but providing code can help to get good reviews because they are able to reproduce your results, which is a good thing.

I would say the most important thing is technical correctness, followed by an easy way to run it (try to use easy build systems and provide instructions for dependencies, when it is complicated to install them and maybe provide binaries in addition to the code).

Coding style comes last and many academic codes are not so easy to read. Especially mathematicians tend to use a lot of non-descriptive one-character variable names, which can only be understood when reading the corresponding paper at the same time.

Answer 7

I am a software developer. I frequently work with research scientists, incorporating their equations and simulations into other programs. I originally had your same concerns, but from the other side (worried it would be detrimental that I only have a low level of knowledge about the underlying science). Someone explained to me that it was like learning a new language. To a native speaker, yes, you'll be harder to understand. It means a lot, though, that you're even making an effort to learn that language. People know you're new at it and they'll generally cut you a lot of slack.

Like you said, you're in the medical field. There's not an expectation that you're proficient in coding, or Russian literary history, or accounting, or anything else that you haven't been trained in. Research papers get published all the time that are clearly not in the author's native language. The writing is rough and hard to read, but the paper still gets published because the science is sound and that's really what it's all about. Your code is really no different. Unless your code is so bad that it makes the paper harder to understand or that it has bugs that cause incorrect results, I highly doubt anyone would think any less of it.

If you're really concerned about the code's quality, create a public repository on GitHub (or similar) and invite others to help clean it up. You might be surprised how much your code can be improved with nothing more than a half hour and a different pair of eyes.

As an aside, I've had a number of my scientist co-workers apologize for the quality of the code they give me. They're shocked when I tell them that their code actually looked better than the absolute garbage that some professional programmers generate. The very fact that you can see quality problems in your own code says that you're more skilled at it than you realize.

Answer 8

You seek to balance the time and effort to develop the code to a degree that it is

• «good enough» to tackle the task you currently address in comparison with an earlier defined set of duties
• easy to maintain by you or (future) colleagues in your group / community, e.g. to add features you did not anticipate in the beginning
• documented well enough that others not directly interacting with you may understand how to use the program and its inner working

while programming not necessarily is your centre of activity.

In your context, I recommend programming classes specifically set up for an audience without prior exposure to computer science. Here, software carpentry is the first site which comes into my mind. Because of its collaborative character designing the courses (survey, includes R), their classes exemplify best practices and standards you may use as a reference which suits places around the globe they taught and teach. It equally may introduce you to peers of similar interests to yours, too. As a result, your code quality improves and is more likely to be accepted by others.

Contrasting to on-line classes on sites like edx.org, improvements of their material is not constrained to the duration of the class itself. Any interested may access their public repositories and suggest improvement.