Is it okay to submit my paper if my code can't be organized now?

Question

Currently, I am working on a model that includes deep neural networks. I am not an expert in coding nor are my colleagues.

I had a hypothesis and it seems to be working. So, I may need to write a research paper and publish it. It takes much time to go through every step and understand in detail the code modules I am using. I somehow managed to join the patches without knowing the coding-related details of each patch. I am taking the patches from the existing GitHub models.

Whenever I see the codes of research papers, they are well-organized by a team of experts. But I am alone and have no team. The research is a part of my thesis. I have no clue about organizing the code.

I have two options:

1. not sharing the code with the journal

   In this case, if the journal asks for the code, then there is no other choice for me and I need to select the second option.

2. posting the managed code on GitHub and then keeping the link in the research paper.

   I am afraid that the readability and the organization of the code impact the decision of reviewers.

Which option should I select now?

Answer 1

I am not an expert in coding nor are my colleagues.

This is not the first time you have made strange statements about "coding." It is true that you do not need to be an "expert in coding"; you are a researcher, not a software engineer. But understanding and organizing your codebase is something that everyone who works with code needs to do, not just the experts.

I somehow managed to join the patches without knowing the coding-related details of each patch. I am taking the patches from the existing GitHub models.

I would be terrified to publish with code I don't understand. How do you know there are no bugs? I assume you have done some "sanity checks" and things made sense....but still, having to retract a paper because the way you "somehow managed to join" them turned out to be wrong would be unfortunate.

Which option I can select now?

Poorly-written code may count against you, but refusing to share the code will probably hurt just as much. Which option you choose probably comes down to personal preference. To the extent there is a global norm, it is that sharing code is an intrinsically good thing and should be done unless there is a good reason not to (and "I did a bad job" is not a good reason).

Answer 2

Most likely NO

It takes much time to go through every step

This left me stunned: didn't you write the code yourself?

I am afraid that the readability and the organization of the code impact the decision of reviewers.

This is likely a correct judgement: If the code is very unstructured, it should not be published because no one — not even you — can claim to understand it.

There are two possible answers.

Understand what happens inside the modules

If by patches you mean invoked modules or libraries, usually reading the docs will be enough to "believe" that they do what they claim to do. No need to understand all of the source code for each module in detail.

If you i) don't understand the docs for the modules or ii) there are no/missing docs for the modules and you aren’t able to understand their source code, then proceed to this third option:

(3): Learn how to code or start collaborating with someone who does

Even people who know how to code can introduce bugs. Low coding quality and logical errors are a serious issue in published research code where even experienced programmers may write software with errors.

Imagine your question the other way around: "I don't really understand math, but I think I've proven something."

No, you didn't.

Answer 3

If you do not understand your code, how do you know that it is actually working, and there is no bug in your code? And how can you publish something you don't understand?

It could be possible that you use synthetic/real data that somehow made the program produce the result you wanted to see. It happened to me before when I tried to modify my correct solution by adding "random" blocks of code in my program to see what might happen.

It was definitely an incorrect solution, yet it somehow produced the correct results for some situations. However, as I tried to spend more time analyzing other settings, the "random" algorithm completely failed to work.

Looking at the results is never enough to know if the program truly works. You have to spend time analyzing your algorithm. There is no other option. Divide your code into blocks, so it will be easier to know what is going on. At least, you need a good theoretical reason to explain why your program works (and under which situations it may fail).

Answer 4

It's pretty much always the case (in situations that aren't security-sensitive, anyway) that making your code available to the public is the best thing to do.

As others have pointed out, you need to get help organising your code. Not only that, but if you're not clear on how your code works you need to do a code review with someone so that you can figure out if it really is doing what you think it's doing. That will likely include organising your code better: as a professional software developer, my first response when trying to understand badly organised code is usually to try to organise it better. (Obviously having some automated tests will help here, but regardless, "rework it so it makes more sense and then see if it still does what it's supposed to do" will both give you insight and possibly even fix problems in the code.)

If your code is freely available on GitHub you can point people at it and ask for help and suggestions. You may not get any response to requests for help, but if you keep it a secret it's certain that nobody can help you.

Also, depending on your paper, it's possible that the reviewers may need to see the code in order to review it properly. If they did need to see the code and couldn't, and let the paper go through anyway, both you and they may be doing a disservice to the entire academic community by keeping the paper from being properly reviewed.

Next time you do this, by the way, make the code publically available from the start and seek help and advice from others early on, while the code is still a small mess. That will help keep the code from growing into a huge mess.

Answer 5

Beyond what everyone else has already said, it is often useful to use domain-specific knowledge to come up with big-picture checks that, if passed, will give high confidence that large portions of the code are working correctly. This is where you, as an academic, can add value beyond standard software engineering testing practices.

For example, here are some big picture checks from my field (computational science):

• In finite elements, you can refine the mesh, plot the error vs mesh size, and verify that the observed convergence rate matches the theoretical convergence rate.

• In Newton's method, you can verify that the observed convergence rate is quadratic.

• In gradient, or higher derivative, computation, you can compare the computed derivative to a sequence of finite difference derivative approximations for smaller and smaller step sizes, and verify that the two converge to each other at the expected rate.

• In the computation of an adjoint code, you can compare (A(x),y) to (x,adjoint_A(y)) and make sure they are the same to roughly machine precision.

These kind of big picture checks are extremely valuable, because bugs (wherever those bugs are) will almost always cause the check to fail.

Answer 6

There are situations where finding a solution is hard, but verifying the solution is easy. There are other cases where verifying a solution is as hard as finding one, and the only way to verify it is to try finding the solution with a different implementation, check that you get the same results, and pray that both algorithms don’t make the same mistakes.

If you can verify the results of your code, that makes problems with it much less important.

Answer 7

On the whole, yes it's okay. If the code works, it works. Data is data even if the method of collection confusing.

Obviously it's better if you could organize it better. It will also make it more likely that people will use your code and build on your research, which means more citations of your paper. But since you assert it is impossible to organize it, that ship has sailed.

Messy code is better than no code. The critical difference is that a paper with messy code can easily be reproduced. A paper with no code is a different matter.