Should I share my horrible software?

Question

After I had published my paper, some people asked me to share the software that I developed. At first, I was very happy that my paper attracted some attention, and I was happy to share not only the binary but also the source code, case studies etc. But looking at my software, I feel very embarrassed.

My software is just horrible: the source code is just a mess, containing several of my unsuccessful attempts; I have never used design patterns, so duplicate code is everywhere; for simplicity and quick implementation, I often prefer recursions to loops etc etc.

I'm always under pressure to produce new results, and cleaning those code would cost me significant effort.

My question is if sharing this horrible software will give people a very negative impression of me? Would it do harm to my career if the people I share are prospect collaborators, employers, as they work in the same field.

Answer 1

Yes, you should.

First, most scientific software is terrible. I'd be very surprised if yours is worse than average: the mere fact you know design patterns and the difference between recursion and loops suggests it's better.

Second, it's unlikely you'll have the incentive or motivation to make it better unless, or until, it's needed by someone else (or you in 6 months). Making it open gives you that incentive.

Potential upsides: possible new collaborators, bugfixes, extensions, publications.

Potential downsides: timesink (maintaining code or fixing problems for other people), getting scooped. I'll be clear: I don't take either of these downsides very seriously.

Answer 2

I would clean it up a little and share it. I've released a lot of code over the years, and also not released code for the reasons you give.

Go through it and and comment it, at whatever level you can. Leave in "failed attempts" and comment them as such. Say why they failed, and what you tried. This is VERY useful info for people coming after you.

Make a README file that says you are releasing it on request in the hope it helps someone. Say that you know the code is ugly, but you hope it's useful anyway.

Far too many people hold things back because it isn't perfect!

Answer 3

Yes! Especially if your paper is e.g. about a new/improved algorithm that you've implemented, or you do significant non-standard data analysis, or basically anything where reproducing your results means re-implementing your software.

Papers seldom have room to give more than an outline. I know I've spent (= wasted) much too much time trying to implement algorithms from papers that left out critical (but not strictly relevant to the paper) details.

Answer 4

¿You think your code is messy? I have seen (and attempted to work with) code that gave me nightmares:

• Five levels of if True nested, scattered at random places through the code.
• Create an array of zeroes, convert it to degrees, take the cosine, and back to radians. Then, throw away the result.
• On a software under heavy development, the list of "supported architectures" is so ancient (and they do say themselves) it would difficult to get your hands on one of these computer nowadays.
• Features broken or modified several versions ago, still recommended in the docs.
• Code that goes from using a standard format input to some format of their own. How to generate it? No one really knows, and the developers handwave a response.
• Releases that don't even compile. (Did you even test it?)
• GUI menus that you have to access in a specific order. Otherwise, you get a segmentation fault and have to start from the beginning.
• Hard-coded paths scattered through the code. So you have to shift through several files finding and changing all the occurences of /home/someguy/absurd_project/working/ to yours.

And, my personal favourite, a certain program of thousands of lines of code, only used comments to eliminate random bits of code, except for one:

Here we punch the cards.

Still, no idea what it was doing.

And this is only leaving outside the classical good practice stuff, like one-letter variables all over the code, algorithms not specified anywhere...

If you are concerned about the quality of your code, it probably means you care enough to have made it better than the average. If you wait until the code is clean, it may never get out, and your scientific contributions will be partially lost.

In my opinion, the important things that you should care about, in order, are:

1. Input and output formats. Use standards when available, make it simple when not. Make using your program as a black box easy.
2. Commented. Brief descriptions of the functions, quick overview of the algorithm.
3. Legibility. Using idiomatic code, good variable names...
4. Structure. This is easier when you know what you want to do, that is usually not the case in research code. Only if there is interest in the community, you may consider refactoring it.

So, release your software whenever you have 1 (2 and part of 3 should come in as you are writing it).

Answer 5

You're asking whether sharing low-quality software would give a bad impression of you. I think that sharing software at all gives a good impression.

1. As a computer scientist, I like when colleagues make their source code available. It makes me more likely to look deeper into their work, maybe contact them, maybe cite them, because there is one more artifact to interact with (not just the paper, but also the code).

2. When a paper reports a result that is "proven" by source code, but the source code is not public, I'm often wondering whether the result is real. Looking at the source code (or just the availability of the source code, without ever looking at it) can convince me.

So sharing your source code, horrible or not, would always give me a good impression of you.

Now, if you want to impress even more, it would help ...

... if you react to issues or pull requests on a site like github, that is, when I see that others try to contact you and you react.

... if your code contains a readme file which relates the claims from your paper to the source code. This way, when I read the paper and want to know more, I can use the readme to jump to the appropriate place in the code. Typical phrases from such a readme could be: "The algorithm from Sec. 3.2 of the paper is in file algorithm/newversion/related/secondtry/foo.c" or "To repeat the run with the small dataset described in Sec. 2 of the paper, run "make; make second_step; foo_bar_2 datasets/christmas.dataset. This run takes about 2 days on my laptop".

You might also be interested in Matthew Might's CRAPL (Community Research and Academic Programming License), available on http://matt.might.net/articles/crapl/. It contains this term: "You agree to hold the Author free from shame, embarrassment or ridicule for any hacks, kludges or leaps of faith found within the Program". It is not clear to me whether this "license" has any legal effect, but the intent is clear: Release your ugly code, and don't think bad of the ugly code of others.

Answer 6

Tangentially related, I will addresses how to share the software given your concerns (not should you share the software which you already have an answer for).

Putting the failed attempts in version control effectively means that nobody will ever see them. The way I handle this is to put each attempt in a method, and each failed attempt in a separate method:

def main():
    get_foobar(x, y)


def get_foobar():
    return x**y


def get_foobar_legacy_1():
    """
    This attempt did not work for values > 100
    """
    return x + y


def get_foobar_legacy_2():
    """
    This attempt did not work on Wednesdays in September
    """
    return x - y

As per the comments below, it may be a good idea to put these methods in a separate FailedAttempts or BadIdeas class. This has the nice effect of compartmentalizing the various stages for the process as per actual need. I find that computer programmers often have a knack for when to break logic off into a method and when not to, but computer scientists often do not. This approach helps the computer scientists break off into a method when necessary.

Answer 7

Of course you should share the source code.

Academically speaking, a software-based result using code that is not readily available is not very valuable, as how would other people be able to verify your claims, if needed? Do you expect them to program on their own for this purpose? Sharing binaries only is much less valuable, and often leads to nightmares for people trying to run them.

Answer 8

I think you should share it. First of all you should do some basic clean up. (e.g.: no earlier code which is not used anymore; no code in comment; valid way of commenting and so on) Moreover if you put some "to do" in the code others can see that you were out of time and they can see your intentions. (e.g.: todo: this should be changed to enum) I also think you should share the most important part of your algorithms. When I share a code I have never share unimportant parts. Everyone can handle reading/writing of files, communication between threads, gui and so on. But don't share unreadable code. It would make no sense. So I think the middle way is the best as many times. :-)

Answer 9

Talk to some of the professors in your computer science department. See if any of them are looking for a project where students can clean up messy code to make it more presentable.

For the students who revise the code, this can be a good learning experience. What happens when coders program with a results-first mindset – or results only mindset? They get to see that first hand. They also get to apply some of those best practices they've been learning about. And they might be motivated to do an especially good job knowing that other professionals are already interested in seeing the code.

A professor might even make this into a contest, where teams of students all take a crack at revising the software, and the best result is shared with the rest of the world.

If their refactoring efforts flop, you're no further behind than you were. If that's the case, disclaimers are a wonderful thing. Simply share the code, but add a caveat: "It isn't pretty. When I wrote this, I was trying to get my research done – I wasn't thinking it would ever go outside my computer lab. But you're welcome to take a look if you really want to."

Answer 10

Lots of points in favour of publishing the code have been named in the other answers, and I completely agree with them. Hence, as the basic desirability of publishing the code has been discussed, I would like to supplement this with a checklist of further points that need to be considered. Many of these issues probably appear in virtually all academic software, so even if you cannot respond "This does not apply to my project." to all of them, you should at least be able to respond "This is a concern, but we can deal with this issue by ..." before publishing your code:

• Are you allowed to publish the code?
  • Can you guarantee you only used code fragments that you are allowed to redistribute? Or did you possibly use code from non-open sources that you may use for your own internal software, but that you are not allowed to publish? Can you guarantee all the code that you used is allowed to be published in one complete package? License compatibility is a non-trivial issue.
  • Can you even reliably find out? Did you outsource any parts of your coding work, or integrate unpublished code from elsewhere? For instance, did you supervise any students during their graduation theses or employ any student research assistants, whose work was based upon your research and thus their code was added to your codebase? Did any co-workers contribute code to your codebase? Did they get some of their code from students? Did all of these people involved properly pay attention to licensing issues (if at all they had the knowledge to make an educated judgement about these licensing questions)? Can it even still be determined where which parts of the code originated? Do the people who contributed each part still know? Are they even still "within contact range" for you?
  • Was the code developed during working time based on third-party funds? If so, do the funding contract terms allow to publish the code, or do they include any requirements that software created within the funded project must be shared exclusively with the project partners?
• Do you have sufficient resources (time and otherwise) to spend the effort to clean up the code and its comments in a way that it is still meaningful, but does not provide any information that must not become public?
  • Do you have any comments giving away who worked on the code? Were the people who contributed code officially allowed to work on the respective research, as per their funding? (Software developers are well aware that teamwork and reuse of components are core aspects of software development. Funding agencies, unfortunately, are typically very unaware of this and assume that if developer A is funded from project X and developer B is funded from project Y, A works exclusively on X and B works exclusively on Y, and revealing that, w.l.o.g., A spent only half an hour doing something that ended up in project Y could lead to severe consequences, such as reclaiming parts of the funding.)
  • Does anything in the published data give away any information about the particularities of how the work was done that must not become public? This is especially important if the whole commit history in a VCS is going to become public (or, practically, means that the commit history should never be published), but may also play a role in other situations. For example: Was any work on the code done outside of the officially assigned working times (e.g. during weekends)? Do working times give away that you worked more than the legal limit of your country for working hours per day? Do working times give away that you did not adhere to legally required breaks? Do working times give away that people assigned to other projects made contributions? Do working times provide any reason to distrust any of the statements you made otherwise about your working times (e.g. in project success reports that required a detailed assignment of working times to pre-defined work packages with certain maximum allotments)? Does anything give away that you worked in situations where you should not have been working (e.g. during a project meeting)? Does anything give away that you worked in locations where you should not have worked (e.g. from home, when your contract does not allow you to do home office, e.g. for insurance-related complications)?
  • Is there any secret information in the code, such as passwords, user account names, or URLs that must not be publicly known (because the servers are not laid out to handle larger amounts of users beyond a small number of select people who were given the URL for the test setup personally)?
• Is the code usable by anyone else?
  • Will the code run, or does it require extensive configuration efforts? Can you spend the effort required to explain what configuration is necessary?
  • Can the code be compiled? Have you used any unofficial modified or custom-built compilers that are not publicly accessible? If so, does the code add anything beyond what may already be provided as a pseudo-code algorithm in your papers?
  • Does the code require any external resources? Will the code only be useful if it can access servers, libraries, or datasets that you cannot publish along with the code for one reason or another? Can at least a description of these resources be provided, or are their interfaces subject to some kind of an NDA?
  • Does the code make any irreversible changes to systems it runs on? For example, does it automatically change any system configuration (such as overwriting the system search path)? Does it perform any low-level access to hardware components that could, in certain constellations (that you internally avoid in your test setups) cause permanent damage to any components? Can you reliably warn users of the code of such possible unwanted side-effects?

Answer 11

Of course. The only way you are going to get better at writing good software is to get feedback (all types). If you're afraid of feedback then you won't really get very far. The three basics to writing great software are practice, practice, and practice.

Now on as to the question of whether it would harm your career if people found out that your software writing skills aren't top notch. I think that no, on the contrary, they would respect you for your academic integrity. And would look forward to collaborating with you.

Answer 12

You may just push it to GitHub and try to maintain a project in case other people who are interested about your project can access your code easily and maybe they can help to improve your code.

Answer 13

Yes, you should. After all, the Linux kernel source code is quite a mess and that haven't prevented many professional developers from studying it and contributing patches and additions to it. Remember also that the Linux kernel is the base of the operating system that runs the fastest and most powerful supercomputers and most devices in the world. P.D: Linus Torvalds, the guy who invented the Linux kernel have a very profitable and successful career which have not been affected negatively or harmed in any way by the fact that the Linux kernel source code is messy.

Answer 14

A reason that no one has mentioned why you should share your code is that you might find someone who is interested in collaborating with you, but who is prepared to spend more time cleaning up the code and making it work on different systems, etc. than on doing the innovative development that you have done.

Lots of people find this kind of work very satisfying and if your code is genuinely useful to them they might be happy to do it. In any case, you might find that getting feedback from people who have tried to use it, but need some kind of help, is a good motivation for you to make it more maintainable/easier to use and understand.

Answer 15

Share it if you want to, don't share it if you don't want to. I know this sounds snarky but I think there is too much pressure nowadays to "share everything" and people will try to make you guilty for not sharing, but really you have no obligation to share anything.

Answer 16

You should definitely share your code.

For sorting things, make regions of the same parts of code like make a region of a failed attempt, and explain why it failed. Also, if you develop in Visual Studio, install the “CodeMaid” extension from Extension Manager and clean your complete solution. It will remove spaces and also remove unused references making most of the things look better.

If you develop in C# then share your code with me. I can also help you with sorting things out :)

Answer 17

Put up a disclaimer that the code is provided "as is" with no promises of support, etc. And then share the code.

Case study: Turning a cloud of isolated points into a watertight surface is an extremely important practical problem, used everywhere from robotics to computer vision to processing data from 3D sensors like the Microsoft Kinect.

Poisson surface reconstruction is 7 years old and has long stopped being the state of the art for solving this problem. But everybody still uses it to this day. Why? Because the author released the code and it has since been incorporated into a bunch of popular geometry processing libraries. The paper now has over a thousand citations.

Answer 18

Yes. You should release your code, probably under the CRAPL license. The goal is to build a better future - and your lousy code will help people do that. A caveat is that you should document how to successfully operate the code well enough for someone to have a decent chance of reproducing any published results.

And, don't worry - one bit of research code I worked on had been developed by 5 postdocs of indifferent programming ability for a series of projects over the course of about 8 years.

The list of global variables (just the names) was roughly 4 pages.

Roughly one third of them were used to set default behavior to change the functionality that functioned at a given moment. Another 20% were parallel data structures - meaning that they stored approximately the same data - and therefore functions in the code pulled from the data structures more or less at random. Yes. They were sometimes out of sync. And sometimes needed to be out of sync.

There were roughly 50 undocumented versions, stored in random portions of the group's server - each of which served at least one specific purpose - and only one admin kept those specific purposes in his head. It was more common than not to have people using the 'wrong' version for a given purpose.

The use of incredibly complex recursive procedures to, eg, write a file, was standard. Seriously - a few thousand lines to save image results.

Oh, and the remains of a butchered attempt to solve a memory leak (actually an invisible figure) by never creating a new variable.

Oh, and the database, that lovely database. About half of the data was unusable owing to (a) database design errors (b) data entry errors (in automatic programs). The code to retrieve files from the database was several hundred lines of logic long... The database itself was also kind enough to contain many copies of the same data, much with broken links between tables. Constraints? No. I watched a statistician proceed from disquiet to fear to tears to quitting within a month of being entrusted with the database...

There were somewhere between 0 and 1 ways to operate the software and retrieve correct results at any given instant...

And yes, there were gotos.

Oh, and in an effort to ensure opaque and nondeterministic operation, a series of computations was performed by calling GUI buttons with associated callbacks.

Approximately 90% of any given function was, quite reliably, not relevant to the result or to debugging of the result - being composed, rather, of short-term projects inserted and then never removed. Seriously - I wrote a feature complete version that actually worked that was 1/10th the size... Significant fractions were copy-pasted inserted functions, many of which differed from each other.

And, no Virginia, there is no documentation. Or descriptive variable names.

Oh, and the undocumented, buggy, dlls and associated libraries - generated using code that no longer existed.

All written in Matlab. In terms of Matlab coding practices, assume that copious use of eval would be the highlight of your day.

Seriously, your code isn't so bad.

That said, if you've done something actually useful, it might be career-enhancing to release a cleaned-up version so that other people will use and cite your library. If you've just done something, then reproduction is probably as far as you'd be well-advised to go.