Why do many talented scientists write horrible software?

Question

I am a software engineer and I have been working with people with academic backgrounds for several years. Many times, I've noticed that (even otherwise brilliant scientists) produce code of extremely low quality (unless their background was precisely Computer Science).

Since those people are very good in doing their research - and eventually obtain remarkable results - it seems they are clever enough to write decent code. Is it just that they don't think it's worth the effort? Plain arrogance? Lack of time?

Examples

It seems to me that in academia the most popular languages are C/C++ and Python (neglecting MATLAB and other vendor-specific languages). The language where I have seen the most amazing pieces of junk is actually C++. The main points are:

• Really, really naive C++ code. They claim they chose C++ over Java/Python/whatever because "it's faster", but they new everything, even an array of 3 floats that is deallocated few lines later, where 3 is known at compile time.

• They have learned pointers from C and they use only them.

• Some of them (not most of them) have read some random blog posts about OOP and now put virtual everywhere, using abnormal levels of abstraction.

• They are convinced of pointless optimization choices.

• They lack proper memory management.

• They copy/paste massive amounts of code from project to project and within the same project as well.

And in this list I am omitting the problems with the process, rather than the product. Scientists use:

• no version control,
• no automated builds,
• no documentation,
• no software process at all (neither agile, nor traditional waterfall).

The workflow is:

I devise the algorithm, I write it as a massive 10k LOC piece of C++ and I click build somewhere.

As this assessment could be probably biased by my own experience, I have inspected some open source projects run by researchers (and maybe a few software engineers) and cited in many important papers. Virtually all of them were:

• crashing on corner cases,
• had ugly GUIs,
• and the code - in my opinion - was ready for a complete rewrite.

Answer 1

I'll have a go at this. This is mainly my personal view based on my use and implementation of academic software. Like many of the comments already mentioned, I don't think bad software is specific to or even more frequent in academia. That said, I think there are a few reasons why it occurs that are specific to the field.

Software is not a priority

In academics, the key performance indicators are all about paper publications. Software, while highly useful in my opinion, has very little value. More often than not, software implementations are side tracks or proof-of-concepts at best used to bump up the citation count (ofcourse exceptions exist).

As a software engineer, I am sure you are aware of the knowledge, time and effort required to produce quality software. Given the publish or perish mantra in academia, this time investment is often spent writing a new publication. It's a risk-reward consideration.

My personal point of view is that quality software is very important. For example, often, the first step in comparing a new algorithm with previous state-of-the-art is reimplementing the state-of-the-art due to a lack of implementation. Obviously this introduces a time delay and can cause a series of bugs. That said, I think in general little will change until software gets valued by performance indicators somehow.

Researchers are not programmers

Most researchers have little or no programming experience, though YMMV depending on the field. I think it's fair to say the majority of researchers learned to program by themselves when confronted with problems where they need it. Superficially learning a language is typically not the problem, but you need more than superficial knowledge to produce quality software (choosing data structures, design patterns, deep knowledge of the language, ...). This is a hurdle for self-taught programmers without a computer science background.

Another problem that less experienced programmers face is not realizing when refactoring is necessary. You can find countless pieces of poorly structured software. In research, this is a natural consequence of iteratively implementing while designing an algorithm which evolves into a monstrous piece of software full of hacks. That monster might still do what it was meant to, though, if you know exactly how to ask it nicely. For many researchers that's where the story ends: get the results and put the beast away forever. It often takes a serious time investment to wash the monster prior to talking it out for a walk in public. This is not always worth it.

Trends in machine learning

My field is machine learning, in which software is being valued increasingly. Examples of this trend include a growing software repository and a position paper by some big names in the field. I am very happy with this evolution, because quality software allows the field as a whole to progress faster and increases reproducibility.

A current patch to the problem is being able to publish papers about peer-reviewed software implementations. I know this is possible in machine learning and statistics. Such software is usually of higher quality.

Answer 2

In addition to @MarcClaesen's answer let me add a chemist's point of view.

• I'm a a programming-affine chemist. From my experience, that's a rare species. Maybe less rare on these sites. Though maybe not more rare than a computer scientist in a chemistry lab who implements good laboratory practice...

• One important point to keep in mind is that students (at least chemists) have no whatsoever introduction to computer programming during their studies. They may have to take an introduction to using spread sheet programs and literature database search, but that's it.
  I meet students that come to do their research practicum and theses in a subfield that is heavy on data analysis. It is extremely rare to meet a student who has had already any kind of programming experience, even though this specialization "concentrates" maths-affine students. I'm still looking for good courses at the university where I could send them to get an introduction to programming.
  I like the software carpentry concept. But note that even there is no introduction to the ideas and mindset of programming. Similarly, the introduction materials I know don't really start where the students would need to start.
  (I've looked hard for good introductions, because I have difficulties to remember how the world looked before I started programming as a teen.)

• So most non-CS scientists I know learned programming in a autodidactic and swim-or-drown way. Note that most natural scientists have a mindset that will explore a programming language just like they explore the behaviour of unknown substances or instruments. As programming languages are deterministic, it is comparably easy to get enough understanding this way to put together some script to calculate something. (I remember a colleague who had his first contact with programming language in Matlab during his Diplom thesis. Some year later, he "invented" the concept of functions.) But during your thesis, you don't have time to learn good programming practice, even if you'd like to. And afterwards you're expected to work and produce results. Learning programming languages is not impossible, but it is usually quite outside the expected scope of what you are doing. The expected scope is usually that you know your way around just enough to get through a some calculations.

• A related practical problem I see is that there are no professional programmers at hand, so no good programming practice tutoring for the students. I think this still is a blind spot in work group organization. None of the groups where I have been working so far had a professional programmer. Some groups were objectively too small to afford one (at least not unless that programmer would have been also good in the chemical/spectroscopy lab... - finding that is even more difficult than finding a chemist who has learned some basics of good programming practice).

• Most of the "monster" programs I know started their life as a tiny little script by someone who just learned enough programming to put together the first useful lines of his life. During the next 2-3 years (usual setting: PhD studies) this steadily grows, and time will always only permit to change just what is needed right now. As people are nice, they give the monster to other people who are even less of a programmer. At the point where one would say there is enough experience to step back and do a thorough refactoring based on the gained experience, the PhD is defended, the student typically moves on to completely different work and the programming project is abandoned.

• As a scientist I have to say that the software development processes you refer to are not very well applicable to much of the scientific programming I do: they require that you already have an idea of how the problem can be solved (How do you produce a deliverable [or design your architecture] if you don't have any idea how it could work?). Often, not even the outcome is known.
  From a basic research point of view when you know how it works you've reached the end of basic research. Then, applied development starts, and there I see how the software development processes can be applied but this is by definition out of the scope for basic research projects.
  The basic research part, OTOH, may be described as a trial-and-error approach to producing the first bare-bones glimpse at a deliverable.
  You may be seeing only the tip of the iceberg of scientific code where it would be (have been) justified to put in the effort of writing properly documented code with a defined interface etc. (other good practice like unit tests I'd prefer to see already with very early attempts...): possibly you don't see the huge amounts of code that are produced for research ideas that then turn out not to work that well and are abandoned.

• There is a major difference in the usage perspective between most scientific programming I see and a traditional software project. Most of that programming occurs in Master's or PhD theses. The scope of the whole thing is rather limited. It will usually be a one-person project, because a thesis by definition is a one-person project. So for the one-and-only developer the scope of possible use of the software is at most a few years or this one project, and usually also just this one developer is going to be the only user. This is radically different from "normal" software development. The contrasting basic research perspective is that even if the method should become widely used and it doesn't turn out that the idea worked just for the one problem it was invented for, the next thing that happens is that someone will improve the algorithm, possibly/usually leading to a totally different implementation. Or find out that it fits into a much more general framework, which would correspond to a totally different interface, etc. In this situation it is not even sure that carefully designing an interface for the method will pay off at all.
  I'm not saying that this couldn't or shouldn't be done with a readable implementation. But it is not the situation that encourages putting in the effort to learn how to write readable code.

• A large part of the programming I do is in scripting data analyses tailored to a specific experiment. Pragmatically, I generalize the code into reusable packages only when I either know from the beginning that I'll need it again, or when I actually encounter this situation. However, this is astonishingly seldom compared to e.g. what I encountered when working as a student as "normal" developer of a database application. However, partly this is probably because for some projects where I know that code will be reused, I set up a package/library from the beginning that I develop in parallel to the data analyses at hand. But then my perspective on that is completely different from the student scope as I expect to keep using this code base for years.

---

• One of the nicest and most astonishing (totally unexpected) experiences wrt. scientific programming I made was: I submitted a paper and released the software implementation in parallel. One of the reviewers asked how I ensure that the calculations are correct - which allowed me to answer that I use unit tests, and the package actually contains about twice as much code for the testing than for calculations. This was unexpected because it is already quite unusual in my field to release the code with the paper - but I had not seen before any paper explaining which automatic tests are provided for the implementation - so I hadn't expected that this information could make it into the actual paper.
  I take this as an extremely promising sign!

• Another very promising sign is that when I explain to my collaborators* the concept of version control systems many like the idea and describe it as something they thought should exist but hadn't known actually exists outside the scope of Word change tracking. (Though for the research workflow I still think the VCS I know (svn, git) work as well as for pure coding projects.)
  update a few years later: getting non-programmer colleagues to use version control is still very much of an uphill discussion (also because VCS dealing with binary files did not improve as fast as I was hoping). I mostly went back one step and we now use nextcloud for sharing data, which while it does not provide real version control, it at least facilitates everyone talking about the same state of the data/files.

* also the not-at-all-programming ones who feed the measured data into the system, or who do e.g. the medical/biological interpretation.

---

update: From the experience I gained since I first wrote that answer, I'd now put the organizational aspects much further up:

I've mostly left academia, I'm freelancing now but still have research projects where I'm in a subcontractor role for (academic) research institutions. For a certain pre-processing procedure I was commissined to supply, I met flat-out refusal to pay for "unneccessary stuff" like unit tests and encapsulating the working code in a library/package with its own namespace. The method is a data analysis procedure, so the most important type of bugs are errors in programming logics (against which unit tests can provide a certain level of guarding). It is to be used in R, i.e. in interactive work scenarios which implies high risk of messing up the user's workspace if third-party functionality is not encapsulated in its own namespace.
That refusal came from the upper management level of the research institute.

@gerrit comments that university burocracy may not allow a group to have a professional programmer. I think this is likely true as a day-to-day reality. However, I also think it is related to this organizational blindness I'm talking about here. If upper management in academia did see the importance of using state-of-the-art working techniques in data treatment and software development, university administration would likely have a different view on this as well. And if grant proposals did include professional data and software management topics, things would maybe improve on that level as well. I think academia is in somewhat of a vicious cycle here: everyone but students are considered very expensive, so project proposals don't dare including any "technical" staff. If the topic of data management or software development was brought up, our academic management did never consider anything but whether "we could have a CS PhD student" which is not a good fit: the projects need well-established and reliable working approaches rather than anything that would be considered sufficiently new to count as CS research so that the student could earn their PhD.
And of course, upper management may be convinced by their colleagues or by examples of how much professional treatment of these aspects helps, but as long as they are not convinced, it will be extremely difficult to find money and permission to try out whether and how much it helps.

Answer 3

There are some great answers out there and I would like to throw in my two cents on the subject as it is a very relevant issue I often think about, or discuss with my colleagues. There will inevitably be some overlaps with parts of existing answers, I only hope that I can give a slightly different perspective in those cases.

---

I have done applied maths as my major and did my masters in biological & medical modeling (whatever that means). I am past half-time on my PhD studies in bioinformatics and systems biology. I almost exclusively work in silico and have come to sire many of those monstrous, ugly and sad pieces of software.

First off I think you are making a small but important mistake in framing your question. You say:

"Why do many talented scientists write horrible software?"

I would instead suggest

"Why do software written by talented scientists end up being horrible?"

The difference is subtle but essential for the rest of my answer. After all it's not like scientists gather around a table and decide to write horrible software.

Many scientists who write code are not educated to write software

There is a serious difference between knowing how to code versus knowing how to write software. I did almost as many courses in the CS dept as I did in maths, during my undergrad and masters, so I felt pretty confident with my programming skills. That is until I was faced with questions like packaging, dependency management, lifecycles, licensing etc. None of these were remotely within the curriculum during my studies. I don't know if those who do CS as undergrads learn these concepts, but I sure as hell never needed to until I all of a sudden had to know them.

Bosses/supervisors of many scientists who write code are not educated on writing software

Not only do you need to learn a bunch of new stuff, but imagine you cannot explain why that is important for you to learn that stuff to your boss. I have this issue pretty often, as writing code is often held comparable to doing labwork at our department. People think writing code just happens on its own and preferably quickly. I have often had discussions with colleagues where they jokingly mentioned that all they want to hear from me is "computer say yes/no?" How long something new might take is very often underrated, having to write tests continuously is typically seen as a waste of time. Which brings me to my next point....

Good software is not valued in academia, at least not in the same way as industry

The measure of competency in academia is publications, and the form of currency is citations. You are constantly in a form of competition to come up with something new and useful, and only the first one out there will get the prize. Clones do not exist or survive particularly long in academia. In contrast, in industry you can win market shares by better advertising, cooler GUI or lower price. In academia, if some method is already published, you need to do something else.

Similarly, if you have already published a method then additional features, clean-up, optimization etc of that proof-of-principle software is often not good enough to warrant a new publication, which practically means that you have wasted months of work for nothing. Sad but true...

Expectations change, you got to expect the unexpected

Might be a small point but I can't stress it enough as it has come to bite me in the back over and over again. You simply don't get proper specifications for a new project. They are often either all too vague, or way too strict (unrealistically so). At times, something that wasn't ever mentioned turns out to be implicitly expected. Then to add insult to injury the specifications change based on a new data format, some other database, new features or just that other cool thing the boss was thinking about when he was away on a conference... You write and rewrite solutions to the same problem, it becomes a clutter.

You typically don't get the support you need

The few programming PhD students at my dept we try to improve ourselves by keeping up to date with the trends. Learning best practises for instance via SO. But more often than not when you want to try something new you see hinders; either the IT dept thinks you are too much of a nuisance, or the boss thinks you are slacking off, or the people you are asking help from think that you don't know sh*t and you are wasting their time. For instance it's taken me several months of negotiations and mailing back and forth in order to be able to access our version control server from home. Eventually it just works faster to skip certain best practices.

The newest coolest CS trends aren't always well documented for people who are not experts

I have tried to get my hands dirty with several "new" technologies, which often have some steep learning curve. Sometimes it's really not worth the effort. Best example I have is Maven. As I often work in Java, I thought I should use modern tools for packaging and dependency management. But my conclusion, after having battled with it for so long, is @%&$ it! I really don't have the energy or time to go through that mess of a documentation.

Bottomline

After giving myself grief over these in the past years, I came up with the following conclusion which gave me some inner peace:

"I am not a software developer. I am neither educated nor paid to write software. Writing software is not my job; learning to solve certain problems is."

Hope this answer gives you some insights as to why software written by scientists (exceptionally talented or otherwise) often don't live up to the standards established by software developers.

Answer 4

Just to supplement great answers by Marc Claesen and cbeleites. In academia, when people write code, it is common that:

• Problems are often open-ended - so perhaps the data structure people start with will be used later for something else, for which it is suboptimal. Also, many things are not properly designed, because everything changes. Compare it to writing a typical commercial software, where things are specified from the beginning and usually far from cutting-edge (even if demanding, not "the first time").
• Maintainability is not a requirement - a small piece of software, not used later, with no-one else going to take over the code (and it is way easier to understand one's own code than code by others - even if it is chaotic, you know what is where). Compare it with situation, where after the author leaves someone else is going to look after the code (or one need to constantly consider hiring one more developer to speed up the progress).
• People work alone or on legacy code - very opposite situations, but giving similar results. In the first case (as above), people can understand their chaotic code; in the second (e.g. modifying pieces in old Fortran code) - people have to adopt in making small, often - not unanticipated, changes, adopting to the existing code base.

Personally (coming from pure academic background), I've learnt most o good coding practices, when collaborating with others:

• by learning from others - some coding practices does not require a lot of brainpower, but a lot of experience - the wisdom telling that a given "smart solution" will become problematic to maintain in longer run (as most of kludges (= ugly hacks)),
• by collaborating - many times I realized that what was reasonably clear for me, was a totally unintelligible cthulhu fhtagn (yet powerful) for others (and the other way as well - a nice code for someone else was a challenging riddle for me),
• all in all, many good practices are in fact going to the least common denominator of skill (and not-smart people are already close it); clever code by one will be difficult for others.

And a dessert - the curse of the gifted, a comment to the last point:

You are a brilliant implementor, more able than me and possibly (I say this after consideration, and in all seriousness) the best one in the Unix tradition since Ken Thompson himself. As a consequence, you suffer the curse of the gifted programmer -- you lean on your ability so much that you've never learned to value certain kinds of coding self-discipline and design craftsmanship that lesser mortals must develop in order to handle the kind of problem complexity you eat for breakfast.

(Source: http://lwn.net/2000/0824/a/esr-sharing.php3; or abbreviated: http://www.linuxtoday.com/infrastructure/2000082800620OPCYKN)

And it was addressed by Eric S. Raymond to Linus Torvalds...

And as a side note (as programming is more and more prevalent), now scientists realize that good practices and workflows are important, see e.g. http://software-carpentry.org/.

Answer 5

Why didn't Roald Amundsen pave a highway to the South Pole?

Why didn't Edmund Hillary build a ski lift on his way up Mount Everest?

The job of academics is to find solutions to problems previously thought impossible, to teach others (and despite the boilerplate in their grant proposals, this target audience is other researchers) how to solve the problem, and to do the above as efficiently as possible.

Academics care about the quality of their code only to the extent that it works "well enough" as a proof of concept of their ideas, and, possibly, that it can be reused in future projects. Refactoring code, writing documentation, carefully error checking, setting up automated builds, etc. is a waste of time unless the time invested improving the software saves them at least as much time generating working results. For the four items I've listed, this is almost never the case.

To be sure, when their research turns out to be practically important, many researchers will go back and write well-engineered implementations of their earlier algorithms (usually as part of a consulting agreement with professional software developers), and they had the training and talent to do so earlier -- it just wasn't worth the time.

Answer 6

Even being a professional software developer, could you develop that you call a good software when requirements vary in unpredictable ways on a weekly basis? This is the world where researchers live an survive.

Doing scientific research means going into unexplored areas. Researchers do not know which features they may need from the monster tomorrow morning. This depends on the results they obtain today midnight. A scientific program accumulates too many iterations, adding features that nobody though would ever be needed.

Any attempts to leave room for new features, add modularity, often even make the things worse when these "generic approaches" must be later hacked around to make further alterations much more drastic than it was expected (and supported) by the "generic framework".

As a result, a program that evolves during research process directly is often only usable as a prototype and must be rewritten before releasing it as commercial or also FOSS software. Professional programmer, if hired, could probably do somewhat better but instability of requirements most likely prevent "arrival" to the really great final design anyway.

Answer 7

In many aspects writing software is an art. Many great painters haven't been born like that, they actually learned the art in years of training and with many bad results in between.

Take a sheet of paper and try to draw anyone you know. Even if you have a picture in front of you, it will most likely look comical. Now a real artist would ask you why you didn't see the shadows, couldn't come up with the right perspective, or put the ears at the completely wrong position even though you had a clear picture in front of you. It is however not your arrogance that caused that, it is your lack of experience in looking at the model in the right way. Technically it has something to do with you using the wrong half of your brain, and you can be trained to change that. Just not till tomorrow.

Scientists work theory-based. They have a theory, they want to prove it, they write code focusing only on the actual theory at hand. That is you seeing a nose, but not the shadows around it. If you'd teach them for month or maybe years to use the right techniques and "strokes" to do it the right way, they might change. However you should ask yourself if that is worth the time. Sometimes it is, but sometimes scientists should just stick to scientific stuff as much as painters should not suddenly start radioactive chemistry out of the blue.

If you decide on teaching them, keep the idea of the artist who starts on his first day in mind: it is not their arrogance, it is their lack of knowledge on how to use the right parts of the brain. There is a reason why "Software Development" is a 3-year apprenticeship, after which you are considered "beginner" level.

Answer 8

Great discussion, and I have thought the same many times. There is another kind of software as well, which isn't really mentioned above - programs developed within academia which aren't part of a research project. There are lots of software development projects that are more focused on logistics, teaching etc (clickers, video capturing, intranets, library software etc). Sometimes these are handled professionally and become collaborative projects etc, but very often they're the result of someone having some graduate assistants that know a bit of programming, who code up something that perhaps works - but of course has no documentation, testing, version control, requirements documentation etc - and when they graduate and move on, good luck for anyone who wants to try to maintain it... Part of this is also the "false economy" of academia, where certain kinds of labor are extremely cheap/free (for the people who benefit from it).

I am personally probably responsible for some crappy software as well, as a PhD student in computer-supported learning. I'm hopefully a bit more aware of best practices, version control etc than many, but I've never had any professional training, and what's perhaps more important, I've never been part of a community of practice, mentored by better researchers etc. In education, it's in a way even more difficult, because there are very few technically inclined students/professors. I of course use SO, mailing lists etc very extensively, but I am sure my code could be massively improved if I had a senior colleague down the hallway who was reviewing my code and providing feedback, etc.

In fact, one of the comments above made me think of this -- it's quite common for universities to have statistical consultants that are to some extent available to researchers. (We have someone in our library whom we can access for free for one or two hours, and then we have to pay a fee, but I know a lot of people take advantage of it, and found it very helpfull sitting down with them and going through their research design, their assumptions, the statistical design etc). It would be an interesting concept to have a "software development consultant" (not sure about the title), who would basically be a professional code reviewer... But could also extend to help people think through their needs, figure out useful frameworks or libraries, navigate version control, open source licenses etc.

And of course, changing the incentive system to rewarding releasing (or improving upon!) high quality code, is incredibly important, but very difficult. I think Mozilla's Academic Code Review exercise is a really interesting experiment in this regard.

Back to writing Python scripts to parse MOOC clicklogs :)

Answer 9

Because programming is like riding a car: Everyone needs it but most of us are not professionals. How one learn programming? Generally buys/lends a book "Python or whatever for beginners". It will be all about how to save a file, how to run a program and how to call a function. After this, I can do most I need urgently now or yesterday.

Where I will learn about design patterns, good software dev practices, agile development, how to write nice code? NOWHERE! Just like after taking a course driving a car I believe I am OK, and I do not read 5 hours a day about how to drive on wet road faster, I don't go to bookshops and read every single CS books, that it may be related to me. Even if I go to the net, maybe Software Carpentry is the only relevant resourse! Seriously, if anyone knows anything similar, please, post is somewhere here!

I will not moonlight to learn a full CS in MIT courseware to hack together my "Hello word" of the day. And I wouldn't be surprised if even CS guys in MIT would learn half of good programming practices outside the school, on job, and not after 4-5 years sitting in school.

Answer 10

summary: replication or lack thereof.

details:

My observations (unfortunately, small n and a single POV) are that the main reason both non- and talented scientists write horrible software is simply "the opposite of replication." They see no value in reproducible research, they don't anticipate their work will be replicated, and they certainly don't desire that their work be replicated. (They just want their work to be cited :-)

I'm a BSCS who did time "in industry," including one well-known faceless acronym. All the coders I knew at least used and valued open-source software, and many contributed (esp @ my last straight-up code gig). OSS is only valued to the extent it is used and extended. (AFAICS--am I missing something? exotic languages that are studied but not used?) Of course, a given OSS is only used if it's robust, tested/testable, well-documented, etc (and only extended if it's public).

Now I've gone back to school as an environmental modeler (mostly atmospheric). The folks with whom I've worked mostly don't even put their code in a public repositories (even the ones much younger than I--this is not a generational issue AFAICS), much less create documentation, modularity, comments (whether in code or in commits), and the other affordances one expects in OSS. This appears to be due (based solely on conversation--not strong empirical data) to their assumption (and, usually, hope) that their code will not only never be used by anyone else (and indeed probably never be used again by the coder), but never even be seen.

Unfortunately I didn't understand this when I "hired on" as a grad student. (I knew almost nothing about graduate academia--I had just "jumped out a window" at the faceless acronym and only knew what I wanted to work on--and was esp clueless regarding the cultural differences between informatics ("computer science" being famously misnamed) and "hard science.") I chose as my advisor the professor whose area of work most interested me. Once I started looking at his code, my vomiting was projectile. I tried to engage him about it, and his attitude was approximately (i.e., not a quote) "papers matter, code does not." He never submitted code with papers or made code public, and used almost exclusively a fairly obscure, {proprietary, expensive} {language, development environment} (as, to be fair, do many of his colleagues, some of whom are Very Big Names in our smallish field). Having some philosophy-of-science background, and knowing that the models on which we work have real public-policy implications (e.g., serious spending), I asked how one might reproduce his results. He said (and this is a quote) "they hafta trust us--we're scientists." He is no longer my advisor ...

While I suspect (again, on small n) the observations above do "measure central tendency," all is not darkness and void :-) In my own field, there is exemplary software like GEOS-Chem. Unfortunately, GEOS-Chem "is what it is" largely (IMHO) due to the GEOS-Chem Support Team, which provides a sort of infrastructure astonishingly rare in my field. Hence I suspect GEOS-Chem is, were software quality being measured over this domain with high coverage (am I missing something?), probably 2-4σ better than the mean.

Answer 11

The short answer to this question is that research scientists are (mostly) not software programmers (although they do publish software from time to time).

I work in a computationally heavy field, which means that a lot of the miscellaneous stuff that I have worked on in the past can be packaged into a software. From my own experience, here are some obstacles that are keeping me from writing a full-fledged software based on my work:

1. A lot of my research involves open-ended investigations, hence the code is also written for that purpose. To write a software implies that I have an essential understanding of everything I have researched so far, which is not the case (and will not be the case in the foreseeable future).

2. Each of the things I have worked on is too small individually to be written as a software. To expand the scope of the things requires research, not software.

3. A lot of the research is outside of my control. There will always be new opportunities and directions for research, which means some research (and the code written along with it) will be aborted, this goes back to the points made above.

4. Most of the time I write MATLAB code on Windows OS. Even if I don't switch my OS (to say Linux), my options are to translate MATLAB code into some .NET language or to export it as a C/C++ object. I might be wrong but I think software development in either language is hard and time-consuming.

5. The reason most research engineers write code using MATLAB is that the turn-around time in research has a high variance. Most of the times, things need to get done on a weekly basis, and this may involve very novel experiments. When it gets busy, the turn-around time might be within the day. These experiments are optimally done using a very stable platform such as MATLAB or Mathematica, whereas, for some other languages, your code can't compile if you accidentally insert a tab somewhere or misses a colon. Again, this fuels the things I have mentioned in the points above and further deteriorates proper software development skills, even though you are writing code.

6. A commenter mentioned that software development is doing great in machine learning. From my perspective, the reason for that is because, in fields such as optimization, machine learning, signal processing, A. things are visualizable, B. things have been studied since the 1950s, C. a lot of people are trying to get into this field, D. a lot of things are honestly easy, computationally speaking and many times an inexact solution (that works in very special cases) suffices. Unfortunately, most of us do not work in such convenient fields.

7. Frankly, we all have lives beyond research. Software development could require personal commitment or collaboration that lasts longer than the entire duration of a research project.

Answer 12

I don't think I'm a talented researcher, but I have done research in software and which actually produced a piece of software. For that piece, I also collaborated with a bunch of folks at a major tech company. Obviously, a lot was different between our approach to software.

• When I wrote the software, I chose the easiest path to demonstrate that my ideas should be correct. I did not spend a lot of time engineering the software, because I did not have time! I tried hard to get it to work, and I did try it to make it readable and hackable (because I know someone else will take over eventually), but I did not spend a whole lot of time engineering it. I was mainly interested in getting it to work so that I can do stuff with it (and show that my ideas are correct!)

• That was actually the good part. In my research, we used a research software library created by another research group in another country. They were actually not aware that people were using their software! As a result, they checked in changes which resulted in the software not to build. Moreover, the code was hard to read and we can't fix it ourselves (it was C++, so error messages aren't that helpful either). We had to contact them personally to get that fixed.

• So, can academics write good software? Yes, at least many could (or they wouldn't be able to teach programming courses). In fact, there was a professor of mine who was a phenomenal programming instructor, but whose personally-written code aren't that pleasant to read. Academics simply do not have time to hack around and make their code good-looking. If they've had more time to improve their software, I am sure they would.

• OTOH, the folks at the tech company were actually interested in producing software they could use (and, by way of that, produce some research papers perhaps). They followed their coding standards. They engineered it deeply. They used build management, integration tests, and coverage tests. They do it because (1) they have time and money, and they're paid to do it, and (2) they're going to actually use it!

Answer 13

Qualities that make for a good scientist don't always make for a good software programmer (except, as the OP pointed out, when the "science" happens to be computer science.

Computer coding is a highly precise art. Many people, including good scientists aren't sufficiently precise to write good code easily. This is particularly true of the more "intuitive" types of scientists.

Many scientists find computer programming "boring" and for this reason, don't do well at it. It's true that "regular" science requires detail work, but not to the degree of programming, which many (including yours truly) find "mind numbing."

Basically, if there is zero (or a very weak) correlation between a good scientist and a good programmer, you will get the whole gamut of programming ability, good, middling, and terrible from a population of scientists.

Answer 14

There are three main reasons.

One is that scientists are not professional software developers. That's true even for computer scientists, and more so for mathematicians, physicists, chemists, biologists, social scientists and so on. Not that they couldn't, most people who are reasonably clever could become professional software developers if they wanted, but most are not.

Two is that scientists are not interested in creating whatever would be the opposite of "horrible software". They are usually only interested in the results. Where this is bad is if their software contains bugs that produce results that are wrong, but close enough to the truth to seem plausible. Fortunately, many bugs will produce results that are obviously wrong. It is also bad if the software is confusing enough that nobody can declare for sure whether it is correct or not, but to my knowledge there are not many complaints about that.

And three is that scientists are often under time pressure. They might write software quickly that they know should be improved, and they might even know how to improve it, but they just don't have the time.

One thing that I really, really hope is not the reason is that some people think anything they understand must be simple and anything they don't understand must be hard. With these assumptions, any scientist writing software that nobody can understand would be assumed to be a genius, while anyone writing software that is easy to understand would be not very impressive at all. So what a professional software developer does, making software that is easy to understand, would be damaging your career in the view of these people. I really hope this is not what happens, but I wouldn't be surprised.