Is it ethical to use proprietary (closed-source) software for scientific computation?

Question

I have data from an ordinary scientific experiment. Now, in order to analyse the data, the most convenient way for me is to use some proprietary, closed-source software.

Is it ethical to use such kind of software in research?

If do use it then this part will be a big black-box in my research. So if someone cannot reproduce my data analysis with the same or with some other software, then it depends on the mercy of the software company if we can find out what went wrong. If the software company does not want to cooperate, then what went wrong might never be found out. I have the feeling that this opposes the principles of science.

Answer 1

I do not think that this is an ethical question in the first place. Reproducibility is not harmed by the requirement that money needs to be spent on buying software or setting up an experiment (or are there open-source particle accelerators?).

While open source solutions are preferable for many reasons, there are clearly cases where using some proprietary software P is appropriate:

• When it is known (and has been verified) that P's implementation of method M is correct while the known open-source implementations of M have not (yet) been established as reliable. Conversely, when open-source implementation O is known to give correct results whereas for P this has not been established, then clearly O should be used when there is a choice.
• When one can reasonably expect that P is widely available in research institutions (and thus to researchers), like for example certain mathematical / computational packages.
• When P allows for a much more (time-/resource-)efficient execution of experiments.
• When P is a software package needed anyway to extract / process data from a big-ticket instrument.

Note that I am assuming that P can actually be bought by anyone. If P is not commercially available but rather a closed-source in-house solution then an open-source package is preferable.

Answer 2

This question has become an important one as the push for greater reproducibility in computational research grows. Use of closed-source software is an acceptable part of research in most fields. Nevertheless, the following viewpoint enunciated by John Claerbout is becoming more widespread:

An article about a computational result is advertising, not scholarship. The actual scholarship is the full software environment, code and data, that produced the result.

From this point of view, if you rely on closed-source code, you can't completely publish your research.

Issues with closed source software in science

There are multiple concerns:

• As you mention, if someone else tries to reproduce your result and gets a different answer, it may be impossible to resolve the discrepancy.
• The software implementation may have bugs. Indeed, all software has bugs and any open source software you use will be susceptible to this too. But with closed source software you do not have the right to inspect the code to find bugs yourself, nor can you fix them yourself if you discover them.

Both of these are matters of principle; in practice, resolving discrepancies and finding/fixing relevant bugs is also a major challenge with open source software (but is at least something you can conceivably do).

Suggestions for mitigating the impact of closed-source code on reproducibility

1. Publish all of your own code. You probably have some scripts that call the proprietary software, or at least some input files that set up the problem for it to solve. Usually, these scripts contain most of what is new in your research, so what is in them is often more important to other researchers than what is in the proprietary source.

2. Use multiple independent approaches to confirm your findings. This reduces the likelihood of a bug affecting your results. In my own research, this extra work has at times paid off by uncovering bugs (that led to erroneous results) in proprietary software.

3. Understand the algorithms encoded in the software, and their weaknesses. Even if you can't see the source, you usually can learn what computational methods they contain. Once you know the circumstances under which those methods are (or are not) reliable, you can check whether they are likely to give trustworthy results for your problem (e.g., by checking the condition number of a matrix that you feed into some linear algebra software).

4. Be aware that some commercial software is just a pretty interface to open-source code. For instance, MATLAB's eigs function for computing eigenvalues of sparse matrices is just a thin wrapper around the open source package ARPACK. If you use that, you're not actually relying on much (if any) closed-source code.

Of course, you should also follow best practices outlined in the literature (e.g., in this article, or the many references linked here) on reproducible computational science.

Answer 3

Law is concerned with what you do; ethics is more about what is a better choice and why it is better. The answer to your question depends on why you use the proprietary software.

• If this is the only way of achieving the result, then of course you can and should use it. It does not matter if the code is proprietary — if this is the only way to solve the problem, then everyone is possible either using it, or looking to use it. However, I am not sure such a software exists, so maybe this is a model example.
• If you can choose between a proprietary software W, and free open-source software L, then the choice is yours to make. If your methods are well documented, your peers can either reproduce them in W (if they have it), or re-create an algorithm in software L (this happens a lot). There are some benefits of using W and some of using L. For academics as a large community it would be better if everyone uses the same open-source software and contributes to its development. However, your personal goals may be different, and it is not unethical to put them first. You can choose W because it is a better written software, it runs faster, it looks better, and you are more productive with it. This is fine.
• However, if you chose W because you are paid by a company M which produces it, and you don't tell this, but instead advertise W in your papers and encourages everyone to buy it, then yes — this is unethical.
• If you choose W because you know that your competitors, who are likely to review your paper, are from a third-world country with no access to W, and you don't particularly like W but use it to hide some dirty statistical voodoo in the code no-one can reproduce — yes, this is unethical.

Answer 4

Note that "Proprietary" and "Closed Source" are not the same thing. For example, MATLAB (a proprietary commercial software) actually ships with readable code for most functions it implements. You're not allowed to reuse this code (hence "Closed Source"), but you can inspect it if you have calculation discrepancies that have to be explained.

Even when the source code is not available, it is usually documented which algorithm is implemented by a given closed-source function. So if somebody cannot reproduce your results using closed-source software, he can find open-source implementations of the algorithm he suspects are incorrect in the software you have used and compare results using those implementations. Sure, it's time consuming, but don't expect that using open source software will somehow make your results 100% reproducible with no effort at all. Your colleagues will still need to run the exact software version on the exact computer environment to be sure they will get exactly the same results, and those versions and environments will likely be unavailable in 10-20 years. After that, the same time consuming process will apply to open source software as well.

I'm assuming here that you don't use software packages which keep secret the algorithms used to implement it. You'll certainly want to avoid using those, not only for the sake of reproducible results, but because results obtained by unknown means don't have any scientific value.

Answer 5

I'll add one additional perspective here - if you manage to get a stunning result based on analysis done in a closed source package for which there is either no open-source implementation available, or for which the same is poorly developed, then you immediately provide a strong incentive for developers in the scientific community without access to that package, but with a desire to reproduce the results, to go out and start working on improving open-source alternatives to said commercial software.

This is a two-fold benefit. In the first case, you are helping to encourage development of new open-source tools and, secondly, the parallel development of such tools help to provide mutual checks for both. I'd say use what works, what is most appropriate, and whatever helps you get the job done best within your budget. If you're not in a position to contribute better open-source tools to the community then let others take up that torch. Producing useful, unique, and interesting scientific results is exactly the sort of thing that stimulates that type of development in the first place.

In a way, using a closed source package can actually be a catalyst for open source development - so you're still helping in an indirect way. If there are, however, open source alternatives to the closed source package you have used then it should be trivial for others to reproduce your results with their own implementations with free tools and there is, therefore, no ethical problem at all. If anything, it provides an opportunity to test the same result with two different tools and this is always better, in science, than reproducing the result using the same tool. You get to check both the tools (against each other) and the science at the same time. Bonus.

Answer 6

If you write "results were processed with the software X v X.X using the following settings and the printed output was", this stops being science and becomes more like a magic. The algorithms must be fully documented somewhere and understandable for the research to be the science.

It is fully possible to satisfy this requirement for the proprietary software, if the algorithms are properly described in the user documentation, or the proprietary software just makes the platform of execution and algorithms themselves are coded on the top of it (Matlab, proprietary C compiler, etc). Ideally, also proprietary code should be available for review under restrictive license.

Unfortunately very often the proprietary tools tend to hide the exact algorithm details, keep the important parts of the code secret, and due that are disliked by the scientific community. Nobody wants to publish an article just to discover later results are only reproducible by the single version of the single proprietary tool, most likely due trivial bug.

Answer 7

Yes, it is unethical^*.

There are four aspects of the software you mention which raise ethical problems: Being closed-source, being proprietary and two aspects which you didn't mention but are characteristic of closed-source proprietary software - requiring (non-neglibile) payment and requiring agreeing to a restrictive license in order to use.

Issues with closed-source software:

• Its use hampers reproducibility (see @DaveKetcheson's answer)
• Its use hampers resolution of contradicting research findings/claims).
• Use of closed-source software hampers the development of software - the specific kind of application you're using, and in general. IMHO.
• It puts software vendors in a position of power over ourselves, and thus indrectly over our research and over the community interested in our research. Famous examples include government espionage, inter-state sabotage and hardware tie-in.

Issues with proprietary software (and its typical licenses):

• It creates artificial restrictions on our freedoms as researchers and users:
  • To run the software any way and any where that's beneficial to our research;
  • To apply lessons learned from studying the program in our own research work, and to possibly modify it to better fit our research (it might be technically possible with source access but legally forbidden);
  • To distribute copies for reproduction and further work to researchers within our group/institution and to others;
  • To distribute copies of our modifications and adaptations within our research groups and to others;
• It often effects similar power-relations between vendor and users even without the source being closed. For example, a vendor affect hardware tie-in by forbidding legal installation on unauthorized hardware.

Issues with software requiring payment for use:

• It creates an undue burden on other researchers who wish to (1) evaluate/review (2) illustrate limitations/flaws or (3) extend and improve your results.
• If increases the pressure on other researchers to buy similar software in order to keep up - and this is an indirect effect on all researchers in your field.
• If decreases the impetus and availability of funding for developing free ( = gratis, libre) software for doing the same.

(*) ... but with all that being said - these unethical aspects weigh against potential public benefit in obtaining and publishing research results. Make sure you fairly stack both sides of the scale when making your decision - and that you think about others as well as yourself.

Answer 8

Decide for yourself:

I can argue from my own experience that using closed-source software leads to shutting a considerable number of potential researchers out (or forcing them into getting illegal copies on a grey market). I worked for a number of years in an area where Matlab is a standard tool (still is, after 20 years :-(), and then I accepted a postdoc at a place which did not have Matlab licences (not all CS Departments have it even in good universities in Germany, as they might not have a campus-wide licence, and while at Engineering they surely have Matlab, it's quite questionable at CS).

I basically had to put on hold every ongoing Matlab-related project (as buying a sufficiently complete Matlab license on a postdoc salary was not an option...)

(And I have once basically stolen a Mathematica license at a department I was visiting, for the same reason. :-))

Another, even harder, case, is Magma (less well-known than Matlab, but still almost standard in areas related to algebraic number theory and group theory, and more), which makes it extremely hard to get a license for anyone outside an established circle of select universities, unless you are in the USA (which perhaps has to do with their biggest customer being NSA :-)).

EDIT: I am told that I am arguing for double standard (mentioning expensive tools for experiments), but I am not. Science software is often much close to scientific publications than it is to scientific tools. In case of a free alternative to an expensive tool, not choosing a free alternative is akin to choosing an expensive and exclusive publisher for your work, rather than publishing on the web.

EDIT2: Also, issues of reliability of closed-source research software are well-documented in the literature; perhaps the most well-known at the moment is the story of a bug in a determinant computation in Mathematica. It took the publication in such a high-profile source like Notices of the AMS for Wolfram Ic to fix it. With open-source normally fixing of such a bug happens much faster, or at least it would be publicly visible on a bug tracker.

Answer 9

I think we need to go to the foundation of the question; the answer to ethical questions begins with morals (the science of right and wrong). Specifically, we should ask ourselves... * are we committing any deception of anyone involved? (e.g. have we disclosed everything related to the research, including all potential conflicts of interest?) Put yourself in the position of others: what would I want someone else to do if faced with my situation, and I had a stake in the research? * are we appropriating anything which does not belong to us?

If we can't identify a specific moral objection, we proceed to the field-specific ethics, namely: * is there a field-specific code of ethics one has committed to, and would be violating? (E.g. the Hippocratic oath, in medicine, lawyer-client privilege, etc.) Have you signed an agreement with your employer?

Finally, we should proceed to "value free" analyses: cost/benefit, relative practicality and availability of alternatives, reasonably expected outcomes.

Note that to place this last category in the highest position is to ignore ethics altogether, and run the risk of justifying questionable means with good ends--always a danger sign.

I hope this helps!

12/08/16 EDIT: I should add that according to the criteria above, and the information and opinions I have seen so far (including this thread), the answer is no, it is not unethical, per se (i.e., in itself).

There are circumstances in which it might become unethical, but that is different. For example, if you chose the closed source software because that choice was to your personal financial advantage, that would clearly be unethical. Or if you knew you could exploit a feature or bug in that software to disadvantage a scientific competitor, or obtain a more favorable result . In other words, motives count heavily.

As an example, someone above mentioned a "marketing" motivation. That would clearly diminish people's acceptance/trust in your results if that were disclosed.

As in politics, sunshine is the best disinfectant. That is, disclose everything that you can reasonably expect would affect the acceptance of your results by non-trusting auditors.

Answer 10

"Free" does not always mean "free" when considering time and effort. Research can be expensive. If someone can buy a package, and that package comes with good support and superb documentation that might save me time in a research project, it would be difficult to believe that it would be cheaper to use open source if it would take more time. In fact, I would go so far as to say that if I didn't spend $2K to speed up my work, leaving my funding agency paying the same amount for less output, that's more ethically questionable than using the proprietary package. (Assuming, of course, that the open source is less convenient, and that the closed source is available to all who can pay -- and yes, I know that those aren't the only situations out there, so please don't bother pointing out exceptions where that's not true).

I run Matlab, and I run Octave. They are similar, but I have a more trouble-free experience running Matlab, so that's what I tend to double click on.

... and yes, I use PLENTY of open source software, where it makes my life easier. I also ONLY recommend open source to students (... except Matlab, which has generous student license), because every time I mention an expensive package to a student, they show up with a pirated version the next day, and I don't want to contribute to or encourage intellectual property theft.

Answer 11

I don't think this is an issue. Many research biologists use Photoshop to process their images, for example. I suppose open-source software is preferred by many since it is more in line with the principles of open research, but making sure your methodology is sound and reproducible is far more important in almost all situations.

Answer 12

It should not matter. Most but not all scientific calculators calculate sines the same way. Do you know how your calculator calculates sines?

Some free software is simply inferior. A specific example is OpenOffice. As wonderful as it is it fails at some tasks that Microsoft Office handles with aplomb and doesn't even attempt others.

What if I use a Hewlett-Packard gas chromotograph because it's the only one I've ever used and someone else gets different results using another brand? Am I responsible?