How can a developer of fundamental software components best get the credit they deserve?

Question

Developing useful software products is becoming an increasingly important research activity in many fields. However, it seems the proper credit/incentive system has not been fully worked out.

The situation becomes particularly problematic when large and complicated software products depends on many individual components developed by different research teams.

Consider a situation in which a researcher spend years of effort to develop a software component C that solves a very specific but very difficult fundamental problem. This researcher then publishes a paper on C.

Another group then develops a more general software B in a few weeks, which is a thin wrapper for C. It basically reduces a more general class of problems to specialized problems that C can solve and then feed to C. With everything being open source, B can directly incorporate C into itself in various ways. Let's assume this is done legally, and the README file in B contains flattering acknowledgement to C. This group then publishes a paper on B, which properly cites the original paper on C. (Best possible situation)

Fast forward a few years. Since C solves only very specific problems, no one uses it directly (other than B). The more general class of problems that B can solve (thanks to C) happened to become a hot topic. And the citation tally is now:

• C: 1 citation (just the paper on B);
• B: 1200 citations.

This seems like a terrible situation for the author of C. And this is not purely hypothetical. I have seen this happen to several people already.

More broadly, this credit/incentive structure may be bad for a field. In particular, any one in tenure track position probably shouldn't be developing software for solving specific but fundamental problem (even though they are likely in a career stage in which they are most capable of doing exactly that).

My question is, taking what I described as given (I know it happens, so there's no point explaining why such situation does not exist), how original developer of fundamental software components could position themselves (other than keeping things closed) to get proper credit?

More broadly, how can a field setup proper structure to incentivize the development of such software?

Answer 1

Create a culture where evaluation of researchers is done by expert judgement rather than crude benchmarks.

Answer 2

One thing is getting journals (and referees) better at referencing software dependencies. I've seen similar situations where large code B depends on some code C, everyone cites B but forgets to cite C. The only way is start yourself, making sure sure to cite everything, then when you referee papers, if they only cite B then request they cite C as well.

At the end of the day people should be citing the relevant literature, when they don't then your job as a referee is to request they do. Also when giving talks make sure to be more explicit that while you used B, that it is based on C to remind people about the link.

Answer 3

how original developer of fundamental software components could position themselves (other than keeping things closed) to get proper credit?

This is a very subjective question, since "proper" will be defined differently by different people. In the original Q OP mentions citations.

But citations are not even a measure of how useful something is. It is a combo of visibility of the original paper, PR campaign, having friends in the field, and how useful is the final product. That is not bad, because awesome but invisible or unusable software should not be rewarded.

So, the original developer should realize that current scientific system rewards very specific behaviors, where generating valuable products is only part of the equation.

If the want to develop fundamental components, they should:

• integrate their work into larger eco-system through collaborations
• spend time not only coding, but building community around their solutions or ideas
• they should not ship just the bare-bones solution, but a valuable toolkit that can be applied, or at least a protocol that will help others develop applications.

Answer 4

Perhaps you should look at your software license. Why shouldn't it have a clause requiring that software that is using it require that papers that cite that (using) software also cite it?

If there was sufficient interest in a project group, or the community at whole, an open source software promoting organization, or a university, could task lawyers with creating such a license and the whole community would benefit.

(It would be not much different than, say, current BSD, which requires all using software, when distributed, to properly cite the copyrights of all BSD-licensed software it depends on, directly or indirectly.)

Answer 5

I support the answer of Alexander Woo, but would like to elaborate on "expert judgement rather than crude benchmarks". In my opinion the benchmarks will always be there, whether we want it or not. This is simply not practical to pull experts from their main jobs to do evaluation of other scientists.

Benchmarks deserve their existence. However, it is up to experts to devise clear criteria for the evaluations. One such criterion could be to exclude engineering and software engineering from the academic evaluations. We should evaluate academicians according to academic criteria, scientific programers according to 0.5 x software engineering + 0.5 x academic criteria, and programers according to industrial standards.

The problem is in the self-perception of scientific programmers. Very often they position themselves both in academic field and industry to maximize their chances of finding a job. Pure mathematicians, string theorists, philosophers, or social scientists do not have such an escape.

Answer 6

The NIH asks applicants to fill out a "Biosketch" that is used to assess the qualifications of the PI (and other key members) of the proposal.

It asks for up to five "Contributions to Science", each of consists of a half-page of text and up to four references to "research products", which need not be publications or even your own work. This would be an ideal place for the authors of C to explain how their work underpins the more popular package B. Indeed, I've heard that people have had success doing something similar with data resources: "I collected/curated data set X, which others have used to do impactful things Y and Z." The one important caveat is that NIH does not allow hyperlinks in grants, so you can't directly link to the README, but I imagine you can allude to it.

Here's the Biosketch Instructions, which include a sample.