11

I am an engineering student, however, my graduate research is more inter-disciplinary (computational science). Which is good for me in some ways, because one of my better skills is programming, since I've been doing it avidly as a hobby for a very long time. Because of this, it would be much easier for me to create a widely used computational science software tool rather than a highly-cited journal article.

With the specific aim of getting a post-doc at a top-ranked university, how does creating a useful piece of software (like an original, optimized quantum monte carlo implementation for example) compare to a widely-cited journal article? In other words, if done well, both would be highly-cited, but do frequent citations of software carry the same weight as top-tier research articles?

  • 1
    If your work is original, there are a number of conferences and workshops which will accept a paper that describes its novelty and why it is better/faster than competing work. – Chris Gregg Mar 18 '13 at 5:15
  • It is a common pattern to ask those who use your software in a publications to cite the manual, which can be published as a technical report, or a short paper describing the software. See for instance wiki.hpc.ufl.edu/doc/FFTW, wiki.octave.org/FAQ#How_can_I_cite_Octave.3F. If the software is useful, those citations pile up to form a huge critical mass. For instance, the manual of the MAGMA computer algebra system has been the top-cited math paper in 2009, 2010, 2011 (according to mathscinet). – Federico Poloni Mar 18 '13 at 7:41
7

The following applies to the neuroscience and the biological sciences.

In almost all cases, the article will win. This is typically due to the attitude that writing software isn't research; it's implementation. In some cases, you will have to overcome technical challenges when implementing the software, but for the most part you'll simply be coding up techniques that were developed and iterated upon by a different research group.

Note that there do exist labs that both developing analytical techniques and then have a team as part of the group who translates that into software development. The Wellcome Trust Center in UCL comes to mind here; they have developed a number of techniques used for detecting brain activity, as well as brain mapping, and they maintain a software suite that allows researchers to use their techniques. There are a number of similar groups in different fields (e.g., medical image analysis (project)(group), MEG analysis (project)(group)). These labs have typically demonstrated over time their strength in developing research software, and are able to obtain grants to specifically support development of their software.

| improve this answer | |
  • 2
    I would agree, except I would add that it doesn't matter to them whether the software writing is research and that collecting data using obvious biological protocols and then interpreting the results in the standard ways is also "just implementation". In the biological sciences, people value biology and are not particularly willing to consider--except when such is in excruciatingly high demand--tools that aid biological research in lieu of the research itself. – Rex Kerr Mar 18 '13 at 15:08
  • @RexKerr - That is completely true, and as you stated yourself, people don't care. To be recognized you have to do research, period. Everything else is icing on the cake. – eykanal Mar 18 '13 at 15:25
  • 1
    "That's not writing. That's typing." — Truman Capote on Jack Kerouac – JeffE Mar 18 '13 at 18:58
  • 1
    My attitude is that if software development is held in low academic esteem in your field, don't do it. My fantasy is that enough of us will vote with our feet until the work is recognized or the fields collapse. – Anon Mar 18 '13 at 22:25
11

There's a thresholding effect for software. If you create a piece of software that has a huge number of downloads and has visible impact, that will count for something. However, with papers, you wouldn't need to have the same level of impact to have the same effect on interviewers.

The problem really is that academics don't know how to evaluate the impact of software development, and would need help doing so. I imagine that if your potential post-doc advisor has used your software, you won't need to do much explaining. But if not, and your research credentials are otherwise weaker, it might be difficult to justify.

| improve this answer | |
3

Attitudes are slowly but surely changing--here is an indication. As of January, the NSF required that the Publications section of NSF biosketches be changed to Products. It is remarkable how important names are to bureaucrats; in this case, the change reflects the awareness that the publications category excluded much of the work of scientific programmers, inventors and the producers of data sets (data sets are routinely referred to as products in the geo and social sciences). The previous biosketch format tended to make the inclusion of software and data sets (not to mention other contributions) under Publications often seem forced. The practice of citing technical reports and papers on software instead of directly citing it ought to be questioned. Given the ever increasing importance of software and data set production for many kinds of scientific research, it is a welcome development not to have to diminish significant and specialized contributions by citing published reports on products that could be cited directly (in contrast to certain inventions) or by hiding them under Synergistic Activities. In time, administrators will get the message that software production ought to count more than it does.

| improve this answer | |
  • "the publications category excluded the much...". The "the" may be redundant. – Faheem Mitha Mar 26 '13 at 8:54
  • Excrescent 'the' removed. – Anon Mar 26 '13 at 9:43
  • Hey, I learned a new word! :-) – Faheem Mitha Mar 26 '13 at 16:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.