3

Four years ago, another group published a paper describing a new method for estimating an important parameter in my field. The method is theoretically elegant, has good accuracy in their benchmarks, and can be applied in cases where every other method I know of gives unreliable estimates.

The authors provided a GitHub repo with the code they used to realize this method. Unfortunately, the code in the repo is very poor. Several scripts do not complete successfully without fixing several lines of code; a key step in the pipeline is left as an exercise for the reader (notably, the paper stated that code for this step would be in the repo); as written, the whole pipeline requires the user to manually run many different scripts in succession; and the code is very inefficient, with e.g. computationally intensive calculations done to define objects that never get used. (The first author was a master's student, which might explain some of this.)

According to the citations of the parent paper, it seems that this method has only ever been used by the same group that wrote it, likely because of the limitations above. The issues page on the GitHub repo demonstrates that a few other people tried to run the code before hitting multiple serious errors and apparently giving up. After a lot of effort and fixes, I was able to run the code, and confirmed that it gives good results in cases where every other method fails. I also decided to reimplement the method almost from scratch as an evening project across a couple of weeks. I would like to believe my rewrite is an improvement: among other things, the runtime is much quicker (e.g. one particular step now takes 10 minutes rather than 40 hours), it takes a greater variety of file formats as input, and the whole process can be run in a single command. I have tested it on multiple machines and Linux distros and confirm that it does actually run out-of-the-box on all of them.

I think it would be good for my field to be able to discover this implementation of a useful method which nobody else has been able to use to date, and to be honest it would also be nice if I could somehow get recognized for this work. I really do think that it would satisfy a significant unmet need.

My idea right now is to put the method up on... JOSS? Zenodo? A preprint server? so it gets a citable DOI. (The venue to publicize this on is another question - I think it would be difficult and unnecessary to somehow turn this rewrite into a more traditional research paper.) But I'm not sure how I would attribute the original authors. I could reach out and offer to make them co-authors on whatever citable DOI I create. I could leave instructions on the GitHub repo to cite both the rewrite's DOI and the original paper when using it - although in practice I think many people would not bother citing both. Or am I wrong for expecting citations for this kind of work? After all, I did not add any functionality, I just made an implementation that is more flexible and much easier to run at all. Should I just make the repo public, tweet about it, add the repo as a line in my CV, and be done with it? I did not get to my current position via a traditional academic route, so I am not as familiar with the norms around publishing and so on.

1 Answer 1

2

I think this sort of work is really useful, especially since academic code is very hard to deal with usually.

To answer your question - does the streamlining/optimization of the code allow you to answer new problems in your field? If your implementation makes the algorithm much faster and easier to work with, you can apply it to a larger problem, or to more cases etc., depending on the domain you work in. So you can write a paper solving these problems that couldn't be tackled before, and at the same time describe and publish your improvements to the code.

2
  • Thank you. It would be possible to contrive to write a paper, but part of my question is whether I can publicize and get credit for the reimplementation without perfunctory paper-writing! I'm not sure that applying it to more cases is a promising route: the "parameter" here is an important kind of measurement error which may or may not be present for a particular observation, and is calculated using a fairly raw version of the data which is tedious to acquire. The original paper was fine. Really I just want to get it out into the world so groups can run it in-house for quality control. Oct 20, 2023 at 10:48
  • You could put it up on github and advertise it on social media, I guess - twitter, or a blogpost? Also, a kind of methods preprint would work so people could cite it, depending on how extensive the changes you made are.
    – user178392
    Oct 20, 2023 at 14:51

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .