7

There was a highly cited paper published earlier this year in my field which introduced a highly theoretical, non trivial, algorithm that relied on extremely heavy theory. The algorithm was exact (Gave the correct solution) but 3 papers were published later on criticizing that it was highly non practical (Its implementation was a nightmare) and proposed approximation algorithms that were simple to implement.

So over the past year, me and 2 of my friends have been trying to implement the exact algorithm and finally we were able to implement it. The algorithm spans 14 files and is implemented in Python. The running time is acceptable for the input size in my field (takes 3-4 minutes on a modern computer to terminate). We also verified its correctness on a data set and it really did predict the exact solution.

We would love to publish the code, but one of our advisors said that it is unlikely that Code only would be publishable in a reputable journal/conference. One option is that we can just put it on Github but that doesn't really count as a publication. Are there conferences/journals which would take a 1-2 page paper that just describes a hard implementation? The only novel thing we did was that we've had to make very minor changes to the algorithm to make it practical but proved that these minor changes don't change the correctness (just makes implementation a lot easier!).

Any ideas?

  • 1
    Does this algorithm have an advantage over others? Is there a problem you can apply your implementation to in order to show (with numbers and not just theory) that this algorithm performs better or faster than others? – Steve Oct 3 '18 at 16:28
  • @Steve That algorithm is exact, all other 3 algorithms are approximations. There are several data sets that the algorithm can calculate the exact answer for. Not sure if that's what you're asking. – Coconut Oct 3 '18 at 16:54
  • In my limited experience: so far, reviewers nor editors ever commented/reacted on "open source" basis/aspect of my research. It could be for several reasons (like: core of the paper was so bad that they never reached paragraph where GitHub was cited) - but my impression is that journals with IF above 0 really don't value code/implementation/open source aspect, but the method, methodology and theoretical foundations. Also, I received only negative remarks on execution time / efficiency reporting in my papers -- algorithmic complexity is of significance, but timing is really useless for others – hardyVeles Oct 3 '18 at 21:07
  • I believe you actually have two separate things here: your implementation, and your updated algorithm (plus the proof that the result is still accurate). Even if the former isn't publishable, it seems like the latter should be. – bta Oct 3 '18 at 22:51
  • 1
    Maybe you can provide deep insights from that algorithm in your paper? That's a good contribution. – Pioneer83 Oct 3 '18 at 23:33
3

This is a judgement call and will be for editors and reviewers. But the main question they will want answered is what is new/novel in your paper? If you can give them that, then they will be more likely to accept it.

The other question is, whether exactness is worth the effort. If the approximation methods give good enough answers then the expense of an exact solution may not be worth it in practice. Can you address that issue in your paper?

You don't describe anything here about scaling if that is an issue with the algorithm. A few minutes on a modern system may be good or bad, depending. If the algorithm needs to be executed once for each Google query, then it is pretty bad. If it has to be executed once per "noticed supernova" then probably fine.

There are a lot of questions your paper could address that might push the editor toward acceptance if you can handle them well.

  • An unprecedented implementation of an algorithm is trivially novel. This however doesn’t mean it’s publishable in a conservative journal. – Konrad Rudolph Oct 3 '18 at 22:15
3

It is hard to say without the details if it is useful for anyone to read such a paper but if the code itself can be beneficial to other researchers in the field, I would encourage you to send it to a proper journal.

Several open-source codes in my field have been published already. Assuming that the novelty of the code is sufficiently high, you can try to submit to journals at least partially specialized to code publishing such as Computer Physics Communications or SoftwareX. However, they have a relatively high reputation and prestige, so I would recommend them only if your work is interesting enough.

2

Well, there seems to be novelty: All experts in your field regarded the original algorithm as correct but unusable / inpractical. You demonstrated, that it can be used, and that approximative solutions are not needed.

You should elaborate on this and show, how the "small differences" improved the algorithm's run time.

1

Yes. However, rather than looking in a computer science journal, look for journals the users of your software read (unless, of course, your end users are computer scientists). For example, if astronomers would use your software, browse and search astronomy journals.

After browse the journals and finding example articles, write your software and manuscript so end users can easily use it. Your results will also need to be polished enough in order to be publishable as software. For example, many R packages get published in journals that either target statistical computing or domain specific journals. To build upon another answer that lists two journals, other ideas might include:

1

In fact, you should not underestimate your effort that you put to make a pure theoretical approach into some practical application. I may not be an export in your field of expertise, but I know sometimes implementation of an algorithm worth more than its theory behind it these days from computational science point of view. If you are working in a pure theoretical department that may be the cause of the underestimation of your adviser. But there are a lot of research paper out there, which are published in reputable journals/conferences just based on their remarkable implementation which made life easier for computational science research community.

I would recommend first of all find a good benchmark dataset, which could show both the correctness of your code and also the power of its accuracy in comparison to approximation approaches. If you choose a benchmark dataset, which has a practical value itself (e.g. a biological dataset to find a relation between some genes of bacteries which may be a hot topic in biology) it could add quite a more value to the novelty of your work.

At the end, I believe it depends on you or your collaborator that how think about your research and how you could make good story out of it. Besides some few other example of journals that are addressed in other answers, you could consider this one: https://joss.theoj.org/, which their editorial board members are quite reputable.

Your Answer

By clicking "Post Your Answer", you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.