17

I recently completed a part of my project and communicated a paper to a conference. Let's call the paper's title as "project x: for this and that". Now, I wish to open-source project x to facilitate reproducible research and to have more (at least some) people use it (and cite it!).

Are there any specific drawbacks or risks involved in open-sourcing project x on, say, Github or Sourceforge? Do note here that I would still be improving on project x, and possibly sending the extended version to a journal (my area of work being Computer Science).

I understand that if a conference/journal requires double-blind review and my project is searchable on the Internet, I am revealing my identity to the reviewers. This is bad, right?

Are there any other cons I should be considering? And are there any pros of open-sourcing before a making it into a publication?

  • 1
    The double-blind review is sort of a joke in a lot of smaller fields. You generally know what everyone is working on, so it's easy to figure out who wrote a paper. – Shep Jan 29 '15 at 0:00
8

Double-blind reviews are usually more common in conferences. For example, I don't know of any journals using double-blind review in my field (machine learning). I'm going to assume revealing your identity is not an issue (if it is, circumstances differ).

Whenever relevant I provide an implementation along with a paper. This also helps reviewers, in case they want to fiddle with an algorithm under slightly different circumstances than those reported in the paper (which is a good thing!). When an implementation is provided, the option is there.

The pros are increased visibility, reproducibility and (in my opinion) credibility since you allow everyone to try for themselves instead of taking your word for it in the paper. On rare occasions, your software may become quite popular during the review period, which may positively impact the paper under review.

A potential con is that someone may discover a critical bug in your implementation. From the perspective of software engineering this is always helpful since you can then improve the software. For the associated paper this may be a good, bad or irrelevant thing, depending on the type of bug:

  1. One that does not influence the results reported in your paper: no big deal. Simply fix and move on. Best case this improves the user experience of your software, worst case you lost a bit of time fixing something unimportant.
  2. One that does influence the results: big deal. This will at least delay a potential publication. Ofcourse it is better to find such errors and fix them instead of publishing erroneous conclusions, but this may have an impact on credibility.
  • 3
    "One that does influence the results: big deal. This will at least delay a potential publication. Ofcourse it is better to find such errors and fix them instead of publishing erroneous conclusions, but this may have an impact on credibility." I understand why that would be unpleasant for the author, but is it acceptable to hide something because you are afraid someone will find a fault in it? – Ri49 Mar 5 '14 at 15:49
  • 1
    @Ri49 I find that unacceptable, but I am sure it happens. This is one of the key reasons why I always provide implementations. – Marc Claesen Mar 5 '14 at 16:45
  • "This may have an impact on credibility": that's a big issue with academia, and is only happening because of software (and human nature) illiteracy. In any software, bugs are bound to happen. Noone can prove a software to be bug-free, Church-Turing proved that. Publishing the sourcecode, even if bugs are found, are in my book proof of seriousness and scientific rigorousness, because you can reproduce experiments and the authors accept the possibility to be wrong and be corrected. This is totally the scientific methodology, so why should this possibly impact credibility is beyond me. – gaborous Jun 3 '16 at 11:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.