41

I and my coauthor wrote a paper and the project involved creating a (small) software library. Part of the novelty of the paper is the output of the code, which is a digital object not intended for by-hand manipulation. The code (open source) would ideally also be useful for others. One journal to which I was considering submitting this requires double-blind peer review, but the GitHub repo where the code is stored, referenced in the paper, identifies one of us simply by looking at the username in the url. We can of course obscure our identities in the paper as authors, but really need to cite the code repository.

I've not had to do double blind review before, and so it's not clear what we should do. My coauthor is going to run into more problems of this sort as they continue research with a similar mix of code-and-paper as output.

Is there anything we can do, at least as a first attempt to soothe the journals worries?

  • 8
    Is this something that you suspect will be a problem, or is it already a problem (e.g., the editors declined your paper)? In the first case, I suggest not worrying. My impression with double-blind reviewing is that it operates on the "honors code"; the veil of author anonymity is easily broken and editors are well aware of that. The point of double-blind is to avoid rubbing the authors' identities in the referees' faces, not to completely rule out the possibility of them discovering them. – darij grinberg Aug 8 at 8:38
  • 7
    @darijgrinberg Writing, "Our code is available on GitHub: https://github.com/AuthorName," seems like rubbing the authors' identities in the referees' faces, noting the GitHub repo...identifies one of us [by] the username in the url. – user2768 Aug 8 at 8:53
  • 26
    @darijgrinberg Link shorteners should never be included in a submitted manuscript, because they allow the author to track who accesses the linked resource. If the author can see in their logs that someone from the network of the Technical University of Sikinia clicked on their link, it's easy to guess who the referee is. – Federico Poloni Aug 8 at 9:21
  • 4
    @FedericoPoloni: Good point!! Better solution: Zenodo hosts snapshots of GitHub repos, and they can be accessed by IDs that don't include the author's name (at least not visibly); so that might be the right thing to do (for long-term preservation reasons as well). – darij grinberg Aug 8 at 9:23
  • 2
62

Censor out the repo's name, and provide code to the referees as an auxiliary file.

  • 6
    If the referees wants to, this is obfuscation is easily circumvented: take any non-trivial part of the code (one expressive function name or comment is enough) and drop it into google. – NichtJens Aug 8 at 15:37
  • 47
    @NichtJens in the age of arxiv, this often works with paper titles too, and it's ok. The authors just have a responsibility to allow the reviewer to proceed double-blind. – usul Aug 8 at 16:05
  • 10
    FWIW, I use this regularly to create a copy of a Git repo without any history at ../repo-name-copy from within that repo: git -C "$(git rev-parse --show-toplevel)" checkout-index --all --prefix="../$(basename "$(git rev-parse --show-toplevel)")-copy/". You may also want to grep -r -e 'Author Name' -e 'Other Author Name' in the resulting directory, and do something like sed -i 's/Jane Doe/Author 1/g;s/Joe Bloggs/Author 2/g' PATH to replace names. – l0b0 Aug 8 at 22:57
  • 2
    @l0b0 Normally I would use git archive HEAD > filename.zip instead of your complicated command --- what's the advantage of this method? – Federico Poloni Aug 9 at 7:21
  • 2
    @FedericoPoloni git rev-parse --show-toplevel gives you the top level directory of the repository, so this command will work when run anywhere within the repository. Other than that, I guess it depends whether you want a copy of the directory structure or an archive. – l0b0 Aug 9 at 7:28
17
  • Make a copy of the repository available at an anonymous URL, e.g., using Google Drive with a fresh account.

  • Submit a copy of the repository with your manuscript (if permitted by the journal), alternatively, send the repository to the editor by email.

  • 6
    If the referees wants to, this is obfuscation is easily circumvented: take any non-trivial part of the code (one expressive function name or comment is enough) and drop it into google. – NichtJens Aug 8 at 15:37
  • 23
    @NichtJens Equally, the referee can drop a sentence-or-two (from the paper) into Google and find the preprint. As noted in another comment, the veil of author anonymity is easily broken and editors are well aware of that. – user2768 Aug 8 at 15:41
  • 1
    Exactly! Wouldn't that be a problem for double-blind reviews as well? EDIT: Apparently it is: statmodeling.stat.columbia.edu/2018/01/15/… – NichtJens Aug 8 at 15:43
  • 16
    @NichtJens The point of double-blind peer review isn't to make it impossible for people to strip the blinding out, but to make it more difficult without some effort by the reviewers or authors. No system is perfect, and the system does need to operate with some minimal assumption of ethical behavior on the part of the reviewers, including not going out of their way to deliberately find who the authors are. – JoshuaZ Aug 8 at 15:54
  • 13
    @NichtJens as a reviewer, the system is more so that I can avoid being unintentionally biased - I won't try to look up the authors, because I don't care, however, if I'd see a name that I recognize on top of the paper or in the github url, there's nothing I can do to unsee it. – Peteris Aug 8 at 19:52
12

I'm literally in the same situation as you right now, and came across this repository/service on GitHub a few days ago:. Since your code and names are already public, it only provides a basic level of obfuscation. However, as long as reviewers are being honest and not actively trying to find out the names of the authors, then it should keep them from accidentally discovering who you are.

Beyond that, the most effective approach is not releasing it publicly until after review, and instead providing the code/documentation/whatever privately through the journal. My concern with this approach is that it depends on removing any name association from the material. So what happens if a reviewer rejects the manuscript, then publishes the code or parts of it as their own before you? The lack of a public record on your part could make it a bit of a headache to resolve.

Ultimately, there's not much you can do about reviewers that intentionally try to circumvent the anonymity. Even without your name anywhere, if you've published before, someone could potentially still get a pretty good idea of who you are through the content and patterns in the manuscript itself.

  • 2
    "the most effective approach is not releasing it publicly until after review," <-- too late – David Roberts Aug 8 at 21:59
  • 1
    @DavidRoberts I saw. I included that second paragraph more for anyone that might stumble on this question in the future – anjama Aug 9 at 0:42
  • 1
    If the authors are given, though separate means, access to an anonymized copy of the code, why should a public release of the code also be avoided? – Curt J. Sampson Aug 10 at 4:26
  • 1
    @CurtJ.Sampson If the reviewers need to do a web search for a term or concept in the paper, documentation in the repository could put in it the search results, especially if it is a specialized area of research. Alternatively, a reviewer might want to see what other work has been done and make sure the paper is properly citing it. Finally, a reviewer might search for the code itself to ensure that someone else hasn't published the code (to ensure that it is original work, and not plagiarized code/violating a copyright) – anjama Aug 10 at 11:22
  • Having both a record of submission, along with putting the code into a public git repository (even if the code is not yet public), ideally preserves the authors' timing. It is possible to arbitrarily set timestamps in git, but the public repo should have its own accounting for the repository. Additionally, one can make public just the latest or a series of commit hashes or another checksum of the codebase. – ti7 Aug 13 at 16:24
9

The simplest thing to do (which I'm surprised has not been suggested before, and is reasonably common) is to create an anonymous GitHub account and duplicate your code there (upload the code in a single commit, don't duplicate the repository itself as you don't want your real username to be present in the commit history).

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.