8

This question may be too specialist to be on topic here. If it is off topic, please feel free to transfer it to another SE site, or close it, as appropriate.

I am planning to publish an applied statistics paper. This paper develops an algorithm and then applies this algorithm to some data. I obtained most of this data from the site http://www.imgt.org. The data I am using are immunoglobulin and T cell receptor nucleotide sequences, in the form of FASTA files. I'm using around 200 of these.

Here is an random example of the data I am using (click on [6 Sequence (FASTA format)] to get the FASTA file).

Now, I have a problem. In Warranty Disclaimer and Copyright Notice, is written

The IMGT® software and data are provided as a service to the scientific community to be used only for research and educational purposes. Individuals may print or save portions of IMGT® for their own personal use. Any other use of IMGT® material need prior written permission of the IMGT director and of the legal institutions (CNRS and Université Montpellier 2).

I just heard from Prof. Marie-Paule Lefranc and she replied:

I have no objection that the data you retrieved for your work from IMGT/LIGM-DB be made available to the reviewers, but unfortunately we cannot authorize a script or a distribution of the IMGT/LIGM-DB files with your code to the users.

You can provide the users with the list of the IMGT/LIGM-DB accession numbers you used, with the source of the data clearly identified: (IMGT/LIGM-DB version number) and reference to NAR 2006.

Well, this just made my life more difficult. To start with, I'm puzzled by this. Isn't biological data like this public domain? Is it really possible to treat immunoglobulin and T cell receptor nucleotide sequence data as proprietary information?

I just wrote back and asked Prof. Lefranc what license the data was published under, which I had not done earlier.

Additionally, how does one make data available to reviewers and not to users? That is awkward, to say the least.

Also, the data is inconvenient to download. As you can see from the example above, the FASTA file is displayed in a web page, and is not downloaded by that button. One needs to clean the web page to get the FASTA file, which is a pain. As you can see Prof. Lefranc also disallowed the use of a script for doing this. What this most likely means in practice is that no user will ever actually test the code, because obtaining the data is too difficult.

I realise that the users of this site may not be comfortable offering what is essentially a legal opinion, and if so, can anyone suggest a more authoritative source to ask about the legalities of this? Thanks.

  • 2
    I guess I do not understand why you chose data that had restrictions for your paper (will the algorithm not work with other data?). It seems you tripped yourself up by assuming everything is free. – Peter Jansson Sep 15 '13 at 22:47
  • It seems like the problem is offering the users the scripts to download the data. But if you keep the algorithm that you used to download the data off your paper, there doesn't seem to be a problem with presenting your results on the imgt sequences (as one of the answers proposed). But since this is an old thread, you could tell us how it went. – BioGeo Dec 20 '16 at 14:13
  • 1
    Hi @BioGeo. It has indeed been a while, but I remember I used the approach suggested by Peter Rice. As you can see, I accepted his answer. SE helpfully tells me that I did so on September 18th 2013. How time flies. – Faheem Mitha Dec 21 '16 at 13:19
9

The copyright is probably on the full database release flatfile and the formatted entries ... you will find similar conditions for UniProt/SwissProt so it is not so unusual.

The restrictions on scripts are common to prevent server performance hits from a large number of requests.

You can simply invite reviewers to download the data from some other server, for example from the EBI SRS server. The URL for entry A00673 would be

"http://srs.ebi.ac.uk/srsbin/cgi-bin/wgetz?[IMGTLIGM-ID:a00673]+-view+FastaSeqs+-ascii"

You can also use a list of accessions, for example A00673 or A01650

"http://srs.ebi.ac.uk/srsbin/cgi-bin/wgetz?[IMGTLIGM-ID:a00673|a01650]+-view+FastaSeqs+-ascii"

If downloading many entries you should pause between requests, but putting lists into the URLs may reduce it to few enough not to cause a problem. I doubts EBI would be upset by 200 requests - they would be concerned about thousands.

There are various fasta formats available for IMGT data, you need to find a server that produces fasta files compatible with your input requirements.

Alternatively of course your reviewers could download the whole database from IMGT or any of the other servers (including ftp://ftp.ebi.ac.uk/pub/databases/imgt/) and generate their own fasta subset from the list of accessions/ids

Hope that helps!

  • Thanks for the super helpful answer, Peter. I'm replying in more detail on the Debian lists. – Faheem Mitha Sep 17 '13 at 19:21
  • 1
    +1 for suggesting to download the entire database. imgt.fasta.Z is only 46 MB, and it's already in a clean machine-readable format, so @FaheemMitha could easily write a script to parse it (or just use an existing tool written for the purpose). – Ilmari Karonen Sep 18 '13 at 12:07
7

Yes, it is making your life a little harder, but it doesn't mean publication of your work is impossible, nor does it make IMGT's action unethical. I cannot comment on the legality of IMGT's copyright claim on the data, but contesting their claim doesn't sound like a great idea in the first place. In any case, talk it through with a lawyer from your university's legal department, before you do anything that deviates from what IMGT asks for.

Now, how can you move forward? Well, separate your existing code (which does the scraping and the analysis) into two separate parts:

  1. The IMGT website scrapper/parser, which will download data and write it to files named after each query (M38103.txt for query “M38103”).

    Do not publish that part (but keep it around, it would be a shame to throw away code that you have already written, and that works: you never know, IMGT's policy may change in the future).

  2. The bulk of your analysis code, which takes these query results as text input files.

You now publish #2, and give the referees access to the files (journal submission websites have an option for “supporting information for reviewers only”, although it may be called differently). In the paper, you indicate clearly (but not aggressively) that “because licensing restrictions do not allow us to redistribute IMGT data, we provide a script that requires query results as text input files”.

You're not the first person to publish valid research results that come from analysis of a proprietary data source. There is no ethical issue here, because the reviewers have enough information to accurately review the validity of your work. Moreover, even the readers will be able to reproduce your work, though it will require a separate download step (and definitely depends on IMGT keeping its database online and freely accessible).

So yeah, it makes your code is little harder to use for others, but it doesn't diminish the values of the results you have obtained with it!

  • Thanks for the helpful response, F'x. If you look at my question, you can see that Prof. Lefranc says "unfortunately we cannot authorize a script..." I find the treatment of biological data as proprietary surprising. I find this prohibition doubling surprising. Is the owner of a web site really in a position to dictate that e.g. curl in my case not be used to download the files? – Faheem Mitha Sep 16 '13 at 8:26
  • If even a script is not allowed, then the difficulty level for obtaining this data is prohibitive. Nobody, except a fanatic is going to manually download 200 web pages and manually clean them. Of course, I suppose they could write their own script and not tell Prof. Lefranc about it. :-) – Faheem Mitha Sep 16 '13 at 8:26
  • 1
    Focus on your results! Do not put so much the focus on the code, but on the algorithm/method and the results… that's what research is about. It's nice to publish the exact code you used, so that others can use it (and also verify your results). But that's an added benefit, the methodology and results are your main contribution to the field. (Anyone can write the code, but fewer people can have the idea behind it.) – F'x Sep 16 '13 at 8:29
  • Sure, the ideas/algorithm/method is important, but making sure people can reproduce the results without excessive difficulty is also important. Apart from anything else working code is important if one wants to understand how the method works in all its details. The paper discussion may not include all necessary details, but working code by definition does. Obviously, working code which does not include necessary input data is not very useful working code. :-) – Faheem Mitha Sep 16 '13 at 9:12
  • 2
    @FaheemMitha the ideas behind reproducible research come at a cost. You cannot use data sets which are not open. – StrongBad Sep 16 '13 at 11:02
4

As F'x already said, from a scientific perspective there is no problem.

The data is available, just not automatically and from you, but from the original source (via the ID numbers). So editor, reviewers and readers can get the data, given they are not too lazy to download it from the original web site. That is much more than is common in many fields of physical/experimental science.
Considering that you are talking about biological data, not having an automated download is so much more convenient than trying to reproduce experimental data (although I have to say that it is a sensible and under-used mid-way checkpoint) ...

However, here are a few more thoughts:

Isn't biological data like this public domain? Is it really possible to treat immunoglobulin and T cell receptor nucleotide sequence data as proprietary information?

  • Facts cannot be copyrighted. But the measured data is subject to copyright. If I go through the effort of doing these measurements, I'm the owner of that data. Just as you are the author of the program code you wrote and the paper you wrote.
    But, as you cannot forbid that someone else to write another program doing the same, or another paper on the same subject, just because I have the rights to my measurements, I cannot forbid you to make your own measurements.
    Of course I could donate my data base to the public domain, just as you can put your program under a FOSS license.

  • Copyright varies considerably depending on jusdiction. So IMHO this question cannot really be answered without taking into account where the database comes from (EU) and where you are located.
    Now, in the EU we have a database copyright given you put enough effort in making the data base (it is not enough to grab an old encyclopedia and scan pages from that. But carefully curating a nucleotide sequence database is clearly enough). Again, this copyright is for the database, not for the facts stored in the database.
    For your nucleotide sequences that means: if you choose to use their nucleotide database, you have to stick to their rules. But again, you are free to measure the nucleotide sequence yourself and use that data set instead.


how does one make data available to reviewers and not to users

You can use the letter to the editor to tell the editor that you'd be happy to supply the reviewers with the curated data set you acutally used for the analysis.


the data is inconvenient to download. [...]
I realise that the users of this site may not be comfortable offering what is essentially a legal opinion

While it is certainly a good idea to learn about copyright, I cannot recommend going for legal loopholes against the database owner's expressed wishes. They do allow enough for you and other scientists to do science. Why would you want to upset them?

I think it would be better to talk to them. What about this instead of asking them to allow scripts, you could offer to produce for them a second version of the data base that is suitable as machine-readable input, and kindly ask them whether they would be willing to make that version available via their server.

In addition to learning about copyright, maybe you could ask them for their reasons for their download policy. There may be a whole lot for you to learn in that answers as well.

I come from one of the physical sciences where good measurements take lots of effort. Here are some reasons why a data base ownder may say that people should download the data base from the original source:

  • The name of "owner"/author of the database is associated with it. As author you may not want to run the risk of getting associated with derivatives that do not follow your strict high-quality policy.
  • One very simple way to ensure that people actually get your data base when they think they do is to tell them always to download the original
    (there are alternatives, such as signing a version for distribution etc.).
  • In addition, the owner may want/need to have at least a rough overview of how many people use the data base. Such information is at the very least extremely helpful when you need to show that you are not doing useless stuff for your wages.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.