I am a researcher, and I would like to scrape a list of scientific libraries sites such as Springerlink.com, ScienceDirect.com, IEEExplore digital library and so on, to analyse them for some purpose. In this task I'm interested in abstracts, and full text of publicly accessible papers, I would like to know the legal framework of this, since that I read before (but I don't remember where and how to find it again) that this is illegal, and if I do that any new request from my IP address to those sites will be rejected. It means that this task will be considered as a Denial of service attack.

To what extent this is true?

  • 4
    EnergyNumbers has given the right answer in general. As for specifics, the sites you mention do not allow scraping or text mining, except under strictly limited circumstances (which rule out almost all cases). You should treat this very carefully. If you try to get permission, you will probably waste quite a bit of time and fail in the end to get the permission you seek. If you try it without permission, you could very well find yourself in court (like Aaron Swartz). If you can carry out your study using a smaller but freely accessible data set, I'd highly recommend trying that instead. Dec 26, 2012 at 2:38
  • 1
    Voting to close: this is a question about law, not academia. Dec 26, 2012 at 7:05
  • 2
    I see it as having both legal and academic aspects (but I don't have a strong opinion regarding whether it is a good fit for this site). The basic issue is what the terms of service for these sites allow, and that's partly a question about what sort of terms are customary for scholarly websites. Dec 27, 2012 at 2:16
  • @NateEldredge I don't see any vote to close cast, was there a technical problem?
    – user102
    Jan 13, 2013 at 18:19
  • @CharlesMorisset: Hmm, I definitely voted to close, but I don't see a number by "close" either, yet if I click it, it says "You have already voted to close this question." Jan 14, 2013 at 3:31

1 Answer 1

  1. I am not a lawyer. This is a legal question. Get proper legal advice.

  2. The legal framework is contract law, and intellectual property law. There may also be implications for other parts of the legal code if your scraping behaves like, or could be interpreted as, a denial-of-service attack or hacking.

  3. Read each site's terms and conditions. Understand the contract of use for each site. Interpret each contract within the legal context of the contract's jurisdiction.

  4. Take into account intellectual property law within your own country, and the publisher's country. You might also need to consider intellectual property law within the country where the web-server is located, too.

  5. Do bear in mind that there have been recent attempts across several jurisdictions to bring intellectual property within criminal, rather than civil law / contracts.

  6. I am not a lawyer. This is a legal question. Get proper legal advice.

  • It is my opinion that the Government should interpret the law for its Citizens... not Lawyers. It would prevent a lot of senseless litigation if permission were granted before instead of second guessed afterwards.
    – Dale
    Aug 22, 2013 at 1:02

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .