2

I'm interested in scraping publicly available information from journal websites (such as article titles, authors, abstracts, and keywords).

After scraping the data, I'm interested in analyzing the data and possibly publishing the findings.

Apart from open access journals for which this is likely less of (or not) an issue, what are the ethics and legality of scraping publicly available information from journal websites?

  • 4
    I don't see any ethical issue whatsoever. In terms of legal issues: you could look at en.wikipedia.org/wiki/Web_scraping#Legal_issues. The laws are just emerging here but very roughly, if you are not damaging the scrapee (either through implementing extracted information in a way that is detrimental to their business or significantly slowing down the running of their site), it seems to be okay. In the case at hand, it sounds like you want information other third parties already collect and post. This makes me think it would not be a problem (but I certainly don't know). – Pete L. Clark Apr 23 '16 at 20:33
  • 1
    In general, the legality will be covered by the terms of use on the websites from which you are scraping. If they forbid scraping, I suppose you might be legally required not to. I say might, because as a non-lawyer I don't know whether such terms are legally permissible and therefore binding. – Corvus Apr 24 '16 at 21:00
  • Hi Josua. I'm interested in doing this kind of analysis too. Would be great to share my ideas with you, so we can mine papers for different purposes, but using similar tools, including the ones you are developing. Let me know, we can have a private conversation about that. – biotech Apr 28 '16 at 8:58
  • That would be good. I'm at jrosen@msu.edu, send me a note! – Joshua Rosenberg Apr 28 '16 at 12:40
  • @PeteL.Clark Can you please turn your comment into an answer so that I can vote for it? – jakebeal Jun 10 '16 at 20:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.