Take the 2-minute tour ×
Academia Stack Exchange is a question and answer site for academics and those enrolled in higher education. It's 100% free, no registration required.

I love Google Scholar as my go-to place to search for papers. Some features like "forward-citations" and their nice-ish autogenerated bibtex are life savers. However, sometimes I (and others) wish we could write scripts to help us:

However, Google Scholar does not provide an API, their robots.txt disallows scrapers on most pages of interest (for instance the cited-by results are not suppose to be accessed by bots), and if you try to make many requests (as a bot would) you will get an CAPTCHA.

Last year they used to have a EULA that said:

You shall not, and shall not allow any third party to: ...

(i) directly or indirectly generate queries, or impressions of or clicks on Results, through any automated, deceptive, fraudulent or other invalid means (including, but not limited to, click spam, robots, macro programs, and Internet agents);

...

(l) "crawl", "spider", index or in any non-transitory manner store or cache information obtained from the Service (including, but not limited to, Results, or any part, copy or derivative thereof);

Some Google services like custom search (for which I could find a EULA) still state this in section 1.4, but the link in the SO answer is now dead and I have not been able to find a new EULA for Scholar. From anecdotal evidence, I know that you can get in a decent amount of trouble of you try to circumvent Google's efforts to prevent scraping of Scholar.

Is there an official source where I could look up Scholar's terms of service? Or does the new unified ToS Google unveiled mean that I should be looking for the terms elsewhere? Is scraping still disallowed?

share|improve this question
    
Publish or Perish uses Google Scholar, so somehow they should be able to "directly or indirectly generate queries". –  Alexander Serebrenik Jul 24 '12 at 7:42
3  
@AlexanderSerebrenik I don't know how Publish or Perish uses Google Scholar, but I know how Mendley uses it: they require you to click a button for each individual search of Google Scholar. If they automatically did the Google Scholar meta-data search for each paper when you import a folder-full then they would violate the old Scholar EULA. That is why they make you click for each query: if each query is accompanied by a click and not part of some script or loop then it is in compliance with the old EULA. Does Publish or Perish do something different? –  Artem Kaznatcheev Jul 24 '12 at 15:03
    
I think that they do exactly as you describe... –  Alexander Serebrenik Jul 24 '12 at 18:27
    
@ArtemKaznatcheev If the answer added is useful, consider accepting it. Else comment or edit the question for better answers. –  Noble P. Abraham Oct 6 '12 at 1:14

1 Answer 1

up vote 4 down vote accepted

Not too sure if you are looking for this.

See One policy, one Google experience released by Google

On March 1, 2012, we changed our Privacy Policy and Terms of Service. We got rid of over 60 different privacy policies across Google and replaced them with one that’s a lot shorter and easier to read. The new policy and terms cover multiple products and features, reflecting our desire to create one beautifully simple and intuitive experience across Google.

That means all of the Google services have the same ToS, which is available here : Google Terms of Service

Here's a quote from that page

Don’t misuse our Services. For example, don’t interfere with our Services or try to access them using a method other than the interface and the instructions that we provide.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.