45

As you maybe aware, Google does not provide any API for Google Scholar product and despite lots of request still there is no interest from Google to provide API for it. So, I am currently working on a product that can utilize publicly available information from Google Scholar page and render them so researchers and academician can use the data and embed them in their own website.

However, while I am developing the system, it happens sometimes that Google blocks my request to fetch scholar page (I parse HTML page). So, I am thinking that there might be a mechanism in their side that blocks request from applications -rather than human. So, it may suggest to get approval from them.

Do you have any idea that:

  1. Do people need approval/permission from Google to use its Google scholar information? The product I am building is probably donation based for basic version and paid for some other version (not yet confirmed).
  2. If people need permission, how should they approach them? I searched the entire scholar page here and found nothing related to get permission. Also there is no forum for Google Scholar according to this place.

Thanks

Improve this question
1

1 Answer 1

Reset to default
32

Google Scholar, as you said, does not provide API, nor a complete documentation. In the comment Davidmh cites this question, and I think that settles everything:

  • Google Scholar robot.txt disallows bots
  • Google service Terms of Service says

    Don’t misuse our Services. For example, don’t interfere with our Services or try to access them using a method other than the interface and the instructions that we provide.

So, in the end, you don't have permission for a script that fetches multiple pages at a time.

Following this Quora question, I found scholar.py, that is not an answer to your question but could be of use (I have not tried it).

Furthermore, I also asked the developer of the H-Index GS Calculator, to get another opinion, but I'm sure the "one-click" nature of the plugin complies with the ToS. A bot does not. Maybe you can change your system in this direction.

UPDATE: the developer says the in the past he wrote the GS team the same questions, and they replied

  1. you can do stuff, respecting the ToS (so, no bot)
  2. they will not provide an API, for legal constraints with affiliated digital libraries (ex. Scopus, WoS, etc.).

Also, his calculator has a throttler that reduces the number of queries per minute, to mitigate the issue. There are few simple techniques to "cheat", but, we are at the boundaries of the ToS :-).

Improve this answer
8
  • 4
    I think "you can't get permission" is not exact. You don't have permission, by the ToS, but they are free to grant them to you, if you convince them. I guess the OP should ask the legal department or become a Google Certified Partner (probably difficult, but who knows).
    – Davidmh
    Commented Dec 30, 2014 at 11:56
  • 2
    I edited the quote. You can't easily get permission should be more appropriate :-)
    – Aubrey
    Commented Dec 30, 2014 at 11:59
  • Thanks Aubrey. Misuse is something I am not doing indeed. I am just letting people better use it. It is not at all misuse. If you can have your live citations in your personal webpage, it helps Google to disseminate its data and also help you to let visitors know your status. So where is misuse that I cannot see?
    – Espanta
    Commented Dec 30, 2014 at 16:30
  • 2
    "For them", misuse is to try to access [data] using a method other than the interface and the instructions that we provide. My personal opinion is the same as yours, but as I wrote before GS has some serious legal constraints (I bet big publishers are really aggressive in defending their data). I think it's a pity, but that is what it is.
    – Aubrey
    Commented Dec 30, 2014 at 16:41
  • 1
    We need to investigate, the only thing I know I read it in this comment.
    – Aubrey
    Commented Dec 30, 2014 at 16:56

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .