I recently started to survey an algorithm described in a paper. I would like to include some variants of this algorithm in my survey.

One natural choice is to use "cited by" feature in Google Scholar. It could return a new search of the papers that cite the previous one (see below). enter image description here

Since there are 1342 papers, it is not feasible to read all of them (even just abstracts) within short period of time. I made a heuristic for selecting "interesting" papers

Only read the papers that have more than some number of citations (say K).

which means I will only have to read papers that in turn have more than K "cited by" number.

Even though this heuristic is unfair for newly published papers, I believe it could indeed help me narrow down my scope.

However, the issue is Google Scholar does not have features to support my heuristic (something like advanced search supported by other databases might). So my problem is

  • How do I download the search results from my Google Scholar query with all the meta data, including "cited by" information.
  • If the previous point is not possible, is there another tool that could help me?

Could someone help me. Thank you in advance!

  • Answer to the similar question with code example: academia.stackexchange.com/a/147115/10624
    – ilyazub
    Commented Apr 1, 2020 at 14:19
  • Google scholar is a commercial product developed by Alpha. Please address your concerns to them. Please remember that if a commercial product is for free, you are paying for it in some other way.
    – EarlGrey
    Commented Jul 19, 2022 at 9:18

2 Answers 2


There is no API for google scholar data. They do that on purpose ... Here are links to some possible workarounds


You can scrape Google Scholar Search Results using BeautifulSoup web scraping library.

Check code in online IDE:

from bs4 import BeautifulSoup
import requests
import re
import json

headers = {
    "User-Agent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.5060.53 Safari/537.36"

params = {
    "hl": "en",     # language of the search
    "q": "covid",   # search query
    "num": 100,     # number of results per page. In this case, 100 results per page

response = requests.get(
    "https://scholar.google.com/scholar", params=params, headers=headers

soup = BeautifulSoup(response.content, "lxml")

organic_results_data = []

for organic_result in soup.select(".gs_r.gs_or.gs_scl"):
    title = organic_result.select_one(".gs_rt a").text
    snippet = organic_result.select_one(".gs_rs").text

    cited_by_match = re.search(
        r"Cited by (?P<cited_by_count>\d+)",
        organic_result.select_one(".gs_or_btn.gs_nph+ a").text,
    cited_by_count = int(cited_by_match.groupdict().get("cited_by_count"))

        {"title": title, "snippet": snippet, "cited_by_count": cited_by_count}

print(json.dumps(organic_results_data, indent=2))

Example output:

    "title": "Thoracic imaging tests for the diagnosis of COVID‐19",
    "snippet": "Background The respiratory illness caused by SARS‐CoV‐2 infection continues to present \ndiagnostic challenges. Our 2020 edition of this review showed thoracic (chest) imaging to be …",
    "cited_by_count": 186
    "title": "An overview of COVID-19",
    "snippet": "… 11, 2020, the World Health Organization (WHO) officially named the disease resulting from \ninfection with SARS-CoV-2 as coronavirus disease 2019 (COVID-19). COVID-19 represents …",
    "cited_by_count": 333
    "title": "COVID-19 and multiorgan response",
    "snippet": "… COVID-19 has demonstrated a wide spectrum of clinical … multiorgan impact of COVID-19 \nreported since its outbreak. … If a paper is reporting on many aspects of the COVID-19, then the …",
    "cited_by_count": 729
  # ...

Alternatively, you can use the free open-source package scholarly.

Example from its documentation.

>>> search_query = scholarly.search_keyword('Haptics')
>>> scholarly.pprint(next(search_query))
{'affiliation': 'Postdoctoral research assistant, University of Bremen',
 'citedby': 56666,
 'email_domain': '@collision-detection.com',
 'filled': False,
 'interests': ['Computer Graphics',
               'Collision Detection',
               'Geometric Data Structures'],
 'name': 'Rene Weller',
 'scholar_id': 'lHrs3Y4AAAAJ',
 'url_picture': 'https://scholar.google.com/citations?view_op=medium_photo&user=lHrs3Y4AAAAJ'}

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .