Google Trends Real Time Search Trends in Python

Terry Tan -

We have already covered scraping Google Trends Explore, and another blog post covered scraping Google Trends Daily Searches. Today we will be using SerpApi's Google Trends Trending Now API to scrape Google Trends Real Time Search Trends, you can try it out in our playground.

If you're interested in scraping Google Trends data, such as interest over time, related topics, and trending searches, we have another tutorial for you. Check out our Google Trends API tutorial using Python.

Why use an API?

  • No need to create a parser from scratch and maintain it.
  • Bypass blocks from Google: solve CAPTCHA or solve IP blocks.
  • No need to pay for proxies, and CAPTCHA solvers.
  • Don't need to use browser automation.

SerpApi take care of everything mentioned above with fast response times under ~2.47 seconds (~1.33 seconds with Ludicrous speed) per request. The result is a well structured data in JSON with only single API call.

Response times and success rates are shown under SerpApi Status page.

What will be scraped

Full code

If you don't need explanation, checkout the full code in replit (online IDE).

import os
from serpapi import GoogleSearch

params = {
    'api_key':
    os.environ['SERPAPI_API_KEY'],  # https://serpapi.com/manage-api-key
    'engine': 'google_trends_trending_now',  # SerpApi search engine
    'frequency': 'realtime',  # The other option is 'daily'
    'geo': 'US',  # Country
    'cat': 'all',  # By default All categories
    'hl': 'en'  # Language
}

search = GoogleSearch(params)
result = search.get_dict()

# Get next page result
pages = result["serpapi_pagination"]["pages"]
next_page_token = pages[0]["next_page_token"]
params["next_page_token"] = next_page_token
next_page_search = GoogleSearch(params)
next_page_result = next_page_search.get_dict()
print(next_page_result)

Prerequisite

Install library:

pip install google-search-results

google-search-results is a SerpApi API package.

Code Explanation

First, we need to import the library

from serpapi import GoogleSearch

Next, we set the required parameters. A list of parameters and their detailed explanation can be found on our documentation.

params = {
    'api_key':
    os.environ['SERPAPI_API_KEY'],  # https://serpapi.com/manage-api-key
    'engine': 'google_trends_trending_now',  # SerpApi search engine
    'frequency': 'realtime',  # The other option is 'daily'
    'geo': 'US',  # Country
    'cat': 'all',  # By default All categories
    'hl': 'en'  # Language
}

Once parameters is defined, everything else will be taken care by the library, for example, making the actual API request. Calling get_dict to make the API request and JSON data will be returned and assigned it to result.

search = GoogleSearch(params)
result = search.get_dict()

Output:

print(result)
"realtime_searches": [
    {
      "id": "US_lnk_VKWqjwAAAAD-qM_en",
      "title": "Daniil Medvedev, Tom Clancy's Rainbow Six Siege, Ubisoft, Tennis player",
      "link": "https://trends.google.com/trends/trendingsearches/realtime?id=US_lnk_VKWqjwAAAAD-qM_en&category=all&geo=US#US_lnk_VKWqjwAAAAD-qM_en",
      "queries": [
        "Daniil Medvedev",
        "Tom Clancy's Rainbow Six Siege",
        "Ubisoft",
        "Tennis player"
      ],
      "explore_links": [
        {
          "link": "https://trends.google.com/trends/explore?q=/g/11bwgvh4_h&date=now+7-d&geo=US",
          "serpapi_link": "https://serpapi.com/search.json?data_type=TIMESERIES&date=now+7-d&engine=google_trends&geo=US&q=%2Fg%2F11bwgvh4_h&tz=420"
        },
        {
          "link": "https://trends.google.com/trends/explore?q=/g/1ymzqr434&date=now+7-d&geo=US",
          "serpapi_link": "https://serpapi.com/search.json?data_type=TIMESERIES&date=now+7-d&engine=google_trends&geo=US&q=%2Fg%2F1ymzqr434&tz=420"
        },
        {
          "link": "https://trends.google.com/trends/explore?q=/m/010vvkbz&date=now+7-d&geo=US",
          "serpapi_link": "https://serpapi.com/search.json?data_type=TIMESERIES&date=now+7-d&engine=google_trends&geo=US&q=%2Fm%2F010vvkbz&tz=420"
        },
        {
          "link": "https://trends.google.com/trends/explore?q=/m/02xhr73&date=now+7-d&geo=US",
          "serpapi_link": "https://serpapi.com/search.json?data_type=TIMESERIES&date=now+7-d&engine=google_trends&geo=US&q=%2Fm%2F02xhr73&tz=420"
        }
      ],
      "sparkline_chart_link": "https://trends.google.com/trends/api/widgetdata/sparkline?hl=en&tz=-480&id=US_lnk_VKWqjwAAAAD-qM_en",
      "articles": [
        {
          "title": "Rainbow Six Siege Names Tennis Star Daniil Medvedev as Game Ambassador",
          "link": "https://news.ubisoft.com/en-us/article/7LDreEjHibASiszLXY6cDI/rainbow-six-siege-names-tennis-star-daniil-medvedev-as-game-ambassador",
          "source": "Ubisoft News",
          "snippet": "With almost 3000 hours played, Medvedev will have his own themed cosmetic items added to the game, provide feedback on upcoming features,...",
          "date": "1 week ago",
          "thumbnail": "https://encrypted-tbn2.gstatic.com/images?q=tbn:ANd9GcSR-UVNnlaYqxVGxrmUzrOpc1hzVQSyAw02mVRl2AuZ2mzEyW0Y3GvrJ7XV9ro"
        },
        {
          "title": "Daniil Medvedev, Tennis Player Turned Rainbow Six Siege Ambassador",
          "link": "https://www.opp.today/news2/smooth-operator-daniil-medvedev-becomes-rainbow-six-siege-game-ambassador/22364/",
          "source": "OPP.Today -",
          "snippet": "Daniil Medvedev, the world No. 3 tennis player, has recently been making waves in the gaming world as a brand ambassador for the popular...",
          "date": "1 week ago"
        },
        {
          "title": "Smooth Operator: Daniil Medvedev becomes Rainbow Six Siege Game Ambassador",
          "link": "https://www.tennis.com/news/articles/smooth-operator-daniil-medvedev-becomes-rainbow-six-siege-game-ambassador",
          "source": "Tennis.com",
          "snippet": "WATCH: Medvedev caught up with the Tennis Channel Live Desk at the Western & Southern Open. NEW YORK—Veteran players of the 2015 video game...",
          "date": "1 week ago"
        },
        {
          "title": "‘Rainbow Six Siege’ Names Tennis Star, Medvedev, Game Ambassador",
          "link": "https://www.licenseglobal.com/video-games/-rainbow-six-siege-names-tennis-star-medvedev-game-ambassador",
          "source": "License Global",
          "snippet": "Long-term partnership with tennis player will include developer access, player onboarding initiatives and custom bundle.",
          "date": "1 week ago"
        },
        {
          "title": "\"Imagine getting teabagged by Daniil Medvedev\" - Fans react to the Russian becoming Rainbow Six Siege game ambassador",
          "link": "https://www.sportskeeda.com/tennis/news-imagine-getting-teabagged-daniil-medvedev-fans-react-russian-becoming-rainbow-six-siege-game-ambassador",
          "source": "Sportskeeda",
          "snippet": "Daniil Medvedev was recently announced as the Game Ambassador for Tom Clancy's Rainbow Six Siege. This news was met with great adulation on...",
          "date": "6 days ago"
        },
        {
          "title": "Is Daniil Medvedev a Gamer? Check Out Former US Open Winner’s Hilarious Exchange With Fellow Gamer as He ...",
          "link": "https://sportsmanor.com/tennis-news-is-daniil-medvedev-a-gamer-check-out-former-us-open-winners-hilarious-exchange-with-fellow-gamer-as-he-becomes-rainbow-six-siege-game-ambassador/",
          "source": "Sportsmanor",
          "snippet": "Russian tennis star Daniil Medvedev was announced as the ambassador of Rainbow Six Siege, and was engaged in a hilarious banter with fans.",
          "date": "5 days ago"
        }
      ]
    },
    ...
],
"serpapi_pagination": {
    "pages": [
      {
        "next_page_token": "WyJVU19sbmtfRUw2MGp3QUFBQUNrc01fZW4iLCJVU19sbmtfWmZPbGp3QUFBQURBX01fZW4iLCJVU19sbmtfM2pQSmpRQUFBQUFYUE1fZW4iLCJVU19sbmtfM3loMGpnQUFBQUNySk1fZW4iLCJVU19sbmtfLWFNaWpnQUFBQURick1fZW4iLCJVU19sbmtfOHFxdmp3QUFBQUJkcE1fZW4iLCJVU19sbmtfRUhDTWp3QUFBQUNjZk1fZW4iLCJVU19sbmtfLWNpM2p3QUFBQUJPeE1fZW4iLCJVU19sbmtfM1llMGp3QUFBQUJwaU1fZW4iLCJVU19sbmtfQldkMWp3QUFBQUJ3YU1fZW4iLCJVU19sbmtfSXd5Rmp3QUFBQUNtQU1fZW4iLCJVU19sbmtfQlUtNmp3QUFBQUNfUU1fZW4iLCJVU19sbmtfRnFudmpnQUFBQUQ1cE1fZW4iLCJVU19sbmtfanJZVWpnQUFBQUNhdU1fZW4iLCJVU19sbmtfdFFBV2p3QUFBQUNqRE1fZW4iLCJVU19sbmtfUU1td2pnQUFBQUR3eE1fZW4iLCJVU19sbmtfMnJJSWp3QUFBQURTdk1fZW4iLCJVU19sbmtfdHhCbGpnQUFBQURTSE1fZW4iLCJVU19sbmtfckhzOGp3QUFBQUNRZE1fZW4iLCJVU19sbmtfcGJiV2pnQUFBQUJ6dU1fZW4iXQ==",
        "next": "https://serpapi.com/search.json?cat=all&engine=google_trends_trending_now&frequency=realtime&geo=US&hl=en&next_page_token=WyJVU19sbmtfRUw2MGp3QUFBQUNrc01fZW4iLCJVU19sbmtfWmZPbGp3QUFBQURBX01fZW4iLCJVU19sbmtfM2pQSmpRQUFBQUFYUE1fZW4iLCJVU19sbmtfM3loMGpnQUFBQUNySk1fZW4iLCJVU19sbmtfLWFNaWpnQUFBQURick1fZW4iLCJVU19sbmtfOHFxdmp3QUFBQUJkcE1fZW4iLCJVU19sbmtfRUhDTWp3QUFBQUNjZk1fZW4iLCJVU19sbmtfLWNpM2p3QUFBQUJPeE1fZW4iLCJVU19sbmtfM1llMGp3QUFBQUJwaU1fZW4iLCJVU19sbmtfQldkMWp3QUFBQUJ3YU1fZW4iLCJVU19sbmtfSXd5Rmp3QUFBQUNtQU1fZW4iLCJVU19sbmtfQlUtNmp3QUFBQUNfUU1fZW4iLCJVU19sbmtfRnFudmpnQUFBQUQ1cE1fZW4iLCJVU19sbmtfanJZVWpnQUFBQUNhdU1fZW4iLCJVU19sbmtfdFFBV2p3QUFBQUNqRE1fZW4iLCJVU19sbmtfUU1td2pnQUFBQUR3eE1fZW4iLCJVU19sbmtfMnJJSWp3QUFBQURTdk1fZW4iLCJVU19sbmtfdHhCbGpnQUFBQURTSE1fZW4iLCJVU19sbmtfckhzOGp3QUFBQUNRZE1fZW4iLCJVU19sbmtfcGJiV2pnQUFBQUJ6dU1fZW4iXQ%3D%3D"
      },
      ...
    ]
}

Next Page Result

Similar code but to get the next page result, we have to get and set the next_page_token.

pages = result["serpapi_pagination"]["pages"]
next_page_token = pages[0]["next_page_token"]
params["next_page_token"] = next_page_token
next_page_search = GoogleSearch(params)
next_page_result = next_page_search.get_dict()

The output will be the same but do note that pagination is only available in the first request because Google doesn't follow the conventional pagination practice for this endpoint.

Documentation

If you have any questions, please feel free to reach out to me.

Join us on Twitter | YouTube

Add a Feature Request💫 or a Bug🐞