Scrape Product Detail Information from Google Shopping

Scraping product details from Google Shopping lets you gather real-time information like prices, reviews, and online sellers directly from search results. In this post, we’ll walk through how to extract this data using a simple API.

When clicking on one of the products from Google Shopping website, you can see a detailed information about the product.

How to scrape the product detail:

• Step 1: Scrape Google Shopping search results using Google Shopping API.
• Step 2: Get token for one of the products.
• Step 3: Use this token to access product details from Google Immersive Product API.

Scraping Google Shopping data

First, we need to scrape the Google Shopping results themselves. We can achieve this by calling the Google Shopping API.

Let's try scraping the Google Shopping website.

Preparation for accessing the SerpApi API in Python

• Create a new main.py file
• Install requests with:

pip install requests

Here is what the basic setup looks like:

import requests
SERPAPI_API_KEY = "YOUR_REAL_SERPAPI_API_KEY"

params = {
    "api_key": SERPAPI_API_KEY, #replace with your real API Key
    # soon
}

search = requests.get("https://serpapi.com/search", params=params)
response = search.json()
print(response)

With these few lines of code, we can access all of the search engines available at SerpApi, including the Google Shopping API.

import requests
SERPAPI_API_KEY = "YOUR_SERPAPI_API_KEY"

params = {
    "api_key": SERPAPI_API_KEY, 
    "engine": "google_shopping",
    "q": "shoes"
}

search = requests.get("https://serpapi.com/search", params=params)
response = search.json()
print(response)

To make it easier to see the response, let's add indentation to the output.

import json

# ...
# ...
# all previous code

print(json.dumps(response, indent=2))

Running this Python file should show you the listing for that keyword from the Google Shopping website.

As you can see from the response, we get a token called "immersive_product_page_token" which we can use to scrape the details of the products.

By the way, here is the complete tutorial for Google Shopping:

Scraping product details from Google Shopping

We're now on the step 3 where we need to call the Google Immersive Product API to get the detailed product data.

params = {
    "api_key": SERPAPI_API_KEY, 
    "engine": "google_shopping",
    "q": "shoes"
}

# Step 1: Access Google Shopping API
search = requests.get("https://serpapi.com/search", params=params)
response = search.json()

# Step 2: Get the token of the product
#   You can get the immersive product page token from any of the shopping results
#   Here we get it from the first result
token_from_first_data = response["shopping_results"][0]["immersive_product_page_token"]

# Step 3: Access Immersive Product API to get product details
params = {
    "api_key": SERPAPI_API_KEY,
    "engine": "google_immersive_product",
    "page_token": token_from_first_data
}

search = requests.get("https://serpapi.com/search", params=params)
response = search.json()
print(json.dumps(response, indent=2))

Here is the result:

Available data on Google Immersive Product API:

• product title
• description
• thumbnails
• brand
• total reviews
• total rating
• price range
• ratings distribution
• stores (online sellers)
• reviews images
• user reviews
• related searches

Google Product API is discontinued

Previously, the product information was also available on the Google Product API. Unfortunately, Google seems to have discontinued this endpoint. So, we can use the Google Immersive Product API as the replacement.