Intro

SerpApi streamlines the process of scraping search results pages. A common use case is scraping product info from Google, HomeDepot, and/or Walmart.

While still much simpler than manually scraping these sites yourself, getting product details is slightly less straightforward than some of the other data you can scrape with SerpApi.

For example, look at this Google Shopping API search example in the SerpApi Playground:


You can see there is some information available here about each product. But how do we get reviews, ratings, buying options, and sellers?

If you were searching in your browser, you could click on the product to expand it, then click “View product details” to get more information.

But how can we do this with SerpApi, since we need to be able to do it programmatically?

Google Shopping API and Google Product API

If you go back to the Playground and click the dropdown in the top left corner, you will see that in addition to the Google Shopping API, we also have a Google Product API:

However, if you select the Google Product API in the Playground, you will notice that the only way to search is with something called a product_id.

If you go back to the Google Shopping API example and click on one of the products to select it, then look over at the selected part of the JSON response, you will notice there is a key named product_id:


This is what we need in order to find the item with the Product API:



Home Depot API

The process for scraping product details from Home Depot is almost identical to the one for  scraping them from Google.

First we have to use the Home Depot Search API to find the product and get the product_id.


Then we can search the Home Depot Product API with the product_id:



Walmart API

There are a few differences when scraping product details from the Walmart API. While product results in a Walmart search do have a product_id, they also have a us_item_id. We need to use the us_item_id if we want the results to include Reviews.

First with the Walmart Search API to get the product_id and the us_item_id:

We can then get the product page with the Product API using the product_id, but there won’t be any reviews:

So if we want the reviews we need to use the us_item_id. Don't be confused by the name of the parameter, it says product_id but we can set it's value to the us_item_id:

If you scroll down, you can see there are reviews:


We also have a separate API for getting Walmart Product Reviews. This API also requires the us_item_id to be supplied as the value of the product_id parameter:

Conclusion

We have seen how to get product details from Google, Home Depot, and Walmart using SerpApi.

I hope you found the tutorial informative and easy to follow. If you have any questions, feel free to contact me at ryan@serpapi.com.