How to Scrape Walmart Product Results

In my previous blog post, we could scrape all product listings from Walmart. We have enough data to do market research, price analysis, and more!

But it's missing a lot of information like manufacturer, product type, full product description... This rich data content will be available on Walmart product detail pages.

Furthermore, the Walmart product page also has detailed specifications and data for many products, customer reviews(rating, reviews, review content, top positive reviews, top negative reviews...), recommendation products, and more.

It will help you enrich your Walmart data. If you've enjoyed our easy-to-use SerpApi Walmart API, why don't check out our SerpApi Walmart Product API?

Setting up a SerpApi account

SerpApi offers a free plan for newly created accounts. Head to the sign-up page to register for an account and do your first search with our interactive playground. When you want to do more searches with us, please visit the pricing page.

Once you are familiar with all results, you can utilize SERP APIs using your API Key.

Scrape your first Walmart Product Result with SerpApi

Head to the Walmart Product documentation on SerpApi for details.

In this tutorial, we will scrape the details of an iPod with Walmart ID 1123329439. The data result contains title, short_description_html, detailed_description_html, product_type, manufacturer, top_positive_review, and top_negative_review

First, you need to install the SerpApi client library.

pip install google-search-results

Set up the SerpApi credentials and search.

import serpapi, os, json

params = {
    'api_key': 'YOUR_API_KEY',         # your serpapi api
    'engine': 'walmart_product',       # SerpApi search engine	
    'product_id': '1123329439'
}

To retrieve Walmart Product details for a given product id, you can use the following code:

client = serpapi.Client()
result = client.search(params)['product_result']

You can store Walmart Product details JSON data in databases or export them to a CSV file.

import csv

header = ['title', 'short_description_html', 'detailed_description_html', 'product_type', 'manufacturer', 'top_positive_review', 'top_negative_review']

with open('walmart_product_ipod.csv', 'w', encoding='UTF8', newline='') as f:
    writer = csv.writer(f)

    writer.writerow(header)

    writer.writerow([result.get('title'), result.get('short_description_html'), result.get('detailed_description_html'), result.get('production_type'), result.get('manufacturer'), result.get('reviews_results', {}).get('reviews', {}).get('top_positive', {}).get('text'), result.get('reviews_results', {}).get('reviews', {}).get('top_negative', {}).get('text')])

This example uses Python, but you can also use your favorite programming languages like Ruby, NodeJS, Java, PHP, and more.

If you have any questions, please feel free to contact me.