How to scrape Yelp results

Yelp is a go-to resource for seeking information about local businesses, restaurants, and services with millions of customer reviews and ratings for businesses across various industries.

Scraping Yelp results can offer valuable insights into the business reputation and review analysis, competitor analysis, local market insights, business listing and contract information, sentiment analysis, and consumer feedback.

By extracting data from Yelp, you can gain a competitive edge, make informed business decisions, and optimize your offerings.

In this tutorial, we will explore how to scrape Yelp results, equipping you with the knowledge and tools to effectively harness the power of Yelp data. Let's get started!

Setting up a SerpApi account

SerpApi offers a free plan for newly created accounts. Head to the sign-up page to register an account and complete your first search with our interactive playground. When you want to do more searches with us, please visit the pricing page.

Once you are familiar with all results, you can utilize SERP APIs using your API Key.

Scrape your first Yelp  Result with SerpApi

Head to the Yelp results from the documentation on SerpApi for details.

In this tutorial, we will scrape and extract organic results from  New York, NY, USA coffee shops. The data contains: "position", "title", "link", "reviews", "rating", "price" and "phone". You can also scrape more information with SerpApi.

First, you need to install the SerpApi client library.

pip install google-search-results

Set up the SerpApi credentials and search.

from serpapi import GoogleSearch
import os, json

params = {
    'api_key': 'YOUR_API_KEY',         # your serpapi api
    'engine': 'yelp',                  # SerpApi search engine	
    'find_desc': 'Coffee',              # query
    'find_loc': 'New York, NY, USA',    # location
}

To retrieve the Yelp organic results for a given search query, you can use the following code:

results = GoogleSearch(params).get_dict()['organic_results']

You can store Yelp organic results JSON data in databases or export them to a CSV file.

import csv

header = ['position', 'title', 'link', 'reviews', 'rating', 'price', 'phone']

with open('yelp_results.csv', 'w', encoding='UTF8', newline='') as f:
    writer = csv.writer(f)

    writer.writerow(header)

    for item in results:
        print(item)
        writer.writerow([item.get('position'), item.get('title'), item.get('link'), item.get('reviews'), item.get('rating'), item.get('price'), item.get('phone')])

There are some common challenges when scraping huge amounts of data from Yelp: bypassing CAPTCHAs, IP blocking, session timeouts, dynamic web structure, pagination and infinite scrolling, data variability complexity, and legal and ethical considerations. Luckily you don't need to tackle these problems yourself. SerpApi takes care of them and provides high-quality Yelp Search Results APIs for your usage. Using SerpApi saves you a lot of time so you can concentrate on your business.

If you have any questions, please feel free to contact me.