How to scrape Yelp data using Python (Places and Reviews)

Have you ever wanted to gather data from Yelp, but found it time-consuming to check every place and review manually? In this blog, we’ll see how to scrape the Yelp website. We’ll show you how to extract useful information like business details and customer reviews in an automated way.

We offer three different APIs to scrape Yelp:
- Yelp Search API: Scrape results from the Yelp search page.
- Yelp Place API: Scrape details of a place.
- Yelp Reviews API: Scrape reviews from a place.

You can use a simple GET request or a dedicated SDK based on your programming language, including Python, Javascript, and more. In this post, we'll use the GET request method in Python.

Preparation for accessing SerpApi API

• Register at serpapi.com for free to get your SerpApi Api Key.
• Create a new main.py file
• Basic setup

Here is what the basic setup looks like

import requests
SERPAPI_API_KEY = "YOUR_REAL_SERPAPI_API_KEY"

params = {
    "api_key": SERPAPI_API_KEY,
    # soon
}

search = requests.get("https://serpapi.com/search", params=params)
response = search.json()
print(response)

With these few lines of code, we can access all of the search engines available at SerpApi, including Yelp. We'll need to adjust the parameters based on our needs.

Scrape Yelp Search Result page

First, we can use the Yelp Search API to get search results based on a particular query. It's useful when scraping many places in a specific location.

Documentation: https://serpapi.com/yelp-search-api

The required parameter is find_loc to define where you want the search to originate. The following location formats are acceptable:
- 706 Mission St, San Francisco, CA
- San Francisco, CA
- San Francisco, CA 94103
- 94103

The search query can be put in the find_desc parameter. Here is the complete code to search for "coffee" in Austin, Texas.

params = {
    "api_key": SERPAPI_API_KEY,
    "engine": "yelp",
    "find_desc": "coffee",
    "find_loc": "Austin, TX, United States"
}

search = requests.get("https://serpapi.com/search", params=params)
response = search.json()
print(response)

You can grab the organic_results or the ads_results when they are available.

organic_results = response["organic_results"]
ads_results = response["ads_results"]

Here is the result:

The organic_results will include:
- place_id
- title
- link
- categories
- price
- rating
- reviews
- neighborhoods
- phone
- snippet
- service_options
- thumbnail

Pagination
By default, Yelp returns 10 results per page. We can paginate the results using the start parameter. It skips the given number of results, e.g., 0 (default) is the first page of the results, 10 is the 2nd page of results, 20 is the 3rd page of results, etc.

params = {
    "api_key": SERPAPI_API_KEY,
    "engine": "yelp",
    "find_desc": "coffee",
    "find_loc": "Austin, TX, United States",
    "start": 10 #example value for 2nd page
}

If you prefer, we also have a video tutorial on this:

Scrape a specific place from Yelp

Next, we can scrape the individual page for a place using the Yelp Place API.

Documentation: https://serpapi.com/yelp-place

We need the place_id as the parameter to run this API. The place_id is available from the previous `organic_results,` or you can also use the last path of the original URL when you open a place directly on Yelp. For example, if you're interested in scraping this page https://www.yelp.com/biz/flora-coffee-austin, then we can use flora-coffee-austin as the place_id .

Here is the code sample:

params = {
    "api_key": SERPAPI_API_KEY,
    "engine": "yelp_place",
    "place_id": "flora-coffee-austin"
}

search = requests.get("https://serpapi.com/search", params=params)
response = search.json()
print(response)

Here is the result:

In the place_result response, you can find this information:

• place name
• description
• reviews
• rating
• categories
• images
• address
• directions
• popular items
• review highlights
• business map
• features
• operation hours
• and more!

Scrape places reviews from Yelp

You are probably interested in scraping reviews from a place as well. We've got you covered! You can use the Yelp Reviews API to scrape all the reviews.

Prefer watching? Here is the video tutorial:

We only need one parameter: place_id. It requires the gibberish place_id of a place. Note that you get two different IDs when using our Yelp Search API. We need the first place_id type for this.

Here is an example:

params = {
    "api_key": SERPAPI_API_KEY,
    "engine": "yelp_reviews",
    "start": 0,
    "num": "49",
    "place_id": "uyYG-z7yiSPhXmkauzb43Q"
}

search = requests.get("https://serpapi.com/search", params=params)
response = search.json()
print(response)

Num is the parameter to define the number of results per page (maximum is 49). We can use the start parameter to paginate the results parameter to paginate the results.

The start parameter defines the result offset. It skips the given number of results. It's used for pagination. (e.g., 0 (default) is the first page of results, 49 is the 2nd page of results, 98 is the 3rd page of results, etc.).

Here is the result:

Under the reviews response, you can find the comments and ratings for this place.

Filter for reviews API
You can filter the reviews based on a certain keyword using the q parameter or filter by the rating number using the rating parameter.

You can also sort the results using sort_by parameter. Possible values for sorting:

relevance_desc - Yelp Sort (default)
date_desc - Newest First
date_asc - Oldest Rated
rating_desc - Highest Rated
rating_asc - Lowest Rated
elites_desc - Elites

I hope it's helpful! Feel free to try!