Scraping Google Maps with SerpApi - an Overview of the Google Maps API

Ryan Schafer -

Intro

SerpApi streamlines the process of scraping search results pages. Today we will take a look inside the powerful Google Maps API. This feature makes it easy to scrape businesses and locations for info, photos, and reviews. You will see that whether you are starting with name of a specific location, a keyword search for a list of related businesses, or a list of IDs you have collected for a location, you can find the data you are looking for using the right query or sequence of queries.  This can be a powerful tool for content aggregators, business owners, entrepreneurs, and others. To see this in action, head to the Playground, where you can try out different search queries and parameters.

This post is intended to provide a general overview of how to scrape Google Maps using SerpApi. We will use the Playground to demonstrate the process and make each step clear. We won't delve into any code or implementation details specific to a particular method or programming language. If you are looking for more details about how to do this with or without code, I recommend this tutorial for scraping Google Maps Local Results with Python, or this post about scraping Google Maps into a Google Sheets spreadsheet. You can also view the Related Post links at the bottom of the page.

Google Maps Local Results API

The usual flow for scraping Google Maps begins with a Google Maps Local Results API search. This is because in order to get detailed information about a business or location, we need one of two unique identifiers: place_id and data_id. Some users may already have a list of these they have collected using a different method, but most need to get them by searching Google Maps for either a specific business, or a keyword.

You can use the Google Maps Local Results API by making sure the Google Maps engine is selected, and setting the type parameter to "Search:

You will also need to specify a q (query) parameter and an ll (GPS coordinates) parameter. The Playground will supply some default values for these after you select the engine and type.  In this example we are using the default paramaters provided by the Playground to search for locations with the keyword "coffee" in New York:

You can see in the example above that a lot of information is available for these businesses just by using the Google Maps Local Results API.

Querying Google Maps Place Results API

If you need more details about a business or location than are available in the Google Local Results, we can use the Google Maps Place Results API.

The easiest way to do this is with the place_id.

Go back to the Google Maps Local Results API and search for McDonald's.

Let's say the top result is the one we are looking for. Scroll down in the JSON output and find the result in position 1. You can see there is a key named place_id.

We will now use the place_id to query the Google Maps Place Results API. For this we can use the regular Google Maps engine and besides place_id we don't need to include any other parameters - though it is generally recommended to use all of the available location parameters with queries to SerpApi. You can see in the example below that we have only provided the place_id :

Playground

What if we don't have the place_id for this location but we do have the data_id? We can also search Google Maps Place Results with the data_id but it is a little more complicated. We have to set type=place and we also need to provide a value for the data parameter. You might think you can just pass the data_id here but to create the data parameter we actually need to use the following format:

!4m5!3m4!1s + data_id + !8m2!3d + latitude + !4d + longitude

So we will also need the latitude and longitude of the location in order to search for it with the data_id. If you look again at our Google Maps search for McDonald`s you will see that the GPS coordinates are also returned in the results for each location listed:

So our `data parameter will be !4m5!3m4!1s + 0x89c25856329e3e85:0x36266aa626175152 + !8m2!3d + 40.762268999999996 + !4d + -73.9837579

or "!4m5!3m4!1s0x89c25856329e3e85:0x36266aa626175152!8m2!3d40.762268999999996 !4d-73.9837579"

Click here to see it in the Playground.

Google Maps Reviews API

So how do we get reviews for a business? Go back to the Playground and select the Google Maps Reviews engine.

At the top of the window you will see there is a required parameter named data_id with a value that looks like a string of random letters and numbers.

We need the data_id in order to get the reviews for a given business. So if we want to get all the reviews for a nearby McDonald's, we first need to go back to the Google Maps Local Results API and search for McDonald's:

Let's say the top result is the one we are looking for. Scroll down in the JSON output and find the result in position 1, then find the data_id. Just below the place_id we extracted to use with the Google Maps Place Results API, you can see there is a key named data_id .

Now we can go back to Google Maps Reviews, and search using the data_id:

Google Maps Photos API

You can scrape photos and info about photos from any business or location with the Google Maps Photos API and the Google Maps Photo Meta API. Google Maps Photos requires the  data_id of the location - this is the same data_id we can use to get Reviews, or construct a data parameter for searching Places. Google Maps Photo Meta also requires a data_id , but it is the data_id of the particular photo, not of the location. Don't get confused and try to use the data_id of a location to search the Google Maps Photo Meta API - it won't work.

The Google Maps Photos API and the Google Maps Photo Meta API also have their own separate engines - google_maps_photos and google_maps_photo_meta respectively.

Instead of looking at photos of McDonald's, lets search for Yellowstone Park. First we need to do a regular Google Maps (or Google Maps Local Results) search to get the data_id for the location:

Now we can search with the Google Maps Photos API:

If we want detailed information about the first photo, we will need to get it's data_id from the photo_meta_serpapi_link:

Then we can use the data_id in a request to the Google Maps Photo Meta API:

Getting the data_id with a place_id or Vice Versa

You might find yourself in a situation where you have one kind of ID and you need the other. For example, you might have a list of place_ids you've collected, but in order to use the Google Maps Reviews API or the Google Maps Photos API you need the data_ids. This is fortunately very easy to remedy. We've already seen that either a data_id or a place_id can be used to query the Google Maps Place Results API.

If you make a request to Google Maps Place Results API you can see that both the data_id and the place_id are included in the JSON:

Conclusion

We've seen how to scrape businesses and locations for info, photos, and reviews using SerpApi's Google Maps API.

I hope you found the tutorial informative and easy to follow. If you have any questions, feel free to contact me at ryan@serpapi.com.

Documentation: