Google Play Store: How to Research the Market

Introduction

The Google Play Store is the most popular application marketplace for Android devices. It suggests thousands of different applications. Therefore, creating a new app that is better and more popular than existing ones has become much more challenging. Luckily, with SerpApi, you can conduct market research and refine your strategies for better success.

Setting Up a SerpApi Account

SerpApi offers a free plan for newly created accounts. Head to the sign-up page to register an account and complete your first search with our interactive playground.

You can make requests programmatically using your Api Key.

Find your competitors

You're developing a new flashcard app that could transform the education market. But have you considered your competition? Use our Google Play Store Api to find them.

Simply visit Playground or send a request directly from your IRB console.

For the rest of this guide, all examples will use Ruby 3.4.1

require 'net/http'
require 'json'

url = URI::HTTPS.build({
  host: "serpapi.com",
  path: "/search.json", 
  query: "engine=google_play&q=flashcards&api_key=#{API_KEY}"
})

response = JSON.parse(Net::HTTP.get(url))

applications = response
  .dig("organic_results")
  .flat_map { it["items"] }
  .tap { puts it.count }
# => 20

A lot of applications, right? And this is just the first page! Check out the serpapi_pagination values to get more.

Extract details

You've already gathered a lot of information here. You can group applications based on rating, downloads, and price.

{
  "title": "Flashcards Deluxe",
  "link": "https://play.google.com/store/apps/details?id=com.orangeorapple.flashcards",
  "product_id": "com.orangeorapple.flashcards",
  "serpapi_link": "http://serpapi.com/search.json?engine=google_play_product....",
  "rating": 4.5,
  "author": "OrangeOrApple.com",
  "category": "Education",
  "downloads": "100,000+",
  "price": "$3.99",
  "extracted_price": 3.99,
  "thumbnail": "https://play-lh.googleusercontent.com/....",
  "feature_image": "https://play-lh.googleusercontent.com/...",
  "description": "Flashcards Deluxe is an easy to use yet...."
}

The good news is that SerpApi makes it easy for you to dive deeper using the serpapi_link. You can find the complete result description here.

# lazy enumerator helps you avoid sending all requests immediately
# and only fetch data when you actually need it
# In the future, you can experiment with Threads to speed up scraping
applications = applications.lazy
  .map { URI(it["serpapi_link"] + "&api_key=#{API_KEY}") }
  .map { JSON.parse(Net::HTTP.get(it)) }

About this app

With the latest update, you can now use these parameters:

• version - Research not only when and what updates were made (check what_s_new), but also determine whether it's a major or minor update.
• released_on - Check when the application was first released.
• requires_android - Find out the minimum Android version required.
• permissions - Review the permissions the app requires.

"about_this_app": {
  "snippet": "Flashcards Deluxe is an easy to use ....",
  "version": "4.72",
  "requires_android": "5.0",
  "released_on": "Nov 26, 2012",
  "updated_on": "Jan 28, 2025",
  "downloads": "100,000+",
  "content_rating": "Everyone",
  "interactive_elements": "Users Interact",
  "offered_by": "OrangeOrApple.com",
  "permissions": [
    {
      "type": "Microphone",
      "details": [
        "record audio"
      ]
    },
    {
      "type": "Other",
      "details": [
        "view network connections",
        "full network access",
        "control vibration",
        "Google Play license check"
      ]
    }
  ]
}

Sometimes, the version or requires_android keys are missing, which means the values are "Varies with device"

Save the result

While you can access your searches later using search_metadata.json_endpoint, keep in mind that we store this data for only 30 days, which may not be sufficient for in-depth research.

If you want to avoid your searches being stored, check out our ZeroTrace mode.

Let's save our results to files forever! I suggest saving the entire results, as they contain a wealth of information you might need in the future.

require 'date'
require 'fileutils'

# `Dir.home` returns the absolute path to your home directory
base_dir = "#{Dir.home}/google_play_research/flashcards"

applications.each do |application|
  app_name = application.dig("product_info", "title")
  app_dir = [base_dir, app_name].join("/")
  # Helpful when you need the scraper to execute daily
  timestamp = Date.today.to_s
  file_path = [app_dir, timestamp + '.json'].join("/")
  content = application.to_json

  # All necessary directories will be created if they don’t exist
  FileUtils.mkdir_p(app_dir)
  File.write(file_path, content)
end

Run daily

Launchd

You can also explore one of our NoCode integrations, or even deploy your code to AWS using CloudWatch Events to schedule your Lambda function.

For simplicity, this example will show how to schedule the job directly on your MacBook. Launchd is Apple's recommended tool for managing background services and scheduled tasks.

Test script

Let’s create a simple test script to verify everything is working correctly before we start sending any real requests.

touch ~/google_play_research/scraper.rb

• touch creates a new, empty file.
• ~ is a shortcut to your home directory.

Now, open the file and paste in this small Ruby script:

`osascript -e 'display notification "#{Time.now}" with title "Time"'`

This command will show a macOS notification with the current time.

To test it, run the script from the terminal:

ruby ~/google_play_research/scraper.rb

You should see a notification pop up in the top-right corner of your screen.

Test Agent

Let’s create our Launchd agent. Agents run on behalf of the currently logged-in user and will be responsible for scheduling our script.

User agents should be placed inside the ~/Library/LaunchAgents directory. We’ll use nano to create and edit the file:

nano ~/Library/LaunchAgents/local.google_play_research.scraper.plist

Paste the following XML configuration into the editor.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
  <dict>
    <key>Label</key>
    <!-- This will be the unique identifier for our new agent -->
    <string>local.google_play_research.scraper</string>

    <key>EnvironmentVariables</key>
    <dict>
      <key>API_KEY</key>
      <!-- You’ll need to add your SerpApi API key here -->
      <string>e2653...</string>
    </dict>

    <key>ProgramArguments</key>
    <array>
      <!-- 
        The absolute path to your Ruby binary. 
        Use `which ruby` to determine it 
      -->
      <string>/opt/homebrew/opt/ruby@3.4/bin/ruby</string>
      <!-- 
        Specify the absolute path to your script.
        Do not use shorthand 
        (e.g., replace 'krugloff' with your own username)
      -->
      <string>/Users/krugloff/google_play_research/scraper.rb</string>
    </array>

    <!-- This script is scheduled to run daily at 1:00 AM -->
    <key>StartCalendarInterval</key>
    <dict>
      <key>Hour</key>
      <integer>1</integer>
      <key>Minute</key>
      <integer>0</integer>
    </dict>
  </dict>
</plist>

Initially, set the time to 5 minutes ahead of your current time. This will help you test if everything is working as expected. If the test doesn't work, try adjusting the Hour value. I've noticed that Launchd on my laptop uses a timezone that's 3 hours behind my local time.

Once done:

• Press CTRL+O to save the file
• Press ENTER to confirm the filename
• Press CTRL+X to exit the editor

Don’t worry - nothing will run just yet.

To run the job, use the following command:

launchctl load ~/Library/LaunchAgents/local.google_play_research.scraper.plist

At the scheduled time, you should see a notification appear in the top-right corner of your screen.

You can check the list of loaded jobs by running:
launchctl list | grep local

Scraper

Let's return to our scraper file and add the final script inside.

require 'net/http'
require 'json'
require 'date'
require 'fileutils'

API_KEY = ENV["API_KEY"]

url = URI::HTTPS.build({
  host: "serpapi.com",
  path: "/search.json", 
  query: "engine=google_play&q=flashcards&api_key=#{API_KEY}"
})

response = JSON.parse(Net::HTTP.get(url))

applications = response.dig("organic_results")
  .flat_map { it["items"] }
  .lazy
  .map { URI(it["serpapi_link"] + "&api_key=#{API_KEY}") }
  .map { JSON.parse(Net::HTTP.get(it)) }

base_dir = "#{Dir.home}/google_play_research/flashcards"

applications.each do |application|
  app_name = application.dig("product_info", "title")
  app_dir = [base_dir, app_name].join("/")
  timestamp = Date.today.to_s
  file_path = [app_dir, timestamp + '.json'].join("/")
  content = application.to_json

  FileUtils.mkdir_p(app_dir)
  File.write(file_path, content)
end

Then we will need to update scheduled time and reload our config.

launchctl unload ~/Library/LaunchAgents/local.google_play_research.scraper.plist
launchctl load ~/Library/LaunchAgents/local.google_play_research.scraper.plist

At the scheduled time, you should see the ~/google_play_research/flashcards folder being created. This indicates that the scraper script ran successfully and produced the expected output.

Research

Imagine how valuable this information can be! You can refine the script to send notifications whenever the data you care about changes. Alternatively, you can collect data over time and create insightful charts to track trends.

There are many questions you can find answers to, such as:

• How often do your competitors update their apps? Do they release frequent small fixes or occasional major updates?
• How quickly does an app reach 1 million downloads after its release?
• Is it worth supporting Android 5.0, or is the user base too small?
• How many permissions do popular apps require? Is it true that users prefer apps with fewer permissions?
• Do your competitors frequently update their app descriptions to improve their marketing impact?
• and more...

Why SerpApi?

Residential Proxies

The biggest advantage is that SerpApi bypasses blocks from Google. It removes the need to figure out how to uses proxies, CAPTCHAs and which providers are good, and there's no need to maintain the parser if Google Play updates again.

Speed

Our goal is to extract information as quickly as possible, and with our Ludicrous Speed technology, we make it even faster (approximately 1-2 seconds per request!).

To meet the needs of applications requiring the highest possible performance, we have introduced a new feature: Ludicrous Speed Max.

US Legal Shield

At SerpApi, we provide a "Legal US Shield":

SerpApi will assume liability of scraping and parsing search engine results for both domestic and foreign companies (— “Legal US Shield” —) unless your usage is otherwise illegal.

You can read more about our Terms of Service and what is in included in the Legal US Shield here.

Relevant Posts

You may be interested in reading more about our Google Play API, and other integrations we have.

Go ahead and grab some ideas from them as well!