When building your own mobile application, understanding the frequent complaints and problems in competitors' offerings can be invaluable. With app store optimization (ASO) tools like the Apple App Store Search API and Apple App Store Reviews API from SerpApi, combined with natural language models, it's easy to analyze user reviews and general information for other apps. That can give you ideas for features in your own project, boost your app's visibility, and improve conversion rates.

This guide will use a Python script, APIs from SerpApi, and additional libraries for running local models for analyzing the reviews.

Setting up the Python project

First, install Python on your computer if you haven't already, then create the folder you want to use for this project. Next, open the Terminal (Mac/Linux) or PowerShell (Windows) application, and change the current directory to that project folder, like this:

cd ~/Desktop/app-store-reviews
📁
Not sure how to find a folder's path? Type cd and a space, then drag the folder into your Terminal or PowerShell window.

Next, create a Python virtual environment for the project, so all dependencies and other data stays within the folder:

python -m venv venv
💻
Python command not found? Try "python3" instead. The name of the executable depends on how it was installed.

Now, activate the Python virtual environment. On a Mac with the default bash/zsh shell, run this:

source venv/bin/activate

On a Windows PC in PowerShell, run this:

venv\Scripts\Activate.ps1

Next, install the transformers, torch, python-dotenv, and requests Python libraries we'll need for this project:

pip install transformers requests python-dotenv

Finally, create a new Python script in the project folder. It can be called main.py or anything else.

Getting started with SerpApi

You need a free SerpApi account before you can use the Google Play Store APIs. You can upgrade to a paid account in the future if you need more search capacity, faster speeds, or other features.

First, create an account and verify your email and other details. After that, get the API key from your account dashboard.

The API key in the dashboard

The best way to store the API key for this project is with environment variables. Create a new text file in your project called .env with these contents, replacing the YOUR_KEY section with your own API key:

SERPAPI_KEY=YOUR_KEY

You can also do that directly from the Terminal on Mac or Linux like this:

echo "SERPAPI_KEY=YOUR_KEY" >> .env

The equivalent for PowerShell on Windows would be this:

Add-Content -Path .env -Value "SERPAPI_KEY=YOUR_KEY"

Make sure to not share your .env file with anyone! If the key is leaked or stolen, you can make a new one from the SerpApi account dashboard.

Writing the Python script

Now it's time to start writing code! Create a new Python script called main.py in your project, and start by importing all the libraries we'll need:

from collections import Counter
from transformers import AutoModelForQuestionAnswering, AutoTokenizer, pipeline
from itertools import chain
import requests
import asyncio
import os
import torch
from dotenv import load_dotenv

Some of these are built-in libraries, some are ones that were downloaded and installed earlier with the pip command. The language models will be downloaded when the script runs.

Collecting product IDs from the Apple App Store

SerpApi's Apple App Store Search Scraper API is a tool used for gathering in-depth metrics, keyword rankings, the number of app downloads, and many more details of an app in the iOS App Store, or Mac App Store. It is an essential tool for keyword tracking and gives you good insight into how the App Store’s ranking algorithm works.

The first function will gather the product IDs of apps related to our search term, using the Apple App Store Search Scraper API. Add this code to the script:

def call_app_store_api(search_term, api_key):
  print("-----")
  print(f"Gathering product ids from SerpApi's Apple App Store Scraper API for search term: {search_term}")
  app_store_url = f"https://serpapi.com/search.json?api_key={api_key}&engine=apple_app_store&term={search_term}"
  response = requests.get(app_store_url)
  product_ids = [organic_result["id"] for organic_result in response.json()["organic_results"]]
  
  return product_ids

This function takes in a search term and API key and returns a list of product IDs for apps related to that search term.

Collecting reviews from the Apple App Store

Once we have the product IDs of the apps related to our search term, we can use the Apple App Store Reviews Scraper API to gather the reviews of each app with automation. This can be done by adding the following code:

async def call_apple_reviews_api(product_id, api_key):
  print("-----")
  print(f"Gathering Reviews from Apple App Store Reviews Scraper API for product id: {product_id}")
  apple_reviews_url = f"https://serpapi.com/search.json?api_key={api_key}&engine=apple_reviews&product_id={product_id}&sort=mostcritical"
  response = requests.get(apple_reviews_url)
  reviews = [review["text"] for review in response.json().get("reviews", [])]
  return reviews

This function takes in a product ID of an App Store Listing and SerpApi API key, and returns a list of reviews for that app, sorted from most negative to least negative with the help of sort=mostcritical parameter.

Collecting all reviews for a search term

To gather all reviews for apps related to our search term, we can use the following function in the code snippet:

async def get_product_reviews(product_ids, api_key):
  reviews = await asyncio.gather(*[call_apple_reviews_api(product_id, api_key) for product_id in product_ids])
  reviews = list(chain(*reviews))
  
  return reviews

This function takes in a list of product IDs and SerpApi API key, and returns a list of all reviews for those apps. Notice that it doesn’t store which reviews belong to which app name. It will serve all the reviews of top apps ranked in a relevant keyword to your app.

Classifying Reviews as Positive or Negative

Sometimes, an app could have a little number of reviews, or a user might give a 1-star but a positive review. These kinds of events might prevent you from seeing genuine negative reviews of your competition.

With the use of natural language processing models, we can classify the reviews as positive or negative, and return only the absolutely negative reviews in the tool. The following code snippet demonstrates this process:

def get_absolutely_negative_product_reviews(classifier, reviews):
  absolutely_negative_reviews = []
  candidate_labels = ['negative', 'positive']
  for review in reviews:
    print("-----")
    print(f"Classifying review: {review}")
    result = classifier(review, candidate_labels)
    index = result["scores"].index(max(result["scores"]))
    classification = result["labels"][index]
    print(f"Classification: {classification}")
    if classification == "negative":
      absolutely_negative_reviews.append(review)
  
  return absolutely_negative_reviews

This function takes in a classifier and a list of reviews and returns a list of only the absolutely negative reviews.

Summarizing Absolutely Negative Reviews

To gain a clearer understanding of the specific problems mentioned in the absolutely negative reviews, we can use a question-answering pipeline to summarize the reviews. This is important to find the main concern of the user in a long review, or in a mixed review with positive and negative aspects. Imagine the following review:

Screenshots are nice. But they don’t represent the app.

What we want is the concern for the honesty of the developer from this review which is the latter sentence. So we want the question-answer pipeline to give us keyword suggestions into what the main problem is in the review. Add this code to the script:

def get_summaries(qa_model, absolutely_negative_reviews):
  tokenizer, model = qa_model
  summaries = []
  for context in absolutely_negative_reviews:
    print("-----")
    print(f"Summarizing review: {context}")
    inputs = tokenizer(
      "What is the problem?",
      context,
      return_tensors="pt",
      truncation="only_second",
      max_length=512,
      return_offsets_mapping=True
    )
    offset_mapping = inputs.pop("offset_mapping")[0]
    context_token_indexes = [
      index for index, sequence_id in enumerate(inputs.sequence_ids(0))
      if sequence_id == 1
    ]

    with torch.no_grad():
      outputs = model(**inputs)

    if context_token_indexes:
      start_index = max(context_token_indexes, key=lambda index: outputs.start_logits[0][index])
      end_index = max(
        [index for index in context_token_indexes if index >= start_index],
        key=lambda index: outputs.end_logits[0][index]
      )
      result = context[int(offset_mapping[start_index][0]):int(offset_mapping[end_index][1])].strip()
    else:
      result = ""

    print(f"Summary: {result}")
    summaries.append(result)
  
  return summaries

This function takes in a question-answering pipeline and a list of absolutely negative reviews and returns a list of summaries for those reviews.

Classifying summaries, and returning distribution of problems

Understanding the ratio of problems in your competition’s reviews is a crucial part of mechanics in app ranking, user acquisition, and overall store intelligence.

To understand the distribution of problems mentioned in the summaries, we can classify the summaries with a preset of potential problems and return the distribution of problems. The following code snippet demonstrates this process:

def get_problems_distribution(classifier, possible_problems, summaries):
  summary_dicts = []
  for summary in summaries:
    print("-----")
    print(f"Classifying summary: {summary}")
    result = classifier(summary, possible_problems)
    index = result["scores"].index(max(result["scores"]))
    classification = result["labels"][index]
    print(f"Classification: {classification}")
    summmary_dict = {"summary": summary, "classification": classification}
    summary_dicts.append(summmary_dict)

  classifications = [d['classification'] for d in summary_dicts]
  counter = Counter(classifications)
  total = sum(counter.values())
  ratios = {problem: counter[problem]/total for problem in possible_problems}
  ratios = dict(sorted(ratios.items(), key=lambda item: item[1], reverse=True))
  
  return ratios

This function takes in a classifier, a list of possible problems, and a list of summaries, and returns the distribution of problems mentioned in the summaries.

Putting it all together

Finally, we need the script to read the SerpApi API key from the .env file, define the term to search and the possible problems, set the classifier model, and run all the functions. Here's the code for that:

load_dotenv()
api_key = os.environ["SERPAPI_KEY"]
term = "to do list"
possible_problems = [
  "bugs",
  "too many advertisements",
  "inefficiency",
  "dishonesty",
  "unenjoyable",
  "incompleteness",
  "expensiveness"
]
classifier = pipeline(
  "zero-shot-classification",
  model="facebook/bart-large-mnli"
)
qa_model_name = "bert-large-uncased-whole-word-masking-finetuned-squad"
qa_model = (
  AutoTokenizer.from_pretrained(qa_model_name),
  AutoModelForQuestionAnswering.from_pretrained(qa_model_name)
)

product_ids = call_app_store_api(term, api_key)
reviews = asyncio.run(get_product_reviews(product_ids, api_key))
absolutely_negative_reviews = get_absolutely_negative_product_reviews(classifier, reviews)
summaries = get_summaries(qa_model, absolutely_negative_reviews)
ratios = get_problems_distribution(classifier, possible_problems, summaries)

# Print results
print("-----")
print(f"Summary of problems for search term: {term}")
for ratio in ratios:
  print(f"{ratio}: {ratios[ratio]*100:.2f}%")

Running the script

You can run the completed script in the Terminal or PowerShell window, like this:

python ./script.py

The script will fetch all the reviews, classify them, and give you a complete report like this:

Summary of problems for search term: to do list
incompleteness: 38.16%
unenjoyable: 22.49%
expensiveness: 19.98%
inefficiency: 8.73%
dishonesty: 5.62%
bugs: 3.35%
too many advertisements: 1.67%

The first run will take the longest, since it will have to download the classification models.

Using the data for app store optimization

With the results of our analysis, we can see that the top problems among the competition's negative reviews for the search term "to do list" are incompleteness, unenjoyable, and expensiveness.

Here are some specific strategies to address these issues:

  • Incompleteness: Identify which features are missing, and add them to your app. This could involve conducting market research, surveying customers, localization demands, notifications for bugs or gathering data on the features of competitors' apps.
  • Unenjoyable experiences: You can gather customer feedback to gain a better understanding of what is causing the negative experiences.
  • Expensiveness: Addressing this could involve conducting market research to understand the prices of similar apps and identifying opportunities to offer competitive pricing. Additionally, implementing in-app purchase options or subscription models can provide alternative revenue streams and make the app more accessible to a wider range of customers.

Conclusion

Outranking the competition in the search results requires gathering and analyzing customer feedback, understanding the competition, and implementing solutions to address the identified problems. By using natural language models and APIs from SerpApi, you can automate some steps of the process, while still gaining insight on possible improvements for your own app.

If you need help using SerpApi, please contact us.

Full code for script

Here is the complete working example of the analysis script:

from collections import Counter
from transformers import AutoModelForQuestionAnswering, AutoTokenizer, pipeline
from itertools import chain
import requests
import asyncio
import os
import torch
from dotenv import load_dotenv

# Collect product_ids from Apple App Store Search Scraper API
def call_app_store_api(search_term, api_key):
  print("-----")
  print(f"Gathering product ids from SerpApi's Apple App Store Scraper API for search term: {search_term}")
  app_store_url = f"https://serpapi.com/search.json?api_key={api_key}&engine=apple_app_store&term={search_term}"
  response = requests.get(app_store_url)
  product_ids = [organic_result["id"] for organic_result in response.json()["organic_results"]]
  
  return product_ids

# Collect reviews of an individual app from most negative to least negative from Apple App Store Reviews Scraper API
async def call_apple_reviews_api(product_id, api_key):
  print("-----")
  print(f"Gathering Reviews from Apple App Store Reviews Scraper API for product id: {product_id}")
  apple_reviews_url = f"https://serpapi.com/search.json?api_key={api_key}&engine=apple_reviews&product_id={product_id}&sort=mostcritical"
  response = requests.get(apple_reviews_url)
  reviews = [review["text"] for review in response.json().get("reviews", [])]
  return reviews

# Collect all reviews of apps in a search term from Apple App Store Reviews Scraper API
async def get_product_reviews(product_ids, api_key):
  reviews = await asyncio.gather(*[call_apple_reviews_api(product_id, api_key) for product_id in product_ids])
  reviews = list(chain(*reviews))
  
  return reviews

# Classify reviews as positive or negative and return absolutely negative reviews
def get_absolutely_negative_product_reviews(classifier, reviews):
  absolutely_negative_reviews = []
  candidate_labels = ['negative', 'positive']
  for review in reviews:
    print("-----")
    print(f"Classifying review: {review}")
    result = classifier(review, candidate_labels)
    index = result["scores"].index(max(result["scores"]))
    classification = result["labels"][index]
    print(f"Classification: {classification}")
    if classification == "negative":
      absolutely_negative_reviews.append(review)
  
  return absolutely_negative_reviews

# Summarize absolutely negative reviews
def get_summaries(qa_model, absolutely_negative_reviews):
  tokenizer, model = qa_model
  summaries = []
  for context in absolutely_negative_reviews:
    print("-----")
    print(f"Summarizing review: {context}")
    inputs = tokenizer(
      "What is the problem?",
      context,
      return_tensors="pt",
      truncation="only_second",
      max_length=512,
      return_offsets_mapping=True
    )
    offset_mapping = inputs.pop("offset_mapping")[0]
    context_token_indexes = [
      index for index, sequence_id in enumerate(inputs.sequence_ids(0))
      if sequence_id == 1
    ]

    with torch.no_grad():
      outputs = model(**inputs)

    if context_token_indexes:
      start_index = max(context_token_indexes, key=lambda index: outputs.start_logits[0][index])
      end_index = max(
        [index for index in context_token_indexes if index >= start_index],
        key=lambda index: outputs.end_logits[0][index]
      )
      result = context[int(offset_mapping[start_index][0]):int(offset_mapping[end_index][1])].strip()
    else:
      result = ""

    print(f"Summary: {result}")
    summaries.append(result)
  
  return summaries

# Classify summaries and return distribution of problems
def get_problems_distribution(classifier, possible_problems, summaries):
  summary_dicts = []
  for summary in summaries:
    print("-----")
    print(f"Classifying summary: {summary}")
    result = classifier(summary, possible_problems)
    index = result["scores"].index(max(result["scores"]))
    classification = result["labels"][index]
    print(f"Classification: {classification}")
    summmary_dict = {"summary": summary, "classification": classification}
    summary_dicts.append(summmary_dict)

  classifications = [d['classification'] for d in summary_dicts]
  counter = Counter(classifications)
  total = sum(counter.values())
  ratios = {problem: counter[problem]/total for problem in possible_problems}
  ratios = dict(sorted(ratios.items(), key=lambda item: item[1], reverse=True))
  
  return ratios

load_dotenv()
api_key = os.environ["SERPAPI_KEY"]
term = "to do list"
possible_problems = [
  "bugs",
  "too many advertisements",
  "inefficiency",
  "dishonesty",
  "unenjoyable",
  "incompleteness",
  "expensiveness"
]
classifier = pipeline(
  "zero-shot-classification",
  model="facebook/bart-large-mnli"
)
qa_model_name = "bert-large-uncased-whole-word-masking-finetuned-squad"
qa_model = (
  AutoTokenizer.from_pretrained(qa_model_name),
  AutoModelForQuestionAnswering.from_pretrained(qa_model_name)
)

product_ids = call_app_store_api(term, api_key)
reviews = asyncio.run(get_product_reviews(product_ids, api_key))
absolutely_negative_reviews = get_absolutely_negative_product_reviews(classifier, reviews)
summaries = get_summaries(qa_model, absolutely_negative_reviews)
ratios = get_problems_distribution(classifier, possible_problems, summaries)

# Print results
print("-----")
print(f"Summary of problems for search term: {term}")
for ratio in ratios:
  print(f"{ratio}: {ratios[ratio]*100:.2f}%")