Scraping Google Ads Transparency Center with SerpApi and Node.js

Introduction

In the current digital era, advertising has evolved into a complex landscape that requires understanding various strategies and tools. This makes the data that Google Ads Transparency Center provides, an invaluable resource for marketers, advertisers, researchers, and business strategists.

This data offers insights into the advertising campaigns that shape consumer perceptions and drive market trends. We can access detailed information about ad creatives, campaign durations, formats, and more using the Google Ads Transparency Center API. In this article, we'll explore this API's different parameters and capabilities. We'll also see their practical application through real-world examples.

Setup

First, register for a free account at SerpApi. You can then retrieve your API key from your account's Dashboard.

Next, we need to install the serpapi package:

npm install serpapi

We're now ready to explore the Google Ads Transparency Center API!

Overview of the API Parameters

In the following sections, we will review key parameters such as advertiser_id, text, region, start_date, end_date, and creative_format. Each section will explain the significance of these parameters, how they can be used to extract specific data and their practical use. By the end of this article, you'll be able to utilize the full potential of the Google Ads Transparency Center API and extract all the data that you need.

Understanding and Utilizing the `advertiser_id` Parameter

The ability to precisely target and analyze specific advertisers' campaigns is invaluable. This is why the advertiser_id parameter in the Google Ads Transparency Center API is crucial. It is a unique identifier assigned to each advertiser within the Google Ads platform.

This ID is critical in distinguishing between the different advertisers and accessing data specific to each one. When you use the advertiser_id in an API query, you are filtering the data to retrieve only information related to the advertisements run by that specific advertiser. This can include details about their ad creatives, campaign durations, ad formats, and more.

Finding the Advertiser ID

First, you need to navigate to the Google Ads Transparency Center website. Then, use the search function on the website to look up the advertiser of interest. For example, to analyze Tesla Inc.'s advertising, search for "Tesla"

Once you find the advertiser, the URL of their specific page will contain the advertiser_id. It typically follows the format https://adstransparency.google.com/advertiser/ADVERTISER_ID. For example, Tesla Inc.'s URL might look like https://adstransparency.google.com/advertiser/AR17828074650563772417, where "AR17828074650563772417" is the advertiser_id.

Example

In this example, the code targets Tesla Inc.'s advertising campaigns using its advertiser_id. Analyzing the output lets you gain insights into Tesla's advertising strategy, preferred ad formats, and campaign durations.

const { getJson } = require("serpapi");

const API_KEY = 'YOUR_API_KEY'; // Replace with your actual SerpApi key
const teslaAdvertiserId = 'AR17828074650563772417'; // Tesla Inc.'s Advertiser ID

async function fetchAdsByAdvertiserId(advertiserId) {
  try {
    const params = {
      engine: "google_ads_transparency_center",
      api_key: API_KEY,
      advertiser_id: advertiserId,
    };

    const response = await getJson(params);

    // Here you can process the response as needed
    console.log("Response: ", response);
  } catch (error) {
    console.error("Error fetching ads: ", error);
  }
}

// Fetching ads by Tesla's Advertiser ID
fetchAdsByAdvertiserId(teslaAdvertiserId);

Understanding the `text` Parameter

The text parameter is important, mainly when specific advertiser IDs are not the focus. Unlike the advertiser_id parameter, which targets a single advertiser's campaigns, the text parameter offers a different approach, especially beneficial when exploring a broader range of advertisers under a common domain or name.

Difference between `text` and `advertiser_id`

The text parameter is designed to perform searches related to a domain or a general advertiser name within the context of Google Ads. This parameter is handy in situations where:

You're interested in retrieving data for advertisers with similar names or who fall under a common domain.
In cases where the specific advertiser_id is unknown or when a broader, more inclusive search is required.
When aiming to compare advertising strategies of multiple entities within the same industry or domain.

Unlike the advertiser_id search, which precisely identifies a single advertiser, the text parameter pulls data from multiple relevant advertisers.

Example

Consider a scenario where you want to analyze advertising trends related to streaming services, focusing on entities similar to or including Netflix. Instead of limiting the search to a specific advertiser ID, you can use the text parameter instead.

const { getJson } = require("serpapi");

const API_KEY = 'YOUR_API_KEY'; // Replace with your SerpApi key

async function fetchAdsByText() {
  try {
    const response = await getJson({
      engine: "google_ads_transparency_center",
      api_key: API_KEY,
      text: 'Netflix', // Text parameter for broader search
    });
    console.log("Ads related to Netflix: ", response);
  } catch (error) {
    console.error("Error: ", error);
  }
}

fetchAdsByText();

In this example, setting the text parameter to 'Netflix' allows the API to fetch ads from various accounts related to or named similarly to Netflix. This approach helps compare the advertising strategies of multiple entities within the same domain.

Utilizing the `region` Parameter to Localize Ad Data

Understanding how advertising strategies are tailored to different regions is essential for businesses aiming to connect with diverse audiences. This can be achieved using the region parameter. It allows for filtering ad data based on specific geographic locations.

Finding the Region Code

A complete list of the supported region codes is available in our Google Ads Transparency Center Regions documentation. In the example below, we'll use the region codes for the United States (2840) and Canada (2124).

Example

In this example, we will retrieve Etsy's advertisements in the United States and Canada using the specific region codes.

const { getJson } = require("serpapi");

const API_KEY = 'YOUR_API_KEY'; // Replace with your SerpApi key
const etsyAdvertiserId = 'AR06057634518662119425'; // Etsy's Advertiser ID

async function fetchEtsyAdsByRegion(regionCode) {
  try {
    const response = await getJson({
      engine: "google_ads_transparency_center",
      api_key: API_KEY,
      advertiser_id: etsyAdvertiserId,
      region: regionCode, // Using region code
    });
    console.log(`Etsy's Ads in region ${regionCode}: `, response);
  } catch (error) {
    console.error("Error: ", error);
  }
}

// Fetching ads for Etsy in the United States (2840) and Canada (2124)
fetchEtsyAdsByRegion('2840'); // United States
fetchEtsyAdsByRegion('2124'); // Canada

You can use the data from these requests to compare Etsy's advertising content, format, and frequency in the United States versus Canada. It can provide insights into how Etsy adapts its advertising to resonate with the cultural and consumer preferences specific to each region and guide how to tailor marketing strategies for each one effectively.

Using `start_date` and `end_date` for Period-Specific Analysis

The start_date and end_date parameters can be used to understand how marketing strategies are tailored to specific timeframes. They allow you to define a particular time range for your search. This functionality is fundamental for separating ad data from particular periods, such as holiday seasons, sales events, or product launches.

Use Cases

By using the start_date and end_date parameters, you can gather useful information about how advertising strategies are adapted for specific events or periods:

Understand how ad strategies change during peak shopping seasons, like Black Friday.
Measure the impact of specific marketing campaigns by analyzing data from the campaign period.
Compare data from similar periods across different years to identify trends and changes in advertising approaches.

Example

To illustrate the power of the start_date and end_date parameters, let's fetch advertising data for the period around Black Friday for ASOS, a famous fashion retailer.

const { getJson } = require("serpapi");

const API_KEY = 'YOUR_API_KEY'; // Replace with your SerpApi key
const asosAdvertiserId = 'AR15983676399282028545'; // ASOS's Advertiser ID

async function fetchASOSBlackFridayAds() {
  try {
    const response = await getJson({
      engine: "google_ads_transparency_center",
      api_key: API_KEY,
      advertiser_id: asosAdvertiserId,
      start_date: '20231120', // One week before Black Friday
      end_date: '20231127', // Black Friday
    });
    console.log("ASOS Black Friday Ads: ", response);
  } catch (error) {
    console.error("Error: ", error);
  }
}

fetchASOSBlackFridayAds();

You can use the ads displayed during the Black Friday week to examine promotional tactics and the intensity of the ad campaign. This data can also be compared with ASOS's advertising at other times of the year to measure how their Black Friday strategy differs from their usual marketing approach. The analysis might also reveal insights into how effective these strategies are regarding consumer engagement and response.

Pagination with `num` and `next_page_token`

The response from Google Ads Transparency Center can contain thousands of ads. We can use the parameters num and next_page_token for effective data retrieval and page navigation.

num: This parameter determines the number of results returned in a single API call. You can maximize the data retrieved per request by setting num to a higher value (e.g., 100 - maximum value).
next_page_token: The parameter provides a token that you use in subsequent API calls to retrieve the next set of results.

Implementing Pagination

You can achieve effective pagination by following the steps below:

Initial Data Retrieval: Execute an initial request without next_page_token.
Sequential Access: Fetch the next results page using the next_page_token from the current response.
End Condition: Implement a check for the existence of next_page_token. If it's not present, it indicates no more pages to retrieve.

Let's see this in practice.

Example

const { getJson } = require("serpapi");

const API_KEY = 'YOUR_API_KEY'; // Replace with your SerpApi key
const appleAdvertiserId = 'AR00604964697900318721'; // Apple's Advertiser ID

async function fetchAppleAds(nextToken = '') {
    try {
      const params = {
        engine: "google_ads_transparency_center",
        api_key: API_KEY,
        advertiser_id: appleAdvertiserId,
        num: 100, // Maximizing results per request
      };
  
      if (nextToken) {
        params.next_page_token = nextToken;
      }
  
      const response = await getJson(params);
  
      console.log("Apple Ads Data: ", response);
  
      // Check for next page
      if (response.serpapi_pagination && response.serpapi_pagination.next_page_token) {
        await fetchAppleAds(response.serpapi_pagination.next_page_token);
      } else {
        console.log("All pages retrieved");
      }
    } catch (error) {
      console.error("Error: ", error);
    }
  }
  
  fetchAppleAds();

The function fetchAppleAds starts with the first page of results, setting num to 100. If next_page_token is present in the response, the function calls itself recursively with the new token, fetching the next set of results. When there is no next_page_token, it indicates that all pages have been retrieved, and the recursive function calls stop.

Retrieving all available data allows for an in-depth analysis of Apple's advertising strategies, including ad formats, campaign durations, and regional targeting.

Exploring `creative_format`

The digital advertising landscape offers a variety of creative formats. Understanding these formats' impact on audience engagement is essential.

Understanding `creative_format` and Its Options

The creative_format parameter allows you to filter ad data based on the type of creative used. It includes three primary options:

Text: Ads that primarily contain text. These are often direct and effectively communicate clear messages.
Image: Ads using visuals to attract and engage the audience. Image ads are vital for creating an immediate visual impact.
Video: Video ads offer a dynamic and immersive experience, ideal for storytelling and emotional engagement.

Example

We’ll examine GoPro's recent campaigns to identify which creative_format they predominantly use.

const { getJson } = require("serpapi");

const API_KEY = 'YOUR_API_KEY'; // Replace with your SerpApi key
const goproAdvertiserId = 'AR06271179193107611649'; // GoPro's Advertiser ID

async function fetchGoProAdsByFormat(format) {
  let totalAds = 0;
  async function paginateAds(nextToken = '') {
    const params = {
      engine: "google_ads_transparency_center",
      api_key: API_KEY,
      advertiser_id: goproAdvertiserId,
      creative_format: format,
      num: 100,
    };

    if (nextToken) {
      params.next_page_token = nextToken;
    }

    const response = await getJson(params);
    totalAds += response.ad_creatives.length;

    if (response.serpapi_pagination && response.serpapi_pagination.next_page_token) {
      await paginateAds(response.serpapi_pagination.next_page_token);
    }
  }

  await paginateAds();
  console.log(`Total number of ${format} ads for GoPro: ${totalAds}`);
}

// Fetching GoPro ads for each creative format
fetchGoProAdsByFormat('text');
fetchGoProAdsByFormat('image');
fetchGoProAdsByFormat('video');

The function fetchGoProAdsByFormat iterates through pages using next_page_token to gather all ads of a particular creative_format. It accumulates the number of ads in the specified format to provide a comprehensive count.

We can evaluate which format is most prevalent in GoPro's recent campaigns by running this function for text, image, and video formats. This will reveal GoPro's preferred method of engaging with its audience. These insights can inform content creators and marketers about the most effective ad formats for products like GoPro.

Final Words

The Google Ads Transparency Center API is a powerhouse for anyone looking to delve into the complexities of digital advertising. It is more than just a data retrieval tool; it unveils the dynamics of digital advertising, making it more transparent. For businesses, marketers, and analysts, this means better-informed strategies, more effective campaigns, and a deeper understanding of the competitive landscape.

As digital advertising continues to evolve, tools like the Google Ads Transparency Center API will be invaluable in staying ahead of the curve. The insights gained from this API can guide decisions, shape strategies, and ultimately lead to more successful and impactful advertising initiatives.

If you have any questions or want to discuss any issues or matters, don't hesitate to contact our team at contact@serpapi.com. We'll be more than happy to assist you and answer your questions!

Introduction

Setup

Overview of the API Parameters

Understanding and Utilizing the advertiser_id Parameter

Finding the Advertiser ID

Example

Understanding the text Parameter

Difference between text and advertiser_id

Example

Utilizing the region Parameter to Localize Ad Data

Finding the Region Code

Example

Using start_date and end_date for Period-Specific Analysis

Use Cases

Example

Pagination with num and next_page_token

Implementing Pagination

Example

Exploring creative_format

Understanding creative_format and Its Options

Example

Final Words

Understanding and Utilizing the `advertiser_id` Parameter

Understanding the `text` Parameter

Difference between `text` and `advertiser_id`

Utilizing the `region` Parameter to Localize Ad Data

Using `start_date` and `end_date` for Period-Specific Analysis

Pagination with `num` and `next_page_token`

Exploring `creative_format`

Understanding `creative_format` and Its Options