How to use SerpApi using PHP and cURL

This tutorial is on scraping search engine results using simple API by SerpApi using PHP and cURL.

Are you new to cURL? Learn the basics of making requests with cURL in PHP.

In this tutorial, we're not going to use SerpApi's PHP library, instead we'll stick to native implementation with cURL.

Here is the complete documentation of the Google Search API we're going to use: https://serpapi.com/search-api

Basic search tutorial in PHP

Step 1: Prepare cURL

Let's prepare our cURL code. Add the SerpApi endpoint as the URL value.

<?php

// Initialize cURL session
$ch = curl_init();

// Set cURL options
curl_setopt($ch, CURLOPT_URL, "https://serpapi.com/search");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);

Step 2: Prepare the parameter

Feel free to adjust this parameter based on search engine and other relevant parameters you want to implement.

// Set the data fields for GET request
$fields = array(
    'api_key' => 'YOUR_API_KEY',
    'engine' => 'google',
    'q' => 'Coffee',
    'location' => 'Austin, Texas, United States',
    'google_domain' => 'google.com',
    'gl' => 'us',
    'hl' => 'en'
);

Step 3: Execute cURL
Let's execute the code above. Don't forget to close the connection at the end.

// Build the URL query string
$queryString = http_build_query($fields);
curl_setopt($ch, CURLOPT_URL, "https://serpapi.com/search?" . $queryString);

// Execute cURL session
$response = curl_exec($ch);

// Check for errors
if(curl_errno($ch)) {
    echo 'Request Error:' . curl_error($ch);
}

// Close cURL session
curl_close($ch);

// Output the response
echo $response;

That's it. You should be able to see the result when running a server. Try php -S localhost:8000 (depend on your OS).

Async search implementation in PHP

Sometimes, you need to perform a "bulk search" or search for multiple keywords at once. We can implement this using the async option.

Make sure to see the basic endpoint of SerpApi Search Archive API here. It looks like this

https://serpapi.com/searches/$searchID.json?api_key=SECRET_API_KEY

Here is a basic example:
- Add an array to store key value of $searchID and temporary $searchData
- Set async to true

$searchQueue = [];
$items = ['green tea', 'black tea', 'oolong tea'];

// Batch search in async mode
foreach ($items as $item) {
    // Set the data fields for GET request
    $fields = array(
        'api_key' => $API_KEY,
        'engine' => 'google',
        'q' => $item,
        'location' => 'Austin, Texas, United States',
        'google_domain' => 'google.com',
        'gl' => 'us',
        'hl' => 'en',
        'async' => true
    );

    // Build the URL query string
    $queryString = http_build_query($fields);
    curl_setopt($ch, CURLOPT_URL, "https://serpapi.com/search?" . $queryString);

    // Execute cURL session
    $response = curl_exec($ch);

    // Check for errors
    if(curl_errno($ch)) {
        echo 'Request Error:' . curl_error($ch);
    }
    
    // save the search id for later retrieval
    $searchResult = json_decode($response, true);
    $searchQueue[$searchResult['search_metadata']['id']] = $searchResult;
}

In above example, I only make the keyword dynamic. Feel free to adjust other parameters based on your use case.

So far, we only store the temporary search information in an array $searchQueue now how do we retrieve it?

Retreive the async result

The idea is:
- Doing an infinite loop of the search (since the search can take some time to finish)
- We keep checking if the id status already updated to success .
- If so, we remove the ID from the search (to make sure we're not running in infinite loop).
- Otherwise, keep checking the search status

while (!empty($searchQueue)) {
    foreach($searchQueue as $search) {
        $searchID = $search['search_metadata']['id'];
        $fields = array(
            'api_key' => $API_KEY
        );

        // Access archive API endpoint
        $queryString = http_build_query($fields);
        curl_setopt($ch, CURLOPT_URL, "https://serpapi.com/searches/$searchID.json?" . $queryString);
        $response = curl_exec($ch);
        
        $search = json_decode($response, true);
        echo $search['search_metadata']['status'];
        if ($search['search_metadata']['status'] == 'Success'
            || $search['search_metadata']['status'] == 'Cached') {
            // Remove the search from the queue based on the search id
            unset($searchQueue[$searchID]);
            echo $search['search_metadata']['id'] . " is done";
            echo "<br>";
            echo "First result title : " .  $search['organic_results'][0]['title'];
            echo "<br>";
        } else {
            echo $search['search_metadata']['id'] . " is not done yet";
            echo "<br>";
        }
    }
}

echo "All searches are done";
// Close cURL session
curl_close($ch);
?>

When the status of the search is succeeded, I print the first organic result's title. Feel free to do anything on this condition block. Since the data is already available here.

That's it! Feel free to try!