Google Search Node.js

google-search-results-nodejs is a MIT-licensed Node.js package that meant to scrape search results from Google, Bing, Baidu, Yahoo and 10+ more search engines with a SerpApi backend. SerpApi provides a Playground to get you started quickly by testing API interactively.

Find SerpApi documentation at: https://serpapi.com/search-api

Find SerpApi package at: https://www.npmjs.com/package/google-search-results-nodejs

• Requirements
• Installation
• Quick Start
• How SerpApi backend works
• How to set SerpApi key
• Google Search API Capability
• Supported Engines
• Example by Specification
• Extra APIs
  • Location API
  • Search Archive API
  • Account API
• Promise and Callback
• Coding Style
• Run Regression
• Error Management
• Changelog

Requirements

• ES6 basic understanding
• Node 7+ and NPM installed

Installation

$ npm install google-search-results-nodejs

Quick start

The following example runs a search for "coffee" using your secret API key which you can find at SerpApi Dashboard page.

Open in the online IDE (Replit).

const SerpApi = require('google-search-results-nodejs')
const search = new SerpApi.GoogleSearch("<your-serpapi-api-key>")

search.json({
 q: "Coffee", 
 location: "Austin, TX"
}, (result) => {
  console.log(result)
})

How SerpApi backend works

How to set SerpApi key

api_key can be set globally using a singleton pattern:

// https://serpapi.com/manage-api-key
const SerpApi = require("google-search-results-nodejs")
const search = new SerpApi.GoogleSearch("<your-serpapi-api-key>")

api_key can be read from the environment variable:

const search = new SerpApi.GoogleSearch(process.env.API_KEY);

api_key can be provided for each request:

const SerpApi = require("google-search-results-nodejs")
const search = new SerpApi.GoogleSearch()

let result = search.json({
 api_key: "<your-serpapi-api-key>", // https://serpapi.com/manage-api-key
 q: "Coffee",                       // search query
 location: "Austin, TX",            // location of the search
}, (data) => {
  console.log(data)
})

Google Search API Capability

const SerpApi = require("google-search-results-nodejs")
const search = new SerpApi.GoogleSearch()

query_params = {
    api_key: "asdewqe1231241asm",              // Your SerpApi API key, https://serpapi.com/manage-api-key                                                                             
    q: "coffee",                               // Search query.                                                                                     
    google_domain: "google.com",               // Google domain to use.                                                                             
    location: "Austin, Texas, United States",  // Location requested for the search.                                                                
    uule: "w+CAIQICINVW5pdGVkIFN0YXRlcw",      // Google encoded location you want to use for the search.                                           
    ludocid: "CID ID",                         // ID (CID) of the Google My Business listing you want to scrape.
    lsig: "AB86z5W5r155sIcs3jqfYkm9Y8Fp",      // Force the knowledge graph map view to show up.
    device: "desktop|mobile|tablet",           // Device used when making a search.                                                                 
    hl: "en",                                  // Language of the search.                                                                           
    gl: "gl",                                  // Country of the search.                                                                            
    lr: "lang_en|lang_fr",                     // One or multiple languages to limit the search to.                                                 
    safe: "active|off",                        // Level of filtering for adult content.                                                             
    nfpr: "1|0",                               // Exclusion of results from an auto-corrected query that is spelled wrong.                          
    num: "100",                                // Number of results per page.                                                                       
    start: "20",                               // Pagination offset.                                                                                
    ijn:"1",                                   // Page number for Google Images.                                                                    
    tbm: "nws|isch|shop|lcl|vid",              // Type of search: news, images, shopping. local, video results.                                                         
    tbs: "custom to be search criteria",       // Advanced search for patents, dates, news, videos, images, apps, or text contents                  
    async: True|False,                         // Allow async request.
    no_cache: True|False                       // Force SerpApi to fetch the Google results even if a cached version is already present             
}

const callback = (data) => {
 console.log(data)                                // create a callback
}
                                                 
search.json(query_params, callback)              // Show result as JSON
search.html(query_params, callback)              // Show result as HTML file

Supported Engines

Example by specification

We love open source, continuous integration and Test Driven Development (TDD). We are using Mocha to test our infrastructure around the clock to achieve the best Quality of Service (QoS).

The directory test/ includes specification/examples.

Set your API key:

export API_KEY="<your-serpapi-api-key>"

Run all tests:

npm test

Extra APIs

Location API

const search = new SerpApi.GoogleSearch("<your-serpapi-api-key>")

search.location("Austin", 3, (data) => {
  console.log(data)
})

Prints the first three (3) locations matching Austin (Texas, Texas, Rochester)

[
   {
      "id":"585069bdee19ad271e9bc072",
      "google_id":200635,
      "google_parent_id":21176,
      "name":"Austin, TX",
      "canonical_name":"Austin,TX,Texas,United States",
      "country_code":"US",
      "target_type":"DMA Region",
      "reach":5560000,
      "gps":[
         -97.7430608,
         30.267153
      ],
      "keys":[
         "austin",
         "tx",
         "texas",
         "united",
         "states"
      ]
   }, ... other results
]

Search Archive API

The first search result returns a search_id which can be provided to get the search result from the archive. The following code will print the search from the archive.

var search = new SerpApi.GoogleSearch("<your-serpapi-api-key>")

search.json({q: "Coffee", location: "Portland" }, (search_result) => {
  // search in archive for the search just returned
  search.search_archive(search_result.search_metadata.id, (archived_search) => {
    console.log(archived_search)
  })
})

Account API

The following code snippet will print your account information.

const search = new SerpApi.GoogleSearch("<your-serpapi-api-key>")

search.account((data) => {
  console.log(data)
})

Promise and callback

This API was developped using basic callback to handle response. Exception are just throw away with interceptor.

If you want to take advantage of the promise to block the request, here is how we'll do:

const util = require('util')

function getJson(parameter, resolve, reject) {  
  const search = new SerpApi.GoogleSearch("<your-serpapi-api-key>")
  try {
    search.json(parameter, resolve)
  } catch (e) {
    reject(e)
  }
}

const blockFn = util.promisify(getJson)
blockFn[util.promisify.custom](parameter).then((data) => {
  expect(data.local_results[0].title.length).toBeGreaterThan(5)
  done()
}).catch((error) => {
  console.error(error)
  done()
})

Open in the online IDE (Replit)

const SerpApi = require("google-search-results-nodejs");
const search = new SerpApi.GoogleSearch(process.env.API_KEY); // your serpapi API key, https://serpapi.com/manage-api-key

const searchQuery = "coffee";

const params = {
  q: searchQuery,   // what we want to search
  engine: "google", // parsing engine
  hl: "en",         // parameter defines the language to use for the Google search
  gl: "us",         // parameter defines the country to use for the Google search
  tbm: "isch",      // parameter defines the type of search you want to do (isch - Google Images)
};

const getJson = () => {
  return new Promise((resolve) => {
    search.json(params, resolve);
  });
};

const getResults = async () => {
  const imagesResults = [];

  while (true) {
    const json = await getJson();
    if (json.images_results) {
      imagesResults.push(...json.images_results);
      params.ijn ? (params.ijn += 1) : (params.ijn = 1);
    } else break;
  }
  return imagesResults;
};

getResults().then((result) => console.dir(result, { depth: null }));

Reference:

• test: test/ExampleSpec.js
• documentation: https://nodejs.org/docs/latest-v8.x/api/util.html#util_util_promisify_original

Coding style

This API is using callback to run in non-blocking code. Here we are trying to follow the spirit of NodeJS.

Reference:

• https://nodejs.org/en/docs/guides/blocking-vs-non-blocking/
• https://nodejs.org/en/docs/guides/dont-block-the-event-loop/

For pratical example, you can see the test located under test/ folder.

Run regression

To run the regression suite.

export API_KEY="your api key"
make test

Error Management

SerpApi keeps error management simple:

1. backend service error or search fail.
2. client error.

If it's a backend error, a simple error message is returned as string in the server response. If it's a client error, then a SerpApiClientException is raised.

Change log

• 2.1
  • add support for Naver, HomeDepot, AppleStoreApp, DuckDuckGo
  • defeature location if it is not supported by the search engine
• 2.0.1
• fix classes loading.
• 2.0
  • Refractor class name: SearchResult -> Search
• 1.2
  • stable version to support all the basic search API.