Google Search Node.js
google-search-results-nodejs
is a MIT-licensed Node.js package that meant to scrape search results from Google, Bing, Baidu, Yahoo and 10+ more search engines with a SerpApi backend. SerpApi provides a Playground to get you started quickly by testing API interactively.
Find SerpApi documentation at: https://serpapi.com/search-api
Find SerpApi package at: https://www.npmjs.com/package/google-search-results-nodejs
- Requirements
- Installation
- Quick Start
- How SerpApi backend works
- How to set SerpApi key
- Google Search API Capability
- Supported Engines
- Example by Specification
- Extra APIs
- Promise and Callback
- Coding Style
- Run Regression
- Error Management
- Changelog
Requirements
- ES6 basic understanding
- Node 7+ and NPM installed
Installation
$ npm install google-search-results-nodejs
Quick start
The following example runs a search for "coffee"
using your secret API key which you can find at SerpApi Dashboard page.
Open in the online IDE (Replit).
const SerpApi = require('google-search-results-nodejs')
const search = new SerpApi.GoogleSearch("<your-serpapi-api-key>")
search.json({
q: "Coffee",
location: "Austin, TX"
}, (result) => {
console.log(result)
})
How SerpApi backend works
How to set SerpApi key
api_key
can be set globally using a singleton pattern:
// https://serpapi.com/manage-api-key
const SerpApi = require("google-search-results-nodejs")
const search = new SerpApi.GoogleSearch("<your-serpapi-api-key>")
api_key
can be read from the environment variable:
const search = new SerpApi.GoogleSearch(process.env.API_KEY);
api_key
can be provided for each request:
const SerpApi = require("google-search-results-nodejs")
const search = new SerpApi.GoogleSearch()
let result = search.json({
api_key: "<your-serpapi-api-key>", // https://serpapi.com/manage-api-key
q: "Coffee", // search query
location: "Austin, TX", // location of the search
}, (data) => {
console.log(data)
})
Google Search API Capability
const SerpApi = require("google-search-results-nodejs")
const search = new SerpApi.GoogleSearch()
query_params = {
api_key: "asdewqe1231241asm", // Your SerpApi API key, https://serpapi.com/manage-api-key
q: "coffee", // Search query.
google_domain: "google.com", // Google domain to use.
location: "Austin, Texas, United States", // Location requested for the search.
uule: "w+CAIQICINVW5pdGVkIFN0YXRlcw", // Google encoded location you want to use for the search.
ludocid: "CID ID", // ID (CID) of the Google My Business listing you want to scrape.
lsig: "AB86z5W5r155sIcs3jqfYkm9Y8Fp", // Force the knowledge graph map view to show up.
device: "desktop|mobile|tablet", // Device used when making a search.
hl: "en", // Language of the search.
gl: "gl", // Country of the search.
lr: "lang_en|lang_fr", // One or multiple languages to limit the search to.
safe: "active|off", // Level of filtering for adult content.
nfpr: "1|0", // Exclusion of results from an auto-corrected query that is spelled wrong.
num: "100", // Number of results per page.
start: "20", // Pagination offset.
ijn:"1", // Page number for Google Images.
tbm: "nws|isch|shop|lcl|vid", // Type of search: news, images, shopping. local, video results.
tbs: "custom to be search criteria", // Advanced search for patents, dates, news, videos, images, apps, or text contents
async: True|False, // Allow async request.
no_cache: True|False // Force SerpApi to fetch the Google results even if a cached version is already present
}
const callback = (data) => {
console.log(data) // create a callback
}
search.json(query_params, callback) // Show result as JSON
search.html(query_params, callback) // Show result as HTML file
Supported Engines
Engine Class name Google Search EngineGoogleSearch()
Google Maps
GoogleSearch()
Google Jobs
GoogleSearch()
Google Trends
GoogleSearch()
Google Autocomplete
GoogleScholarSearch()
Google About This Result
GoogleSearch()
Google Lens
GoogleSearch()
Google Finance
GoogleSearch()
Google Related Questions
GoogleScholarSearch()
Google Scholar
GoogleScholarSearch()
Google Play Store
GoogleSearch()
Google Product
GoogleSearch()
Google Immersive Product
GoogleSearch()
Google Reverse Image
GoogleSearch()
Google Events
GoogleSearch()
Google Local Services
GoogleSearch()
Bing
BingSearch()
Baidu
BaiduSearch()
DuckDuckGo
DuckDuckGoSearch()
Yahoo
YahooSearch()
Yandex
YandexSearch()
eBay
EbaySearch()
Youtube
YoutubeSearch()
Walmart
WalmartSearch()
HomeDepot
HomeDepotSearch()
Apple App Store
AppleAppStoreSearch()
Naver
NaverSearch()
Yelp
YelpSearch()
Example by specification
We love open source, continuous integration and Test Driven Development (TDD). We are using Mocha to test our infrastructure around the clock to achieve the best Quality of Service (QoS).
The directory test/
includes specification/examples.
Set your API key:
export API_KEY="<your-serpapi-api-key>"
Run all tests:
npm test
Extra APIs
Location API
const search = new SerpApi.GoogleSearch("<your-serpapi-api-key>")
search.location("Austin", 3, (data) => {
console.log(data)
})
Prints the first three (3) locations matching Austin (Texas, Texas, Rochester)
[
{
"id":"585069bdee19ad271e9bc072",
"google_id":200635,
"google_parent_id":21176,
"name":"Austin, TX",
"canonical_name":"Austin,TX,Texas,United States",
"country_code":"US",
"target_type":"DMA Region",
"reach":5560000,
"gps":[
-97.7430608,
30.267153
],
"keys":[
"austin",
"tx",
"texas",
"united",
"states"
]
}, ... other results
]
Search Archive API
The first search result returns a search_id
which can be provided to get the search result from the archive. The following code will print the search from the archive.
var search = new SerpApi.GoogleSearch("<your-serpapi-api-key>")
search.json({q: "Coffee", location: "Portland" }, (search_result) => {
// search in archive for the search just returned
search.search_archive(search_result.search_metadata.id, (archived_search) => {
console.log(archived_search)
})
})
Account API
The following code snippet will print your account information.
const search = new SerpApi.GoogleSearch("<your-serpapi-api-key>")
search.account((data) => {
console.log(data)
})
Promise and callback
This API was developped using basic callback to handle response. Exception are just throw away with interceptor.
If you want to take advantage of the promise to block the request, here is how we'll do:
const util = require('util')
function getJson(parameter, resolve, reject) {
const search = new SerpApi.GoogleSearch("<your-serpapi-api-key>")
try {
search.json(parameter, resolve)
} catch (e) {
reject(e)
}
}
const blockFn = util.promisify(getJson)
blockFn[util.promisify.custom](parameter).then((data) => {
expect(data.local_results[0].title.length).toBeGreaterThan(5)
done()
}).catch((error) => {
console.error(error)
done()
})
Open in the online IDE (Replit)
const SerpApi = require("google-search-results-nodejs");
const search = new SerpApi.GoogleSearch(process.env.API_KEY); // your serpapi API key, https://serpapi.com/manage-api-key
const searchQuery = "coffee";
const params = {
q: searchQuery, // what we want to search
engine: "google", // parsing engine
hl: "en", // parameter defines the language to use for the Google search
gl: "us", // parameter defines the country to use for the Google search
tbm: "isch", // parameter defines the type of search you want to do (isch - Google Images)
};
const getJson = () => {
return new Promise((resolve) => {
search.json(params, resolve);
});
};
const getResults = async () => {
const imagesResults = [];
while (true) {
const json = await getJson();
if (json.images_results) {
imagesResults.push(...json.images_results);
params.ijn ? (params.ijn += 1) : (params.ijn = 1);
} else break;
}
return imagesResults;
};
getResults().then((result) => console.dir(result, { depth: null }));
Reference:
- test:
test/ExampleSpec.js
- documentation: https://nodejs.org/docs/latest-v8.x/api/util.html#util_util_promisify_original
Coding style
This API is using callback to run in non-blocking code. Here we are trying to follow the spirit of NodeJS.
Reference:
- https://nodejs.org/en/docs/guides/blocking-vs-non-blocking/
- https://nodejs.org/en/docs/guides/dont-block-the-event-loop/
For pratical example, you can see the test located under test/
folder.
Run regression
To run the regression suite.
export API_KEY="your api key"
make test
Error Management
SerpApi keeps error management simple:
- backend service error or search fail.
- client error.
If it's a backend error, a simple error message is returned as string in the server response. If it's a client error, then a SerpApiClientException
is raised.
Change log
- 2.1
- add support for Naver, HomeDepot, AppleStoreApp, DuckDuckGo
- defeature location if it is not supported by the search engine
- 2.0.1
- fix classes loading.
- 2.0
- Refractor class name: SearchResult -> Search
- 1.2
- stable version to support all the basic search API.