A common problem when using any third-party API with a frontend application is how to handle CORS and keep your API key safe. In the case of SerpApi, the CORS policy doesn't permit the APIs to be called from the frontend since that would expose the user's API key. A short term or development solution to this is using a CORS proxy like the one found here - however this isn't recommended for long term or production since it includes a variety of security risks.

In this tutorial we will build a proxy server that can act as a go-between for your frontend application and the third party API you want to call. When your frontend app sends a request to the proxy server, your proxy server will fetch data from the external or third party API you want to query, then return it to your frontend application in a response.

We will build the proxy server with Node.js and Express. We will include methods for fetching data from SerpApi that you can request from your frontend app.

If you don't need the explanation and just want to use the project code as a template, you can find the full code here:

GitHub - schaferyan/serpapi-nodejs-proxy-server: Proxy server you can use to query SerpApi from a front-end web application.
Proxy server you can use to query SerpApi from a front-end web application. - GitHub - schaferyan/serpapi-nodejs-proxy-server: Proxy server you can use to query SerpApi from a front-end web applica…
GitHubschaferyan

Prerequisites

You will need to be somewhat familiar with basic Javascript. You will have an easier time if you are familiar with ES6 syntax and features, as well as Node and Npm.

You need to have the latest versions of Node and Npm installed on your system. If you don’t already have Node and Npm installed you can visit the following link for help with this process:

https://docs.npmjs.com/downloading-and-installing-node-js-and-npm

You need to have an IDE installed on your system that supports JavaScript and Node.js. I recommend VSCode, as it provides an extension called REST Client that is helpful for testing API routes but any IDE that supports Node.js will do.

You will also need to sign up for a free SerpApi account at https://serpapi.com/users/sign_up.

Preparation

First we need to create a package.json file inside our project directory:

mkdir my-nodejs-server
cd my-nodejs-server
npm init

Npm will walk you through the process of creating a package.json file.
After this we need to make a few changes to the package.json file.
We will be using the ES Module system rather than CommonJs, so you need to add the line type="module" to your package.json.

{
  "name": "track-google-search-rankings-nodejs",
  "version": "1.0.0",
  "description": "Track Google Search Rankings with NodeJs and SerpApi",
  "type": "module",
  "main": "index.js",

We will also add a start script for convenience:

  "scripts": {
    "start": "node index.js",
    "test": "echo \"Error: no test specified\" && exit 1"
  },

Next we need to install our dependencies:

npm install express dotenv serpapi cors

If you haven’t already signed up for a free SerpApi account go ahead and do that now by visiting https://serpapi.com/users/sign_up and completing the signup process.

Once you have signed up, verified your email, and selected a plan, navigate to https://serpapi.com/manage-api-key . Click the Clipboard icon to copy your API key.

Then create a new file in your project directory called .env and add the following lines:

SERPAPI_KEY = “PASTE_YOUR_API_KEY_HERE”
PORT = 3000

Creating an Express Server

Create a file named server.js:

import express, { json } from "express";
import dotenv from "dotenv";
dotenv.config();

const app = express();

app.use(json());

const PORT = process.env.PORT;

app.listen(PORT, () => {
console.log(`Server listening on port ${PORT}\n-------------------------`);
});

If you go to your terminal and type npm start you should see the following:

Server listening on port 3000

Great, your server is working.

Now let's add a generic response. Add the following line to your server.js:

app.get("/", (req, res) => {
    res.status(200).json({ message: "I am an Express Server!" });
  });

Now restart your server so changes can take effect. Then go to your browser and type http://localhost:3000/ into your address bar.

You should see something like this:

{"message":"I am an Express Server!"}

Adding 'Routes'

Right now the server can respond, but we don't have a way to request specific information from it. For that, we need to add what are called "api routes". If we needed to add a lot of options, or give access to hierarchical information, we might do this using the express.Router class. However, for this tutorial we only need a few routes, so let's keep it simple. In your server.js file, underneath where you added the default response, add the following code:

app.get("/another-route", (req, res) => {
  res.status(200).json({ message: "here is another route" });
});

If you go back to your browser and type http://localhost:3000/another-route into your address bar you should see this:

{"message":"here is another route."}

Querying SerpApi

Let's add a route that takes a GET request with a single parameter like the one above, then pass the parameter as a query in a request to SerpApi's Google Search API. We can then return the results in our server's response.

First let's create the function we will use to call SerpApi. Create a file called search.js and export the following function:

export async function search(req, res, next){
    const params ={}
    params.q=req.params.keyword;
    try{
        const data = await getJson("google", params);
        res.locals.result = data;
        res.status(200);
    }catch{
        res.status(400);
    }
    next();
}

Import the function in server.js:

import {search} from './search.js';

Now lets add a route that accepts a keyword parameter, and calls the search function:

app.get("/keyword-search/:q", search, (req, res) => {
  res.status(200).json(res.locals.result);
});

Restart the server and try typing http://localhost:3000/search/coffee into your browser address bar. You should see a JSON object with search results data for the query "coffee".

Multiple Parameters

What if we want our API server to be able to receive more than one parameter?There's a few ways we can deal with this. One way is to set up the route so that it takes a query string, and include all of our search parameters in it. Query strings begin with a ?, and include key-value pairs in the format key=value. Multiple query parameters are separated with an &.

Let's add another route to server.js that will accept a query string instead of a parameter:

app.get("/search", getResults, search, (req, res) => {
  res.status(200).json(res.locals.result);
});

Notice we aren't using : to indicate a parameter. We've just given the route a different name. Also, notice we are still using the search function here we defined previously, we've just added another function, getResults, ahead of it in the chain. That's because we want to do basically the same thing, only we need to get the parameters from the query string.

Be sure to also import getResults at the top of Server.js:

import {search, getResults} from './search.js';

How do we process the query string? Fortunately Express provides an easy way to do this. Express automatically parses the parameters in a query string, if the request contains one, and stores them in an Object called req.query . Let's add the getResults() function to Search.js:

export async function getResults(req, res, next) {
  req.params = req.query;
  next();
}

All we have to do here is move our parameters from req.query to req.params where our next function, search() expects to find them.

Now restart the server, and try the following request in your browser:

http://localhost:3000/search/?q=Coffee&location=Austin,+Texas,+United+States&hl=en&gl=us&google_domain=google.com

Another way to submit a request with multiple parameters is to use a POST request, and include the parameters in the JSON body. To do this we will create a route that is identical to the one we just created, except that is post rather than get. Also, instead of getResults we will write a new function postResults to extract the parameters from the JSON body of the request.

app.post("/search", postResults, search, (req, res) => {
  res.status(200).json(res.locals.result);
});

We import the new function above:

import {getResults, postResults, search } from './search.js';

And define it in search.js:

export async function postResults(req, res, next) {
  req.params = { ...req.body };
  next();
}

Here we are doing the same thing as in getResults , only this time we are copying the parameters from req.body instead of from req.query.

POST requests require a bit more setup to test than GET requests, since you can't just enter a URL in your browser. A common tool for testing API routes is Postman. If you are using VSCode, you can install an extension called REST Client, that makes it very easy to test API routes. I won't cover how to set up or use either of these tools in this tutorial, but whichever method you decide to use, you want to send a POST request that looks something like this:

POST http://localhost:3000/search HTTP/1.1
content-type: application/json

{
    "q": "coffee",
    "location": "Austin, Texas, United States"
}

You can add as many parameters as you'd like in the JSON body.

Note that it is generally against REST principles to use POST for a read operation, as POST is intended for writing data to a database. You may want to stick to a GET method with a query string as we went into above. However, later on if you decide to add a database to your stack to keep track of your requests and store results for later, POST is the correct method to use.

CORS

Currently your proxy server can be queried from any domain. We only this server to be accessible from the domain you will use - this will make it harder to exploit. To do this we need to set the Access-Control-Allow-Origin CORS header to the name of the domain where your frontend app is hosted. We will use the cors Node.js package for this. Import it at the top of server.js:

import cors from "cors";

Then add the following line after app.use(json());

app.use(cors({ origin: "http://yourdomain.com" }));

Your server.js file should now look like this:

import express, { json } from "express";
import dotenv from "dotenv";
import { config } from "serpapi";
import cors from "cors";
import {getResults, postResults, search } from './search.js';



dotenv.config();
// console.log(process.env);
config.api_key = process.env.SERPAPI_KEY;


const app = express();
app.use(cors({ origin: "http://yourdomain.com" }));
app.use(json());

const PORT = process.env.PORT || 3000;

app.get("/", (req, res) => {
  res.status(200).json({ message: "response to default get request" });
});

app.get("/another-route", (req, res) => {
  res.status(200).json({ message: "here is another route" });
});

app.get("/keyword-search/:q", search, (req, res) => {
  res.status(200).json(res.locals.result);
});

app.get("/search", getResults, search, (req, res) => {
  res.status(200).json(res.locals.result);
});

app.post("/search", postResults, search, (req, res) => {
  res.status(200).json(res.locals.result);
});

app.listen(PORT, () => {
  console.log(`Server listening on port ${PORT}\n-------------------------`);
});

search.js should look like this:

import { getJson } from "serpapi";
import dotenv from "dotenv";
dotenv.config();

const API_KEY = process.env.SERPAPI_KEY;

export async function postResults(req, res, next) {
  req.params = { ...req.body };
  next();
}

export async function getResults(req, res, next) {
  req.params = req.query;
  next();
}

export async function search(req, res, next) {
  const params = req.params;
  try {
    const data = await getJson("google", params);
    res.locals.result = data;
    res.status(200);
  } catch (error) {
    console.log(error)
    res.status(400);
  }
  next();
}

Conclusion

You now have a very simple proxy server that will allow you to request search results from SerpApi without directly exposing your API key. However, before putting this code into production it's important you're aware of a rather large caveat. Without adding some method of authentication, anyone who knows your domain name can still gain access to the API key and exploit it.

We won't cover authentication in this tutorial, but if you are interested in developing a more secure proxy server a fairly straightforward option is the Basic authorization schema provided by the HTTP authentication framework. Express provides the express-basic-auth middleware for adding HTTP Basic authorization to an Express app. You can learn more about implementing this here. Other common authorization methods include cookies, JWTs or other tokens, and OAuth.

Of course, the server we built also only supports Google Search with SerpApi. You can add routes for other search engines, modify the project to support another API, or add processing logic so that your server will return something more specific than the full search results JSON file.

I hope you found the tutorial informative and easy to follow. If you have any questions, feel free to contact me at ryan@serpapi.com.