Scrape Google Jobs to easily make Job Lists using SerpApi

Introduction

Nowadays it's very important to have an online presence and promote your work in creative ways. However, creating content is not easy, especially when you need to be consistent in order to stay relevant.
It's important that creating content doesn't become a burden, so a good strategy for creating content in a sustainable way is to choose a specific niche you enjoy to focus on, and find content you have fun making while not overwhelming yourself, so you can stay consistent without burning out.

But what exactly are the benefits in creating content, and why should anyone start?
There are many benefits to creating content online, and personally, I find the biggest benefits are:

• Helps establishing your professional brand.
• Increases your network organically.
• Attracts recruiters and professionals in your field to follow your work.
• Exposes you to opportunities you didn't even know exist.
• Enriches the community around you creating value and igniting relevant discussions.
• Connects you to brands and companies that are relevant in your industry.
• Empowers people who follow you and inspire them to explore beyond their comfort zone.

The biggest challenge in creating content though is... How and where to start?
I have an idea! Start by creating job board lists! They're easy and fun to make, and generate lots of value to the community you're part of. Additionally, there are always new job opportunities being posted every day, which makes Job Board Lists always relevant!
To help you get started, I created the job-post-maker-using-serpapi script which allows you to make job lists for any industry and in any location using a single command.

Setting up

I'll walk you through downloading and using the job-post-maker-using-serpapi script, and if you prefer, you can also follow along with our video:

Before starting, make sure you have Ruby installed in your computer.

First, you will need to clone the job-post-maker-using-serpapi repository to your computer:

Go to the directory you just created:

cd job-post-maker-using-serpapi

Now we need to install all gems required for this script to run.
In the terminal, in the same directory of the project, run:

bundle install

Creating a SerpApi account

In order to use this script, we need to create a SerpApi account so we can get a SerpApi secret API key.
Head over to serpapi.com, click Register:

Sign up for a free account using your email, then confirm your email and phone number to access your account.
Right in the Dashboard, you can copy your secret API key to use in the script.

Now open your Text Editor of choice (mine is VS Code), and create a file called .env in the main directory of the script.
Inside the newly create file, you will paste your secret API key like the following:

Save the file, and now you're ready to create outstanding job lists with ease.

Warning: Your SerpApi key must always be kept secret. In case you expose your API key accidentally, you can easily generate a new one by going to your Dashboard, pressing Manage API key under your API key, then click Regenerate API Key and press OK when prompted to confirm.

Variables we can change

We are ready to create some beautiful job lists using the script default values, however, it's more interesting to run a script that will generate jobs according to your own needs, so let's check the variables we can change to make the jobs relevant to you:

Open the main script file called job_post_maker.rb.
In this file you will find all variables that you can change to create your job lists:

• job_title: Required parameter
  The query used to find jobs using SerpApi's Google Jobs API.

Value Examples:

job_title = 'Ruby on Rails'
job_title = 'Game developer'
job_title = 'React developer full time'
job_title = 'developer internship'
job_title = 'Lead software engineer contract'

• location: Optional parameter Defines from where you want the search to originate. If several locations match the location requested, the most popular one will picked. Download a list of all supported locations.

Value Examples:

location = 'Sao Paulo,State of Sao Paulo,Brazil'
location = 'Mumbai,Maharashtra,India'
location = 'New York,New York,United States'

• remote: Optional parameter Only fetches remote jobs when set to true, and only fetches on-premise jobs when set to false. If not included, the script will fetch both remote and on-premise jobs.
• language_code: Optional parameter Two-letter language code parameter that defines the language to use for the Google Jobs search. See a list of Supported Languages. If this parameter is not provided, then the default language code for English will be used.

Value Examples:

language_code = 'en'    # for English
language_code = 'es'    # for Spanish
language_code = 'pt-br' # for Portuguese

• number_of_jobs: Optional parameter Number of unique jobs to fetch for the job list. If not used, it will default to returning 15 jobs. If the number provided is greater than the number of unique jobs available for your specific parameters, then the script will only return the number of jobs it can find.
  Attention: The greater the number of jobs requested the more requests will be necessary to make the list. Usually, Google returns 10 jobs per request, but sometimes it returns less. Also, duplicate jobs are not added to the list. It's important to keep this in mind because SerpApi's Free tier offers 100 requests per month, so you may want to adjust this parameter based on how many times you'd like to create a job list per month, or you can consider upgrading your SerpApi account to a paid tier to get more requests per month.
• board_relevance: Optional parameter Array where you list your favorite job boards. Each job has options for links where to apply for that opportunity. If any of the available options are listed in this Array, then the link to apply for the job will be for the job board listed in the Array depending on the order listed. The first job boards listed have higher relevance. Also, every time you run the script, a list of all job boards associated with the jobs on the list will be printed on the terminal to make it easier to find job boards available and to make it easier to copy the names of job boards you like most.

Value Examples:

board_relevance = [
  'LinkedIn',
  'GoRails Jobs',
  'Ruby On Remote',
  'We Are Hiring',
  'We Work Remotely'
]

• fun_bullet_point: Optional parameter The string provided in this parameter will be used as a bullet point when generating the job list. It's recommended to use an emoji to make the list look more appealing, fun and engaging. If this parameter is not provided, then the list will use the default 🟢 as a bullet point.

Creating the Job List using the script

Now that you have your SerpApi key in the .env file and set all the variables to accommodate your needs you can run the script to generate the list.

In the terminal, in the same directory of the project, run:

ruby job_post_maker.rb

You will see the progress of the script running on the terminal.

Example of output on terminal:

Starting to scrape jobs
Scraping... 9 unique jobs out of 10 jobs scraped.
Scraping... 15 unique jobs out of 20 jobs scraped.
COMPLETED: 15 jobs added to the list.

Job boards covered:
Built In NYC
DailyRemote
LinkedIn
Ruby On Remote
Ruby-Forum
Work At A Startup

You can see the list of jobs generated in the file called GENERATED_JOB_LIST.txt.

Example of generated list:

🦖 Junior Fullstack Engineer at @SerpApi: https://serpapi.com/careers/junior-fullstack-engineer
🦖 Senior Fullstack Engineer at @SerpApi: https://serpapi.com/careers/senior-fullstack-engineer
🦖 Customer Success Engineer at @SerpApi: https://serpapi.com/careers/customer-success-engineer
🦖 Outbound Sales Engineer at @SerpApi: https://serpapi.com/careers/outbound-sales-engineer
🦖 Senior Accountant at @SerpApi: https://serpapi.com/careers/senior-accountant
🦖 Talent Acquisition Specialist at @SerpApi: https://serpapi.com/careers/talent-acquisition-specialist
...

Now that you have a list of job opportunities, you can share it with your network.

Making a beautiful art for your Job List

I personally like to start the art for my job board posts with DALL-E, then I add GIFs I find on Pinterest or Google Images, and fun text I make using TextStudio to it using Canva.

Here's an example of a complete job board post:

Conclusion

Creating content is a very good way to show your work, network with professionals in your field, and become more relevant to a social media platform's algorithm connecting you to recruiters and opportunities. Though, it's easy to feel overwhelmed and intimidating to start due to the commitment of having to post consistently to stay relevant.
By using the job-post-maker-using-serpapi repository script, we can easily leverage the power of SerpApi to help us create amazing job board posts in less than 3 minutes, allowing us to create consistent value without taking much of our time.

Thank you very much for reading this article. I hope it helps you get started in creating content to rip the immense benefits of creating content and generating value to your community.
We'd be very happy if you tag SerpApi on your posts, and we'd love to see your phenomenal creations using SerpApi.