Create a Telegram News Feed by Scraping Google News

Telegram is an increasingly popular messaging app that is proving a worthy rival to WhatsApp. One of the ways Telegram stands out is how easy they make it to create custom bots that can automate various processes in Telegram Groups. In this tutorial, we'll look at how to use SerpApi to scrape Google News and publish the results to a Telegram Group once per hour.

We'll use the popular no-code platform Make.com to create this Telegram news feed so you don't even know how to code in order to build this. I won't go over all the basics of Make.com here since I've covered that heavily elsewhere. If you're new to using Make.com and SerpApi together, you can check out my introductory guide in Announcing SerpApi's Make App.

Our plan will be to scrape Google once an hour for the top three Formula 1 news results. We'll publish each news result one-by-one to a Telegram Group that we can add any of our friends to.

Let's get started.

Telegram Setup

You'll need a Telegram account to follow along with this tutorial and create a Telegram bot. You can create a Telegram account through their apps on Android and iOS.

Once your account is setup, you can then log into your account on Telegram's web version or one of their desktop apps. Working on a Telegram bot on your actual computer is a bit easier than using your mobile.

All Telegram bots are descendants, so to speak, of the BotFather. We'll start a chat with @BotFather to ask him for a token.

Kick things off by sending /start as a message to the BotFather.

Trigger the creation of a new bot by sending /newbot. BotFather will then ask a few questions to set up your bot.

Finally, you'll get this message confirming creation of your token. Keep this secret or anyone can hijack your bot.

Create a new Group.

You'll then be prompted to add members. I'll just add my bot.

Give your Group a name.

Finally, check the URL in your browser to get the Group ID. We'll need this later for the API. Note that it's inclusive of - so we'll use -4728726997 later to message this Group via API.

Scrape the Latest News from Google

This step requires a SerpApi account. You can create an account and get 100 free searches per month through here: https://serpapi.com/users/sign_up

First, I'll note Google has two different platforms for getting news content. SerpApi mirrors this by offering two different APIs. The first is the Google News platform available on news.google.com.

The second is what we call Google News Results which is actually normal Google Search filtered to news results only. When you run a normal Google query and click on the "News" filter, you're on what we call Google News Results.

We'll use Google News Results because it offers a more straightforward option for filtering for news published within the last hour.

In order to filter Google Search for news only, we add tbm=nws to our request URL.

For example: https://serpapi.com/search.json?engine=google&q=formula+1&tbm=nws (API Playground Sample)

Our bot will run once an hour and fetch only results published within the last hour. We can filter for the last hour by adding tbs=qdr:h

For example: https://serpapi.com/search.json?engine=google&q=formula+1&tbm=nws&tbs=qdr:h (API Playground Sample)

If you're interested in other time filters, please refer to Filtering Google Search and Google News Results.

We can check our results in the Playground to ensure we're getting what we want. Looking at the JSON response, we get a news_results object indicating our news filter is working and checking the date field shows each result being published less than an hour ago.

Next, we'll wire this up in Make.

Add SerpApi to Make

You'll need a Make account for this step. You can create an account and get 1,000 free operations per month through here: https://www.make.com/en/register

Once you're in, create a new Make Scenario and give it a descriptive name like "Telegram News Feed".

Search for SerpApi and select the Search Google Module.

Create a SerpApi connection if you don't have one already.

You'll need to grab your SerpApi API key and paste it in the form. Your API key is available here: https://serpapi.com/manage-api-key

Make will do a test call to the SerpApi Account API to confirm your API key is valid. Assuming it's valid, you'll see your SerpApi account email appear next to your newly created connection.

Now we can configure the search query. We'll search for formula 1.

We'll filter for news results only and request news articles published in the last hour. These parameters are under advanced settings so please open advanced settings.

To filter only for the last hour, find the tbs parameter and set it to qdr:h as we did in our API Playground query.

To filter only for news, find the tbm parameter and set it to nws. You can use the drop-down menu to select the parameter.

Hit "Run once" to give the query a try.

Check the SerpApi Module's output. You should see a single Bundle with a news_results array containing 10 news articles.

We need to unpack this array so we can process each news result one-by-one. Look for the Iterator Module under Flow Control.

Populate the Iterator form with 1. news_results[].

Try running the Scenario and you should see Bundles for each news result in the Iterator Output.

Add Telegram to Make

The last Module we need is to Telegram's Send a Text Message or a Reply Module.

Create a connection like we did previously for SerpApi.

Paste in your Telegram Token here.

You should see your Telegram Connection appear in the Connection drop-down menu.

Enter the Chat ID we found in our Telegram Group's URL.

For the Text field, select the link field from the green Iterator output; not the blue SerpApi output. Feel free to include any other free form text or fields in your message. This will be the content we send to Telegram.

We want Telegram to display link previews of all the news articles so we'll make sure to not disable link previews.

Note that link previews won't work for all news sites. In those cases, the message will just be a URL. Unfortunately, it's impossible to tell if this will happen until trying the link in Telegram.

Before you try running this, let's put a cap on the number of articles we're sending Telegram each run. With the current setup, our Iterator will process all 10 items in news_results. Let's limit this to only the top 3 news results.

Between the Iterator and Telegram Modules, there will be 3 little dots. Click on the middle one to open the create filter menu.

Build a filter condition for the position field of the Iterator Module. The condition should be a numeric operator for "Less than or equal to."

We want only the top 3 so we'll set the filter to allow news results when the position is less than or equal to 3.

Now try running the Scenario. You should see 3 operations in the bubble coming off the Telegram Module.

More importantly, you should see 3 news articles appear in your Telegram Group.

Looks good! Here's what the Scenario should look like:

Schedule Hourly Scrapes

Finally, we want this to run every hour. Start the setup by clicking where it says "Every 15 minutes."

Set it to execute every 60 minutes.

You can also use the Advanced Scheduling feature to set your Scenario to only execute on certain days or during certain times. In the example below, I've set it to run between 8am and 8pm on weekdays.

Once you save those settings, Make will prompt you to activate the Scenario so it will start following the schedule. Hit "Activate scenario" if you're ready.

Once activated, you should see something like below. Simply hit the toggle to deactivate the Scenario.

Want to Import This?

If you'd like to skip all this manual configuration, you can download the blueprint of this Scenario and then import it into an empty Scenario in your Make account. All you'll need to do is configure your own SerpApi and Telegram connections. Then update the Telegram Module to reference your own Chat/Group ID.