Google and Bing offer Knowledge Graph results and I already have 2 blog posts guiding you to scrape all of this data effortlessly. Yahoo is one of the big search engines, and Yahoo offers Knowledge Graph results.

If we wonder which sources of data and how Yahoo built their Knowledge Graph results, you can visit their research page. https://research.yahoo.com/publications/9194/yahoo-knowledge-graph

Yahoo results contain slightly different data compared to Google and Bing. It may help your content to be more diverse and more informative.

Setting up a SerpApi account

SerpApi offers a free plan for newly created accounts. Head to the sign-up page to register an account and complete your first search with our interactive playground. When you want to do more searches with us, please visit the pricing page.

Once you are familiar with all results, you can utilize SERP APIs using your API Key.

Scrape your first Yahoo Knowledge graph result with SerpApi

Head to the Yahoo Knowledge Graph Results from the documentation on SerpApi for details.

In this tutorial, we will scrape all albums of "Sia"(Singer) using Yahoo Knowledge Graph results. The data contains: "name", "link", "thumbnail" and "year". You can also scrape even more information with SerpApi!

First, you need to install the SerpApi client library.

pip install google-search-results

Set up the SerpApi credentials and search.

import serpapi, os, json

params = {
    'api_key': 'YOUR_API_KEY',         # your serpapi api
    'engine': 'yahoo',                 # SerpApi search engine	
    'p': 'Sia'
}

To retrieve Yahoo Knowledge Graph Results for a given search term, you can use the following code:


results = serpapi.Client().search(params).get_dict()['knowledge_graph']['albums']

You can store Yahoo Knowledge Graph Results JSON data in databases or export them to a CSV file.

import csv

header = ['name', 'link', 'image', 'year']

with open('sia_albums.csv', 'w', encoding='UTF8', newline='') as f:
    writer = csv.writer(f)

    writer.writerow(header)

    for item in results:
        print(item)
        writer.writerow([item.get('name'), item.get('link'), item.get('image'), item.get('extensions')])

This example is using Python, but you can also use your all your favorite programming languages like Ruby, NodeJS, Java, PHP....

If you have any questions, please feel free to contact me.