How to scrape Bing search results

Bing, one of the leading search engines, offers valuable data for businesses, marketers and researchers. By scraping Bing search results, you can identify the latest market trends, gather data on keyword rankings, conduct SEO analysis, Competitor analysis, and market and consumer research.

Each search engine has strengths and unique features, e.g., Google is known for its comprehensive search index and advanced search features. Yahoo offers very good news, email, and finance search results. Bing offers deep integration with Microsoft Products, Social Media integration, showing real-time updates, trending topics, or social media profiles. Furthermore, Bing has different search algorithms, results slightly different from other search engines, and provides more information.

Setting up a SerpApi account

SerpApi offers a free plan for newly created accounts. Head to the sign-up page to register an account and do your first search with our interactive playground. When you want to do more searches with us, please visit the pricing page.

Once you are familiar with all results, you can utilize SERP APIs using your API Key.

Scrape your first Bing Search Result with SerpApi

Head to the Bing search results from the documentation on SerpApi for details.

In this tutorial, we will scrape and extract the timeline of Elon Musk from Bing knowledge graph results when searching “Elon Musk”. The data contains: "year", "text" and "link"

We will be using Ruby in this example, and the process is similar to other languages like Python, Node, Java, Javascript, etc.

First, you need to install the SerpApi client in your Gemfile.

gem 'google_search_results'

Then install the gem.

bundle install

Set up the SerpApi credentials and search.

BingSearch.api_key = "YOUR_API_KEY"
search = BingSearch.new(q: "Elon Musk")

puts search.get_hash

To retrieve the Bing knowledge graph for a given search query, you can use the following code:

results = search.get_hash[:knowledge_graph][:timeline]

You can store Elon Musk's timeline JSON data in databases or export them to a CSV file.

require 'csv'

CSV.open("elon_musk_timeline.csv", "w") do |csv|
  csv << ["year", "text", "link"]
  results.each do |item|
    csv << [item[:year], item[:text], organic_result[:link]]
  end
end

You can scrape Elon Musk's other information like social profiles, quotes, social projects and innovations... Or from other people like Bill Gates, Steve Jobs, ...

We also have some guides on scraping the Bing organic results, Images Results, and News Results with other languages. We also have a plan to support Bing Chat(AI) results in the future.

If you have any questions, please feel free to contact me.