How to Scrape DuckDuckGo Search Results

DuckDuckGo has different results compared to Google and Bing search engines. DuckDuckGo focuses on privacy.

When you need results with prioritized user privacy, scraping DuckDuckGo results can be useful.

Besides that, you can scrape DuckDuckGo results to do competitor analysis, improve your SEO, and aggregate content.

In my previous blog post, we explored how we can easily scrape Google and Bing results with APIs from SerpApi. It would be easy with DuckDuckGo too.

Setting up a SerpApi account

SerpApi offers a free plan for newly created accounts. Head to the sign-up page to register an account and do your first search with our interactive playground. When you want to do more searches with us, please visit the pricing page.

Once you are familiar with all the results, you can utilize SERP APIs using your API Key.

Scrape your first DuckDuckGo Search Result with SerpApi

Head to the DuckDuckGo search results from the documentation on SerpApi for details.

In this tutorial, we will scrape and extract the organic results of "web ranking" keyword. The data contains: "position", "title", "link", "favicon", "snippet"

First, you need to install the SerpApi client library.

pip install google-search-results

Set up the SerpApi credentials and search.

import serpapi, os, json

params = {
    'api_key': 'YOUR_API_KEY',         # your serpapi api
    'engine': 'duckduckgo',            # SerpApi search engine	
    'q': 'web ranking'
}

To retrieve DuckDuckGo Search Results for a given search term, you can use the following code:

results = serpapi.Client().search(params).get_dict()['organic_results']

You can store DuckDuckGo Search Results JSON data in databases or export them to a CSV file.

import csv

header = ['position', 'title', 'link', 'favicon', 'snippet']

with open('duckduckgo.csv', 'w', encoding='UTF8', newline='') as f:
    writer = csv.writer(f)

    writer.writerow(header)

    for item in results:
        print(item)
        writer.writerow([item.get('position'), item.get('title'), item.get('link'), item.get('favicon'), item.get('snippet')])

This example is using Python, but you can also use your all your favorite programming languages like Ruby, NodeJS, Java, PHP, and more.

If you have any questions, please feel free to contact me.