For those who don’t use Chrome, Firefox or even Internet Explorer for their browser may have opted to search with DuckDuckGo.
Since SerpApi’s goal is to scrape Google and other search engines, we found it to be in our wheelhouse to scrape these privatized search results from DuckDuckGo that differ from that of Google’s search engine.
Taken from DuckDuckGo’s Wikipedia:
DuckDuckGo is an internet search engine that emphasizes protecting searchers' privacy and avoiding the filter bubble of personalized search results.DuckDuckGo does not show search results from content farms.[5] It uses various APIs of other websites to show quick results to queries and for traditional links it uses the help of its partners (mainly Bing) and its own crawler
When initially looking at the JSON response for the oh-so-typical SerpApi response: “coffee”.
I see common datasets compared to the Google Search: like ads
organic_results
inline_images
inline_images_link
related_searches
and the normal data that SerpApi provides: search_metadata
search_parameters
search_information
serpapi_pagination
.
Here’s a quick snapshot of differences of datasets for “coffee” between the DuckDuckGo Search and Google Search API:
One thing that jumps out to me is the amount of organic_results
that are provided in a default search. There are 27 items provided in DuckDuckGo’s search for coffee.
There is a serpapi_pagination
link available for the next page of results. And the offset in the link accounts for the 27 previous results:
- “https://serpapi.com/search.json?engine=duckduckgo&kl=us-en&q=Coffee&start=27"
Checking out the list of parameters, they differ a bit from the other engines. The localization parameter kl
defines the region to use for the DuckDuckGo search. Region code examples: us-en
for the United States, uk-en
for United Kingdom, or fr-fr
for France. Head to the DuckDuckGo regions for a full list of supported regions.
There are also the advanced filters for this search. The safe search safe
. This parameter defines the level of filtering for adult content. It can be set to 1
(Strict), -1
(Moderate - default), or -2
(Off). The df
parameter defines results filtered by date.
It can be set to:d
: Past day,w
: Past week,m
: Past month,y
: Past year,
Or you can pass a custom date following the next format: from_date
+ ..
+ to_date
(e.g. 2021-06-15..2021-06-16
).
This API does not have html response, just a text. search_metadata.prettify_html_file
contains a prettified version of the result. It is displayed in playground.
You can sign-up for SerpApi here: https://serpapi.com/
You can find the SerpApi user forum here: https://forum.serpapi.com/
You can find the API documentation here: https://serpapi.com/search-api/