> ## Content Index
> Fetch the complete content index at: https://serpapi.com/blog/llms.txt
> Use this file to discover other available public pages before exploring further.

# Scrape Related Searches from DuckDuckGo using Python
- URL: https://serpapi.com/blog/scrape-related-searches-from-duckduckgo-using-python/
- Published: 2021-07-30T05:22:54.000Z
- Updated: 2022-12-21T07:25:24.000Z
- Description: Scrape query name and a link to that query from DuckDuckGo related search results using Python.
- Author: Dmitriy Zub
- Tags: DuckDuckGo, Python, Web Scraping

### What will be scraped

![image](https://res.cloudinary.com/practicaldev/image/fetch/s--Y-H5LKia--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_880/https://dev-to-uploads.s3.amazonaws.com/uploads/articles/jvkb166m77st9qbabl0k.png)

### [DuckDuckGo Related Searches API](https://serpapi.com/duckduckgo-related-searches)

The difference in using an API solution and DIY solution is that you'll get a faster response since there's no need to render the page. 

Additionally, iterating over structured `JSON` is a faster process rather than writing parser from scratch and maintaining it overtime, and figuring out how to bypass blocks.

```python
import json
from serpapi import GoogleSearch

params = {
  "api_key": "...",          # https://serpapi.com/manage-api-key
  "engine": "duckduckgo",
  "q": "fus ro dah",
  "kl": "us-en"
}

search = GoogleSearch(params)
results = search.get_dict()

for result in results['related_searches']:
    print(json.dumps(result, indent=2))

```

![](https://i.giphy.com/media/FDEwcpBrUe5sXMINWe/giphy.gif)

### DIY Process

For some reason `request-html` can't locate elements at the bottom of the page when using `xpath` or `css` selectors, and `scrolldown=` parameter didn't help either.

This time `selenium` was used since it is the easiest way to get the data but at the same time, not the fastest.

Selecting `CSS` selector to grab **query** and a **link** and running the script.  

![](https://i.giphy.com/media/0tKMZpqOCHzIm3lSLF/giphy.gif)

*Note №1: running `selenium` in headless mode didn't return any results, or I was doing something wrong.*

*Note №2: the data could be extracted from the* `*<script>*` *tag without `selenium` use, but it will be a much more time-consuming process.*

### Full DIY Code

```python
from selenium import webdriver

driver = webdriver.Chrome(executable_path='/path/to/chromedriver.exe')
driver.get('https://duckduckgo.com/?q=fus ro dah&kl=us-en&ia=web')

for result in driver.find_elements_by_css_selector('.result__a.related-searches__link'):
    query = result.text
    link = result.get_attribute('href')
    print(f'{query}\n{link}\n')
driver.quit()

------------------
'''
fus ro dah meme
https://duckduckgo.com/?q=fus%20ro%20dah%20meme&kl=us-en

fus ro dah sound
https://duckduckgo.com/?q=fus%20ro%20dah%20sound&kl=us-en

fus ro dah skyrim
https://duckduckgo.com/?q=fus%20ro%20dah%20skyrim&kl=us-en
...
'''

```

### Links

- [Code in the online IDE](https://replit.com/@DimitryZub1/DuckDuckGo-Scrape-Related-Searches-python#main.py)
- [DuckDuckGo Related Searches API](https://serpapi.com/duckduckgo-related-searches)