> ## Content Index
> Fetch the complete content index at: https://serpapi.com/blog/llms.txt
> Use this file to discover other available public pages before exploring further.

# Build Simple CLI-Based Voice Assistant with PyAudio, Speech Recognition, pyttsx3 and SerpApi
- URL: https://serpapi.com/blog/build-simple-cli-based-voice-assistent-with-serpapi/
- Published: 2022-11-28T14:05:09.000Z
- Updated: 2022-12-07T13:42:44.000Z
- Description: Build a simple CLI voice assistant as an introduction to speech recognition and API usage. A step-by-step tutorial.
- Author: Dmitriy Zub
- Tags: Projects, Python

## Intro

As you saw by the title, this is a demo project that shows a very basic voice-assistant script that can answer your questions in the terminal based on Google Search results.

You can find the full code in the GitHub repository: [dimitryzub/serpapi-demo-projects/speech-recognition/cli-based/](https://github.com/dimitryzub/serpapi-demo-projects/tree/4789cf3c2a5cb7af6c29ba13a4222a0eec05eadf/speech-recognition/cli-based)

The follow-up blog post(s) will be about:

- Web-based solution using [Flask](https://flask.palletsprojects.com/en/2.2.x/), some HTML, CSS and Javascript.
- Android & Windows based solution using [Flutter](https://flutter.dev/) and [Dart](https://dart.dev/).

## What we will build

## Prerequisites

First, let's make sure we are in a different environment and properly install the libraries we need for the project. The hardest (possibly) will be to install `pyaudio`.

### Virtual Environment and Libraries Installation

Before we start installing libraries, we need create and activate new environment for this project:

```bash
# if you're on Linux based systems
$ python -m venv env && source env/bin/activate
$ (env) <path>

# if you're on Windows and using Bash terminal
$ python -m venv env && source env/Scripts/activate
$ (env) <path>

# if you're on Windows and using CMD
python -m venv env && .\env\Scripts\activate
$ (env) <path>

```

|                                  | Explanation                                                                                                                                                                      |
| -------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| python -m venv env               | tells Python to run module ([\-m](https://docs.python.org/2/using/cmdline.html#cmdoption-m)) [venv](https://docs.python.org/3/library/venv.html) and create a folder called env. |
| &&                               | Stands for AND.                                                                                                                                                                  |
| source <venv\_name>/bin/activate | will activate your environment and you'll be able to install libraries only in that environment.                                                                                 |

Now install all needed libraries:

```bash
pip install rich pyttsx3 SpeechRecognition google-search-results

```

Now to [pyaudio](https://pypi.org/project/PyAudio/). Please, keep in mind that `pyaudio` may throw an error while installing. An additional research may be needed on your end.

If you're on Linux, we need to [install some development dependencies to use pyaudio](https://stackoverflow.com/a/50409644/15164646):

```bash
$ sudo apt-get install -y libasound-dev portaudio19-dev
$ pip install pyaudio

```

If you're on Windows, it's simpler (tested with CMD and Git Bash):

```bash
pip install pyaudio

```

## Full Code

```python
import os
import speech_recognition
import pyttsx3
from serpapi import GoogleSearch
from rich.console import Console
from dotenv import load_dotenv

load_dotenv('.env')
console = Console()

def main():
    console.rule('[bold yellow]SerpApi Voice Assistant Demo Project')
    
    recognizer = speech_recognition.Recognizer()

    while True:
        with console.status(status='Listening you...', spinner='point') as progress_bar:
            try:
                with speech_recognition.Microphone() as mic:
                    recognizer.adjust_for_ambient_noise(mic, duration=0.1)
                    audio = recognizer.listen(mic)
                    
                    text = recognizer.recognize_google(audio_data=audio).lower()
                    console.print(f'[bold]Recognized text[/bold]: {text}')
                    
                    progress_bar.update(status='Looking for answers...', spinner='line')
                    params = {
                        'api_key': os.getenv('API_KEY'),
                        'device': 'desktop',
                        'engine': 'google',
                        'q': text,
                        'google_domain': 'google.com',
                        'gl': 'us',
                        'hl': 'en'
                    }

                    search = GoogleSearch(params)
                    results = search.get_dict()
                    
                    try:
                        if 'answer_box' in results:
                            try:
                                primary_answer = results['answer_box']['answer']
                            except:
                                primary_answer = results['answer_box']['result']
                            console.print(f'[bold]The answer is[/bold]: {primary_answer}')
                            
                        elif 'knowledge_graph' in results:
                            secondary_answer = results['knowledge_graph']['description']
                            console.print(f'[bold]The answer is[/bold]: {secondary_answer}')
                        else:
                            tertiary_answer = results['answer_box']['list']
                            console.print(f'[bold]The answer is[/bold]: {tertiary_answer}')
                        progress_bar.stop() # if answered is success -> stop progress bar.
                        
                        user_promnt_to_contiune_if_answer_is_success = input('Would you like to to search for something again? (y/n) ')
                        
                        if user_promnt_to_contiune_if_answer_is_success == 'y':
                            recognizer = speech_recognition.Recognizer()
                            continue # run speech recognizion again until `user_promt` == 'n'
                        else:
                            console.rule('[bold yellow]Thank you for cheking SerpApi Voice Assistant Demo Project')
                            break
                    except KeyError:
                        progress_bar.stop()
                        
                        error_user_promt = input("Sorry, didn't found the answer. Would you like to rephrase it? (y/n) ")
                        
                        if error_user_promt == 'y':
                            recognizer = speech_recognition.Recognizer()
                            continue # run speech recognizion again until `user_promt` == 'n'
                        else:
                            console.rule('[bold yellow]Thank you for cheking SerpApi Voice Assistant Demo Project')
                            break
                            
            except speech_recognition.UnknownValueError:
                progress_bar.stop()
                user_promt_to_continue = input('Sorry, not quite understood you. Could say it again? (y/n) ')
                
                if user_promt_to_continue == 'y':
                    recognizer = speech_recognition.Recognizer()
                    continue # run speech recognizion again until `user_promt` == 'n'
                else:
                    progress_bar.stop()
                    console.rule('[bold yellow]Thank you for cheking SerpApi Voice Assistant Demo Project')
                    break

                
if __name__ == '__main__':
    main()

```

### Code Explanation

Import libraries:

```python
import os
import speech_recognition
import pyttsx3
from serpapi import GoogleSearch
from rich.console import Console
from dotenv import load_dotenv

```

| Library                                                                          | Purpose                                                                                                                                                               |
| -------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| [rich](https://github.com/Textualize/rich)                                       | Python library for beautiful formatting in the terminal.                                                                                                              |
| [pyttsx3](https://github.com/nateshmbhat/pyttsx3)                                | Python's Text-to-speech converter that works in offline.                                                                                                              |
| [SpeechRecognition](https://github.com/Uberi/speech%5Frecognition)               | Python library to convert speech to text.                                                                                                                             |
| [google-search-results](https://github.com/serpapi/google-search-results-python) | SerpApi's Python API wrapper that parses data from 15+ search engines.                                                                                                |
| [os](https://docs.python.org/3/library/os.html)                                  | To read secret environment variable. In this case it's SerpApi API key.                                                                                               |
| [dotenv](https://github.com/theskumar/python-dotenv)                             | To load your environment variable(s) (SerpApi API key) from .env file. .env file could renamed to any file: .napoleon . (dot) represents a environment variable file. |

Define `rich` [Console()](https://rich.readthedocs.io/en/stable/console.html). It will be used to prettify terminal output (animations, etc):

```python
console = Console()

```

Define `main` function where all will be happening:

```python
def main():
    console.rule('[bold yellow]SerpApi Voice Assistant Demo Project')

    recognizer = speech_recognition.Recognizer()

```

At the beginning of the function we're defining `speech_recognition.Recognizer()` and `console.rule` will create the following output:

```
───────────────────────────────────── SerpApi Voice Assistant Demo Project ─────────────────────────────────────

```

The next step is to create a while loop that will be constantly listening for microphone input to recognize the speech:

```python
while True:
    with console.status(status='Listening you...', spinner='point') as progress_bar:
        try:
            with speech_recognition.Microphone() as mic:
                recognizer.adjust_for_ambient_noise(mic, duration=0.1)
                audio = recognizer.listen(mic)
                
                text = recognizer.recognize_google(audio_data=audio).lower()
                console.print(f'[bold]Recognized text[/bold]: {text}')

```

| Code                                                                                                                                                                                     | Explanation                                                                                               |
| ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------- |
| [console.status](https://github.com/Textualize/rich/blob/5f4e93efb159af99ed51f1fbfd8b793bb36448d9/rich/console.py#L1144-L1163)                                                           | A rich progress bar, it's used only for cosmetic purpose.                                                 |
| speech\_recognition.Microphone()                                                                                                                                                         | To start picking input from the microphone.                                                               |
| [recognizer.adjust\_for\_ambient\_noise](https://github.com/Uberi/speech%5Frecognition/blob/353d4ef74a55082de1f32796601899506d1b8bc7/speech%5Frecognition/%5F%5Finit%5F%5F.py#L560-L567) | Intended to calibrate the energy threshold with the ambient energy level.                                 |
| [recognizer.listen](https://github.com/Uberi/speech%5Frecognition/blob/353d4ef74a55082de1f32796601899506d1b8bc7/speech%5Frecognition/%5F%5Finit%5F%5F.py#L636-L649)                      | To listen for actual user text.                                                                           |
| [recognizer.recognize\_google](https://github.com/Uberi/speech%5Frecognition/blob/353d4ef74a55082de1f32796601899506d1b8bc7/speech%5Frecognition/%5F%5Finit%5F%5F.py#L859-L874)           | Performs speech recognition using Google Speech Recongition API. lower() is to lower recognized text.     |
| console.print                                                                                                                                                                            | A rich print statement that allows to use text modification, such as adding **bold**, *italic* and so on. |

`spinner='point'` will produce the following output ([use python -m rich.spinner to see list of spinners](https://github.com/Textualize/rich/blob/5f4e93efb159af99ed51f1fbfd8b793bb36448d9/rich/console.py#L1157)):

![rich-loading-progress](https://user-images.githubusercontent.com/78694043/203987922-937460b4-2754-429a-af82-f8f7491cb33e.gif)

After that, we need to initialize SerpApi search parameters for the search:

```python
progress_bar.update(status='Looking for answers...', spinner='line') 
params = {
    'api_key': os.getenv('API_KEY'),  # serpapi api key   
    'device': 'desktop',              # device used for 
    'engine': 'google',               # serpapi parsing engine: https://serpapi.com/status
    'q': text,                        # search query 
    'google_domain': 'google.com',    # google domain:          https://serpapi.com/google-domains
    'gl': 'us',                       # country of the search:  https://serpapi.com/google-countries
    'hl': 'en'                        # language of the search: https://serpapi.com/google-languages
    # other parameters such as locations: https://serpapi.com/locations-api
}

search = GoogleSearch(params)         # where data extraction happens on the SerpApi backend
results = search.get_dict()           # JSON -> Python dict

```

`progress_bar.update` will, well, update `progress_bar` with a new `status` (text printed in the console), and `spinner='line'` will produce the following animation:

![rich-loading-line-progress](https://user-images.githubusercontent.com/78694043/204004803-6a8412eb-70f8-4060-82be-5df9ce076133.gif)

After that, the data extraction happens from Google search using SerpApi's [Google Search Engine API](https://serpapi.com/search-api).

The following part of the code will do the following:

![image](https://user-images.githubusercontent.com/78694043/204011946-9f4ff6ea-20f3-4021-b57b-b783f7c2c71a.png)

```python
try:
    if 'answer_box' in results:
        try:
            primary_answer = results['answer_box']['answer']
        except:
            primary_answer = results['answer_box']['result']
        console.print(f'[bold]The answer is[/bold]: {primary_answer}')

    elif 'knowledge_graph' in results:
        secondary_answer = results['knowledge_graph']['description']
        console.print(f'[bold]The answer is[/bold]: {secondary_answer}')
    else:
        tertiary_answer = results['answer_box']['list']
        console.print(f'[bold]The answer is[/bold]: {tertiary_answer}')
    progress_bar.stop()  # if answered is success -> stop progress bar

    user_promnt_to_contiune_if_answer_is_success = input('Would you like to to search for something again? (y/n) ')

    if user_promnt_to_contiune_if_answer_is_success == 'y':
        recognizer = speech_recognition.Recognizer()
        continue         # run speech recognizion again until `user_promt` == 'n'
    else:
        console.rule('[bold yellow]Thank you for cheking SerpApi Voice Assistant Demo Project')
        break
except KeyError:
    progress_bar.stop()  # if didn't found the answer -> stop progress bar

    error_user_promt = input("Sorry, didn't found the answer. Would you like to rephrase it? (y/n) ")

    if error_user_promt == 'y':
        recognizer = speech_recognition.Recognizer()
        continue         # run speech recognizion again until `user_promt` == 'n'
    else:
        console.rule('[bold yellow]Thank you for cheking SerpApi Voice Assistant Demo Project')
        break

```

The final step is to handle error when no sound was picked up from the microphone:

```python
# while True:
#     with console.status(status='Listening you...', spinner='point') as progress_bar:
#         try:
            # speech recognition code
            # data extraction code
        except speech_recognition.UnknownValueError:
                progress_bar.stop()         # if didn't heard the speech -> stop progress bar
                user_promt_to_continue = input('Sorry, not quite understood you. Could say it again? (y/n) ')

                if user_promt_to_continue == 'y':
                    recognizer = speech_recognition.Recognizer()
                    continue               # run speech recognizion again until `user_promt` == 'n'
                else:
                    progress_bar.stop()    # if want to quit -> stop progress bar
                    console.rule('[bold yellow]Thank you for cheking SerpApi Voice Assistant Demo Project')
                    break

```

`console.rule()` will provide the following output:

```
───────────────────── Thank you for cheking SerpApi Voice Assistant Demo Project ──────────────────────

```

Add `if __name__ == '__main__'` idiom which protects users from accidentally invoking the some script(s) when they didn't intend to, and call the `main` function which will run the whole script:

```python
if __name__ == '__main__':
    main()

```

## Links

- [rich](https://github.com/Textualize/rich)
- [pyttsx3](https://github.com/nateshmbhat/pyttsx3)
- [SpeechRecognition](https://github.com/Uberi/speech%5Frecognition)
- [google-search-results](https://github.com/serpapi/google-search-results-python)
- [os](https://docs.python.org/3/library/os.html)
- [python-dotenv](https://github.com/theskumar/python-dotenv)

Join us on [Twitter](https://twitter.com/serp%5Fapi) | [YouTube](https://www.youtube.com/channel/UCUgIHlYBOD3yA3yDIRhg%5Fmg)

Add a [Feature Request](https://github.com/serpapi/public-roadmap/issues)💫 or a [Bug](https://github.com/serpapi/public-roadmap/issues)🐞