Connect Assistant API to the Internet (OpenAI x Google)

Many developer dreams come true! Now, we can connect custom assistant (AI) API with external APIs. This way, we can combine the smart AI brain with more fresh data from the internet.

Connect Assistant API by OpenAI to the Internet

Understand the components

OpenAI released assistant API at the end of 2023. With it, developers can build their custom AI assistant. You can add "tools" to your assistant. It can be PDF files or external APIs.

Here is our intro blog post on the basic of assistant API by OpenAI

We will use function calling to use the external API function.

Here is the intro blog post for basic of function calling by OpenAI

I assume you're already familiar with these two features from OpenAI (If not, please read the related post first).

What we're going to build

We'll continue our work from the previous post, where we have two simple endpoints to create a thread and a new message on the backend side.

You can get the complete code here:

We are going to build a blogger AI assistant:
- It can fetch top-ranking websites from Google search
- It can recommend a blog title based on that

Preparation and Setup

Let's prepare a few things before we write the actual code.

External API: Google Search API
We're going to use Google search API from SerpApi. So you need to register at serpapi.com and get your API Key for free.

Create a new assistant
Visit OpenAI website and create a new assistant.

Explanation
- I'm creating a new assistant called "SEO assistant"
- I give an instruction that includes information about the function_call tools I'm going to use.

You are an SEO assistant. A user will provide a keyword and you will analyze this keyword, SEO-wise. 

The function_call tools will return an organic_results based on the keyword. Analyze this organic_results and provide the user with information like what is the top 3 ranking results and what would be a good title idea if user want to be in the top rank as well.

Next, you can add a function to this assistant by clicking the + Add button. Here is my function for this. If you're unfamiliar with this structure, I suggest reading our function-calling blog post.

adding function on assistant tools OpenAI

Notes: creating a new assistant and attaching a function can be done programmatically as well.

Video tutorial

If you prefer to watch a video, I hope you enjoy this one:

Code tutorial

Here is a step-by-step tutorial on how to add an external API to the OpenAI assistant API.

Install package
We'll install a new package for this external API on the sample project we've started before.

npm install serpapi --save

Library documentation

Add API Key
- Copy your API key from serpapi.com dashboard.
- Add it to your .env file SERPAPI_KEY=XXXXX
- Call it from your code

const { OPENAI_API_KEY, ASSISTANT_ID, SERPAPI_KEY } = process.env;

Function for external API
First, I prepare the function to call our Google Search API

async function getSearchResult(query) {
    console.log('------- CALLING AN EXTERNAL API ----------')
    const json = await getJson({
        engine: "google",
        api_key: SERPAPI_KEY,
        q: query,
        location: "Austin, Texas",
    });

    return json["organic_results"];
}

It's up to you on what response to return, but I only need the organic_results from Google SERP in this case. The API response includes things like ads, knowledge graphs, people asking questions, and so on.

Later, we will pass this response to the assistant to extend it's knowledge.

Update checkingStatus method

Since we're adding a function-calling, we need to update our checkingStatus method. We're not just checking when it's completed; we need to add a conditional statement if any action is required.

Here is the conditional statement after checking if the status is completed:

if(status == 'completed') {
        clearInterval(pollingInterval);

        const messagesList = await openai.beta.threads.messages.list(threadId);
        let messages = []
        
        messagesList.body.data.forEach(message => {
            messages.push(message.content);
        });

        res.json({ messages });
    }

    // + Addition for function calling
    else if(status === 'requires_action') {
        console.log('requires_action.. looking for a function')

        if(runObject.required_action.type === 'submit_tool_outputs') {
            console.log('submit tool outputs ... ')
            const tool_calls = await runObject.required_action.submit_tool_outputs.tool_calls
            // Can be choose with conditional, if you have multiple function
            const parsedArgs = JSON.parse(tool_calls[0].function.arguments);
            console.log('Query to search for: ' + parsedArgs.query)

            const apiResponse = await getSearchResult(parsedArgs.query)
            
            const run = await openai.beta.threads.runs.submitToolOutputs(
                threadId,
                runId,
                {
                  tool_outputs: [
                    {
                      tool_call_id: tool_calls[0].id,
                        output: JSON.stringify(apiResponse)
                    },
                  ],
                }
            )

            console.log('Run after submit tool outputs: ' + run.status)
        }
    }

Code explanation
- We check the run status to see if it requires an action
- We check if the actions are submit_tool_outputs
- We parse the argument from the user's message (in this sample is the keyword we want to search for)
- We pass that parsed argument and call our external API getSearchResult
- Running the external API with beta.threads.runs.submitToolOutputs

Since our program keeps looping (polling) by the setInterval, it will revisit this checkingStatus method until the status is completed and return the response.

Let's try it

Now let's try our new code in Postman (HTTP Client)

Creating a thread
First, let's create a new thread:

We receive the threadId to use for sending a message.

Sending a message
Now, let's send our first message to ensure everything is working fine:

We can see that the basic functionality is still running fine, and the assistant can still respond to regular messages.

I'm just saying Hi and the AI response normally.

Trigger a function
Now, let's trigger the function to search for a particular keyword.
I'm asking Can you help me research for X ? Thanks!

The AI is smart enough that the user now provides a keyword that they should research for. It's based on the instruction we gave originally during the first creation.

Let's check our log:

We can see the function is triggered well. We still need to verify the real SERP result to make sure our AI is not hallucinating.

The top organic results are reflected in the response we received in the Postman.

Asking for ideas
The data is accurate, but let's see if the AI can think based on this data. I'm sending this message:
Give me 3 title ideas for the blog post please

Final conversation with assistant API by OpenAI

Since it's an assistant API, it understands our context as long as we share the same thread ID. It doesn't call the function again since it already saved that information.

That's it!
This is how we can connect the assistant API by OpenAI to the internet or external API.

Here is the final code from this tutorial

Feel free to contact me if you have any questions at hilman(d0t)serpapi(dot)com

Connect Assistant API to the Internet (OpenAI x Google)

Hilman Ramadhan

Understand the components

What we're going to build

Preparation and Setup

Video tutorial

Code tutorial

Let's try it

Free Plan · 100 searches / month

APIs

Easy Integrations

Features

Use Cases

Understand the components

What we're going to build

Preparation and Setup

Video tutorial

Code tutorial

Let's try it

Free Plan · 100 searches / month