> ## Content Index
> Fetch the complete content index at: https://serpapi.com/blog/llms.txt
> Use this file to discover other available public pages before exploring further.

# Create a super fast AI assistant with Groq (Without a database)
- URL: https://serpapi.com/blog/create-super-fast-ai-assistant-with-groq/
- Published: 2024-05-16T01:55:33.000Z
- Updated: 2024-09-20T07:39:38.000Z
- Description: Here is how I create an AI assistant without a database or special API. We'll rely on the AI model itself using Groq for a super fast response.
- Author: Hilman Ramadhan
- Tags: Artificial Intelligence

Last week, I tried to build a voice AI assistant using OpenAI AI assistant. It takes a while to generate a response, which is not suitable for a voice assistant. So, I'm looking for an alternative to make my assistant faster. That's how I found out about Groq. This post will cover how I built an AI assistant using Groq. 

![](https://storage.ghost.io/c/a5/00/a5004977-0dd2-4bcd-9292-dd0e05d4c59e/content/images/2024/05/build-a-fast-ai-assistant-with-groq-1.png)

Build a fast AI assistant with Groq and llama model illustration

**Pros and Cons summary**  
Pro: Easy to implement with only one API (Groq API).  
Respond is fast.  
  
Cons: The longer we chat, the higher the chance that we might lose some context along the way.

## What is Groq?

Groq is a service that provides a super fast engine to run AI applications. **It's not an AI model!** We can run different AI models like Llama, Mixtral, Gemma and more!

[Why Groq - GroqAn LPU Inference Engine, with LPU standing for Language Processing Unit™, is a new type of end-to-end processing unit system that provides the fastest![](https://wow.groq.com/wp-content/uploads/2024/02/android-icon-192x192-1.png)GroqGroq, Inc.![](https://wow.groq.com/wp-content/uploads/2024/02/Groq-Product-Graphics2.png)](https://wow.groq.com/why-groq/)

## How I built a fast AI assistant

Many AI models exist, but only OpenAI offers an easy way to implement a chat-like experience using the [Assistants API](https://platform.openai.com/docs/assistants/overview). By default, these models won't know or understand the context of our previous chat. So, we have to re-explain everything if we want the AI to understand the context of each message.

Some alternatives exist, such as using [LangChain chat history](https://python.langchain.com/docs/how%5Fto/qa%5Fchat%5Fhistory%5Fhow%5Fto/) or [ConversationBufferMemory](https://python.langchain.com/v0.1/docs/modules/memory/types/buffer/). But I prefer to find a simple way (\*with the caveat, of course). Luckily, I found some ideas on the internet (Thank you, Internet!).

> The idea below can be implemented for any AI model/engine, not just Groq. You can try this with OpenAI itself, Mixtral, Claude, and so on.

![](https://storage.ghost.io/c/a5/00/a5004977-0dd2-4bcd-9292-dd0e05d4c59e/content/images/2024/05/CleanShot-2024-05-16-at-08.28.20@2x.png)

chat flow illustration

Here is the flow:

- The user sends the initial message
- The AI responds to the message
- We ask AI to summarize the conversation
- We send the response and summary back to the user
- The user will send the summary back later alongside the new message
- AI now will reply based on the fresh message and with help of the conversation summary to provide some context.

**The caveat of this method**  
By summarizing a conversation, we may lose some information along the way. That's why it's a good idea in certain cases to store the message history on a database (Vector database). 

> One way I can reduce this shortage is by attaching the recent reply from AI. I've also read an article that suggests keeping the latest 2-3 conversations and providing them as additional context later.

## Code implementation

I'll use NodeJs for this tutorial. Feel free to use any language you want. The final code is available at GitHub:

[GitHub - hilmanski/assistants-api-with-groq-aiContribute to hilmanski/assistants-api-with-groq-ai development by creating an account on GitHub.![](https://github.githubassets.com/assets/pinned-octocat-093da3e6fa40.svg)GitHubhilmanski![](https://opengraph.githubassets.com/1a8ec93aa50be1be795b2ae1665c44c0936686962eefa324f0ea47350276a96f/hilmanski/assistants-api-with-groq-ai)](https://github.com/hilmanski/assistants-api-with-groq-ai)

1. **Install dependencies**

```
npm i express groq-sdk dotenv --save
```

- Express for creating a route for the endpoint
- Groq-sdk is the official package for using Groq in Javascript
- dotenv to store our API key safely.
1. **Add API Key**

Create a new `.env` file. Add your Groq API key in this file like this:

```
GROQ_API_KEY=YOUR_GROQ_API_KEY
```

Make sure to sign up to Groq and get your API key [here](https://console.groq.com/keys).

1. **Basic Setup**

Let's create a new `index.js`file, and we'll write everything in this file. We prepare one endpoint called `chat` where we'll send these parameters:  
\- message: user's message  
\- latestReply: The latest reply from AI   
\- messageSummary: The conversation summary so far

In this endpoint, we'll do two things:  
\- Respond to new user message (with latestReply and messageSummary as context)  
\- Create a new conversation summary by providing the fresh reply from AI.

```javascript
const express = require('express');

// Express Setup
const app = express();
app.use(express.json());
const port = 3000

require("dotenv").config();
const { GROQ_API_KEY } = process.env;

// GROQ Setup
const Groq = require("groq-sdk");
const groq = new Groq({
    apiKey: GROQ_API_KEY
});

async function chatWithGroq() { } // soon
async function summarizeConversation() { } // soon

app.post('/chat', async (req, res) => {
    const { message, latestReply, messageSummary } = req.body;

    // request chat completion
    const reply = await chatWithGroq(message, latestReply, messageSummary)
    
    // request chat summary
    const summary = await summarizeConversation(message, reply, messageSummary)
    
    // Always return chat history/summary
    res.send({
        reply,
        summary
    })
})

app.listen(port, () => {
  console.log(`Example app listening on port ${port}`)
})
```

1. **Chat with Groq method**

Here is the chatWithGroq method implementation:

```javascript
async function chatWithGroq(userMessage, latestReply, messageHistory) {
    let messages = [{
        role: "user",
        content: userMessage
    }]

    if(messageHistory != '') {
        messages.unshift({
            role: "system",
            content: `Our conversation's summary so far: """${messageHistory}""". 
                     And this is the latest reply from you """${latestReply}"""`
        })
    }

    console.log('original message', messages)

    const chatCompletion = await groq.chat.completions.create({
        messages,
        model: "llama3-8b-8192"
    });

    const respond = chatCompletion.choices[0]?.message?.content || ""
    return respond
}
```

- We only provide a conversation summary when we have one (look at the if statement). So, it won't be included in our first message.
1. **Conversation summary method**

Here is the `summarizeConversation` method implementation:

```javascript
async function summarizeConversation(message, reply, messageSummary) {
    let content = `Summarize this conversation 
                    user: """${message}""",
                    you(AI): """${reply}"""
                  `

    // For N+1 message
    if(messageSummary != '') {
        content = `Summarize this conversation: """${messageSummary}"""
                    and last conversation: 
                    user: """${message}""",
                    you(AI): """${reply}"""
                `
    }

    const chatCompletion = await groq.chat.completions.create({
        messages: [
            {
                role: "user",
                content: content
            }
        ],
        model: "llama3-8b-8192"
    });

    const summary = chatCompletion.choices[0]?.message?.content || ""
    console.log('summary: ', summary)
    return summary
}
```

In this method, we ask the AI to create a summary based on the latest summary and recent reply.

## Demo Time!

You can use any API client, like Postman, Thunder (VS Code), etc.

> Don't forget to run your program with `node index.js`

Create a POST request for the `/chat` endpoint and provide `message` endpoint and provide the first message parameter.

![](https://storage.ghost.io/c/a5/00/a5004977-0dd2-4bcd-9292-dd0e05d4c59e/content/images/2024/05/CleanShot-2024-05-16-at-09.39.32@2x.png)

initial message illustration

We can display the `reply` from the `response` on our user interface. This is the actual reply to our message.

We'll save the `summary` for the next request.

Now, this is how the JSON looks like for the N+1 message:

![](https://storage.ghost.io/c/a5/00/a5004977-0dd2-4bcd-9292-dd0e05d4c59e/content/images/2024/05/CleanShot-2024-05-16-at-09.41.55@2x.png)

N+1 message parameters

The next messages should include the `latestReply` and `messageSummary` as parameters.

- message: \*Don't forget to add a new message. This is you talking to the AI. Notice that I use `here` on my question, to validate that the AI knows what's the previous context here.
- latestReply: Send the latest reply from AI (from previous response)
- messageSummary: Send the conversation summary so far (from previous response)

Here is the result to this request:

![](https://storage.ghost.io/c/a5/00/a5004977-0dd2-4bcd-9292-dd0e05d4c59e/content/images/2024/05/CleanShot-2024-05-16-at-09.44.28@2x.png)

Summary conversation and reply example

As you can see, the AI knows that when I said `here` I was talking about `Indonesia`. You can try to send a follow-up message (create a new request) by asking something like "Can you tell me more about number 4?" as an example. But don't forget that we always need to update the `latestReply` and `summaryConversation` on each request.

> To return the response and summarize the conversation, I only need to wait around `2s`. This is much faster than using OpenAI AI assistants.

## FAQ

**Why don't we store all the conversation history?**  
The longer we talk with the assistant, the more tokens we'll need. It's to prevent us from paying a lot of money for the service. This method might work for the open source model that you run on your own server.

**Reference:**  
\- [Build a smart AI voice assistant](https://serpapi.com/blog/build-a-smart-ai-voice-assistant-connect-to-the-internet/)  
\- [Basic tutorial: Assistants API by OpenAI](https://serpapi.com/blog/assistant-api-openai-beginner-tutorial/)