All AI models, including Gemini by Google, are limited by a knowledge cutoff at a specific year. This means they can't access or provide updates on events or developments after their last training update. This limitation is crucial to keep in mind when seeking information about recent events.
![](https://serpapi.com/blog/content/images/2024/07/gemini-API-cannot-access-real-time-data.png)
But don't worry! If you want to use the Gemini API while still needing real-time data from the internet, we have something called "Function Calling." With it, we can call a custom function or an external API.
What is a function calling?
Here is what Google says about function-calling:
Custom functions can be defined and provided to Gemini models using the Function Calling feature. The models do not directly invoke these functions, but instead generate structured data output that specifies the function name and suggested arguments. This output lets you write applications that take the structured output and call external APIs, and the resulting API output can then be incorporated into a further model prompt, allowing for more comprehensive query responses. - Gemini API Docs
![](https://serpapi.com/blog/content/images/2024/07/google-gemini-function-calling-python-1.png)
Using function calling
, we want to achieve 2 things:
- Parse specific data from the user's prompt. *Turn human language into structured data
- Call a custom function and pass data we received as arguments
Video tutorial
If you prefer watching a video, here is the video explanation:
Basic function calling sample in Gemini API using Python
Here is a step-by-step tutorial on how to use Gemini API with function calling.
Source code is available on GitHub: https://github.com/hilmanski/gemini-api-python-function-calling-sample
Step 1: Project Setup
Create a new Python environment for your project.
python -m venv env
source env/bin/activate
Install the Gemini API Python package
pip install google-generativeai
Step 2: Get and export the Gemini API Key
Export your Gemini API Key
Get your API Key in Google AI Studio and then export it.
export GEMINI_API_KEY=YOUR_API_KEY_HERE
Step 3: Code setup
Create a new Python file. You can name it whatever you want, for example main.py
. Import the genai
package and use the API Key
import os
import google.generativeai as genai
genai.configure(api_key=os.getenv('GEMINI_API_KEY'))
Step 4: Declare a custom function
Let's create a new customer function and schema to share with Gemini later.
To use the function calling in Gemini, we need a function declaration
. It is a schema explanation of the actual function.
# continue at main.py file ...
# Sample custom function
def score_checker(score):
if score > 0.5:
return "Student passed the test"
else:
return "Student failed the test"
# function declaration
score_checker_declaration = {
'name': "score_checker",
'description': "Check if a score is a pass or fail",
'parameters': {
"type": "object",
"properties": {
"score": {
"type": "number",
"description": "The score to check"
}
},
"required": [
"score"
]
},
}
In this example, I have a simple function called score_checker
that will return a text depending on the score. We will call this function.
You can also see a function declaration where we describe the function and its properties. We will share this with Gemini so Gemini can be aware of any available "tools" it has.
Step 5: Test the AI
Now, the important part is to run Gemini API!
prompt = "I saw my test score was 0.7. Did I pass?"
model = genai.GenerativeModel('gemini-1.5-flash')
response = model.generate_content(
prompt,
tools=[{
'function_declarations': [score_checker_declaration],
}],
)
Explanation:
- I prepared a prompt.
- setup a new model
- use generate_content
method while attaching the prompt and the custom function declaration
Our goal is to make sure Gemini can parse the score from the human natural language we provide on the prompt.
We can get the argument and the function name Gemini needs to call by:
function_call = response.candidates[0].content.parts[0].function_call
args = function_call.args
function_name = function_call.name
Assume we get the correct arguments and correct function name. Let's call the actual function:
if function_name == 'score_checker':
result = score_checker(args['score'])
Now we have the results from score_checker
function.
Step 6: Respond in a natural language
We can return the result as it is, but we can also return the response in a human natural language by providing a response from the AI. To do this, we'll call generate_content
method again with the result.
response = model.generate_content(
"Based on this information `" + result + "` respond to the student in a friendly manner.",
)
print(response.text)
Instead of static text between: Student passed the test
or Student failed the tests
we can get something more dynamic like this:
![](https://serpapi.com/blog/content/images/2024/07/CleanShot-2024-07-16-at-06.12.39.png)
Get answers from the internet in Gemini API
We've looked at the basics. Let's replace the static function with calling an external API, which enables our AI to grab knowledge from the internet.
In this example, I will use the Google direct answer box API from SerpApi. This API allows us to grab any real-time data provided by Google SERP. For example, when asking about the weather, Google provides this answer in the answer box
:
![](https://serpapi.com/blog/content/images/2024/06/CleanShot-2024-06-24-at-14.44.27.png)
So, the idea is that we grab this data programmatically and pass the answer to the AI model. But before that, we need to grab the users' questions and turn them into a query to search in Google via SerpApi.
Let's dive in!
*The first three steps for the setup are the same.
Step 1: Project Setup
Create a new Python environment for your project.
python -m venv env
source env/bin/activate
Install the Gemini API Python package.
pip install google-generativeai
Step 2: Get and export the Gemini API Key
Export your Gemini API Key. Get your API Key in Google AI Studio and then export it.
export GEMINI_API_KEY=YOUR_API_KEY_HERE
Step 3: Code setup
Create a new Python file. You can name it whatever you want, for example main.py
. Import the genai
package and use the API Key.
import os
import google.generativeai as genai
genai.configure(api_key=os.getenv('GEMINI_API_KEY'))
Step 4: Install SerpApi and get your SerpAPI API key
Make sure to register at serpapi.com. Grab your API key from the dashboard and export it like before.
export SERPAPI_API_KEY=your_api_key
Install SerpApi python package
pip install google-search-results
Step 5: Declare a custom function
Let's declare the function and the function schema we want the Gemini to use.
To use the function calling in Gemini, we need a function declaration
. It is a schema explanation of the actual function.
# Continue at main.py file
from serpapi import GoogleSearch
def get_answer_box(query):
print("Parsed query: ", query)
search = GoogleSearch({
"q": query,
"api_key": os.getenv('SERPAPI_API_KEY')
})
result = search.get_dict()
if 'answer_box' not in result:
return "No answer box found"
return result['answer_box']
# function declaration
get_answer_box_declaration = {
'name': "get_answer_box",
'description': "Get the answer box result for real-time data from a search query",
'parameters': {
"type": "object",
"properties": {
"query": {
"type": "string",
"description": "The query to search for"
}
},
"required": [
"query"
]
},
}
Explanation:
- I'm printing the query in the get_answer_box
so we can see the parsed query returned by the AI.
Step 6: Call the AI
It's time to call the AI with the function calling feature. Please note, the first call to AI is only to parse the arguments and function name to use, not for providing the answer.
prompt = "It's freezing!, what is the temperature in London today? do you know it?"
model = genai.GenerativeModel('gemini-1.5-flash')
response = model.generate_content(
prompt,
tools=[{
'function_declarations': [get_answer_box_declaration],
}],
)
We can get the arguments, in this case, the query and the correct function name to call.
function_call = response.candidates[0].content.parts[0].function_call
args = function_call.args
function_name = function_call.name
Get the answer from SerpApi by calling the actual get_answer_box
function and the query as a parameter.
if function_name == 'get_answer_box':
result = get_answer_box(args['query'])
Step 7: Respond in human natural language
Now that we have the data let's pass this data on to AI and return a human response.
# Don't forget to `import json` at the top of this file
# Turn API response into string
# We're limiting the characters since the response can be very long.
data_from_api = json.dumps(result)[:500]
# Call AI model to generate natural language
response = model.generate_content(
"""
Based on this information: `""" + data_from_api + """`
and this question: `""" + prompt + """`
respond to the user in a friendly manner.
""",
)
print(response.text)
I shared the original prompt from the user for more context.
Run the program with python realtime.py
. Here is the result I got:
![](https://serpapi.com/blog/content/images/2024/07/CleanShot-2024-07-16-at-08.32.06.png)
As you can see, the AI responds based on the user prompt (contradicting the user's statement) and provides the current temperature that we got from Google answer box.
![](https://serpapi.com/blog/content/images/2024/07/CleanShot-2024-07-16-at-08.33.26.png)
Let's try a different prompt. Let's ask about Stock price
# prompt = "It's freezing!, what is the temperature in London today? do you know it?" # comment out
prompt = "Morning! how is Tesla stock doing today? I just bought some shares yesterday."
Result:
![](https://serpapi.com/blog/content/images/2024/07/CleanShot-2024-07-16-at-08.46.01@2x.png)
That's it; feel free to take a look at the final source code https://github.com/hilmanski/gemini-api-python-function-calling-sample
References:
- Intro function calling in Gemini API
- SerpApi Google answer box API
- Codelabs Gemini API