RubyLLM 2.x: What's new for your agents

RubyLLM 2.0 and 2.1 are out and come with new APIs for video, speech, OCR, reranking, batches, conversation controls, expanded list of supported providers and even better Rails integration. But that's not all. If you build chat-based agents in Rails, tools can now wait for a human to approve them, you can run the agent loop one step at a time, and the agents can live in the database and survive a restart. So let's have a look on how to upgrade our old agents and give them new capabilities.

A quick look on the changes

The short version of both releases include the following main additions:

• Tool approvals. Mark a tool with requires_approval and the agent pauses until a human says yes or no.
• Caller-controlled loop. ask_later, generate, run_tools, and step let you run the agent loop one piece at a time.
• Durable agents. Agents can be backed by your Chat model and resumed across requests, jobs, and restarts.
• Provider-hosted tools. Web search, code execution, and remote MCP servers through one provider_tools API, with citations.
• Batches. Submit many chats at once at the provider's batch rates and collect results later.
• Fallbacks, prompt caching, and cost tracking. with_fallbacks, with_caching, and normalized tokens and cost on every response.
• MCP client. Connect agents to Linear, GitHub, Notion, or any MCP server with a class, the same way you describe a tool.
• Tool progress. Tools report what they're doing while they run, and your UI can show it.
• Tool search. Mark tools as deferred and the model searches for the ones it needs instead of receiving every definition.
• Evaluations. Run your agent against a dataset of reference answers and get a graded report.
• Cache lifetimes per boundary. cache_until_here(ttl: "1h") for the prefix that never changes.
• Faster by default. Streaming parses incrementally, replies stop retaining requests, and connections can be reused.

New agent capabilities

Tool approvals

One of the biggest additions to RubyLLM is the ability to approve tool calls. The obvious problem is that nobody wants a language model repricing the catalog on its own. RubyLLM 2.0 solves this with requires_approval:

class UpdatePrice < RubyLLM::Tool
  description "Updates the price of a product in our catalog"
  requires_approval

  param :sku, desc: "Product SKU"
  param :price, desc: "New price in USD"

  def execute(sku:, price:)
    product = Product.find_by!(sku: sku)
    product.update!(price: price)
    "Updated #{product.name} to $#{price}"
  end
end

Add it to the agent's tools and ask for a change:

class PriceMonitorAgent < RubyLLM::Agent
  # ...
  tools LookupProduct, SearchGoogleShopping, UpdatePrice
end

agent = PriceMonitorAgent.new
agent.ask "Lower the Chemex 6-Cup to match the market average."
agent.awaiting_approval? # => true

The agent still runs LookupProduct and SearchGoogleShopping on its own. When it gets to UpdatePrice, it stops. The tool call stays pending and ask returns. You can inspect what the model wants to do before anything touches the database:

agent.pending_approvals.each do |call|
  puts "#{call.name} #{call.arguments.inspect}"
end
# update_price {"sku"=>"CHX-6", "price"=>44.97}

Then approve or deny and let the agent finish:

agent.approve(agent.pending_approvals.first)
agent.complete

deny works the same way. The model gets told its request was rejected and can respond accordingly, usually by explaining what it would have done.

This is the single most useful change in 2.0 for anyone running agents against production data. Read-only tools backed by SerpApi can run freely. Tools that adjust live prices might need a human in the loop, and now that's possible.

Controlling the agent loop

In RubyLLM 1.x, ask ran the whole thing: send the message, run tools, send results back, repeat until the model answers. That's convenient until you want to be more specific and choose the number of rounds, log every step, or run each turn in a separate background job.

RubyLLM 2.0 breaks the loop into 4 parts:

• ask_later stages the question without sending it.
• generate asks the model for one response.
• run_tools executes any pending tool calls.
• step advances by one generation or one tool execution, whichever is next.

So the previous loop can be run one step at the time:

agent = PriceMonitorAgent.new
agent.ask_later "Which of our coffee products are priced more than 15% above market?"

agent.generate  # model decides to call LookupProduct and SearchGoogleShopping
agent.run_tools # the SerpApi searches happen here
agent.generate  # model reads the results and answers

Nothing new happens compared to ask, but you can now put a counter, a timeout, or a job boundary between any two lines. For a pricing agent that might run dozens of Google Shopping searches, being able to say "stop after ten tool rounds" is a feature, not a limitation.

Rails durable agents

Previously we could persist chats with acts_as_chat. Now we can go further and lets the agent itself be the record. Point the agent at your chat model with chat_model:

class PriceMonitorAgent < RubyLLM::Agent
  chat_model Chat
  model "gpt-5.4-mini"

  instructions <<~PROMPT
    You are a pricing analyst for the merchandising team.
    Always show specific numbers when comparing prices.
  PROMPT

  tools LookupProduct, SearchGoogleShopping, UpdatePrice
end

Then use create! instead of new:

chat = PriceMonitorAgent.create!
chat.ask "Lower the Chemex 6-Cup to match the market average."
chat.awaiting_approval? # => true

This is where approvals get practical. The request that triggered the agent has long returned. The merchandiser sees the proposed price change in the UI, maybe hours later, and clicks approve. Your controller or job just reloads the agent and continues:

chat = PriceMonitorAgent.find(params[:chat_id])
chat.approve(params[:tool_call_id])
chat.complete

The transcript, the pending tool call, and the decision all live in your database. If a job dies halfway, you can resume from the last saved step.

Your app still owns the chats and messages tables. RubyLLM 2.0 takes ownership of the supporting tables for the model registry, tool calls, usage, and batches, so it can evolve them without you maintaining the models.

Provider tools

The next useful addition is a shared API for tools hosted by the provider. Things like web search, code execution, and remote MCP servers. You enable them with with_provider_tools on a chat or provider_tools on an agent, and the results come back with citations:

class PriceMonitorAgent < RubyLLM::Agent
  model "claude-sonnet-5"
  provider_tools :web_search
  tools LookupProduct, SearchGoogleShopping, UpdatePrice
end

response = PriceMonitorAgent.new.ask "Any recent news about Chemex pricing?"
response.citations.each { |c| puts "#{c.title} - #{c.url}" }

Provider web search is great starting point for open-ended questions. The tool returns answers with links.

Provider web search vs. SerpApi tools

Previously, I showed you how to build RubyLLM agents with the examples of wrapping SerpApi SDK to give your agents access to 100+ search engines. While it's now easier than ever to just use the default search, a SerpApi-backed tool gives you structured data from a very specific engine.

For example, SerpApi's Google Shopping API always returns a title, a price, and a source every single time. You pick the engine, so swapping to Amazon, Walmart, or eBay is a one-line change. You control the parameters, so you can pin the location or currency. And you can see every query the agent ran in the search inspector.

This gives you accuracy and flexibility the default search cannot.

Batch processing

Checking one product price in a chat is one thing. But what about checking the whole catalog every night? RubyLLM now adds batch processing on top of ask_later which lets you use providers batch rates for more affordable queries.

An example of this could be submitting a chat question in bulk:

chats = Product.where(category: "coffee").map do |product|
  market = SerpApi.search(q: product.name, engine: "google_shopping")
                  .shopping_results.first(5)
                  .map { |r| "#{r[:source]}: #{r[:price]}" }
                  .join(", ")

  RubyLLM.chat(model: "gpt-5.4-mini")
    .with_instructions("You are a pricing analyst. Answer in one sentence.")
    .ask_later("Our #{product.name} is $#{product.price}. Market: #{market}. " \
               "Are we more than 15% above market?")
end

batch = RubyLLM.batch(chats)

Save batch.id somewhere and another background job can check on it later:

batch = RubyLLM::Batch.find(batch_id, provider: :openai)

if batch.complete?
  batch.messages.each { |message| puts message.content }
end

Results come back in submission order, so zipping them with the products list is trivial. The Batches guide covers failures, embedding batches, and how tool calls behave inside a batch if you'd rather let the model call the tool itself.

Fallbacks, caching, and cost

Finally let's go over three smaller (but neat) additions that matter once the agent runs on a schedule rather than in a console.

Fallbacks. When the primary model fails with a transient error, try another one:

chat = RubyLLM.chat(model: "gpt-5.4-mini")
             .with_tools(LookupProduct, SearchGoogleShopping)
             .with_fallbacks("claude-sonnet-5")

The conversation keeps its tools and settings, so pick a fallback that supports what you use. You need credentials for both providers.

Prompt caching. Our agent sends the same instructions on every request. Mark them as a cacheable prefix:

chat = RubyLLM.chat(model: "claude-sonnet-5").with_caching
chat.with_instructions(File.read("pricing-policy.txt")).cache_until_here

response = chat.ask "Is the Aeropress competitively priced?"
response.tokens.cache_read

Providers still decide the minimum prefix length and lifetime, and a cache hit isn't guaranteed. But for a long pricing policy that doesn't change, it adds up.

Cost tracking. Every response now reports normalized tokens and cost, including the failed attempts behind a fallback:

response.tokens.input
response.tokens.output
response.cost.total
chat.cost.total

Unknown pricing stays nil rather than showing as zero. In Rails, usage is recorded in its own ledger, so changing model prices later doesn't rewrite history. If you've ever been asked "how much does this agent cost us per day?", this is the answer.

Upgrading from 1.x

Install or upgrade the gem with the following Bundler call:

bundle add ruby_llm --version 2.0.0

Applications on 1.x go to 2.0 first, then 2.1. From 2.1 on, each release ships the upgrade from the release before it:

bundle update ruby_llm --version 2.1.0

The 2.0 step is mostly renames, with one name per concept across the library. For tools, param became parameter, desc: became description:, and params do ... end became parameters do ... end.

A 1.x tool:

class LookupProduct < RubyLLM::Tool
  description "Looks up a product in our catalog by name"
  param :name, desc: "Product name to search for"

  def execute(name:)
    # ...
  end
end

Becomes:

class LookupProduct < RubyLLM::Tool
  description "Looks up a product in our catalog by name"
  parameter :name, description: "Product name to search for"

  def execute(name:)
    # ...
  end
end

The agent class itself needs no changes. Other renames you'll hit first:

• max_tokens is now max_output_tokens.
• with_params and provider-specific options go through with_provider_options.
• response.input_tokens and friends live under response.tokens.
• Callbacks follow Rails naming: on_tool_call is before_tool_call, on_end_message is after_message.
• Tools can no longer halt the loop; use approvals or the caller-controlled loop instead.
• Tool results that return a Hash or Array are sent as JSON text instead of Ruby's inspect output.

In Rails, version 2.0 moves the model registry, tool calls, usage, and batches into RubyLLM-owned tables. bin/rails generate ruby_llm:upgrade creates the migrations in phases, with cleanup in a later deploy, and an optional --mode copy keeps a route back to 1.16 while you verify. Then run the generator again on 2.1 to add the MCP credentials table and the new columns.

The official upgrade guide has the full list.

Also new

Here are some other things I skipped because they don't change how agents work, but which you might want to start using too:

• RubyLLM.animate is now available for requesting video
• RubyLLM.speak for speech
• RubyLLM.transcribe for transcription with speaker labels and timestamps
• RubyLLM.ocr for PDFs and scans
• RubyLLM.rerank for search results
• RubyLLM.research for provider-hosted deep research
• RubyLLM.workflow for grouping steps in your instrumentation

Plus there are now four new providers too: Cohere, Deepgram, ElevenLabs, and Ollama Cloud.

Conclusion

RubyLLM 2.x makes agents safer to run against real data. Approvals keep humans in charge of writes, loop control lets you bound what an agent does, and durable agents mean a pending decision can wait in your database until someone makes it.

SerpApi fits in the same place it did before: as an agent tool that gives your agent structured, inspectable search data from Google Shopping, Amazon, Walmart, and other engines. Give it a try with 250 free searches a month.