Connect your backend with custom tools

Custom tools let your agent reach into your own backend during a live call. The caller says something — “What’s the status of my order?” or “Book me for Thursday at 2 PM” — the agent decides which tool to call, builds the arguments from the conversation, sends a POST to your HTTPS endpoint, and speaks the result back to the caller within seconds. No human intervention, no call transfers required. Typical uses include customer lookups, appointment booking, CRM updates, inventory checks, and knowledge base queries.

How a tool call works

Caller speaks → AI decides to call tool → POST to your endpoint → AI speaks result
                                                ↓
                                  You respond within the tool's timeout

Phase	Typical latency
AI decision	< 500 ms after caller finishes speaking
Your handler	< 3 s ideal — the tool’s Timeout is the hard limit (default 30 s, 5–300 s)
Agent speaks the result	~1 s after receiving your JSON

By default the caller waits in real time throughout this flow, so keep your handler fast. For slow tools, see While the tool runs.

Add a tool in the Agent Editor

In the dashboard, open your agent and navigate to Tools → Add Tool. Fill in the following fields:

Tool name — a snake_case identifier the AI uses internally (e.g. lookup_customer, book_appointment). Keep it short and descriptive.
Description — plain-English instructions that tell the AI when to call this tool. Be explicit. Start with “Call this tool when…” and list trigger phrases. The description is how the AI decides whether to invoke the tool.
HTTP method — use POST for writes and lookups that need call context; use GET for simple lookups.
Endpoint URL — your endpoint’s full URL, https:// (recommended) or http://.
Headers — add auth headers here (e.g. Authorization: Bearer your-secret). They are sent verbatim on every request.
Argument schema — define the arguments the AI should extract from the conversation (see below).
While this tool runs / When the result arrives — whether the agent waits for your endpoint or keeps talking, and when it gives the result (see While the tool runs).
Timeout (seconds) — how long Verbeo waits for your endpoint (5–300 s, default 30 s).

The description is the single most important field. A vague description like “Look up customer info” leads to the AI calling the tool at the wrong time or not at all. A precise description like “Call this tool at the start of every call to look up the customer by their phone number” gives the AI a clear trigger.

While the tool runs

Two settings on each custom tool control how the conversation continues while your endpoint works. They apply to agents on Gemini 3.8 Live. While this tool runs

Option	What the caller experiences
Wait for the result (default)	The agent pauses until your endpoint answers, then gives the caller the result. Best for quick lookups the caller is waiting for.
Keep talking	The agent stays free while your endpoint works: if the caller says something, the agent answers. It does not fill the silence on its own. Best for slow tools.

When the result arrives (only with Keep talking)

Option	What the caller experiences
Say it when the agent has finished talking (default)	If the agent is in the middle of answering the caller, it finishes, then gives your result.
Don’t say it	The agent confirms the request right away, without waiting for your endpoint, and never announces the result. When your endpoint answers, the result is added to the conversation silently, so the agent can use it if the caller asks (for example for a reference number). For tools that only save or send something, such as a CRM note.

Gemini 3.1 ignores these settings. They are kept on the tool and apply once the agent runs on Gemini 3.8 Live.
Built-in tools (end call, transfer, SMS, voicemail, calendar, knowledge base) are managed by Verbeo and have no such settings.

With Keep talking, when the caller speaks again while your endpoint is still working and the same tool is then called with different details, only the newest result is given. That covers a correction (“no, order 222”) but also an addition (“and order 222 too”): the earlier result stays in the conversation, and the agent gives it if the caller asks. Both requests reach your endpoint and both are completed, so keep it idempotent (see Security recommendations).

With Don’t say it, the agent has already confirmed the request, so a failure (a non-2xx response or a timeout) is never announced: it is added to the conversation silently, like a success. Use it only where a silent failure is acceptable, and check your endpoint’s own logs.

A request with Keep talking that is still running when the call ends is allowed to finish (up to its timeout, at most 30 seconds after the call). The call’s call_analyzed webhook may be sent before it finishes; such a tool then appears in tools[] without a result.

Argument schema

The argument schema tells the AI what data to extract from the conversation and pass to your endpoint. Each argument has:

Field	Options	Description
Name	—	JSON key in snake_case (e.g. `appointment_date`)
Type	`string`, `number`, `boolean`, `enum`	Determines JSON type sent to your handler
Description	—	Instructions the AI follows when extracting this value. Be specific.
Required	true / false	If true, the AI asks the caller for the value before invoking
Choices	—	For `enum` type only — the allowed values

Writing good argument descriptions:

appointment_date — The date the caller requested, in ISO-8601 format (YYYY-MM-DD).
Convert relative phrases like "tomorrow" or "next Monday" to an absolute date
using today's date. If the caller is ambiguous, ask them to clarify before calling
the tool.

Avoid one-word descriptions like “date” — the AI will guess, and often guess wrong.

Implement your endpoint

Your handler receives a POST with the tool_invoked event envelope. Parse payload.args for the extracted arguments and payload.call for call metadata including the live transcript. Here is a minimal Node.js/Express example that handles an appointment booking tool:

const express = require('express');
const app = express();
app.use(express.json());

app.post('/tools/book-appointment', async (req, res) => {
  // Verify the request is from Verbeo
  const authHeader = req.headers['authorization'];
  if (authHeader !== 'Bearer your-shared-secret') {
    return res.status(401).json({ error: 'Unauthorized' });
  }

  const { event, payload } = req.body;
  if (event !== 'tool_invoked') {
    return res.status(400).json({ error: 'Unexpected event type' });
  }

  const { appointment_date, appointment_time, customer_name } = payload.args;

  try {
    // Call your calendar system
    const booking = await createBooking({
      date: appointment_date,
      time: appointment_time,
      name: customer_name,
    });

    // Return a short result the AI can speak back
    res.json({
      msg: `Booked for ${customer_name} on ${appointment_date} at ${appointment_time}. Confirmation ID is ${booking.id}.`,
      data: { booking_id: booking.id },
    });
  } catch (err) {
    res.json({
      error: `Booking system unavailable. Apologize and offer to call back with a confirmation.`,
    });
  }
});

app.listen(3000);

Response format

Return JSON within the tool’s timeout. Your response shapes what the agent says next.

Field	Type	Purpose
`msg`	string	One-sentence summary. The agent uses this as the basis for what it says. Keep it concise and natural-language-friendly.
`data`	object	Optional structured data the agent may reference.
`say`	string	Optional verbatim text the agent speaks. Use sparingly.
`error`	string	Set on failure. The agent apologizes and mentions the error gracefully.

Examples:

// Successful booking
{
  "msg": "Appointment confirmed for Thursday April 24th at 2 PM. Booking ID is BK-7821.",
  "data": { "booking_id": "BK-7821" }
}

// Nothing found
{
  "msg": "No account found for that phone number. Ask the caller to spell their email address."
}

// Hard error
{
  "error": "Calendar service is down. Apologize and offer to send a confirmation by text."
}

Always return valid JSON, even on failure. An empty body or non-JSON response is treated as an error — the agent apologizes and the call continues without the tool result.

The `tool_invoked` webhook payload

This is the full payload your endpoint receives on every tool invocation:

{
  "event": "tool_invoked",
  "call_id": "8eb0e5f2-e8b5-44cb-beba-9b802f39c340",
  "agent_id": "a2e0146d-6b37-4b09-b33b-f2c72f22684d",
  "timestamp": "2026-04-18T12:19:20.789Z",
  "payload": {
    "tool_name": "book_appointment",
    "args": {
      "customer_name": "Sarah Chen",
      "appointment_date": "2026-04-19",
      "appointment_time": "11:00",
      "appointment_type": "service"
    },
    "call": {
      "call_type": "outbound",
      "agent_name": "Support Concierge",
      "workspace_id": "34c4a847-7516-4277-8fc1-1b33ff0126d4",
      "from_number": "+15550001234",
      "to_number": "+12125550001",
      "customer_number": "+12125550001",
      "dynamic_variables": { "customer_name": "Sarah", "plan": "premium" },
      "transcript": "[12:18:55] User: I'd like to book a visit.\n[12:19:02] Agent: What day works for you?\n[12:19:08] User: Tomorrow at 11 AM.",
      "transcript_object": [
        { "role": "user", "content": "I'd like to book a visit.", "timestamp": "2026-04-18T12:18:55.000Z" },
        { "role": "agent", "content": "What day works for you?", "timestamp": "2026-04-18T12:19:02.000Z" },
        { "role": "user", "content": "Tomorrow at 11 AM.", "timestamp": "2026-04-18T12:19:08.000Z" }
      ]
    }
  }
}

payload.args contains exactly the arguments you defined in the schema. payload.call gives you the current call state including the live transcript and any dynamic variables you passed at call creation.

Timeout and error handling

Failure	Agent behavior
Non-2xx response	Speaks a generic apology. Tool is not automatically retried. (With Don’t say it nothing is spoken: see the warning above.)
Response takes longer than the tool’s timeout	Same — the request times out and the agent apologizes.
Malformed JSON	Treated as a non-2xx error.
Empty body	Treated as an error. Always return at least `{"msg": "Done."}`.

If your real operation takes more than a few seconds:

On Gemini 3.8 Live, set the tool to Keep talking (see While the tool runs): the caller can keep talking to the agent while your endpoint works, and the agent gives the result when it arrives. The agent does not fill the silence on its own if the caller says nothing.
On Gemini 3.1 (where the setting is ignored), return an immediate holding message and handle the result asynchronously:

{ "msg": "Let me look that up — just one moment." }

Then use a follow-up action (SMS, callback, or the next conversational turn) to close the loop.

Security recommendations

Add an auth header — configure a shared secret in the tool’s Headers field (e.g. Authorization: Bearer your-secret). Verbeo sends it verbatim on every request. Reject any request missing it.
Use HTTPS — every request carries the tool’s headers, your keys included. An http:// endpoint is accepted, but it sends them unencrypted.
Be idempotent — network conditions can duplicate requests. Deduplicate by call_id + tool_name + args. If you have already processed an identical invocation, return the same response.
Handle PII carefully — payload.call.transcript may contain names, phone numbers, and other personal data. Only log or forward it to systems that meet your privacy requirements.

Tool invocations in post-call data

Every tool call during a call appears in the call_analyzed webhook event — in payload.data.tools[] with the full args, result, and timestamp, and inline in payload.data.transcript[] with role: "tool". This gives you a complete audit trail of what the agent did on every call.

Phone Numbers

Web Widget

Call data

Billing

Connect your backend with custom tools

How a tool call works

Add a tool in the Agent Editor

While the tool runs

Argument schema

Implement your endpoint

Response format

The `tool_invoked` webhook payload

Timeout and error handling

Security recommendations

Tool invocations in post-call data

​How a tool call works

​Add a tool in the Agent Editor

​While the tool runs

​Argument schema

​Implement your endpoint

​Response format

​The tool_invoked webhook payload

​Timeout and error handling

​Security recommendations

​Tool invocations in post-call data

How a tool call works

Add a tool in the Agent Editor

While the tool runs

Argument schema

Implement your endpoint

Response format

The `tool_invoked` webhook payload

Timeout and error handling

Security recommendations

Tool invocations in post-call data