LatentKit

Tool Calling

Define tools on /v1/chat, receive normalized tool calls, and return tool results.

LatentKit supports OpenAI-style tool (function) calling on POST /v1/chat. You define tools in the request; the routed model decides when to call them; LatentKit normalizes tool calls to one shape regardless of which provider executed — so the same handling code works whether the route lands on OpenAI, Anthropic, Gemini, or another tool-capable model.

Capability requirement

Any request that includes tools, a non-none tool_choice, or tool/function role messages requires the function_calling capability. Only route models with that capability are eligible, and routing fails with NO_HEALTHY_PROVIDER if none exist.

1. Define tools

curl https://ai.latentkit.com/v1/chat \
  -H "Authorization: Bearer $LATENTKIT_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "messages": [{ "role": "user", "content": "What is the weather in Dubai?" }],
    "tools": [
      {
        "type": "function",
        "function": {
          "name": "get_weather",
          "description": "Get current weather for a city",
          "parameters": {
            "type": "object",
            "properties": {
              "city": { "type": "string" }
            },
            "required": ["city"]
          }
        }
      }
    ],
    "tool_choice": "auto"
  }'
FieldDescription
toolsArray of OpenAI-style function definitions with JSON Schema parameters
tool_choiceauto (model decides), none, required, or a specific function selector
parallel_tool_callsAllow or disallow multiple tool calls in one turn, when the provider supports it

2. Read normalized tool calls

When the model calls a tool, the response contains tool_calls in OpenAI function-call shape:

{
  "content": "",
  "tool_calls": [
    {
      "id": "call_abc123",
      "type": "function",
      "function": {
        "name": "get_weather",
        "arguments": "{\"city\": \"Dubai\"}"
      }
    }
  ]
}

function.arguments is a JSON string. Parse it, validate it against your schema, allowlist the function name, and apply your normal user authorization before executing anything. Treat all model-produced arguments as untrusted.

3. Return tool results

Execute the tool in your backend, then continue the conversation with a tool role message referencing the call ID:

{
  "messages": [
    { "role": "user", "content": "What is the weather in Dubai?" },
    {
      "role": "assistant",
      "content": null,
      "tool_calls": [
        { "id": "call_abc123", "type": "function",
          "function": { "name": "get_weather", "arguments": "{\"city\": \"Dubai\"}" } }
      ]
    },
    {
      "role": "tool",
      "tool_call_id": "call_abc123",
      "content": "{\"temp_c\": 41, \"condition\": \"sunny\"}"
    }
  ],
  "tools": [ ... ]
}

Repeat until the model returns a plain text answer instead of more tool calls.

SDK example

const response = await client.chat.create({
  messages,
  tools,
  tool_choice: 'auto',
});

for (const call of response.tool_calls ?? []) {
  const args = JSON.parse(call.function.arguments);
  // run your tool, append the result as a `tool` role message, call again
}

The Python SDK accepts the same tools and tool_choice keyword arguments on client.chat.create(...).

When the model refuses to call a tool

Some Anthropic models reject forced tool choice. LatentKit adapts a single forced tool to structured JSON output when its schema is small. Schemas with more than eight optional fields use native tools with an explicit call instruction to avoid slow schema compilation. Your schema and optional fields are preserved. If the model returns no required tool call, LatentKit tries the next eligible route within the request deadline.

If your application requires a tool call (for example tool_choice: "required") but the model returns plain text, LatentKit returns the typed error TOOL_CALL_NOT_PRODUCED (category: model_output, retryable). Recommended handling:

  1. Retry once — possibly with a stricter tool_choice or larger max_tokens.
  2. If retries keep failing, revise the prompt or tool descriptions; repeated failures usually mean the model cannot map the request to your schema.

See Error reference for the full model_output category guidance.

Routing note

Tool-heavy workloads may require credits or BYOK even when plain chat is within a Free managed allowance, and route changes can shift which provider executes. If tool behavior must stay stable, pin the preferred model in the routing policy.

On this page