Skip to content
InferencePassInferencePass

Tools and structured JSON

Function tools are declared in the request and are validated before funds are reserved.

{
  "tools": [{
    "type": "function",
    "function": {
      "name": "lookup_status",
      "description": "Look up a service status.",
      "parameters": { "type": "object", "properties": {}, "additionalProperties": false }
    }
  }],
  "tool_choice": "auto"
}

Tool names must be stable identifier-style strings. Up to 64 tools are accepted; each schema is bounded to 128 KiB. tool_choice: "required" or a named function requires at least one tool. parallel_tool_calls is accepted when the selected capability supports the request.

Structured output

Use response_format: {"type":"json_object"} for JSON mode or response_format: {"type":"json_schema","json_schema":{...}} for a named schema. The gateway enforces bounded depth, node count, branch count, local $ref only, and a 128 KiB serialized schema limit. Unsupported policy keywords fail closed instead of being silently ignored.

The model route must affirmatively support tools or structured_output. The gateway repeats this check even if a browser control was disabled correctly, so edited requests cannot bypass capability gating.