Streaming and cancellation

Set stream: true to receive server-sent events from POST /v1/chat/completions or POST /v1/responses.

curl -N "$INFERENCEPASS_API_BASE/v1/chat/completions" \
  -H "Authorization: Bearer $INFERENCEPASS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"gpt-5.6-sol","messages":[{"role":"user","content":"Say hello."}],"stream":true}'

The native transcript starts with response.created, carries zero or more text/tool/refusal deltas, emits one finalized response.usage, then ends with response.completed, response.failed, or response.cancelled. The wire terminator is data: [DONE]. Sequence numbers are contiguous and every event carries the same request ID.

Cancellation

Abort the HTTP request or close the response reader. The gateway aborts upstream generation when possible, reconciles known usage, and terminates with response.cancelled. A cancellation is not a promise of zero cost: usage can be partial or require review. Do not automatically retry after customer-visible output.

Client behavior

Consume with an incremental SSE parser; chunks can split anywhere, including inside UTF-8 characters. Ignore comment heartbeats. Apply backpressure by pausing reads. Set a bounded client timeout and surface the request ID on any terminal error.

The TypeScript SDK exposes for await...of iteration and an AbortSignal option.