Text generation in the OpenAI Chat Completions, OpenAI Responses and Anthropic Messages formats.
Create a chat completion
OpenAI Chat Completions compatible. Works with every model whose mode is chat. Supports
streaming, tools, JSON output, images (vision models) and reasoning_effort (reasoning models).
Unknown fields are passed to the upstream provider as-is, except the gateway-controlled fields
listed below.
Billing. Before the call, the worst-case cost (prompt estimate + output limit) is reserved
against your balance and the key's credit limit. If your credit can't cover the requested
max_tokens / max_completion_tokens (or the default 4,096 when unset), the output limit is
lowered to what you can afford; below 16 affordable tokens the request is refused with 402.
After the call you are billed the actual usage and the rest of the reservation is returned.
Streaming. With "stream": true the response is text/event-stream with OpenAI
chat.completion.chunk events ending in data: [DONE]. The gateway always sets
stream_options.include_usage, so the last chunk carries usage.
Removed or refused by the gateway. Upstream-control fields are silently dropped: api_key,
api_base, base_url, api_version, api_type, organization, user_config, model_list,
mock_*, fallbacks, context_window_fallbacks, num_retries, max_retries, timeout,
caching, cache, proxy_server_request, secret_fields, guardrails, extra_headers,
headers, custom_llm_provider, deployment_id, base_model, model_info, *_cost_per_*,
and anything prefixed litellm, aws_, vertex_, azure_ or watsonx_; metadata keys
starting with litellm / model_ and metadata.tags are dropped too. Requests are refused
(400) when they use provider-hosted tools (web_search*, web_fetch*, code_interpreter*,
code_execution*, mcp*, image_generation*, computer_use*, file_search*, local_shell*)
or web_search_options, reference objects stored upstream (file_id, vector_store_ids,
item_reference, a string container, top-level conversation / prompt), contain more than
64 URLs, or contain URLs that resolve to private / internal addresses. Inline data: URLs are fine.
If the key has redact secrets enabled, credentials found in the prompt are masked before
forwarding.
Create a chat completion › Request Body
modelModel slug from GET /v1/models (a model with mode chat).
streamStream server-sent events. stream_options.include_usage is forced on.
max_tokensOutput limit; lowered automatically when your credit can't cover it.
max_completion_tokensOutput limit (preferred over max_tokens when both are set); lowered automatically when your credit can't cover it.
temperaturetop_pnNumber of choices; each is reserved and billed.
seedpresence_penaltyfrequency_penaltyFunction tools. Provider-hosted tools (web search, code interpreter, MCP, …) are refused.
parallel_tool_calls{"type":"json_object"} or a json_schema.
reasoning_effortlow, medium or high on models with supports_reasoning.
logprobstop_logprobsCreate a chat completion › Responses
The completion. JSON when stream is false, an SSE stream otherwise.
idobjectcreatedmodelCreate a response (OpenAI Responses API)
OpenAI Responses API compatible. Works with every chat model: models whose provider has no
Responses route are served transparently through Chat Completions (including streaming and tools),
and the result is converted back to the Responses format.
previous_response_id is accepted only for response ids previously returned to your own
account by this endpoint; any other id returns 404. Billing, output-limit capping, stripped
fields and refused tools / references are the same as for
Chat Completions (POST /v1/chat/completions); here the output limit field is
max_output_tokens.
Streaming ("stream": true) returns the Responses event stream (response.created, …,
response.completed).
Create a response (OpenAI Responses API) › Request Body
modelText, or a list of input items (input_text, input_image, …). Must not be empty.
instructionsstreammax_output_tokensOutput limit; lowered automatically when your credit can't cover it.
{"effort": "low"} on reasoning models.
tool_choicetemperaturetop_pprevious_response_idOnly ids of responses previously returned to your own account.
storeCreate a response (OpenAI Responses API) › Responses
A Responses API response object, or an SSE stream when stream is true.
Create a message (Anthropic Messages API)
Anthropic Messages API compatible, for Anthropic SDKs and Claude Code. Works with every chat model, not only Claude: the request is translated to Chat Completions and the result (including the stream) is translated back.
Supported: 'system (string or text blocks; cache_control is kept for claude* models),'
text / image / document / tool_use / tool_result blocks, client tools with input_schema,
tool_choice (auto, any, none, tool, disable_parallel_tool_use), max_tokens,
temperature, top_p, stop_sequences, stream, and thinking (enabled with
budget_tokens, or adaptive) on reasoning models — mapped to reasoning_effort
(budget_tokens → low / medium / high). Other fields are ignored; Anthropic server tools (without
input_schema) are dropped.
Dated model names such as claude-haiku-4-5-20251001 fall back to the undated catalog slug when
only that one exists.
Errors use the Anthropic shape. Credit errors are 402 with type billing_error.
Claude Code:
Code
Headers
anthropic-versionSent by Anthropic SDKs (e.g. 2023-06-01). On GET /v1/models its presence switches the response to the Anthropic list shape.
Create a message (Anthropic Messages API) › Request Body
modelAny chat model slug; dated Claude names map to the catalog slug.
max_tokensRequired by Anthropic SDKs; lowered automatically when your credit can't cover it.
System prompt; text blocks may carry cache_control (kept for claude* models).
streamtemperaturetop_pstop_sequencesClient tools with name, description, input_schema.
{"type": "auto" | "any" | "none" | "tool", "name"?, "disable_parallel_tool_use"?}
{"type": "enabled", "budget_tokens": N} or {"type": "adaptive"}; used on reasoning models only.
Create a message (Anthropic Messages API) › Responses
An Anthropic message, or an Anthropic SSE stream (message_start … message_stop) when stream is true.
idtyperolemodelthinking, text and tool_use blocks.
stop_reasonstop_sequenceCount input tokens (estimate)
Anthropic-compatible token counting. Returns a rough estimate (about 4 characters per token over
system, messages and tools); it is not billed, but the call still requires a valid key
and counts toward rate limits.
Headers
anthropic-versionSent by Anthropic SDKs (e.g. 2023-06-01). On GET /v1/models its presence switches the response to the Anthropic list shape.
Count input tokens (estimate) › Request Body
modelAny chat model slug; dated Claude names map to the catalog slug.
max_tokensRequired by Anthropic SDKs; lowered automatically when your credit can't cover it.
System prompt; text blocks may carry cache_control (kept for claude* models).
streamtemperaturetop_pstop_sequencesClient tools with name, description, input_schema.
{"type": "auto" | "any" | "none" | "tool", "name"?, "disable_parallel_tool_use"?}
{"type": "enabled", "budget_tokens": N} or {"type": "adaptive"}; used on reasoning models only.
Count input tokens (estimate) › Responses
Estimated input tokens.
input_tokens