GatewayResponses

Create Response

Create an LLM response.

Supports both streaming and non-streaming modes via the stream parameter.

Request

This endpoint expects an object.
inputlist of objectsRequired
Conversation history
modelstring or nullOptional

Model ID in format ‘provider/model-name’. Optional if a routing policy is configured.

toolslist of objects or nullOptional
Available tools
max_tokensinteger or nullOptional
Maximum tokens to generate
temperaturedouble or nullOptional0-2
Sampling temperature
top_pdouble or nullOptional0-1
Nucleus sampling parameter
stoplist of strings or nullOptional
Stop sequences
response_formatobject or nullOptional
Response format
streambooleanOptionalDefaults to false
Whether to stream the response
tagslist of objects or nullOptional

Tags to attach to this request for categorization (key-value pairs)

project_idstring or nullOptional

Optional project ID (UUID) to associate with this request

routing_policy_idstring or nullOptional

Override the default routing policy (for testing)

include_routing_metadatabooleanOptionalDefaults to false
Include detailed routing metadata in response

Response

Successful Response

Errors