Module requests 

Request and response types for inference

Structs

ApiChatMessage

ApiChatRequest

ApiChatResponse

ApiCompletionRequest

ApiCompletionResponse

ApiFunction

ApiFunctionCall

ApiJsonSchema

ApiResponseFormat

ApiStreamOptions

ApiTool

ApiToolCall

ApiToolChoiceFunction

BatchRequest

Batch request for processing multiple requests together

EngineDecodeStageInterval

A measured request-local engine decode stage in the request’s monotonic clock domain. These are explicit producer boundaries, not intervals inferred by filling gaps between other profile events.

EngineTokenTimingEvidence

InferenceEvidenceRequest

Explicit request for execution evidence that is expensive or sensitive to retain.

InferenceExecutionEvidence

Evidence captured at the engine execution boundary.

InferenceRequest

Inference request

InferenceResponse

Inference response

ScheduledRequest

Scheduled request with additional state information

StreamChunk

Streaming response chunk

Enums

ApiFunctionCallChoice

ApiMessageRole

ApiRequest

ApiResponse

ApiToolCallProtocol

ApiToolChoice

EngineDecodeStage

Engine-boundary timing for one inference request.

RequestState

Request state in the scheduler

StructuredOutputBranch

Branch established by the structured-output grammar for a complete result. When both the final and tool languages accept the same bytes, the grammar owner must select Final before invoking the classified response helper.

Constants

DEFAULT_MAX_TOKENS_METADATA_KEY

PROMPT_TOKENS_METADATA_KEY

Functions

api_response_from_classified_generated_text

Preserve an authoritative grammar decision through product response shaping.

api_response_from_generated_text

chat_api_may_emit_tool_or_function_call

chat_api_response_from_generated_text