Skip to main content

Crate ml_runtime_provider

Crate ml_runtime_provider 

Source

Structs§

ChatMessage
ChatMetadata
Execution evidence recorded by the runtime/provider (not by the model).
ChatRequest
ChatResponse
ChatStreamSummary
Terminal stream summary. It deliberately omits the full text, so a stream never forces the runtime to buffer unbounded output.
GenerationCapabilities
Optional features of a generation provider. The OpenAI-compatible adapter cannot discover these, so they are configured and enforced up front.
GenerationModelInfo
GenerationOptions
ProviderCapabilities
ProviderCapability
ToolCall
A tool-call proposal produced by a model. It is not evidence that the operation ran, was authorized, or would succeed; the calling application decides whether and how to execute it.
ToolDefinition
A tool the model may propose to call. The runtime never executes it: the schema constrains the proposed arguments and grants no authority.
Usage
Token accounting as reported by the provider; never estimated.

Enums§

ChatRole
ChatStreamEvent
ExecutionLocation
FinishReason
Why generation stopped, when the provider says. Other preserves an unrecognized provider value verbatim.
ResponseFormat
Requested output shape. JsonSchema is a request to the provider; it is never a guarantee. The runtime validates the result separately.
StructuredOutput
Result of checking generated text against the requested format.
ToolChoice
Provider-neutral tool selection policy. Absent means the provider default.

Constants§

DEFAULT_MAX_OUTPUT_BYTES
Default upper bound on accumulated generated output (4 MiB).
MAX_TOOL_NAME_BYTES
Longest tool name accepted (matches common provider limits).

Traits§

GenerationProvider
A provider of generative inference. Separate from crate::Provider, which serves typed inference.
Provider

Functions§

is_valid_tool_name
Tool names are restricted to [A-Za-z0-9_-]{1,64} so they are safe on every provider wire and in logs.