Module provider

Expand description

Provider: the interface for communicating with an LLM.

This module defines the Provider trait and its companion data types: requests (ChatRequest), responses (ChatResponse / StreamEvent), errors (ProviderError), and usage (Usage). The trait itself is vendor-agnostic; the implementations (OpenAiProvider / FakeProvider / RetryProvider) all implement the same trait, and any implementation can be wrapped by RetryProvider to gain retry capability.

Structs§

ChatRequest: A single conversation request.
ChatResponse: The reply to one conversation.
FakeProvider: A programmable fake Provider — a script is a sequence of per-turn replies, consumed in order by chat / stream_chat.
ModelOptions: Model options for one conversation.
OpenAiProvider: Provider implementation for OpenAI-compatible APIs.
RetryPolicy: Retry policy (with defaults; Default is “exponential backoff + jitter, 3 attempts”).
RetryProvider: Retry wrapper: implements Provider and retries inner failures per RetryPolicy.
Usage: Token usage for one conversation.

Backoff: Backoff strategy: how long to wait after each failure before the next attempt.
FakeReply: One turn of reply from the script.
FinishReason: Why the model ended its reply.
ProviderError: Why a Provider call failed.
Retryable: Retry judgment: which errors are worth retrying.
StreamEvent: An event in a streamed conversation reply.
StructuredOutputMode: Transport shape for structured output (an OpenAiProvider construction setting): decides how response_format is sent.
TimeoutStage: The stage at which a timeout occurred: one-to-one with OpenAiProvider’s four timeouts (connect / non-streaming total / streaming event interval / streaming total), plus the error-response-body read timeout and a generic transport timeout.