Crate llmsdk_provider

Expand description

Provider trait abstractions for llmsdk.

This crate is the Rust port of @ai-sdk/provider at specification version v4. It defines the contract every model provider (OpenAI, Anthropic, …) implements, plus the shared message / content / streaming types and the unified error type.

The crate is intentionally minimal: no HTTP, no retry, no middleware. Those live in llmsdk-provider-utils and downstream provider crates.

§Layout

language_model: chat / completion models with streaming.
embedding_model: vector embedding models.
image_model: image generation models.
video_model: video generation models.
reranking_model: document reranking models.
files_model: file-upload models (e.g. Anthropic’s POST /files).
skills_model: skill-upload models (Anthropic skills bundles).
middleware: decorators for stacking cross-cutting concerns (retry / logging / caching) on top of any LanguageModel.
provider: top-level factory returning model instances by id.
error: unified ProviderError.
shared: provider options / metadata / warnings reused across models.

§Example

use llmsdk_provider::SPECIFICATION_VERSION;
assert_eq!(SPECIFICATION_VERSION, "v4");

§Stability

Until 1.0, expect breaking changes; the spec version pins compatibility with the matching @ai-sdk/provider major.

Modules§

embedding_model: Embedding model trait and supporting types.
error: Unified error type for all provider operations.
files_model: File upload model trait and supporting types.
image_model: Image generation model trait and supporting types.
json: JSON value re-exports used across the provider surface.
language_model: Language model trait and supporting types.
middleware: Middleware layer for the language model surface.
provider: Top-level Provider trait — the factory for model instances.
reranking_model: Reranking model trait and supporting types.
shared: Shared types reused by language / embedding / image models.
skills_model: Skill upload model trait and supporting types.
speech_model: Text-to-speech model trait and supporting types.
transcription_model: Speech-to-text model trait and supporting types.
video_model: Video generation model trait and supporting types.

Structs§

ApiCallErrorBuilder: Builder for API call errors with optional fields.
CacheMiddleware: Middleware that memoizes generate / stream responses keyed by CallOptions.
CallOptions: Options for one model invocation.
DynEmbeddingModel: Type-erased embedding model handle. See DynLanguageModel for rationale.
DynImageModel: Type-erased image model handle. See DynLanguageModel for rationale.
DynLanguageModel: Type-erased language model handle.
EmbedOptions: Options for one EmbeddingModel::do_embed call.
EmbedResult: Result of EmbeddingModel::do_embed.
EmbeddingUsage: Token usage for an embedding call.
FilePart: File attachment part.
FinishReason: Why the model stopped generating.
FunctionTool: Client-defined function tool with a JSON schema input.
GenerateResponse: Response-side info attached to GenerateResult.
GenerateResult: Result of super::LanguageModel::do_generate.
GeneratedImage: One image returned by the provider.
ImageOptions: Options for one ImageModel::do_generate call.
ImageResult: Result of ImageModel::do_generate.
ImageUsage: Token usage reported by an image-generation model.
ImageUsageInputDetails: Input-token breakdown by modality.
InputTokenUsage: Input-side token counters.
LoggingMiddleware: Middleware that emits Logger events around every call.
MemoryCacheStore: In-memory CacheStore with optional TTL and LRU eviction.
MemoryCacheStoreBuilder: Builder for MemoryCacheStore with TTL / LRU options.
MiddlewareContext: Structured fields carried across the middleware chain.
OutputTokenUsage: Output-side token counters.
ProviderError: Unified error returned by every provider operation.
ProviderMiddlewareSet: Two middleware chains: one for language models, one for image models.
ProviderTool: Provider-defined tool referenced by id.
RankingEntry: One entry in RerankingResult::ranking.
ReasoningPart: Reasoning trace produced by the model.
RequestInfo: Request metadata for telemetry / debugging.
RerankingOptions: Options for one RerankingModel::do_rerank call.
RerankingResult: Result of RerankingModel::do_rerank.
ResponseInfo: Response metadata for telemetry / debugging.
ResponseMetadata: Response metadata reported by a provider mid-stream or post-call.
RetryMiddleware: Middleware that retries failed calls with exponential backoff.
RetryMiddlewareBuilder: Builder for RetryMiddleware; create via RetryMiddleware::builder.
SkillFile: One file within a skill bundle.
SpeechOptions: Options for one SpeechModel::do_generate call.
SpeechResponseInfo: Response metadata attached to SpeechResult.
SpeechResult: Result of SpeechModel::do_generate.
StderrLogger: Minimal Logger that writes one line per event to stderr.
StreamResponse: Headers / metadata available when the stream opens.
StreamResult: Result of super::LanguageModel::do_stream.
TextPart: Plain text part.
ToolApprovalRequest: Approval request for a provider-executed tool call.
ToolApprovalResponsePart: User decision on a provider-executed tool call.
ToolCallPart: Tool call requested by the assistant.
ToolInputExample: One input example for FunctionTool::input_examples.
ToolResult: Tool result emitted by a provider-executed tool.
ToolResultPart: Result of a tool invocation.
TranscriptionOptions: Options for one TranscriptionModel::do_generate call.
TranscriptionResponseInfo: Response metadata for transcription results.
TranscriptionResult: Result of TranscriptionModel::do_generate.
TranscriptionSegment: One transcription segment.
UploadFileOptions: Options for one FilesModel::upload_file call.
UploadFileResult: Result of FilesModel::upload_file.
UploadSkillOptions: Options for one SkillsModel::upload_skill call.
UploadSkillResult: Result of SkillsModel::upload_skill.
UrlPattern: One supported-URL regex pattern entry.
Usage: Aggregated token usage for a call.
VideoOptions: Options for one VideoModel::do_generate call.
VideoResponseInfo: Response metadata for VideoModel::do_generate.
VideoResult: Result of VideoModel::do_generate.

Enums§

AssistantPart: Content part allowed in a Message::Assistant.
CachedEntry: A cached call result.
CallKind: Discriminates the active call kind passed to LanguageModelMiddleware::transform_params.
Content: One generated content unit.
FileBytes: Either raw bytes or a base64-encoded string.
FileData: File data carried in prompts or tool results.
FinishReasonKind: Normalized reason variants.
JsonValue: Represents any valid JSON value.
Message: One turn in a conversation.
ReasoningEffort: Reasoning effort level.
RerankingDocuments: Documents to rerank. Two-state tagged union over plain text or JSON objects.
ResponseFormat: Response format directive.
Source: Citation / grounding source returned by the model.
StreamPart: One unit emitted on the stream returned by super::LanguageModel::do_stream.
Tool: Either a client-defined function tool or a provider-defined tool.
ToolChoice: How the model should choose among the configured tools.
ToolMessagePart: Content part allowed in a Message::Tool.
ToolOutputPart: One part inside ToolResultOutput::Content.
ToolResultOutput: Payload variants for a tool result.
UploadFileData: Payload variants accepted by FilesModel::upload_file.
UserPart: Content part allowed in a Message::User.
VideoData: One video returned by the provider.
VideoFile: Input image or video for image-to-video / video editing endpoints.
Warning: Provider-emitted warning about a model call.

Constants§

SPECIFICATION_VERSION: Specification version this crate implements.

Traits§

CacheStore: Backing store for CacheMiddleware.
EmbeddingModel: Contract every text-embedding model implements.
EmbeddingModelMiddleware: Contract for middleware that decorates an EmbeddingModel.
FilesModel: Contract every file-upload model implements.
ImageModel: Contract every image-generation model implements.
ImageModelMiddleware: Contract for middleware that decorates an ImageModel.
LanguageModel: Contract every chat / completion model implements.
LanguageModelMiddleware: Contract for middleware that decorates a LanguageModel.
Logger: Sink for LoggingMiddleware events.
Provider: Factory returning model instances by id.
RerankingModel: Contract every reranking model implements.
RerankingModelMiddleware: Contract for middleware that decorates a RerankingModel.
SkillsModel: Contract every skill-upload model implements.
SpeechModel: Contract every text-to-speech model implements.
TranscriptionModel: Contract every speech-to-text model implements.
VideoModel: Contract every video-generation model implements.
VideoModelMiddleware: Contract for middleware that decorates a VideoModel.

Functions§

wrap_embedding_model: Compose an embedding model with one or more middlewares.
wrap_image_model: Compose an image model with one or more middlewares.
wrap_language_model: Compose a model with one or more middlewares.
wrap_provider: Wrap a provider so every returned model is decorated with the matching middleware chain.
wrap_reranking_model: Compose a reranking model with one or more middlewares.
wrap_video_model: Compose a video model with one or more middlewares.

Type Aliases§

BoxStream: Boxed Send stream alias used for streaming results.
Embedding: One embedding vector.
Headers: HTTP headers attached to a request or response.
JsonObject: JSON object map ({ string: JsonValue }), matching serde_json::Map<String, Value>.
JsonSchema: JSON schema describing a tool’s input or a JSON response format.
Prompt: An ordered list of Messages sent to the model.
ProviderMetadata: Provider-specific metadata returned by a provider, keyed by provider id.
ProviderOptions: Provider-specific options, keyed by provider id.
ProviderReference: Mapping of provider names to provider-specific file / skill identifiers.
Result: Convenience alias used across the crate.
SupportedUrls: Native-URL support map returned by super::LanguageModel::supported_urls.