Crate modelplease 

Core language model types, traits, and provider implementations.

Provides the foundational types used across the predict engine and any other crate that communicates with language models. Includes concrete implementations for Anthropic and OpenAI APIs, plus a spec-compliant SSE parser for streaming responses.

Providers

• AnthropicLanguageModel (anthropic) — Anthropic Messages API
• OpenAiLanguageModel (openai) — OpenAI Chat Completions API, including compatible endpoints through OpenAiConfig::base_url
• OllamaLanguageModel (ollama) — local Ollama daemon
• BedrockProvider (bedrock) — AWS Bedrock via Converse / ConverseStream
• DummyLM — test mock

OpenAI-Compatible Providers

With the openai feature, OpenAiLanguageModel works with any provider that implements the OpenAI Chat Completions API:

use std::sync::Arc;
use modelplease::{ApiKey, OpenAiConfig, OpenAiDeps, OpenAiLanguageModel, RetryConfig};

let lm = OpenAiLanguageModel::new(
    OpenAiDeps { client: Arc::new(reqwest::Client::new()) },
    OpenAiConfig {
        api_key: ApiKey::parse("local")?,
        base_url: "http://localhost:8080/v1".to_owned(),
        retry_config: RetryConfig::default(),
    },
);

Quick Start

use modelplease::{ContentPart, HttpsUrl, MediaSource, Message, Role};

// Simple text message
let _msg = Message::user("What is the capital of France?");

// Message with mixed content parts
let url = HttpsUrl::parse("https://example.com/photo.jpg").unwrap();
let _msg = Message::with_parts(
    Role::User,
    vec![
        ContentPart::text("Describe this image:"),
        ContentPart::image(MediaSource::Url { url }),
    ],
);

Modules

• Messages — Message, Role, ContentPart
• Providers — LanguageModelProvider, GenerateRequest
• Configuration — LanguageModelConfig, ResponseFormat
• Responses — LanguageModelResponse, StreamDelta, Usage
• Errors — LanguageModelError
• Retry — RetryConfig, with_retry
• SSE — SseEvent, parse_sse_stream
• Provider implementations — AnthropicLanguageModel, OpenAiLanguageModel, and DummyLM

Structs

AnthropicConfig

Configuration for AnthropicLanguageModel.

AnthropicDeps

Injected dependencies for AnthropicLanguageModel. The HTTP client is constructed once at the application composition root and shared across HTTP-based providers.

AnthropicLanguageModel

Anthropic Messages API language model.

ApiKey

Provider API key. Validated at parse time; redacted on Display.

AwsAccountId

A 12-digit AWS account id. Used as the optional bucket_owner on MediaSource::S3 for cross-account S3Location references.

BedrockMantleProvider

AWS Bedrock Mantle language-model provider.

BedrockMantleProviderConfig

Per-family region routing + retry knobs for BedrockMantleProvider.

BedrockMantleProviderDeps

Injected dependencies for BedrockMantleProvider.

BedrockProvider

AWS Bedrock language model provider.

BedrockProviderConfig

Configuration for BedrockProvider. Pure value-shaped data — AWS SDK client construction (credential chain probing, profile resolution) happens in the application composition root, not here.

BedrockProviderDeps

Injected AWS SDK clients for BedrockProvider.

ChatModelInfo

Catalog metadata for one model offered by a provider.

DummyLM

A test mock that implements LanguageModelProvider.

GenerateRequest

Per-call inputs for LanguageModelProvider::generate and LanguageModelProvider::generate_stream.

HttpsUrl

A URL pinned to the https:// scheme.

InvalidReasoningEffort

Error returned when parsing a ReasoningEffort from a string that doesn’t match one of the six accepted names.

LanguageModelConfig

Configuration for a language model generation call.

LanguageModelResponse

A response from a language model generation call.

MediaSupport

What a model accepts for one MediaKind.

MediaType

An RFC 6838 media type / MIME (type/subtype[;parameters]).

Message

A single message in an LM conversation.

ModelCapabilities

What a single model can carry across every MediaKind.

ModelId

Provider model identifier (e.g. "claude-sonnet-4-6", "gpt-4o-mini"). Wraps an arbitrary String so it cannot be passed where an ApiKey is expected.

OllamaConfig

Configuration for OllamaLanguageModel.

OllamaDeps

Injected dependencies for OllamaLanguageModel. The HTTP client is constructed once at the application composition root and shared across HTTP-based providers.

OllamaLanguageModel

Ollama language model.

OpenAiConfig

Configuration for OpenAiLanguageModel.

OpenAiDeps

Injected dependencies for OpenAiLanguageModel.

OpenAiLanguageModel

OpenAI Chat Completions API language model.

ProviderFileId

An opaque, provider-issued file identifier (Anthropic Files API, OpenAI Files API).

ReasoningCapability

What reasoning a single model supports.

ReasoningParamConflicts

Sampling-parameter restrictions that apply when reasoning is on.

RetryConfig

Configuration for retry behavior.

S3Uri

A canonical AWS S3 URI: s3://<bucket>/<key>.

SseEvent

A parsed Server-Sent Event.

SseStream

Stream adapter that transforms a byte stream into SSE events.

StreamDelta

An incremental chunk from a streaming language model response.

Usage

Token usage statistics for a generation call.

Enums

ApiKeyError

Failure mode returned by ApiKey::parse.

AwsAccountIdError

Failure mode returned by AwsAccountId::parse.

BedrockMantleAuth

Authentication for BedrockMantleProvider.

CacheTtl

Prompt-cache time-to-live for cache breakpoints.

CapabilityError

Reasons a request can fail capability validation, by precision.

ContentPart

A single content part within a message.

HttpsUrlError

Failure mode returned by HttpsUrl::parse.

LanguageModelError

Errors that can occur when calling a language model.

LatencyMode

Latency tier for the generation call.

MediaKind

Bucket for media content parts.

MediaSource

Where the bytes of one media content part come from.

MediaTypeError

Failure mode returned by MediaType::parse.

PromptCaching

Prompt-caching mode for the generation call.

ProviderFileIdError

Failure mode returned by ProviderFileId::parse.

ReasoningConfig

Resolved reasoning intent passed to providers via LanguageModelConfig::reasoning.

ReasoningEffort

Reasoning effort hint for thinking models.

ReasoningMode

Provider wire-mode classes a model accepts.

ReasoningValidationError

Reasons a ReasoningConfig can fail ReasoningCapability::validate.

ResponseFormat

Constrains the output format of a language model response.

ResponseFormatKind

Mirror of ResponseFormat without the per-variant payload.

Role

Message role in a conversation.

S3UriError

Failure mode returned by S3Uri::parse.

SourceKind

Discriminant for MediaSource variants, suitable for use in enumset::EnumSet-backed capability masks (e.g. MediaSupport.sources).

StopReason

Normalized reason the model stopped generating.

Traits

AcceptsAudioBytes

Implementor accepts at least one model that takes MediaSource::InlineBytes for MediaKind::Audio. (OpenAI gpt-audio family, Bedrock Voxtral.)

AcceptsDocumentBytes

Implementor accepts at least one model that takes MediaSource::InlineBytes for MediaKind::Document. (Anthropic, OpenAI file API, Bedrock.)

AcceptsImageBytes

Implementor accepts at least one model that takes MediaSource::InlineBytes for MediaKind::Image. (All providers except text-only ones.)

AcceptsImageS3

Implementor accepts at least one model that takes MediaSource::S3 for MediaKind::Image. (Bedrock only.)

AcceptsImageUrl

Implementor accepts at least one model that takes MediaSource::Url for MediaKind::Image. (OpenAI, Anthropic.)

AcceptsVideoBytes

Implementor accepts at least one model that takes MediaSource::InlineBytes for MediaKind::Video. (Bedrock Nova Pro/Lite only.)

AcceptsVideoS3

Implementor accepts at least one model that takes MediaSource::S3 for MediaKind::Video. (Bedrock only.)

LanguageModelProvider

A chat / completion provider — one impl per backend.

Functions

all_source_kinds

EnumSet<SourceKind> containing every variant — useful as a default accepted_sources for callers that haven’t narrowed the slot.

parse_sse_stream

Parse a byte stream (e.g. from reqwest::Response::bytes_stream()) into a stream of SseEvents following the WHATWG SSE specification.

with_retry

Execute an async operation with retry and exponential backoff.