Module model 

Calling a model.

A completion is an effect like any other — journaled once, replayed from the record, untrusted on the way back. What is different is the meter.

A failed completion is not a free one

Every other outward call this crate makes either happens or does not. A model call has a third state: it ran, generated four hundred tokens, and then the stream died. The provider will bill those tokens. The answer is unusable.

Two consequences, and both are easy to get backwards.

The spend must be reported anyway. EffectError::Metered carries what was consumed, and the runtime bills it on the failure path. Without that, the token and cost ceilings — the ones that exist to bound exactly this — count zero while a retry loop against a flaky provider spends real money.

A died-mid-stream call is Disposition::Landed, not InDoubt. The usual reasoning about reaching the peer is inverted here: we know perfectly well that it reached the provider, because we watched it generate. What we lack is the answer, and repeating the call buys a second bill for the same question. InDoubt would invite Recovery to resolve an outcome that is not in doubt at all. Because a completion changes nothing in the world, a landed failure is still repeated when the call’s RetryPolicy permits — whether a second bill is worth an answer is the caller’s call, and the default policy declines.

Determinism

A model is the least deterministic thing a run touches, which is exactly why the completion is journaled: replay reads the recorded answer rather than asking again. The prompt is part of the effect key, so a changed prompt is a changed effect and shows up as divergence rather than as a quietly different run.

Modules

anthropic

A ModelProvider for the Anthropic Messages API.

bedrock

Amazon Bedrock Runtime through the provider-neutral Converse API.

chat_completions

A ModelProvider for the OpenAI-compatible Chat Completions wire.

embeddings

The embeddings wire, beside the drivers it shares a transport with. Embedding drivers, for the seam semantic retrieval needs.

env

A shipped driver, by its provider name, configured from the environment.

fake

The deterministic stand-in provider. See fake for why it ships. A model provider with no model behind it.

gemini

A ModelProvider for the Google Gemini Developer API.

openai

A ModelProvider for the OpenAI Responses API.

Structs

Completion

What came back.

ModelCall

One completion.

ModelId

Which model, from which provider.

ModelRole

One model role, as a declaration resolves it: which model, and the ceilings the reviewer put beside it.

Pricing

What a model’s tokens cost, as the deployment states it.

Prompt

An instruction and a question as text parts, any of which may close a cached prefix — written once, sent to whichever driver is linked.

ProviderContinuation

Opaque, self-contained provider state for one continuation turn.

Request

One request to a provider.

ToolCall

A tool the model asked to call.

ToolDeclaration

A tool the model may ask for, as the request declares it.

ToolExchange

A tool the model asked for, and what came back.

Transcript

A tool loop’s conversation so far, and the label of everything in it.

Usage

What a completion cost.

Enums

CacheTtl

How long a provider keeps a cached prompt prefix.

ModelError

Why a completion failed.

ModelStreamEvent

A live, non-durable model-stream event.

ReasoningEffort

Provider-neutral reasoning depth.

SchemaMode

How a driver should obtain a schema-conforming answer.

StrictSubset

A provider’s strict-mode subset of JSON Schema.

Traits

ModelProvider

Talks to a provider.

ModelStreamObserver

Receives live model progress while one terminal completion remains canonical.

Functions

strict_schema_problem

Why schema cannot be used with strict constrained decoding on every provider this crate drives, if it cannot.