Module llm 

The embedder-supplied seam for submilli:llm — gated model calls from inside a Submilli program.

call, batch, and models() all dispatch through the single embedder-provided LlmProvider: the actual HTTP, key resolution, retry, and provider-SDK dependencies live in the embedder (submilli-server), keeping this crate free of them — the same split crate::runtime::mcp::McpTransport makes.

Two taxonomies, answering different questions. LlmCallError is the dispatch-level Err: the request never reached the model at all, or the transport died before any element ran. FailureReason is per-element: the request reached the model and that one element came back unusable or incomplete. Neither is a subset of the other, and a per-element failure must never be raised as an Err — a whole-batch Err would discard every element outcome including successes that were already billed, and would leave per-element budget reconciliation with nothing to reconcile against.

No prompt or completion text appears here. Not in an error Display, not in a log line, not in a permission-check context, and not in an LlmFailure::message. The taxonomies carry the model name, the numbers, a status code, and a fixed classification string — never the payload. Provider response bodies are dropped at this boundary specifically: they carry completion fragments, request echoes, and account identifiers, so unlike McpCallError::Upstream no variant here interpolates one.

Re-exports

pub use super::token_ledger::SharedTokenBudget;

Structs

ExecutionTokenBudget

One execution’s token budget, reserving against its own ceiling and the server-wide one together.

LlmFailure

The failure-arm payload of an LlmOutcome: everything that exists only when ok is false.

LlmLimits

The ceilings a dispatch is checked against.

LlmModel

A model the provider serves.

LlmOutcome

One element’s result, positionally matched to the prompt that produced it.

Enums

FailureReason

Why one element of a dispatch is not a clean completion.

LlmCallError

Why a model dispatch failed — the request never reached the model, or the transport died before any element ran.

LlmLimitKind

Which token ceiling a refused dispatch ran into. Carries the numbers, never the prompts — the same rule SessionKvLimitKind follows for values.

PromptBoundKind

Which prompt bound a dispatch ran into. Carries the numbers, never the prompts — the same rule SessionKvLimitKind follows for values.

Constants

DEFAULT_MAX_ALL_EXECUTIONS_TOKENS

Tokens every live execution may spend between them.

DEFAULT_MAX_EXECUTION_TOKENS

Tokens one execution may spend across every model call it makes.

DEFAULT_MAX_HELD_TOKENS

Tokens one execution may hold as indeterminate (null-usage) reserve before further dispatch is refused. See LlmLimitKind::IndeterminateSpend.

DEFAULT_MAX_PROMPT_BYTES

UTF-8 bytes in a single prompt element.

DEFAULT_MAX_PROMPT_COUNT

Elements in one batch slice.

DEFAULT_OUTPUT_CAP

Output tokens reserved per prompt when the model declares no outputReserve.

LLM_MODULE_NAME

The single internal host module every submilli:llm call dispatches through.

Traits

LlmProvider

The embedder-provided model provider: resolves the credential, performs the completions, and reports what each one cost.