Trait LlmProvider 

pub trait LlmProvider: Send + Sync {
    // Required methods
    fn call<'a>(
        &'a self,
        model: &'a str,
        prompts: &'a [String],
        schema_json: Option<&'a str>,
    ) -> Pin<Box<dyn Future<Output = Result<Vec<LlmOutcome>, LlmCallError>> + Send + 'a>>;
    fn models<'a>(
        &'a self,
    ) -> Pin<Box<dyn Future<Output = Result<Vec<LlmModel>, LlmCallError>> + Send + 'a>>;

    // Provided method
    fn output_reserve(&self, _model: &str) -> Option<u64> { ... }
}

The embedder-provided model provider: resolves the credential, performs the completions, and reports what each one cost.

Object-safe boxed-future trait (rather than async fn) so the interpreter stays free of the embedder’s async stack while still await-ing the call from within the async host function — the shape McpTransport uses.

Obligations on every implementor

These are contractual, not enforced by the interpreter. A second implementor inherits them:

1. Positional ordering. The returned Vec<LlmOutcome> has exactly one element per input prompt, in input order. Element i is the outcome of prompts[i], including when it failed.
2. Bounded concurrency. call takes a slice rather than a single prompt because the guest is single-threaded Wasm holding one host call at a time: fan-out cannot live above this boundary, so it lives below it. The implementor is responsible for bounding that concurrency — the interpreter does not limit it.
3. Per-element failure never becomes Err. One bad element is an LlmOutcome with ok: false. Err(LlmCallError) is only for dispatch that never ran, or a transport that died before any element started; an Err discards the siblings that did succeed and were billed.
4. No prompt or completion text in errors. Nothing an implementor puts in an LlmCallError may quote the payload.

Required Methods

fn call<'a>( &'a self, model: &'a str, prompts: &'a [String], schema_json: Option<&'a str>, ) -> Pin<Box<dyn Future<Output = Result<Vec<LlmOutcome>, LlmCallError>> + Send + 'a>>

fn models<'a>( &'a self, ) -> Pin<Box<dyn Future<Output = Result<Vec<LlmModel>, LlmCallError>> + Send + 'a>>

Returns the locally declared model catalog without network I/O or other side effects. The caller must inspect these names before it can check policy for each model, so this runs before per-model authorization.

Provided Methods

fn output_reserve(&self, _model: &str) -> Option<u64>

The output cap declared for one model, consulted before dispatch.

Separate from Self::models because the reservation needs one model’s reserve on the hot path of every call; using the whole catalog here would add unnecessary work to each dispatch. An implementor that already holds its declarations (the blueprint case) answers from memory, and one that does not should return None rather than block.

None means the model declared no reserve, or the implementor cannot answer cheaply; the caller then applies LlmLimits::default_output_cap. Returning None is always safe, which is why this defaults rather than being required of every implementor.

Dyn Compatibility

This trait is dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementors