Struct ModelCall 

pub struct ModelCall { /* private fields */ }

One completion.

Implementations

impl ModelCall

pub const DEFAULT_MAX_OUTPUT_TOKENS: u32 = 16_384

Per-call output ceiling used when the caller does not set one.

Sized for reasoning models, whose thinking tokens count against the same limit as the answer: at 4096 a model reasoning at medium effort regularly spends the whole allowance thinking and returns a truncated turn with no answer in it. A role that needs a tighter bound declares max_tokens; the spend ceilings, not this default, bound cost.

pub fn new( provider: Arc<dyn ModelProvider>, model: ModelId, prompt: Value, ) -> Self

A completion from this provider.

pub fn with_tools( self, tools: impl IntoIterator<Item = ToolDeclaration>, ) -> Self

Tell the model which tools it may ask for.

What comes back is a request, not an action: the chosen name is matched against the operator’s grants exactly — never resolved to a near neighbour — and the arguments stay untrusted until they pass the sink’s field-provenance rules. Declaring is telling; authorizing is separate.

Declare only what is granted. Offering the model a tool the manifest does not grant produces a call that is refused after the model has been paid for choosing it, and teaches nobody anything.

pub fn continuing(self, transcript: &Transcript) -> Self

Continue a tool loop: show the model what its tools answered, with the provider state its last turn left.

The prompt stays what it was. A continuation is the same question with more known, so re-stating it would change the effect key and make each turn of a loop a different call for replay purposes.

The transcript’s label travels with it: the sink judges it beside the prompt’s, its relayed outputs are read by the sink’s content rules, and the answer is labelled with it. That is why a continuation can only be handed over as a Transcript — a bare list of outputs would reach the provider outside every gate.

pub fn with_compaction(self, compaction: Compaction) -> Self

Ask the provider to compact the conversation once its input reaches compaction.trigger_tokens: the earlier turns are replaced by a summary inside the provider’s continuation, which the next call sends back, and Completion::compacted says it happened.

Anthropic and OpenAI Responses compact; every other wire refuses the call rather than letting the conversation run into its window. The instruction (/system) is never compacted. Part of the effect key: the provider is asked something else.

pub fn streaming_to(self, observer: Arc<dyn ModelStreamObserver>) -> Self

Observe visible model text as it arrives during live execution.

pub fn stopped_by(self, stop: Stop) -> Self

End this call on stop, while it streams, as a recorded completion of what had arrived (Completion::stopped).

Stopping ends the bill where the stream is rather than at the answer’s end. A buffered call, and a call that declares a schema, run to their end. The signal is not effect identity; replay returns the recorded completion.

pub const fn with_max_output_tokens(self, max_output_tokens: u32) -> Self

Bound how many tokens this call may generate.

pub const fn with_max_input_tokens(self, max_input_tokens: u32) -> Self

Bound how many tokens this call may send, cached ones included.

Input cannot be bounded before the call — it is whatever the conversation has grown to — so this is held against the usage the provider reports: a completion that sent more is a failed call, billed as reported. That keeps the per-call figure a tenant’s spend quota reserves a bound on every answer a run goes on to use. Not part of the effect key: it changes which answers are accepted, not what the provider is asked.

pub const fn with_reasoning_effort(self, effort: ReasoningEffort) -> Self

Request an explicit reasoning depth.

pub const fn with_reasoning_summary(self) -> Self

Ask the provider for a readable summary of the model’s reasoning.

It arrives as Completion::reasoning and, live, as ModelStreamEvent::ReasoningDelta. On Anthropic it turns adaptive thinking on for a model that would not otherwise think. Part of the effect key: the provider is asked something else.

pub const fn with_pricing(self, pricing: Pricing) -> Self

Price this call’s usage, so its spend carries money as well as tokens.

Applied at the effect boundary to every usage the call reports — the answer’s and a metered failure’s alike — so a money ceiling counts a stream that died after generating exactly as it counts one that finished. Not part of the effect key: a price changes what the run is billed, not what the provider is asked.

pub const fn with_max_sensitivity(self, s: Sensitivity) -> Self

The highest sensitivity this model may be shown.

The control that matters for a hosted model: a prompt assembled from a secret is an exfiltration whether or not anyone meant it.

It is also the floor of the completion’s own label — see with_output_sensitivity for why the two are one decision.

pub const fn with_output_sensitivity(self, s: Sensitivity) -> Self

Declare the completion’s sensitivity floor, above the derived one.

The derived floor is max_sensitivity, always: a caller who raised the egress ceiling to show the model a confidential prompt has told the runtime what class of data the answer was derived from, and a completion labelled below its own prompt is a laundering primitive — ask the model to restate the secret and read it back a level down. So this setter can only raise the floor; a value below the ceiling is kept and simply loses to it at Effect::output_sensitivity, where the two are joined in one place rather than compensated for by hand at every call site.

pub const fn with_retry(self, r: RetryPolicy) -> Self

pub fn with_media<'a>( self, media: Arc<dyn BlobStore>, artifacts: impl IntoIterator<Item = &'a FetchedMedia>, ) -> Self

Permit these exact FetchedMedia artifacts to materialize from this blob store immediately before live dispatch.

The prompt and effect key contain only media digests. Strict replay does not execute perform, so it reads neither blob storage nor the network. A prompt marker without a matching digest/type grant is refused; knowing another case’s digest is not authority to read that blob. Model output remains journaled for replay and may itself reproduce media content; digest-only input storage is not an output-redaction promise.

pub fn expecting(self, schema: Value) -> Self

Require the answer to conform to a JSON Schema.

The schema goes into the effect key, which is the point: editing a schema changes the effect, so a replayed run reports divergence instead of quietly reading back an answer shaped to different rules. A schema that lived outside the key would let today’s shape re-interpret last year’s stored answer.

Enforcement happens at the provider, during generation, where a constraint can prevent a malformed answer rather than reject one already paid for. What this crate adds on top is the parse — see Completion::structured.

Trait Implementations

impl Debug for ModelCall

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Effect for ModelCall

fn mutates(&self) -> bool

A completion does not change the world.

Said plainly because it is the one outward call in this crate where that is true, and it is what makes retrying a rate-limit sane. It is not free — see spend — but a second completion does not move money twice.

fn protected_fields(&self) -> &[ProtectedField]

/system when the prompt has an instruction.

A model reads its instruction and its data as the same undifferentiated text, so text arriving as data that reads like a directive is obeyed like one. Every other control here bounds what the model may then do; this is the only one that asks who was allowed to give the order. Untrusted material belongs in messages.

fn output_sensitivity(&self) -> Sensitivity

Never below max_sensitivity.

A sink’s result inherits the class of what was sent into it: a caller who raised the egress ceiling to get a confidential prompt out has declared what the answer derives from, and returning that answer at a lower label would make one round trip through a model a release nobody authorized. with_output_sensitivity still raises the floor further; nothing lowers it below the ceiling.

fn carried(&self) -> Option<Tainted<Value>>

The relayed tool outputs, under the transcript’s label.

The provider continuation is not offered to content rules: it is the model’s own earlier output, often encrypted, and its label is already in the transcript’s.

fn outbound_bytes(&self) -> u64

Everything the provider is sent, not only the prompt the sink gate binds.

The prompt of a tool-loop turn can be two bytes while the request carries every declared tool, every earlier tool result, the provider’s continuation state and each granted image — and all of it leaves the plane. So the figure is the canonical size of every provider-visible input (the same set the effect key holds) plus each referenced media artifact at its encoded size, since the marker in the prompt is a few dozen bytes standing in for megabytes.

fn trust(&self) -> Trust

Model output is untrusted, and this is the case the rule was written for.

A completion is a plausible-sounding string produced from whatever was in the context window — including anything untrusted that got there. It is the canonical prompt-injection carrier.

fn source(&self) -> SourceId

Which model answered — model:{provider}/{model} — the same spelling the live stream label has always carried, now derived in one place so the terminal completion and the stream cannot name one answer two ways.

type Output = Completion

What comes back. Must round-trip through JSON: replay reconstructs it from the journal rather than from the driver.

fn gen_ai_operation(&self) -> Option<&'static str>

Which OpenTelemetry GenAI operation this effect is, if it is one.

fn descriptor(&self) -> EffectDescriptor

What this effect does. Hashed (with position) into the effect key.

fn recovery(&self) -> Recovery

What to do when the outcome is unknown — after a crash, or after an InDoubt failure. The two are the same situation reached from different directions.

fn retry(&self) -> RetryPolicy

How many times to repeat this effect when it fails, and how far apart.

fn max_sensitivity(&self) -> Sensitivity

The highest data sensitivity this sink may receive.

fn sink_arguments(&self) -> Option<&Value>

The exact value this effect will send to its sink.

fn rebind(&mut self, arguments: Value) -> bool

Bind arguments in place of the ones this effect was built over, where it can — what a declared redaction at a sink needs.

fn rebind_carried(&mut self, carried: Value) -> bool

Bind a redacted copy of carried, where the effect can.

fn gen_ai_request(&self) -> Option<GenAiRequest>

Who this operation asks, and for what — the request half of the GenAI span attributes.

fn gen_ai_response(&self, output: &Completion) -> Option<GenAiResponse>

What came back, split the way the convention reports it.

fn spend(&self, output: &Completion) -> Spend

What this effect consumed.

fn perform<'life0, 'async_trait>( &'life0 self, ) -> Pin<Box<dyn Future<Output = Result<Completion, EffectError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Do the thing. Called at most once per key per run; never called during replay.

fn attach(&mut self, _provenance: &Provenance)

Receive the run-scoped provenance for this call.

fn retries_landed(&self) -> bool

Whether a failure that landed may be tried again, within retry.

fn credential_binding(&self) -> Option<CredentialBinding>

Whom the credential this effect presents names, for an effect that presents one to another party.

fn delegation_depth(&self) -> Option<usize>

The resulting delegation depth when this effect hands authority onward.

fn reconcile<'life0, 'async_trait>( &'life0 self, ) -> Pin<Box<dyn Future<Output = Result<Reconciliation<Self::Output>, EffectError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Ask the provider whether a call landed.