Enum OutputCap 

pub enum OutputCap {
    Tokens(usize),
    WindowPercent(f64),
    RegionPercent {
        percent: f64,
        region: String,
    },
}

How large one reply may be, as a stage’s parameters.max_output_tokens says it.

A bare number is the classic form and is sent as written. The other two are relative, because a fixed number is the wrong shape for the question: a stage that rewrites a whole report needs “as much as this model can give”, and a stage that fills a region needs “as much as the region holds”. A fixed 24000 on the report stage was smaller than the report, and every reply was cut off. A relative cap is clamped to the model’s own maximum, since asking for more than that is refused by the provider.

parameters = { max_output_tokens = 8000 }              # tokens
parameters = { max_output_tokens = "40%" }             # of the model's context window
parameters = { max_output_tokens = "100% of claims" }  # of the `claims` region's budget
parameters = { max_output_tokens = { percent = 100, of = "claims" } }

Variants

Tokens(usize)

A fixed number of output tokens, sent as written.

WindowPercent(f64)

A fraction (0.0..=1.0) of the model’s context window.

RegionPercent

A fraction (0.0..=1.0) of a named region’s token budget.

Fields

percent: f64

The fraction.

region: String

The region whose budget it is a fraction of.

Implementations

impl OutputCap

pub fn parse(value: &Value) -> Result<Self, String>

Read the cap from the parameter’s JSON value. Err carries the reason in the words a blueprint author needs, for the manifest loader to surface: a cap that does not parse must fail the load, because the alternative (no cap) is exactly the silent no-op that hides a typo.

pub fn resolve( &self, model_window: usize, model_max_output: usize, region_budget: impl Fn(&str) -> Option<usize>, ) -> usize

The cap in tokens for one request.

model_window and model_max_output are the model’s own limits; region_budget answers “how many tokens may region X hold” for the window the request is built from. A relative cap is clamped to the model’s maximum. A region cap naming a region the stage does not carry falls back to the model’s maximum, which is the same “as much as you can” the author was reaching for; the loader already warned about the name.

Trait Implementations

impl Clone for OutputCap

fn clone(&self) -> OutputCap

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for OutputCap

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl PartialEq for OutputCap

fn eq(&self, other: &OutputCap) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl StructuralPartialEq for OutputCap