Skip to main content

FINAL_OUTPUT_REGION_TOKENS

Constant FINAL_OUTPUT_REGION_TOKENS 

Source
pub const FINAL_OUTPUT_REGION_TOKENS: usize = 2_000;
Expand description

Token budget for FINAL_OUTPUT_REGION.

Deliberately far smaller than a maximal answer. The region is a convenience, not the storage: it exists so a later stage can see what was submitted and revise it, while the authoritative copy lives on the component and on disk. Sized at the whole answer, a maximal submission would pin ~65k tokens into every subsequent inference for the rest of the run, which costs more than the convenience is worth. A long answer is mirrored as a preview instead.