Struct ReasoningBudget 

pub struct ReasoningBudget { /* private fields */ }

A sampler that caps how long a model may think.

Reasoning models emit an unbounded <think>…</think> block before their answer, and nothing in the model stops them running to the context limit. This watches for the start tag, counts down, and — when the budget is spent — forces the closing sequence token by token, masking everything else.

It is the piece llama.cpp pairs with a lazy grammar: the grammar leaves reasoning unconstrained by design, so something else has to bound it.

Build one with ReasoningBudget::new and add it to a LlamaSampler::chain, or let CommonSampler assemble it for you via CommonSamplerParams::set_reasoning_budget.

Implementations

impl ReasoningBudget

pub fn new( model: &LlamaModel, starts: &[Vec<LlamaToken>], ends: &[Vec<LlamaToken>], forced: &[LlamaToken], budget: i32, ) -> Result<Self, ShimError>

Build a budget sampler.

• starts — tokenized opening tags; any one arms the countdown.
• ends — tokenized closing tags; any one disarms it naturally.
• forced — emitted when the budget runs out, usually a short message followed by a closing tag.
• budget — tokens allowed inside the block.

Errors

Returns CommonSamplerError::Failed if llama.cpp returns null.

pub fn with_initial_state( model: &LlamaModel, starts: &[Vec<LlamaToken>], ends: &[Vec<LlamaToken>], forced: &[LlamaToken], budget: i32, initial_state: ReasoningBudgetState, ) -> Result<Self, ShimError>

Self::new with an explicit starting state — use ReasoningBudgetState::Counting when the prompt already opened a reasoning block, which is what a template with thinking_forced_open produces.

Errors

Returns CommonSamplerError::Failed if llama.cpp returns null.

pub fn state(&self) -> Option<ReasoningBudgetState>

Where the state machine currently is.

pub fn force_end(&mut self) -> bool

Cut the reasoning block short now. Returns false if it was not counting.

pub fn into_sampler(self) -> LlamaSampler

Take the underlying sampler, to add to a chain.

Trait Implementations

impl Debug for ReasoningBudget

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.