Struct AllowedTokens 

pub struct AllowedTokens { /* private fields */ }

A fixed allowed-token-set constraint: the same ids every step, regardless of decode state.

The bread-and-butter case for the tool-loop — e.g. “the next token must be one of the ids that spell a valid tool name from the enum”, or “pick one of these N candidate-file ids”. Because the set never changes, it is a BTreeSet<u32> fixed at construction and AllowedTokens::allowed just hands it back.

The clone-per-step in AllowedTokens::allowed is intentional and cheap: a tool-name set is tens of ids, and cloning a small BTreeSet once per generated token is nothing next to a whole transformer forward pass. If a future caller pins a large fixed set, switch its storage to a shared Arc<BTreeSet<u32>> — the TokenConstraint contract does not change.

Implementations

impl AllowedTokens

pub fn new(ids: impl IntoIterator<Item = u32>) -> Result<Self, ConstraintError>

Build a fixed-set constraint from an iterator of allowed ids.

Errors

ConstraintError::EmptyConstraint if ids is empty — a constraint that allows nothing would mask the entire vocab at every step and can only ever produce ConstraintError::NoTokenAllowed. Rejecting it here turns a guaranteed-later failure into an obvious construction-time one.

Trait Implementations

impl Clone for AllowedTokens

fn clone(&self) -> AllowedTokens

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for AllowedTokens

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl TokenConstraint for AllowedTokens

fn allowed(&self, _state: &DecodeState<'_>) -> AllowedSet

The set of token ids that keep the output valid if emitted next.