Skip to main content

TokenConstraint

oxibonsai_runtime::constrained_decoding

Trait TokenConstraint 

Source 
pub trait TokenConstraint: Send + Sync {
    // Required methods
    fn allowed_tokens(
        &self,
        generated: &[u32],
        vocab_size: usize,
    ) -> Option<Vec<bool>>;
    fn advance(&mut self, token: u32) -> bool;
    fn is_complete(&self) -> bool;
    fn reset(&mut self);
    fn name(&self) -> &str;
}
Expand description

A constraint that restricts which tokens are valid at each decoding step.

Implementors maintain internal state representing how far through the constrained structure the generation has progressed.

Required Methods§

Source

fn allowed_tokens( &self, generated: &[u32], vocab_size: usize, ) -> Option<Vec<bool>>

Given the tokens generated so far, return a bitmask of allowed next tokens.

vocab_size is the total vocabulary size. Returns None if all tokens are allowed (no active constraint).

Source

fn advance(&mut self, token: u32) -> bool

Called after a token is committed.

Returns false if the constraint is now violated (generation should stop).

Source

fn is_complete(&self) -> bool

Returns true if the current state is a valid terminal state.

Source

fn reset(&mut self)

Reset the constraint to its initial state.

Source

fn name(&self) -> &str

Human-readable name for debugging and logging.

Implementors§