Struct Bitmask 

pub struct Bitmask { /* private fields */ }

A dense allowed-token bitmask over vocab_size tokens, packed into little-endian u32 words.

Implementations

impl Bitmask

pub fn zeros(vocab_size: usize) -> Self

Allocates an all-zero mask sized for vocab_size tokens (ceil(vocab_size/32) words).

pub fn from_words(words: &[u32], vocab_size: usize) -> Self

Builds a mask from packed words, clearing unused high bits.

pub fn set(&mut self, id: TokenId) -> Result<(), CompileError>

Sets the bit for id. Fails (never truncates) when id is outside 0..vocab_size.

pub fn get(&self, id: TokenId) -> bool

Returns whether the bit for id is set (false for out-of-range ids).

pub fn as_words(&self) -> &[u32]

The packed words (the wire format): ceil(vocab_size/32) little-endian u32s.

pub fn vocab_size(&self) -> usize

The number of tokens this mask covers.

pub fn count_ones(&self) -> usize

Returns the number of set token bits.

Trait Implementations

impl Clone for Bitmask

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for Bitmask

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Eq for Bitmask

impl PartialEq for Bitmask

fn eq(&self, other: &Self) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl StructuralPartialEq for Bitmask