Struct MaskGenerator 

pub struct MaskGenerator { /* private fields */ }

Generates logit masks from TØR-G builder state.

This struct combines a token mapping with a vocabulary size to produce LogitMask instances from Builder::valid_next_tokens().

Implementations

impl MaskGenerator

pub fn new(mapping: TokenMapping, vocab_size: usize) -> Self

Create a new mask generator.

Arguments

• mapping - Token mapping from TØR-G tokens to LLM vocab IDs
• vocab_size - Size of the LLM vocabulary

pub fn mapping(&self) -> &TokenMapping

Get the token mapping.

pub fn vocab_size(&self) -> usize

Get the vocabulary size.

pub fn generate(&self, valid_tokens: &[Token]) -> LogitMask

Generate a logit mask from a list of valid tokens.

Tokens that cannot be mapped are silently ignored.

pub fn generate_from_builder(&self, builder: &Builder) -> LogitMask

Generate a logit mask from builder state.

This is a convenience method that calls builder.valid_next_tokens() and then generates the mask.

Trait Implementations

impl Clone for MaskGenerator

fn clone(&self) -> MaskGenerator

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for MaskGenerator

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.