Struct AllowedTokensCache 

pub struct AllowedTokensCache { /* private fields */ }

LRU cache mapping Earley state hashes → token masks.

Default capacity: 256 entries. At 150 k tokens (Qwen3 vocab) × 1 byte each, that is ≈38 MB worst-case; typical grammars cycle through far fewer states.

Thread safety

AllowedTokensCache is not Sync on its own. In GrammarConstraint it is wrapped in a std::sync::Mutex to satisfy the &self signature of TokenConstraint::allowed_tokens.

Implementations

impl AllowedTokensCache

pub fn with_capacity(capacity: usize) -> Self

Create a cache with the given capacity.

The capacity is clamped to a minimum of 1 to keep invariants simple.

pub fn get(&mut self, state_hash: u64) -> Option<Arc<[bool]>>

Try to get a cached mask for the given state hash.

On a hit the key is promoted to the back of the LRU queue (most recently used) and the hit counter is incremented. On a miss the miss counter is incremented.

pub fn insert(&mut self, state_hash: u64, mask: Vec<bool>)

Insert a mask for the given state hash, evicting LRU if at capacity.

If the hash is already present the call is a no-op (the existing entry is kept; this is safe because a single-threaded Mutex prevents races).

pub fn hits(&self) -> u64

Number of cache hits since creation (for testing / observability).

pub fn misses(&self) -> u64

Number of cache misses since creation (for testing / observability).

pub fn len(&self) -> usize

Number of entries currently held in the cache.

pub fn is_empty(&self) -> bool

True if the cache holds no entries.