Module cache 

Tokenizer Caching Layer

Provides a caching wrapper around any tokenizer implementation to speed up repeated tokenization of the same strings (e.g., system prompts).

Architecture

• L0 Cache: Whole-string exact match (90% of wins)
• L1 Cache: Prefix matching at fixed boundaries (future work)

Usage

let tokenizer = Arc::new(HuggingFaceTokenizer::from_file("tokenizer.json")?);
let cached = Arc::new(CachedTokenizer::new(tokenizer, CacheConfig::default()));
let encoding = cached.encode("Hello world")?;

Structs

CacheConfig

Configuration for the tokenizer cache

CacheStats

CachedTokenizer

A caching wrapper around any tokenizer

L0Cache

L0 cache implementation using DashMap for lock-free reads.

L1Cache

L1 cache implementation with special-token-boundary prefix matching

L1CacheStats

TokenizerFingerprint

A fingerprint of a tokenizer’s configuration