Function code_bits_for_num_tokens 

pub fn code_bits_for_num_tokens(num_tokens: usize) -> u8

Minimum code width needed to address num_tokens distinct tokens: ceil(log2(num_tokens)).

This is the runtime width of OnPair token codes for a dictionary with num_tokens entries. It is independent from the training-time MaxDictBits budget.