Function load_tokenizer_with_truncation 

pub fn load_tokenizer_with_truncation(
    model_path: &Path,
    max_len: usize,
) -> Result<Tokenizer>

Loads a tokenizer with truncation enabled for a maximum sequence length.

This is important for cross-encoder models that have a fixed maximum sequence length. Inputs exceeding max_len will be truncated to fit.