Function truncate_to_token_limit 

pub fn truncate_to_token_limit(text: &str, max_tokens: usize) -> String

Truncate text to at most max_tokens tokens.

Decodes the truncated token sequence back to text so the result is always valid UTF-8 with no mid-token corruption. Falls back to byte-level truncation if BPE decode fails (should not happen in practice).