pub fn tokenize(text: &str) -> Vec<String>
Tokenize text with code-awareness: splits camelCase, snake_case, punctuation boundaries, lowercases, and filters short tokens.