Function lex 

pub fn lex(source: &str) -> Vec<Token>

Tokenizes source into a vector of span-based tokens.

Post-processes the Logos output:

• Coalesces consecutive lexer errors into single Garbage tokens
• Splits StringLiteral tokens into quote + content + quote
• Splits RegexPredicateMatch/RegexPredicateNoMatch into operator + whitespace + regex