pub fn tokenize_lossless(source: &str) -> Vec<(Token<'_>, Span)>Expand description
Tokenize source code losslessly: every byte of source appears in
exactly one emitted (Token, Span) entry. This is the input to
the CST builder.
Differs from tokenize in that Token::Whitespace tokens are
preserved (the AST-style parser drops them; the CST keeps them so
the round-trip stays byte-identical).