pub fn classify_token(kind: &str, is_named: bool, text: &str) -> TokenKindExpand description
The shared C-family token classification.
Grammar kind names drive the mapping; anonymous (non-named) tokens are
keywords when their kind is purely alphabetic and punctuation otherwise
(operators, delimiters, and directive introducers like #include). Named
leaves outside the known kinds — notably the opaque preproc_arg
replacement text — classify as TokenKind::Unknown.