pub fn tokenize(input: &str) -> Result<Vec<Token>, LexError>
Tokenize input into a Vec<Token> ending in Token::Eof, with PG string semantics (backslash is a literal byte inside '…'; '' is the only escape).
input
Vec<Token>
Token::Eof
'…'
''