Struct Lexer 

pub struct Lexer { /* private fields */ }

SQL Lexer for tokenizing input

Uses byte-based indexing for efficiency. SQL is predominantly ASCII, so we can work with bytes directly and only decode UTF-8 when needed.

Implementations

impl Lexer

pub fn new(input: &str) -> Self

Create a new lexer for the given input

pub fn next_token(&mut self) -> Token

Get the next token