Struct ParserToken 

pub struct ParserToken<'a>(/* private fields */);

One parser-observed token from a parsed statement.

Returned by [ParsedStatement::tokens]. This is useful when building token-aware tooling such as:

• Semantic syntax highlighting.
• Identifier/function/type classification.
• Statement-level token diagnostics.

Requires collect_tokens: true in ParserConfig.

Examples

use syntaqlite_syntax::{Parser, ParserConfig, TokenType};

let parser = Parser::with_config(&ParserConfig::default().with_collect_tokens(true));
let mut session = parser.parse("SELECT max(x) FROM t;");
let stmt = session.next().transpose().unwrap().unwrap();

let tokens: Vec<_> = stmt.tokens().collect();
assert!(!tokens.is_empty());
assert!(tokens.iter().any(|t| t.token_type() == TokenType::Select));

// Flags expose parser-inferred role information (identifier/function/type).
let _has_semantic_role = tokens.iter().any(|t| {
    let f = t.flags();
    f.used_as_identifier() || f.used_as_function() || f.used_as_type()
});

Implementations

impl<'a> ParserToken<'a>

pub fn text(&self) -> &'a str

Exact source text for this token.

Preserves original casing and quoting from input SQL.

pub fn token_type(&self) -> TokenType

Token kind from the SQLite SQL grammar.

This is the lexical class (keyword, identifier, operator, etc.).

pub fn flags(&self) -> ParserTokenFlags

Semantic usage flags inferred by the parser.

Use this to distinguish contextual role, for example:

• Keyword text used as an identifier.
• Function-call names.
• Type names.

pub fn offset(&self) -> u32

Byte offset of the token start within the statement source.

pub fn length(&self) -> u32

Byte length of the token text.

Trait Implementations

impl Debug for ParserToken<'_>

Available on crate feature sqlite only.

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter.