Module lexer 

Structs

Lexer

SQL lexer that produces a stream of tokens from source text.

TokenizeDurationSecondsHistogram

Histogram buckets for fsqlite_tokenize_duration_seconds.

TokenizeMetricsSnapshot

Point-in-time tokenize metric snapshot.

Functions

reset_tokenize_metrics

Reset tokenize metrics (used by tests/diagnostics).

set_force_interner_scan_bench

Force the O(entries) retained_bytes fold on the current thread (bd-5310l bench A/B only).

set_tokenize_metrics_enabled

Enable or disable tokenize metrics collection on the hot path.

tokenize_metrics_enabled

Return whether tokenize metrics collection is enabled.

tokenize_metrics_snapshot

Point-in-time snapshot of tokenize metrics.