pub enum Token {
Show 144 variants
Select,
From,
Where,
As,
Null,
True,
False,
And,
Or,
Not,
Create,
Table,
Insert,
Into,
Values,
Index,
On,
Begin,
Commit,
Rollback,
Order,
By,
Limit,
Ident(String),
QuotedIdent(String),
SessionVar(String),
Integer(i64),
Float(f64),
Numeric(String),
String(String),
HexBytes(String),
Plus,
Minus,
Star,
Slash,
Percent,
Eq,
NotEq,
Lt,
LtEq,
Gt,
GtEq,
InetContainedBy,
InetContainedByEq,
InetContains,
InetContainsEq,
InetOverlap,
OverLeft,
OverRight,
LParen,
RParen,
LBracket,
RBracket,
Comma,
Semicolon,
Dot,
DotDot,
Bang,
At,
JsonGet,
JsonGetText,
JsonGetPath,
JsonGetPathText,
JsonDeletePath,
JsonContains,
JsonPathExists,
JsonContainedBy,
JsonKeyExists,
JsonKeysAny,
GeomParallel,
GeomPerp,
GeomSameAs,
ClosestPoint,
GeomHoriz,
JsonKeysAll,
TsMatch,
TsMatchOld,
AtMinusAt,
Intersects,
IsBelow,
IsAbove,
PatternLt,
PatternLtEq,
PatternGt,
PatternGtEq,
L2Distance,
InnerProduct,
CosineDistance,
DoubleColon,
ColonEq,
FatArrow,
Colon,
Concat,
Pipe,
Amp,
Tilde,
TildeStar,
NotTilde,
NotTildeStar,
DoubleTilde,
DoubleTildeStar,
NotDoubleTilde,
NotDoubleTildeStar,
Caret,
CaretAt,
Hash,
Adjacent,
DoubleBang,
Is,
Between,
In,
Like,
Group,
Distinct,
Union,
All,
Join,
Inner,
Left,
Cross,
Outer,
Right,
Full,
Default,
Savepoint,
Release,
To,
Having,
Show,
Extract,
Offset,
Asc,
Desc,
Interval,
Placeholder(u16),
Drop,
For,
Tables,
Except,
Publication,
Subscription,
Connection,
Partition,
Eof,
}Variants§
Select
From
Where
As
Null
True
False
And
Or
Not
Create
Table
Insert
Into
Values
Index
On
Begin
Commit
Rollback
Order
By
Limit
Ident(String)
QuotedIdent(String)
SessionVar(String)
v7.14.0 — MySQL session / user variable reference
(@VAR / @@VAR). The wrapped string is the verbatim
source form (including the @ / @@ prefix). Used by
mysqldump preamble (SET @OLD_FOREIGN_KEY_CHECKS = @@FOREIGN_KEY_CHECKS, …); SPG accepts the token and
the SET parser treats the assignment as a no-op apart
from any second LHS that targets a real session
parameter (e.g. FOREIGN_KEY_CHECKS=0).
Integer(i64)
Float(f64)
Numeric(String)
String(String)
HexBytes(String)
v7.39 (round 367, M20) — a MySQL 0x… hexadecimal literal in the
MySQL dialect: a BINARY STRING, not an integer. Carries the raw hex
digits (parser decodes, left-padding an odd count). The PG dialect
never emits this — there 0x… stays a radix-16 Integer.
Plus
Minus
Star
Slash
Percent
v7.37.7 C.1.7 — PG % integer modulo operator (also short for
mod(y, x)). MySQL accepts MOD keyword + %; SPG follows
the PG form here. Token alone (no %= etc., kept simple).
Eq
NotEq
Lt
LtEq
Gt
GtEq
InetContainedBy
v7.17.0 Phase 3.P0-47 — PG INET / CIDR strict contained-in
<<. LHS is strictly inside RHS (no equality).
InetContainedByEq
v7.17.0 Phase 3.P0-47 — PG INET / CIDR contained-in-or-equal
<<=. LHS network ⊆ RHS network.
InetContains
v7.17.0 Phase 3.P0-47 — PG INET / CIDR strict contains >>.
LHS strictly contains RHS.
InetContainsEq
v7.17.0 Phase 3.P0-47 — PG INET / CIDR contains-or-equal >>=.
LHS network ⊇ RHS network.
InetOverlap
v7.17.0 Phase 3.P0-47 — PG INET / CIDR network overlap &&.
Either side contains any address of the other.
OverLeft
v7.39 — range &< / &>.
OverRight
LParen
RParen
LBracket
RBracket
Comma
Semicolon
Dot
DotDot
v7.37.20 (20.4) — .. range operator, used by PL/pgSQL
FOR i IN 1..10 LOOP bounds. Emitted by the lexer as a
single token so parse_expr doesn’t have to distinguish
range-. from struct-field-..
Bang
v7.39 (round 353, M10) — MySQL’s ! (logical negation). Its own
token because its precedence is nothing like NOT’s.
At
v7.17.0 Phase 2.6 — standalone @ punctuation. Emitted when
@ is NOT followed by an ident-start byte (i.e. the
@VAR / @@VAR SessionVar path doesn’t match). Lets the
parser stitch the MySQL 'user'@'host' DEFINER form back
together as String + At + String. Pre-2.6 this same shape
surfaced as a LexErrorKind::UnknownChar('@') and broke
every mysqldump CREATE VIEW with a DEFINER clause at lex
time.
JsonGet
pgvector L2 distance operator <->. Lexed as one token so the
parser can give it its own precedence rung.
v4.14 -> — JSON object/array element access, returns json.
JsonGetText
v4.14 ->> — same access, returns text.
JsonGetPath
v6.4.5 #> — JSON path walk, returns json. Path is the
right-hand TEXT with PG {a,b,0} syntax.
JsonGetPathText
v6.4.5 #>> — same walk, returns text.
JsonDeletePath
#- — delete the value at a nested JSON path. RHS is a PG
text-array literal {a,b}.
JsonContains
v6.4.5 @> — JSON containment. j @> sub returns true if
every key/value in sub is present in j with structural
containment for objects + arrays.
JsonPathExists
@? jsonpath existence operator.
JsonContainedBy
v7.37.6-A <@ — JSON contained-by. a <@ b ⇔ b @> a.
JsonKeyExists
v7.37.6-A ? — JSON key exists (object), or element-as-text
exists (array). j ? 'key' returns BOOL.
JsonKeysAny
v7.37.6-A ?| — JSON any-key-exists. j ?| ARRAY['a','b']
returns BOOL; true if any one of the listed keys exists in j.
GeomParallel
v7.39 (read01 geo_ops.c) — ?|| “is parallel” (lseg / line).
GeomPerp
v7.39 (read01 geo_ops.c) — ?-| “is perpendicular” (lseg / line).
GeomSameAs
v7.39 (read01 geo_ops.c) — ~= “same as” (geometric equality).
ClosestPoint
v7.39 (read01 geo_ops.c) — ## closest point.
GeomHoriz
v7.39 (read01 geo_ops.c) — ?- “is horizontal” (binary points /
prefix lseg-line).
JsonKeysAll
v7.37.6-A ?& — JSON all-keys-exist. j ?& ARRAY['a','b']
returns BOOL; true if every listed key exists in j.
TsMatch
v7.12.2 @@ — tsvector / tsquery match. Either ordering
(vec @@ q or q @@ vec) parses; engine eval normalises
before matching.
TsMatchOld
v7.39 (round 508) — @@@, PG’s deprecated spelling of @@. Kept
because pg_operator still carries it and old application SQL still
writes it.
AtMinusAt
v7.39 (round 508) — @-@, “length of” (lseg, path).
Intersects
v7.39 (round 508) — ?#, “do these intersect” (box / line / lseg /
path, in every combination PG defines).
IsBelow
v7.39 (round 508) — <^ “is strictly below” and >^ “is strictly
above” (point, box).
IsAbove
PatternLt
v7.39 (round 508) — the text_pattern_ops comparisons ~<~, ~<=~,
~>~, ~>=~. They compare BYTES, ignoring collation, which is what
makes them index-usable for LIKE prefixes: 'A' ~<~ 'a' is true
where 'A' < 'a' is false under a non-C collation. pg_dump emits
them, so a dump of an ordinary database would not restore.
PatternLtEq
PatternGt
PatternGtEq
L2Distance
InnerProduct
pgvector inner-product operator <#> (returns negative dot product
so smaller still means more similar — same semantics as pgvector).
CosineDistance
pgvector cosine distance operator <=>.
DoubleColon
PG-style cast expr::type — single token because we want it to bind
at postfix precedence.
ColonEq
v7.12.4 — PL/pgSQL assignment operator :=.
Outside PL/pgSQL bodies this token has no SQL-side meaning.
FatArrow
v7.38 (read01, T14) — => names a function argument
(make_date(year => 2024, …)).
Colon
v7.12.4 — bare : separator. Used inside tsvector external-form
literals ('cat:1 dog:2'::tsvector) and as the fallback path for
the PL/pgSQL assignment lexer.
Concat
Standard SQL string concatenation ||.
Pipe
Bitwise OR | (single pipe — || lexes as Concat first).
Amp
Bitwise AND & (single amp — && lexes as InetOverlap first).
Tilde
Bitwise NOT ~ (prefix); regex match in binary position.
TildeStar
Case-insensitive regex match ~*.
NotTilde
Negated regex match !~.
NotTildeStar
Negated case-insensitive regex match !~*.
DoubleTilde
LIKE operator ~~ (PG’s operator form of LIKE).
DoubleTildeStar
ILIKE operator ~~* (case-insensitive LIKE).
NotDoubleTilde
NOT LIKE operator !~~.
NotDoubleTildeStar
NOT ILIKE operator !~~*.
Caret
Power operator ^.
CaretAt
Starts-with operator ^@ (PG 11+).
Hash
Integer XOR operator #.
Adjacent
Range “is adjacent to” operator -|-.
DoubleBang
tsquery prefix negation operator !!.
Is
IS keyword — postfix IS NULL / IS NOT NULL predicates.
Between
In
Like
Group
Distinct
Union
All
Join
Inner
Left
Cross
Outer
Right
Full
Default
Savepoint
Release
To
Having
Show
Extract
Offset
Asc
Desc
Interval
INTERVAL — followed by a string literal carrying the span text
(e.g. INTERVAL '1 day 2 hours').
Placeholder(u16)
v6.1.1 — $N parameter placeholder for the extended query
protocol. The number N is 1-based per PostgreSQL convention.
0 and $0 are not valid; the lexer rejects them.
Drop
v6.1.2 — DROP keyword. Used by DROP PUBLICATION <name>.
Reserved for future DROP TABLE / DROP INDEX / DROP USER
surface that currently goes through SHOW-shaped admin SQL.
For
v6.1.2 — FOR keyword (publication scope).
Tables
v6.1.2 — TABLES plural keyword (FOR ALL TABLES,
FOR ALL TABLES EXCEPT …). The existing TABLE keyword
stays a separate token so CREATE TABLE’s single-table
form keeps lexing as today.
Except
v6.1.3 (reserved at v6.1.2 to keep the AST shape stable) —
EXCEPT keyword for FOR ALL TABLES EXCEPT t1, t2.
Publication
v6.1.2 — PUBLICATION keyword.
Subscription
v6.1.4 (reserved at v6.1.2) — SUBSCRIPTION keyword.
Connection
v6.1.4 — CONNECTION keyword (for
CREATE SUBSCRIPTION … CONNECTION '<conn_str>' …).
Partition
v7.37.6-B(sentori Epic 2 P0)— PARTITION keyword. Drives
both CREATE TABLE p (…) PARTITION BY RANGE (key) (declarative
parent) and CREATE TABLE c PARTITION OF p FOR VALUES FROM (a) TO (b) | DEFAULT (child). OF / MINVALUE / MAXVALUE
stay PG-context-sensitive identifiers — the parser matches them
as case-insensitive Token::Ident strings off the back of this
reserved keyword, mirroring how INSERT … RETURNING handles
RETURNING without burning a global keyword slot.