pub enum TokenKind<'a> {
Integer(i64),
Real(f64),
Name(Vec<u8>),
LiteralString(Vec<u8>),
HexString(Vec<u8>),
Keyword(&'a [u8]),
ArrayStart,
ArrayEnd,
DictStart,
DictEnd,
}Expand description
Token payload. References into the input slice are zero-copy.
Variants§
Integer(i64)
Integer (no ., no e/E). The bytes are the original
digits + optional sign.
Real(f64)
Real number (has .). Stored as the original string so the
parser can re-format with the same precision policy.
Name(Vec<u8>)
Name /foo — the bytes (without the leading /), already
#xx-decoded.
LiteralString(Vec<u8>)
Literal string (...). Inner bytes only, with PDF escape
sequences (\n, \r, \t, \\, \(, \), octal \nnn,
line-continuation \<EOL>) already decoded. Balanced parens
inside a literal string are preserved verbatim per §7.3.4.2.
HexString(Vec<u8>)
Hex string <...>. Bytes are the decoded payload (every
pair of hex digits produces one byte; an odd trailing nibble
is left-aligned per §7.3.4.3).
Keyword(&'a [u8])
Bare ASCII identifier — keywords like obj, endobj, R,
true, false, null, xref, trailer, startxref,
stream, endstream, plus content-stream operators (m,
l, c, cm, q, Q, f, f*, S, B, B*, n,
re, RG, rg, w, etc.). The parser dispatches on the
keyword text.
ArrayStart
[ — array start.
ArrayEnd
] — array end.
DictStart
<< — dictionary start.
DictEnd
>> — dictionary end.