Skip to main content

Module lexer

Module lexer 

Source
Expand description

Tokenizer for PDF object syntax (ISO 32000-2 §7.2 lexical conventions).

The lexer turns a byte slice into Tokens. It is deliberately dumb: it knows nothing about indirect objects, streams or the object graph, so the same tokens serve crate::parser, the content-stream reader and the cross-reference reader.

Structs§

Lexer
A cursor over PDF tokens.

Enums§

Token
One lexical unit of a PDF file.

Functions§

is_delimiter
Delimiters: ( ) < > [ ] { } / % (§7.2.2).
is_regular
Regular characters: everything that is neither whitespace nor a delimiter.
is_whitespace
Whitespace: NUL, HT, LF, FF, CR and SP (§7.2.2).