Module anysexpr::parse

Translating a character stream to a token stream. This is (currently) called “parser” because it fully parses atoms (like strings, symbols, etc.), thus “tokenizer” may be selling it short (?). The only tokens denoting nesting are Token::Open and Token::Close. See read if interested in trees rather than atoms / tokens.

Structs

• ParseErrorWithPos
• TokenWithPos

Enums

• CommentStyle
• Context
• ParseError
• Token

Functions

• maybe_open_close
  Possibly return opening or closing token for a given character.
• parse
  Parse a stream of characters and their positions into a stream of tokens (atoms or opening/closing tokens).