Crate antlr4rust

Antlr4 runtime

This is a Rust runtime for ANTLR4 parser generator. It is required to use parsers and lexers generated by ANTLR4 parser generator

This documentation refers to particular api used by generated parsers,lexers and syntax trees.

For info on what is ANTLR4 and how to generate parser please refer to:

• ANTLR4 main repository
• README for Rust target

Customization

All input and output can be customized and optimized for particular usecase by implementing related trait. Each of them already has different implementations that should be enough for most cases. For more details see docs for corresponding trait and containing module.

Currently available are:

• CharStream - Lexer input, stream of char values with slicing support
• TokenFactory - How lexer creates tokens.
• Token - Element of TokenStream
• TokenStream - Parser input, created from lexer or other token source.
• ParserRuleContext - Node of created syntax tree.

Zero-copy and lifetimes

This library supports full zero-copy parsing. To allow this 'input lifetime is used everywhere inside to refer to data borrowed by parser/lexer. Besides references to input it also can be TokenFactory if it returns references to tokens. See ArenaFactory as an example of such behavior. It allocates tokens in Arena and returns references.

Using generated parse tree you should be careful to not require longer lifetime after the parsing. If that’s the case you will likely get “does not live long enough” error on the input string, despite actual lifetime conflict is happening much later

If you need to generate owned versions of parse tree or you want simpler usage, you can opt out zero-copy by requiring 'input to be static. In this case it is easier to also use types that contains “owned” in their name or constructor function like OwningTokenFactory or InputStream::new_owned().

Visitors and Listeners

Parse listeners must outlive 'input because they have to be stored inside of the parser. It still allows to retrieve borrowed data from parse tree which should be enough to cover 99% use cases.

ParseTreeWalker can accept listeners with arbitrary lifetime.

Visitors also can have arbitrary lifetime.

Downcasting

Rule context trait object support downcasting even for zero-copy case. Also generic types(currently these are H:ErrorStrategy and I:TokenStream) that you can access in generated parser from embedded actions also can be downcasted to concrete types. To do it TidExt::downcast_* extension methods should be used.

Re-exports

pub use input_stream::InputStream;

Modules

char_stream

IntStream extension for Lexer that allows subslicing of underlying data

common_token_stream

Channel based TokenStream

error_listener

Error reporting

error_strategy

Error handling and recovery

errors

Error types

input_stream

Input to lexer

int_stream

<ost generic stream of symbols

lexer

Lexer implementation

lexer_atn_simulator

Implementation of lexer automata(DFA)

parser

Base parser implementation

parser_atn_simulator

Base parser implementation

parser_rule_context

Full parser node

rule_context

Minimal parser node

token

Symbols that parser works on

token_factory

How Lexer should produce tokens

token_stream

IntStream that produces tokens for Parser

tree

General AST

trees

A set of utility routines useful for all kinds of ANTLR trees.

vocabulary

Mapping from symbol type to its string representation

Structs

BailErrorStrategy

This implementation of ANTLRErrorStrategy responds to syntax errors by immediately canceling the parse operation with a ParseCancellationException. The implementation ensures that the ParserRuleContext.exception field is set for all parse tree nodes that were not completed prior to encountering the error.

BaseLexer

Default implementation of Lexer

BaseParser

Main underlying Parser struct

DefaultErrorStrategy

This is the default implementation of ErrorStrategy used for error reporting and recovery in ANTLR parsers.

ListenerId

Allows to safely cast listener back to user type

Enums

PredictionMode

This enum defines the prediction modes available in ANTLR 4 along with utility methods for analyzing configuration sets for conflicts and/or ambiguities.

Traits

CoerceFrom

Stable workaround for CoerceUnsized

CoerceTo

Stable workaround for CoerceUnsized

ErrorStrategy

The interface for defining strategies to deal with syntax errors encountered during a parse by ANTLR-generated parsers. We distinguish between three different kinds of errors:

Lexer

Lexer functionality required by LexerATNSimulator to work properly

Parser

parser functionality required for ParserATNSimulator to work

TokenSource

Produces tokens to be used by parser. TokenStream implementations are responsible for buffering tokens for parser lookahead