Trait antlr_rust::TokenSource

source · [−]
pub trait TokenSource<'input> {
    type TF: TokenFactory<'input> + 'input;

    fn next_token(&mut self) -> <Self::TF as TokenFactory<'input>>::Tok;
    fn get_input_stream(&mut self) -> Option<&mut dyn IntStream>;
    fn get_source_name(&self) -> String;
    fn get_token_factory(&self) -> &'input Self::TF;

    fn get_line(&self) -> isize { ... }
    fn get_char_position_in_line(&self) -> isize { ... }
}
Expand description

Produces tokens to be used by parser. TokenStream implementations are responsible for buffering tokens for parser lookahead

Required Associated Types

TokenFactory this token source produce tokens with

Required Methods

Return a {@link Token} object from your input stream (usually a {@link CharStream}). Do not fail/return upon lexing error; keep chewing on the characters until you get a good one; errors are not passed through to the parser.

Returns underlying input stream

Returns string identifier of underlying input e.g. file name

Gets the TokenFactory this token source is currently using for creating Token objects from the input.

Required by Parser for creating missing tokens.

Provided Methods

Get the line number for the current position in the input stream. The first line in the input is line 1.

Returns the line number for the current position in the input stream, or 0 if the current token source does not track line numbers.

Get the index into the current line for the current position in the input stream. The first character on a line has position 0.

Returns the line number for the current position in the input stream, or -1 if the current token source does not track character positions.

Implementations on Foreign Types

Implementors