Trait antlr_rust::TokenSource

pub trait TokenSource<'input> {
    type TF: TokenFactory<'input> + 'input;
    pub fn next_token(&mut self) -> <Self::TF as TokenFactory<'input>>::Tok;
    pub fn get_input_stream(&mut self) -> Option<&mut dyn IntStream>;
    pub fn get_source_name(&self) -> String;
    pub fn get_token_factory(&self) -> &'input Self::TF;

    pub fn get_line(&self) -> isize { ... }
    pub fn get_char_position_in_line(&self) -> isize { ... }
}

Produces tokens to be used by parser. TokenStream implementations are responsible for buffering tokens for parser lookahead

Associated Types

type TF: TokenFactory<'input> + 'input

TokenFactory this token source produce tokens with

Loading content...

Required methods

pub fn next_token(&mut self) -> <Self::TF as TokenFactory<'input>>::Tok

Return a {@link Token} object from your input stream (usually a {@link CharStream}). Do not fail/return upon lexing error; keep chewing on the characters until you get a good one; errors are not passed through to the parser.

pub fn get_input_stream(&mut self) -> Option<&mut dyn IntStream>

Returns underlying input stream

pub fn get_source_name(&self) -> String

Returns string identifier of underlying input e.g. file name

pub fn get_token_factory(&self) -> &'input Self::TF

Gets the TokenFactory this token source is currently using for creating Token objects from the input.

Required by Parser for creating missing tokens.

Loading content...

Provided methods

pub fn get_line(&self) -> isize

Get the line number for the current position in the input stream. The first line in the input is line 1.

Returns the line number for the current position in the input stream, or 0 if the current token source does not track line numbers.

pub fn get_char_position_in_line(&self) -> isize

Get the index into the current line for the current position in the input stream. The first character on a line has position 0.

Returns the line number for the current position in the input stream, or -1 if the current token source does not track character positions.

Loading content...

Implementations on Foreign Types

impl<'input, T> TokenSource<'input> for &mut T where
    T: TokenSource<'input>, 

type TF = T::TF

pub fn next_token(&mut self) -> <Self::TF as TokenFactory<'input>>::Tok

pub fn get_line(&self) -> isize

pub fn get_char_position_in_line(&self) -> isize

pub fn get_input_stream(&mut self) -> Option<&mut dyn IntStream>

pub fn get_source_name(&self) -> String

pub fn get_token_factory(&self) -> &'input Self::TF

Loading content...

Implementors

impl<'input, T, Input, TF> TokenSource<'input> for BaseLexer<'input, T, Input, TF> where
    T: LexerRecog<'input, Self> + 'static,
    Input: CharStream<TF::From>,
    TF: TokenFactory<'input>, 

type TF = TF

pub fn next_token(&mut self) -> <Self::TF as TokenFactory<'input>>::Tok

pub fn get_line(&self) -> isize

pub fn get_char_position_in_line(&self) -> isize

pub fn get_input_stream(&mut self) -> Option<&mut dyn IntStream>

pub fn get_source_name(&self) -> String

pub fn get_token_factory(&self) -> &'input TF

Loading content...