Struct antlr_rust::lexer::BaseLexer [−][src]
pub struct BaseLexer<'input, T: LexerRecog<'input, Self> + 'static, Input: CharStream<TF::From>, TF: TokenFactory<'input> = CommonTokenFactory> {
pub interpreter: Option<Box<LexerATNSimulator>>,
pub input: Option<Input>,
pub token_start_char_index: isize,
pub token_start_line: isize,
pub token_start_column: isize,
pub token_type: isize,
pub token: Option<TF::Tok>,
pub channel: isize,
pub mode: usize,
pub text: Option<<TF::Data as ToOwned>::Owned>,
// some fields omitted
}
Expand description
Default implementation of Lexer
Public fields in this struct are intended to be used by embedded actions
Fields
interpreter: Option<Box<LexerATNSimulator>>
LexerATNSimulator
instance of this lexer
input: Option<Input>
CharStream
used by this lexer
token_start_char_index: isize
token_start_line: isize
token_start_column: isize
token_type: isize
Overrides token type emitted by lexer for current token
token: Option<TF::Tok>
Make it Some
to override token that is currently being generated by lexer
channel: isize
Channel lexer is currently assigning tokens to
mode: usize
Mode lexer is currently in
text: Option<<TF::Data as ToOwned>::Owned>
Make it Some
to override text for token that is currently being generated by lexer
Implementations
impl<'input, T, Input, TF> BaseLexer<'input, T, Input, TF> where
T: LexerRecog<'input, Self> + 'static,
Input: CharStream<TF::From>,
TF: TokenFactory<'input>,
impl<'input, T, Input, TF> BaseLexer<'input, T, Input, TF> where
T: LexerRecog<'input, Self> + 'static,
Input: CharStream<TF::From>,
TF: TokenFactory<'input>,
Current position in input stream
Used from lexer actions to override text of the token that will be emitted next
Add error listener
Remove and drop all error listeners
pub fn new_base_lexer(
input: Input,
interpreter: LexerATNSimulator,
recog: T,
factory: &'input TF
) -> Self
pub fn new_base_lexer(
input: Input,
interpreter: LexerATNSimulator,
recog: T,
factory: &'input TF
) -> Self
Creates new lexer instance
Trait Implementations
impl<'input, T, Input, TF> Deref for BaseLexer<'input, T, Input, TF> where
T: LexerRecog<'input, Self> + 'static,
Input: CharStream<TF::From>,
TF: TokenFactory<'input>,
impl<'input, T, Input, TF> Deref for BaseLexer<'input, T, Input, TF> where
T: LexerRecog<'input, Self> + 'static,
Input: CharStream<TF::From>,
TF: TokenFactory<'input>,
impl<'input, T, Input, TF> DerefMut for BaseLexer<'input, T, Input, TF> where
T: LexerRecog<'input, Self> + 'static,
Input: CharStream<TF::From>,
TF: TokenFactory<'input>,
impl<'input, T, Input, TF> DerefMut for BaseLexer<'input, T, Input, TF> where
T: LexerRecog<'input, Self> + 'static,
Input: CharStream<TF::From>,
TF: TokenFactory<'input>,
impl<'input, T, Input, TF> Lexer<'input> for BaseLexer<'input, T, Input, TF> where
T: LexerRecog<'input, Self> + 'static,
Input: CharStream<TF::From>,
TF: TokenFactory<'input>,
impl<'input, T, Input, TF> Lexer<'input> for BaseLexer<'input, T, Input, TF> where
T: LexerRecog<'input, Self> + 'static,
Input: CharStream<TF::From>,
TF: TokenFactory<'input>,
type Input = Input
type Input = Input
Concrete input stream used by this parser
Same as TokenStream::get_input_stream
but returns concrete type instance
important for proper inlining in hot code of LexerATNSimulator
Read more
Sets channel where current token will be pushed Read more
Pushes current mode to internal mode stack and sets m
as current lexer mode
`pop_mode should be used to recover previous mode Read more
Sets type of the current token Called from action to override token that will be emitted by lexer Read more
Used to informs lexer that it should consider next token as a continuation of the current one
impl<'input, T, Input, TF> TokenSource<'input> for BaseLexer<'input, T, Input, TF> where
T: LexerRecog<'input, Self> + 'static,
Input: CharStream<TF::From>,
TF: TokenFactory<'input>,
impl<'input, T, Input, TF> TokenSource<'input> for BaseLexer<'input, T, Input, TF> where
T: LexerRecog<'input, Self> + 'static,
Input: CharStream<TF::From>,
TF: TokenFactory<'input>,
type TF = TF
type TF = TF
TokenFactory this token source produce tokens with
Return a {@link Token} object from your input stream (usually a {@link CharStream}). Do not fail/return upon lexing error; keep chewing on the characters until you get a good one; errors are not passed through to the parser. Read more
Get the line number for the current position in the input stream. The first line in the input is line 1. Read more
Get the index into the current line for the current position in the input stream. The first character on a line has position 0. Read more
Returns underlying input stream
Returns string identifier of underlying input e.g. file name
Gets the TokenFactory
this token source is currently using for
creating Token
objects from the input. Read more