Struct Scanner

pub struct Scanner { /* private fields */ }

A Scanner. It consists of multiple DFAs that are used to search for matches.

Each DFA corresponds to a terminal symbol (token type) the lexer/scanner can recognize. The DFAs are advanced in parallel to search for matches. It further constists of at least one scanner mode. Scanners support multiple scanner modes. This feature is known from Flex as Start conditions and provides more flexibility by defining several scanners for several parts of your grammar. See https://www.cs.princeton.edu/~appel/modern/c/software/flex/flex.html#SEC11 for more information.

To create a scanner, you can use the ScannerBuilder to add scanner mode data. At least one scanner mode must be added to the scanner. This is usually the mode named INITIAL.

Implementations

impl Scanner

pub fn try_new(scanner_modes: Vec<ScannerMode>, use_nfa: bool) -> Result<Self>

Creates a new scanner. The scanner is created with the given scanner modes. The ScannerImpl is created from the scanner modes and the use_nfa flag determines if the scanner uses a DFAs or an NFAs for the pattern matching.

pub fn find_iter<'h>(&self, input: &'h str) -> FindMatches<'h>

Returns an iterator over all non-overlapping matches. The iterator yields a Match value until no more matches could be found.

pub fn log_compiled_automata_as_dot(&self, modes: &[ScannerMode]) -> Result<()>

Logs the compiled DFAs as a Graphviz DOT file with the help of the log crate. To enable debug output compliled DFA as dot file set the environment variable RUST_LOG to scnr::internal::scanner_impl=debug.

pub fn generate_compiled_automata_as_dot( &self, modes: &[ScannerMode], target_folder: &Path, ) -> Result<()>

Generates the compiled DFAs as a Graphviz DOT files. The DOT files are written to the target folder. The file names are derived from the scanner mode names and the index of the DFA.

Trait Implementations

impl Debug for Scanner

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl ScannerModeSwitcher for Scanner

fn current_mode(&self) -> usize

Returns the current scanner mode.

fn set_mode(&mut self, mode: usize)

Sets the current scanner mode.

A parser can explicitly set the scanner mode to switch to a different set of DFAs. Usually, the scanner mode is changed by the scanner itself based on the transitions defined in the active scanner mode.

fn mode_name(&self, index: usize) -> Option<&str>

Returns the name of the scanner mode with the given index. If the index is out of bounds, None is returned.