Struct Scanner 

pub struct Scanner { /* private fields */ }

Multi-pattern scanner compatible with vscode-oniguruma’s OnigScanner.

Example

use ferroni::scanner::{Scanner, ScannerFindOptions};

let mut scanner = Scanner::new(&["\\d+", "[a-z]+"]).unwrap();
let m = scanner.find_next_match("hello42", 0, ScannerFindOptions::NONE).unwrap();
assert_eq!(m.index, 1); // "[a-z]+" matched first
assert_eq!(m.captures()[0].start, 0);
assert_eq!(m.captures()[0].end, 5);

Implementations

impl Scanner

pub fn new(patterns: &[&str]) -> Result<Scanner, RegexError>

Create a scanner from a list of pattern strings using the vscode-oniguruma defaults (Oniguruma syntax and capture groups enabled).

Enabling capture groups preserves unnamed captures and numbered backreferences when a pattern also contains named groups. Pass an explicit ScannerConfig to select different compile-time options.

pub fn with_config( patterns: &[&str], config: &ScannerConfig, ) -> Result<Scanner, RegexError>

Create a scanner with custom configuration (syntax and compile-time options).

Example

use ferroni::scanner::{Scanner, ScannerConfig, ScannerSyntax, ScannerFindOptions};
use ferroni::oniguruma::OnigOptionType;

let config = ScannerConfig::default()
    .options(OnigOptionType::IGNORECASE)
    .syntax(ScannerSyntax::Oniguruma);
let mut scanner = Scanner::with_config(&["hello"], &config).unwrap();
let m = scanner.find_next_match("HELLO", 0, ScannerFindOptions::NONE);
assert!(m.is_some());

pub fn with_backtracking_optimization( patterns: &[&str], config: &ScannerConfig, ) -> Result<Scanner, RegexError>

Create a scanner with the conservative, experimental AST rewrites described by crate::api::RegexBuilder::optimize_backtracking. Successful captures and pattern priority are preserved; retry, stack, and timeout outcomes may differ. See Scanner::backtracking_rewrites.

pub fn with_pattern_cache( patterns: &[&str], config: &ScannerConfig, cache: &mut ScannerPatternCache, ) -> Result<Scanner, RegexError>

Scanner::with_config, sharing compiled patterns through cache.

A pattern the cache already holds for this config is reused without compiling it again; the others are compiled and added. The scanner behaves exactly like one from Scanner::with_config. If construction fails (a pattern does not compile, or the set is refused), the error is the one Scanner::with_config returns and the cache is left as it was. See ScannerPatternCache for lifetime and threads.

Example

use ferroni::scanner::{Scanner, ScannerConfig, ScannerPatternCache};

let config = ScannerConfig::default();
let mut cache = ScannerPatternCache::new();
let rules: [&[&str]; 3] = [&[r"\d+", r"\w+"], &[r"\w+", r"\s+"], &[r"\d+", r"\s+"]];
let scanners: Vec<Scanner> = rules
    .iter()
    .map(|patterns| Scanner::with_pattern_cache(patterns, &config, &mut cache))
    .collect::<Result<_, _>>()
    .unwrap();
// Six patterns, three of them distinct, each compiled once.
assert_eq!((scanners.len(), cache.len()), (3, 3));

// A pattern that does not compile leaves the cache unchanged.
assert!(Scanner::with_pattern_cache(&["[a-z]+", "("], &config, &mut cache).is_err());
assert_eq!(cache.len(), 3);

pub fn with_backtracking_optimization_and_pattern_cache( patterns: &[&str], config: &ScannerConfig, cache: &mut ScannerPatternCache, ) -> Result<Scanner, RegexError>

Scanner::with_backtracking_optimization, sharing compiled patterns through cache as Scanner::with_pattern_cache does. Patterns compiled with and without the optimization are cached separately.

pub fn warnings(&self) -> &[Vec<BacktrackWarning>]

Findings of the compile-time backtracking check, one list per pattern in the order they were given. Grammar loaders can surface these to grammar authors. See crate::backtrack_lint for limits.

use ferroni::scanner::Scanner;

let scanner = Scanner::new(&["ok+", "(a+)+$"]).unwrap();
assert!(scanner.warnings()[0].is_empty());
assert_eq!(scanner.warnings()[1].len(), 1);

pub fn backtracking_rewrites(&self) -> &[Vec<BacktrackingRewrite>]

Rewrite reports for each pattern, in the original pattern order. Unsupported shapes have no report; original warnings stay in Scanner::warnings, including for successfully rewritten patterns.

pub fn stats(&self) -> ScannerStats

Get current scanner counters.

pub fn prefilter_stats(&self) -> PrefilterStats

What the scanner’s DFA pre-filter covers (ScannerConfig::prefilter).

use ferroni::scanner::{Scanner, ScannerFindOptions};

let mut scanner = Scanner::new(&[r"\d+", r"[a-z]+", r"(?<=\))"]).unwrap();
let stats = scanner.prefilter_stats();
assert_eq!(stats.covered + stats.own, 3);
// The automata are built by the first search, not the constructor.
assert!(!stats.built);
scanner.find_next_match("f(x) 1", 0, ScannerFindOptions::NONE);
let stats = scanner.prefilter_stats();
// The look-behind reads nothing at its position, so that pattern is
// searched on its own. Without the `dfa-prefilter` feature, all are.
if stats.built {
    assert_eq!((stats.covered, stats.own), (2, 1));
}

pub fn reset_stats(&mut self)

Reset scanner counters.

pub fn find_next_match( &mut self, text: &str, start_position: usize, options: ScannerFindOptions, ) -> Option<ScannerMatch>

Find the next match starting at start_position (byte offset).

One-off searches (without a stable string ID) use the RegSet path. Use find_next_match_with_id to enable per-regex cache reuse when repeatedly advancing through the same string.

A search that stops at a process-wide limit (time, retry or stack) before reaching a match is reported as None. Unlike Regex::find_with, a scanner has no variant that reports the limit as an error.

pub fn find_next_match_with_id( &mut self, text: &str, str_id: u64, start_position: usize, options: ScannerFindOptions, ) -> Option<ScannerMatch>

Find the next match with a string ID for caching.

When searching the same string repeatedly (advancing start_position), pass the same str_id to enable cache hits that skip redundant searches.

A search that stops at a process-wide limit (time, retry or stack) before reaching a match is reported as None. Unlike Regex::find_with, a scanner has no variant that reports the limit as an error.

pub fn find_next_match_utf16( &mut self, string: &OnigString, start_position: usize, options: ScannerFindOptions, ) -> Option<ScannerMatch>

Find the next match using UTF-16 positions (for vscode-textmate/Shiki compatibility).

start_position is in UTF-16 code units. The returned CaptureIndex values (start, end, length) are also in UTF-16 code units.

A search that stops at a process-wide limit (time, retry or stack) before reaching a match is reported as None. Unlike Regex::find_with, a scanner has no variant that reports the limit as an error.

Example

use ferroni::scanner::{Scanner, ScannerFindOptions, OnigString};

let mut scanner = Scanner::new(&["Y", "X"]).unwrap();
let s = OnigString::new("a💻bYX");
// 💻 is 2 UTF-16 code units, so Y is at UTF-16 position 4
let m = scanner.find_next_match_utf16(&s, 0, ScannerFindOptions::NONE).unwrap();
assert_eq!(m.captures()[0].start, 4);
assert_eq!(m.captures()[0].end, 5);

pub fn find_next_match_utf16_with_id( &mut self, string: &OnigString, str_id: u64, start_position: usize, options: ScannerFindOptions, ) -> Option<ScannerMatch>

Find the next match using UTF-16 positions with a string ID for caching.

A search that stops at a process-wide limit (time, retry or stack) before reaching a match is reported as None. Unlike Regex::find_with, a scanner has no variant that reports the limit as an error.

Trait Implementations

impl Debug for Scanner

Shows the pattern count and the settings the patterns were compiled with.

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.