Struct PatternSet 

pub struct PatternSet { /* private fields */ }

A set of patterns, asked together.

Implementations

impl PatternSet

pub fn new(pats: Vec<Pattern>) -> Self

A set over pats, in the order given. That order is the index every answer is reported under.

pub fn named(pats: Vec<Pattern>, names: Vec<String>) -> Self

A set over pats with a name each, in the order given; a name short of the count is the member’s index as text.

pub fn from_text( text: &str, shapes: &mut ShapeSet, ) -> Result<PatternSet, ShapeError>

The members a pattern file declares, into shapes: a let NAME = PATTERN line is a member under its name, and any line that is not a declaration is a member under its line number, parsed against the declarations so far; kind, shape, shape-after and test lines are declared as ShapeSet::declare_text declares them, so the file’s kinds and shapes serve the members and lib --test checks them. Every member is lexed under the file’s declarations, as a scan under --lib is.

Errors

The first line that is neither a declaration nor a pattern, or whose declaration is refused, with its line number in the message.

pub fn from_file( path: &Path, shapes: &mut ShapeSet, ) -> Result<PatternSet, ShapeError>

Self::from_text over the pattern file at path, a relative @file set in it read from beside the file.

Errors

The file cannot be read, or a line of it is refused.

pub fn under(self, shapes: ShapeSet) -> Self

This set with its members lexed under shapes, as a set built from a pattern file is lexed under the file’s declarations.

pub fn names(&self) -> &[String]

The members’ names, one each, where the set was built from a pattern file, and nothing where it was built from patterns alone.

pub fn name(&self, i: usize) -> String

The name the member at i goes by: its name where the set has them, else its index as text.

pub fn shapes(&self) -> &ShapeSet

The declarations the members are lexed under.

pub fn scan(&self, input: &[u8]) -> Vec<(usize, Span)>

Every match of every member: each member’s leftmost, non-overlapping matches over input, tagged with the member’s index and ordered by position, then by member. The members a byte route answers never reach the lexer; the rest share one lex and each is walked over it as itself, so every span keeps the member that made it. A set built from a pattern file lexes every member under the file’s declarations, since a shape decides boundaries a byte route never sees.

pub fn scan_from(&self, input: &[u8], at: usize) -> Vec<(usize, Span)>

Self::scan from the first token starting at or after byte at, each member’s leftmost, non-overlapping selection re-run from there.

pub fn scan_from_over( &self, input: &[u8], at: usize, toks: &[Token], ) -> Vec<(usize, Span)>

Self::scan_from over a lex of input the caller already holds.

A streaming push lexes its retained buffer before it scans, and a set scanning the same bytes would otherwise lex them again - measured at 7.3 MB in 57 pushes, the stream’s lex ran 57 times and the set’s 56 more over the same buffer. Lending the tokens removes the second.

toks must be a lex of the whole of input taken under no declared shapes, which is what crate::lexer::lex_into gives. A set that declares shapes ignores them and lexes under its own, since those decide boundaries the caller’s lex never saw.

pub fn scan_matches(&self, input: &[u8], lists: bool) -> Vec<(usize, Match)>

Self::scan with each match’s registers resolved under its own member’s names, over the lex the member was found on where it shared one, so a set of binding members pays no lex beyond the scan’s; under lists, with every binding a register made under a repetition, as crate::captures_with_lists resolves them.

pub fn first_matches(&self, input: &[u8], lists: bool) -> Vec<(usize, Match)>

Each member’s first match over input, with its registers resolved as Self::scan_matches resolves them, ordered by position, then by member. A member stops at its first match and the lexer stops with it: a byte route reads no token, a member that lexes alone reads through a cursor that lexes as it goes, and the rest share one prefix that widens only while a member is still open. A set built from a pattern file with declarations lexes every member under them.

pub fn max_tokens(&self) -> Option<usize>

The most tokens a match of any member can span, for a set every member of which is bounded: what a stream over the set commits under.

pub fn depends_on_whole_input(&self) -> bool

Whether any member depends on input outside a single match span, so a stream over the set commits nothing before its end.

pub fn depends_on_more_than_its_lines(&self) -> bool

Whether any member depends on input beyond the lines its match spans, so a stream over the set, which cuts only just after a newline, commits nothing before its end.

pub fn reads_whitespace(&self) -> bool

Whether any member reads whitespace, so a stream over the set cannot take a line’s end as a boundary.

pub fn probed(self, on: bool) -> Self

Whether the literals the members require are probed once per input through a filter over its n-grams, or searched for once per member, which is how a set is built: measured on a thousand members over 2 MiB, the filter cost 40 ms more than the searches when the literals were present and saved 13 ms when they were absent. The answers are the same either way; a caller whose lists are mostly absent turns the probe on.

pub fn walked_as_one(self, on: bool) -> Self

Whether the members the single-pass engine takes are walked as one program over the shared lex, which is how a set is built, or each as itself. The answers are the same either way; this is the switch the two forms are timed against each other through.

pub fn with_names(self, names: Vec<String>) -> Self

This set with the members’ names replaced.

pub fn len(&self) -> usize

How many patterns the set holds.

pub fn is_empty(&self) -> bool

Whether the set holds none.

pub fn patterns(&self) -> &[Pattern]

The patterns themselves, in index order.

pub fn matches(&self, input: &[u8]) -> Vec<usize>

The indices of the patterns that match input, in index order.

Every pattern is asked. A route that answers without the lexer answers first and costs nothing; what is left shares one lex.

pub fn is_match(&self, input: &[u8]) -> bool

Whether any pattern in the set matches input.

Stops at the first that does, so a set whose early members are byte-routable can answer without lexing even when later ones would have needed it, and a set that must lex stops widening its prefix the moment one member matches.

pub fn matched(&self, input: &[u8]) -> SetMatches

Which patterns match input, as a bitset over the set’s indices.

The counterpart of the regex crate’s matches, which returns its SetMatches. Self::matches answers the same question as a list of the indices that matched; this reports every index with its verdict, so a caller asking about one pattern does not scan a list to find it.

pub fn matches_at(&self, input: &[u8], at: usize) -> SetMatches

Which patterns match at or after byte at, as a bitset.

The counterpart of the regex crate’s matches_at. The bytes before at are still read, so an assertion that looks back sees them.

This lexes once and whole rather than widening the prefix Self::matches shares. A prefix grows forward from the start of the input, which is the wrong shape for a question anchored partway through it: the bytes a member would settle from are the ones already passed.

pub fn is_match_at(&self, input: &[u8], at: usize) -> bool

Whether any pattern in the set matches at or after byte at.

pub fn matches_with_spans(&self, input: &[u8]) -> Vec<(usize, Span)>

Which patterns match input, and where each one first does.

The regex crate’s RegexSet cannot answer this: it reports which patterns match and states that it does not report where. The reason is that its saving comes from carrying every pattern in one automaton, which loses the identity of the pattern that reached an accepting state.

A token set’s saving is the shared lex rather than a shared automaton, so each member is still walked as itself and its match keeps its span. The position costs nothing beyond the walk that decided the verdict.

pub fn matched_all(&self, input: &[u8]) -> bool

Whether every pattern in the set matches input.

Trait Implementations

impl Clone for PatternSet

fn clone(&self) -> Self

The same members, names and declarations; the union is built again on first use.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for PatternSet

fn fmt(&self, f: &mut Formatter<'_>) -> Result

The members and their names; the union built over them is not shown.