RegexSolver
The regex crate tells you whether a string matches a pattern. RegexSolver treats patterns as the sets of strings they match — so you can intersect, subtract, compare, complement, and enumerate them, and get the result back as a regex.
use ;
let a: Term = "(ab|xy){2}".parse?;
let b: Term = ".*xy".parse?;
// Which strings match BOTH patterns? Get the answer as a regex:
let both = a.intersection?;
assert_eq!;
// Test a concrete string against the result (matching is anchored):
assert!;
// ...and sample them:
assert_eq!;
What would you use this for?
- Safe migrations -
old_rule.subset(&new_rule)?: does the new validation pattern accept everything the old one did? - Test-data generation -
term.generate_strings(100, 0, (PathOrder::Shuffled, CharacterOrder::Shuffled))?: produce realistic-looking strings matching any pattern, spread over the cases the pattern allows, reproducible by seed, restricted to the characters you can use (with_charset) and to a band of lengths (with_min_length/with_max_length), with pagination. - Rule analysis: find shadowed or overlapping routes, firewall rules, and validators with
intersection/difference. - Equivalence proofs -
a.equivalent(&b)?: show that two differently-written patterns match exactly the same strings. - Pattern simplification: every operation returns a
Termyou can turn back into a regex pattern withto_pattern().
Under the hood, every pattern compiles to a finite automaton:
Try it
&&
# How do two patterns relate? (equivalence, subsets, intersection, differences)
# Generate n sample strings matching a pattern
Or in your own project:
By default the parallel feature is enabled: automaton-backed unions/intersections with more than 3 argument operands and parts of the automaton-to-regex conversion run on rayon (purely regex-backed operations stay sequential). Disable it for a leaner dependency tree on single-threaded workloads:
= { = "1", = false }
Semantics in 30 seconds
RegexSolver implements pure regular languages, which differs from typical regex engines in two ways:
- Everything is anchored:
abcmatches the string "abc", not "xabc" or "abcx". Patterns describe whole strings. .matches any character, including line feed (\n).
The rest follows from regular-language theory:
- Backreferences (
\1,\2, ...) go beyond regular languages and return an error, as do lookahead/lookbehind assertions ((?=...),(?<=...)). - Anchors and word boundaries: since matching is already full-string, a leading
^/\Aand a trailing$/\zare accepted as redundant no-ops. Anchors anywhere else, and word boundaries (\b,\B), would constrain matching in ways a pure regular language can't express, so they returnEngineError::UnsupportedRegexFeaturerather than silently changing the language. - Inline flags (
(?i),(?m),(?s),(?x)) returnEngineError::UnsupportedRegexFeature: the engine matches character ranges uniformly and can't honor them, and silently dropping them would diverge from standard regex semantics (e.g.(?i)abcwould no longer matchABC). - All quantifiers are greedy: ungreedy markers (
*?,+?,??) are ignored as sets of strings,a*anda*?are the same language. - The empty language (matches no string at all) is written
[](empty character class). This is distinct from the empty string"".
RegexSolver is based on the regex-syntax library for parsing patterns. Features that don't affect the language as a set of strings (such as ungreedy markers) are accepted and ignored; features that would change matching in a way the engine can't represent (backreferences, lookaround, inline flags, and unsupported anchor/boundary positions) return an EngineError instead of being applied incorrectly.
A tour of the API
Term is the type you'll interact with: it wraps either a regular expression or an automaton and picks the best representation for each operation. The essentials:
| Method | Description |
|---|---|
Term::from_pattern(pattern) |
Parses a pattern into a term. |
intersection(&self, terms) / union(&self, terms) |
Set operations over any number of terms. |
difference(&self, other) / complement(&self) |
What self matches and other doesn't / everything self doesn't match. |
concat(&self, terms) / repeat(&self, range) |
Sequence and repeat languages; range is any Rust range expression (2..=5, 1.., ..3, ...). |
equivalent(&self, other) / subset(&self, other) |
Compare languages. |
is_empty() / is_total() / length() / cardinality() |
Analyze a language: matches nothing? everything? string lengths? how many strings? |
generate_strings(limit, offset, options) |
Enumerate matching strings eagerly (call determinize() or minimize() once first when paginating). |
iter_strings(options) |
Lazy iterator equivalent; computes the deterministic automaton once and yields strings in batches. options.with_min_length(n)/.with_max_length(n) confine the walk to a band of lengths — with a max, even an infinite language yields a finite iterator. |
to_pattern() / to_automaton() / to_regex() |
Convert back out. |
All fallible operations return Result<_, EngineError>.
Building automata by hand
FastAutomaton is used to directly build, manipulate and analyze automata. To convert an automaton to a RegularExpression the method to_regex() can be used.
States are created with new_state() and transitions with add_transition_from_range, which labels the transition with a plain CharRange:
use CharRange;
use FastAutomaton;
use Char;
// Build an automaton matching "[a-c][0-9]*" by hand:
let mut automaton = new_empty;
let s1 = automaton.new_state;
automaton.accept;
let a_to_c = new_from_range;
let digits = new_from_range;
automaton.add_transition_from_range?;
automaton.add_transition_from_range?;
assert!;
assert!;
assert_eq!;
Internally, transition labels are bitvector Conditions over the automaton's SpanningSet of disjoint character ranges, that is what makes label union/intersection/complement O(1) (article). add_transition_from_range maintains that representation for you; for full manual control over conditions and spanning sets, see the add_transition documentation.
Everything Term does is also available directly on FastAutomaton, including determinize, minimize, the set operations, equivalent/subset, the analyses, generate_strings, to_regex, plus low-level construction (new_state, accept, add_epsilon_transition, ...) and inspection (states, transitions_from, to_dot, ...).
Working with patterns as ASTs
RegularExpression is the parsed pattern itself: a plain AST enum (Character / Repetition / Concat / Alternation) you can analyze and walk directly. Set operations like intersection and difference live on FastAutomaton (or, more conveniently, on Term); convert with to_automaton().
use RegularExpression;
// A validation pattern for an order id, e.g. "ORD-2024-12345".
let pattern = new?;
// How long can matching ids get? Size your database column accordingly.
assert_eq!;
// The AST is a plain enum: walk it to lint patterns, e.g. reject
// validation rules that accept unboundedly long input.
assert!;
assert!;
The variants are freely constructible too; a hand-built repetition whose maximum is below its minimum denotes no valid language and is rejected with EngineError::InvalidRepetitionBounds when converted by to_automaton().
Parsing (new, parse), the simplifying combinators (concat, union, repeat, simplify) and the analyses (length, cardinality, evaluate_complexity) are documented on RegularExpression.
Bound Execution
Automaton operations can blow up on adversarial inputs, so the engine is built to run untrusted patterns safely: a thread-local ExecutionProfile caps runtime and state explosion, and controls when the engine may determinize or minimize on its own. Hitting a limit returns a specific EngineError instead of hanging or panicking.
Time-Bounded Execution
use ;
let term = from_pattern?;
let execution_profile = new
.execution_timeout // limit in milliseconds
.build;
// Asking for 100 million strings cannot finish within the budget, so the
// generation aborts instead of running to completion.
execution_profile.run;
State-Limited Execution
use ;
let term1 = from_pattern?;
let term2 = from_pattern?;
let execution_profile = new
.max_number_of_states // we set the limit
.build;
// We run the operation with the defined limitation
execution_profile.run;
Disabling Implicit Determinization
FastAutomaton operations that require a deterministic automaton (minimize, complement, difference, equivalent, subset, cardinality, ...) determinize a non-deterministic input on their own by default. Since subset construction can blow up exponentially, this can be disabled: those operations then return EngineError::DeterministicAutomatonRequired instead, and determinization only happens through an explicit determinize() call. Deterministic inputs are always accepted, and the whole Term API keeps working since that layer manages the underlying representation itself, so its determinizations count as explicit.
use Term;
use ExecutionProfileBuilder;
use EngineError;
// Any non-deterministic FastAutomaton; ".*abc" compiles to one.
let nfa = from_pattern?.to_automaton?.into_owned;
assert!;
let execution_profile = new
.implicit_determinization // default is true
.build;
execution_profile.run;
How it works
- Patterns are parsed with regex-syntax and simplified into a small regular-expression AST; set operations run on finite automata; results convert back to patterns via state elimination.
- Transition labels are bitvectors over a per-automaton "spanning set" of disjoint character ranges, making label union/intersection/complement O(1): see Optimizing Automaton Representation with Transition Conditions.
- Correctness is cross-validated against the
regexcrate and exercised by property-based tests over randomly generated automata and expressions, with brute-force oracles for the analyses.
Cross-Language Support
If you want to use this library with other programming languages, we provide a wide range of wrappers:
For more information about how to use the wrappers, you can refer to our guide.
License
This project is licensed under the MIT License.