regex-charclass

Manipulate and convert regular expression character classes.

A character class is a set of char values. This crate stores one as a sorted collection of non-overlapping inclusive ranges, gives it the usual set algebra, and renders it back as the shortest character class it can find — a named Unicode property where one matches, and a bracketed class otherwise.

The set is an irange::RangeSet. RangeSet, AnyRange and Char are re-exported at the crate root, so a dependent needs neither its own dependency on irange nor a version kept in step with this crate's. The whole crate is also re-exported as regex_charclass::irange.

Installation

[dependencies]
regex-charclass = "1.1"

The optional serde feature implements Serialize and Deserialize for a set of char:

[dependencies]
regex-charclass = { version = "1.1", features = ["serde"] }

Example

use regex_charclass::{AnyRange, Char, CharacterClass, RangeSet};

let letters = RangeSet::new_from_range_char('a'..='z');
assert_eq!(26, letters.get_cardinality());
assert_eq!("[a-z]", letters.to_regex());

let hex = RangeSet::new_from_ranges(&[
    AnyRange::from(Char::new('0')..=Char::new('9')),
    AnyRange::from(Char::new('A')..=Char::new('F')),
    AnyRange::from(Char::new('a')..=Char::new('f')),
]);
// A set matching a named Unicode property is rendered as that property.
assert_eq!("\\p{ASCII_Hex_Digit}", hex.to_regex());
assert_eq!("\\P{ASCII_Hex_Digit}", hex.complement().to_regex());

assert_eq!(".", hex.union(&hex.complement()).to_regex());
assert_eq!("[]", hex.intersection(&hex.complement()).to_regex());
assert_eq!("[g-z]", letters.difference(&hex).to_regex());

The generated regular expression

to_regex returns the shortest form it finds for a set, in this order:

Three of them need care from whoever reads them back:

• [] is a placeholder for the empty set, not a pattern. An empty bracketed class is a parse error in every engine, so a set that may be empty has to be handled before the output reaches one. [^\s\S] is the usual pattern for a class matching nothing.
• . stands for the total set, which includes \n, so the pattern needs dot-matches-newline ((?s)).
• \p{Name} and the Perl classes stand for Unicode sets, so the pattern needs Unicode mode, which is the default in most engines. The tables here are generated from UCD 17.0.0, so an engine built against an older version may resolve a name to a slightly different set.

Operations

n is the number of ranges in the set. The set operations come from irange and are documented there; the table below is what this crate adds.

m is the number of named Unicode classes, which is a constant: to_regex binary searches them for a set with the same ranges.

See the API documentation for the full list, and CHANGELOG.md for notable changes.

The surrogate block

char is not contiguous: the codepoints U+D800..=U+DFFF are surrogates and hold no char. Char counts by index rather than by codepoint and skips that block, so '\u{D7FF}' and '\u{E000}' are adjacent and the total set is one range of 1,112,064 values rather than two.

Minimum supported Rust version

regex-charclass builds with Rust 1.63 and later. Raising this version is a breaking change.

Enabling the serde feature pulls in serde_derive, whose own dependencies require a more recent compiler.

Regenerating the Unicode tables

src/tokens/unicode/ is generated by ucd-generate; generate-classes.sh has the commands and the UCD download it needs. The generated files are exempt from rustfmt, so a regeneration is a readable diff.

Contributing

Bug reports and pull requests are welcome, see CONTRIBUTING.md.

License

Licensed under the MIT License.