Expand description
characterminator – find and eliminate characters outside an allowed
set, per file type and per file, so text costs fewer tokens.
The crate is federated: every directory under src/ is one Rust module
and one spec node, and each node’s SPEC.md states what it owns. This
file composes and re-exports; it holds no logic of its own (src:V38).
The nodes are PRIVATE. What a library user may name is the short list
below, re-exported here and nowhere else (src:V98): enough to check a
string against a set, rewrite it, and read the hazards in it. Everything
else – rule resolution, config assembly, rendering – is the binary’s
business until a release decides to promise it.
Every function that judges a character takes the caller’s rule as one
predicate, impl Fn(char) -> bool (.:I): a set’s contains, or any
closure.
use characterminator::{Map, SetCatalog, fix, scan_str};
let ascii = SetCatalog::builtin().resolve("ascii", "text")?;
let hits = scan_str("a\u{2014}b", |c| ascii.contains(c));
assert_eq!(hits.len(), 1);
let fixed = fix("a\u{2014}b", &Map::builtin(), |c| ascii.contains(c))?;
assert_eq!(fixed.output, "a--b");Structs§
- Char
Range - An inclusive range of code points, the unit a set is built from.
- CharSet
- A named set of code points.
- FixReport
- What
fixfound, with no text to write: thefix --checkanswer. - Fixed
- The rewritten text together with the report that explains it.
- Hazards
- Every hazard class, resolved once per run, paired with its lint.
- Hit
- One located character. Whether it is ALLOWED is not decided here: this node reports what is where, and the rules and lint nodes decide what it means.
- Lint
- A named check. The name is what a rule line and the json output carry, so it is the stable identifier rather than the message text.
- Map
- The effective map: the entries, classes and families these lines
declared, plus the candidate list
resolve_atwalks. - Position
- Where a character sits, in the three units a reader or a tool needs.
- Rewrite
- A rewrite that
fixwould apply: what was found, and what replaces it. - SetCatalog
- Every declared set, by name.
Enums§
- Compose
Error - Why a name could not be resolved into a set.
- FixError
- Why a map cannot be used, or a fix cannot be trusted.
- Group
- The groups a lint can belong to. The group carries the default level: hazard forbids, charset denies, pedantic allows until asked.
- Level
- The rustc and clippy levels, in the order of increasing severity.
- Unreadable
- Why a file could not be scanned. Neither case is silent: an unreadable file that reported nothing would be indistinguishable from a clean one.
Functions§
- fix
- Rewrite every disallowed span of
textthat the map covers. - scan_
bytes - Scan raw bytes, refusing by name when they are not text.
- scan_
str - Locate every character the caller’s test rejects.
Type Aliases§
- Scanned
- What
scan_byteshands back: the decoded text, and the hits in it.