Skip to main content

Crate characterminator

Crate characterminator 

Source
Expand description

characterminator – find and eliminate characters outside an allowed set, per file type and per file, so text costs fewer tokens.

The crate is federated: every directory under src/ is one Rust module and one spec node, and each node’s SPEC.md states what it owns. This file composes and re-exports; it holds no logic of its own (src:V38).

The nodes are PRIVATE. What a library user may name is the short list below, re-exported here and nowhere else (src:V98): enough to check a string against a set, rewrite it, and read the hazards in it. Everything else – rule resolution, config assembly, rendering – is the binary’s business until a release decides to promise it.

Every function that judges a character takes the caller’s rule as one predicate, impl Fn(char) -> bool (.:I): a set’s contains, or any closure.

use characterminator::{Map, SetCatalog, fix, scan_str};

let ascii = SetCatalog::builtin().resolve("ascii", "text")?;
let hits = scan_str("a\u{2014}b", |c| ascii.contains(c));
assert_eq!(hits.len(), 1);
let fixed = fix("a\u{2014}b", &Map::builtin(), |c| ascii.contains(c))?;
assert_eq!(fixed.output, "a--b");

Structs§

CharRange
An inclusive range of code points, the unit a set is built from.
CharSet
A named set of code points.
FixReport
What fix found, with no text to write: the fix --check answer.
Fixed
The rewritten text together with the report that explains it.
Hazards
Every hazard class, resolved once per run, paired with its lint.
Hit
One located character. Whether it is ALLOWED is not decided here: this node reports what is where, and the rules and lint nodes decide what it means.
Lint
A named check. The name is what a rule line and the json output carry, so it is the stable identifier rather than the message text.
Map
The effective map: the entries, classes and families these lines declared, plus the candidate list resolve_at walks.
Position
Where a character sits, in the three units a reader or a tool needs.
Rewrite
A rewrite that fix would apply: what was found, and what replaces it.
SetCatalog
Every declared set, by name.

Enums§

ComposeError
Why a name could not be resolved into a set.
FixError
Why a map cannot be used, or a fix cannot be trusted.
Group
The groups a lint can belong to. The group carries the default level: hazard forbids, charset denies, pedantic allows until asked.
Level
The rustc and clippy levels, in the order of increasing severity.
Unreadable
Why a file could not be scanned. Neither case is silent: an unreadable file that reported nothing would be indistinguishable from a clean one.

Functions§

fix
Rewrite every disallowed span of text that the map covers.
scan_bytes
Scan raw bytes, refusing by name when they are not text.
scan_str
Locate every character the caller’s test rejects.

Type Aliases§

Scanned
What scan_bytes hands back: the decoded text, and the hits in it.