characterminator 0.1.2

find and eliminate characters outside an allowed set, per file type and per file, so text costs fewer tokens
Documentation
# SPEC

## §G GOAL

`characterminator` (bin `ctrm`) — Rust toolkit: find & eliminate chars outside allowed set, per file type & per file, so text costs fewer tokens. Default = pure ASCII; extended sets (locales, i18n data) granted only where declared.

## §F FEDERATION

dir|owns|⊥owns|tokens
src|the tool: charset presets, rule resolution, scan, fix, lint levels, token facade, render, CLI|repo gate, dogfood waves, release|-
docs|human reference: memory, LLM disclaimer, third-party notices|rules (the specs are the law)|-
tests|integration tests that drive the `ctrm` binary|unit tests (beside their module)|-
dev|`characterminator-dev`, `publish = false`: README badges, notices closure & their drift checks|anything a consumer installs; the gate's ops (`hk.pkl`)|-

## §N NAV

rel|path|lens
up|-|-
self|.|-

## §C CONSTRAINTS

- Rust **edition 2024**, stable, MSRV **1.95** = fleet pin (`nixpkgs-lock` → nixos-26.05). One crate: lib + bin. MIT. `unsafe_code` FORBID.
- crate `characterminator`, bin `ctrm` (V92).
- CPU only, offline, deterministic. ⊥ network, ⊥ model.
- SPEC.md FORMAT = cavekit **4.1.0** as vendored by `microlith` (upstream rev `c322f0b`) + its `FORMAT-EXTENSIONS.md` (`§F`/`§N`). Form gated by `mth`, ⊥ restated here (V14).
- token counting = `itok` lib dep (crates.io 0.3, `default-features = false`, `features = ["bpe"]`). ⊥ own tokenizer, ⊥ own bytes/4.
- fs walk = `itok::walk::tracked`. `itok::glob::matches` & `itok::estimate::select_paths` = `pub(crate)` ∴ OWN glob matcher (zero-dep, `src/rules:T42`) & explicit-path selection lives in `src/tokens`.
- federation = `sherd`: dir = node = Rust module (dir + `mod.rs`), ⊥ 2018 `foo.rs`+`foo/`.
- gate runner = `hk` from `nix-hk`; ops in `hk.pkl`; schema vendored `pkl/Config.pkl` ∴ gate runs w/ ⊥ network.
- dev shell = `flake.nix`; ∀ inputs follow `nixpkgs-lock`; pins: `microlith` `v0.7.3`, `itok` `v0.3.1`, `sherd` `v0.5.3`. pin policy = V44.
- source ASCII-only (dogfood, V13).
- deps minimal; each direct dep justified in `docs/THIRD-PARTY-NOTICES.md`. direct: `itok` · `unicode-normalization` & `unicode-security` (0.1, `default-features = false`; UCD tables for pedantic, `src/lint/pedantic:V58`).
- ⊥ non-public repo named in spec, source, or commit message. private repos ? scanned & improved; their data cited as anonymous counts only. named only once verified public; unknown → private (fail closed).

## §I INTERFACES

- cmd: `ctrm check | fix | stats | explain | sets | guard`; each verb's contract → `src/cli` §I. exit codes below.
- adopt: `.pre-commit-hooks.yaml` (ids `ctrm-check`, `ctrm-fix`) · `action.yml` (composite; inputs `args`, `output`, `working-directory`). both run the SAME `ctrm` binary (V52).
- file grammars live w/ their parser: `.ctrm` → `src/rules` §I · `.ctrm-map` → `src/fix` §I · `.ctrm-sets` & builtin presets → `src/charset` §I.
- flag: full list, per-verb `--format` & refusal rules → `src/cli` §I.
- lib: `characterminator::{scan_str, scan_bytes, fix}`: pure, ⊥ I/O; rule = `impl Fn(char) -> bool`. rest of surface → `src:V98`. runner: doc-test per fn.
- exit: 0 ok · 1 violation | drift · 2 = ⊥ verdict reached: usage, config (`src/cli:V74`), write error (`src/cli:V47`), dir w/ ⊥ tracked file (`src/tokens:V43`), ⊥ git work tree (`src/tokens:V69`) · `guard`: 0 decided | 1 adapter failure, 2 ⊥ EVER (`src/cli/guard:V53`).

## §V INVARIANTS

V13: dogfood: `ctrm check` gates own tree (`hk.pkl` step `ctrm`). `.ctrm` grants, last match wins: `*` `ascii` · `*.md` `ascii+spec` (custom set, `.ctrm-sets`) ∵ specs DOCUMENT the chars they govern · `README.md` `LICENSE` `docs/**/*.md` back to `ascii` ∵ the front page argues plain text · `AGENTS.md` `ascii+caveman` ∵ its glossary decodes spec notation · `.context-limits` `ascii+caveman` ∵ it cites `§` · `pkl/Config.pkl` `any` ∵ vendored upstream schema, ⊥ ours to rewrite.
V14: SPEC.md form gated by `mth fmt --check SPEC.md` & `mth check --records .spec-records SPEC.md`. `mth` absent → gate FAILS hard, ⊥ skip. ∀ node SPEC.md, ⊥ root only; node records → `.spec-records` beside its SPEC.md ∵ ids repeat across specs.
V15: SPEC.md capped from commit one: `.context-limits` row gated by `itok check`; ceiling ~12% over measured.
V16: `sherd check`, `sherd budget` & `sherd sync --check` gate; federation live since the split ∴ ⊥ deferred.
V40: dev shell installs a hook ONLY into its OWN repo (crate-name marker @ worktree root) & ONLY into an UNTRACKED hooks dir ∵ `core.hooksPath` ? be tracked. refusal LOUD. runner: hk step `hook-guard` (text: both guards precede every `install`).

V44: ∀ flake input pins a TAG, bumped in its OWN reviewed commit. FOLLOWING a branch REJECTED ∵ what the gate enforces ? then change w/ ⊥ diff to read, & a gate whose rules move unreviewed gates ⊥. `nixpkgs-lock` & `nix-hk` = the fleet authorities, pinned by their own lock ∴ exempt.

V46: line coverage GATED, ⊥ reported. FLOOR = hard min; `.coverage` = CEILING, the figure the badge CLAIMS. measured < floor → FAIL. measured < claim → FAIL ∵ badge OVERSTATES. measured − claim > 0.5 → FAIL ∵ badge stale. `cargo llvm-cov` needs llvm tools matching `rustc`s LLVM ∴ gate asserts the majors agree: a mismatch reads as a crash, ⊥ as a pin.

V52: adoption surfaces (pre-commit, Action) WRAP the `ctrm` binary, built from the consumer-pinned rev; ⊥ own rule logic ∵ a wrapper judging files drifts from its tool. action: inputs → shell via `env` ONLY, ⊥ `${{ }}` in `run:` ∵ template injection; exit status kept when `output` redirects; rustc PINNED = MSRV via `RUSTUP_TOOLCHAIN` ∵ runner's Rust unverified. CI runs the action via `uses: ./` ∵ an unexercised wrapper ? rot unseen.

V92: crate `characterminator`, bin `ctrm` (`rg`/`mth` shape: crate carries meaning, bin carries muscle memory). `ctr` REJECTED ∵ containerd ships `cmd/ctr`. `ctrm` free on crates.io, ⊥ found on PATH. runner: hk step `ctrm` = `cargo run --bin ctrm` ∴ a rename fails the gate; the record → `.spec-records`.

V103: a ceiling has a CAP: chain (dir row) ≤ 5500 tok, `SPEC.md` file ≤ 3500 tok. below the cap a ceiling rises by the ~12% band; a node that would pass it SPLITS (`sherd split`), ⊥ raises ∵ a chain = what 1 worker loads every turn & raising was the only move ever made (10 raises in 1 wave). cap moves only in its own reviewed commit. runner: hk `ceiling-cap`.

V105: every release tag ships exactly three target archives (`ctrm`, `LICENSE`, `README.md`) and one `SHA256SUMS` asset: x86_64 Linux, aarch64 Linux, and aarch64 macOS. The release workflow checks out the tag, builds from it, and uploads to the existing release with `gh release upload --clobber`; it never creates or edits release notes. runner: `scripts/check-release-workflow.sh`.

## §T TASKS

id|status|task|cites
T17|~|release: `release.toml` (`cargo-release`) + crates.io publish, ∵ T13 & T16 land|-
T27|x|ARCHIVED to SPEC-ARCHIVE.md|`src/tokens:V10`,`src/cli:V7`
T28|.|dogfood wave 2: fleet (`src/charset:R4`), presets confirmed (`src/charset:R19`); mine adopters' `.ctrm-sets` (human edits only) → promote per `src/charset:V138`|`src/charset:V138`
T45|x|ARCHIVED to SPEC-ARCHIVE.md|V14,V16
T48|x|ARCHIVED to SPEC-ARCHIVE.md|V44
T51|x|ARCHIVED to SPEC-ARCHIVE.md|V46
T54|x|ARCHIVED to SPEC-ARCHIVE.md|V52
T55|x|release assets: `.github/workflows/release.yml` builds and uploads the three tagged target archives plus checksums; the workflow contract is checked by `scripts/check-release-workflow.sh`|V105

## §B BUGS

id|date|cause|fix
B1|2026-09-18|dev shell installed hooks at `git rev-parse --git-path hooks` w/ ⊥ repo check ∴ entered from a sibling, overwrote its TRACKED hooks & downgraded that gate refuse → skip|V40