characterminator 0.1.2

find and eliminate characters outside an allowed set, per file type and per file, so text costs fewer tokens
Documentation
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
//! The hazard group: which lint, if any, a character fires, and the
//! emoji sequences that let a joiner, a tag or VS16 off.
//!
//! See `src/lint/hazard/SPEC.md`. This file COMPOSES: detection lives in
//! `hazards`, the RGI sequence list in `sequence`, and the parent
//! `src/lint` re-exports what a sibling node may name.

mod hazards;
mod mojibake;
mod sequence;

pub use hazards::Hazards;
pub(crate) use hazards::carries_no_text;
pub(crate) use mojibake::{
    DOUBLE_ENCODED, Run, inside, runs as double_encoded,
};