1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
//! Layer 2 (part of [`crate::api`]) — the precompiled pipeline presets and the
//! named-policy-profile registry.
use Cow;
use crateError;
// ── Precompiled pipeline presets ──────────────────────────────────────────────
/// Canonicalize text for security-sensitive *comparison* (homoglyph / bidi /
/// zero-width / control neutralization).
///
/// Pipeline: NFKC → strip bidi/format → strip invisible classes (#413) →
/// strip control → strip zero-width → collapse whitespace → cap combining marks
/// (anti-zalgo, #429) → NFC → confusables → NFC. (The confusable fold is
/// sandwiched between two NFC passes
/// so TR39 skeletoning is normalization-stable and the preset is idempotent —
/// #416.) Fallible only through the confusables stage, whose target script is
/// fixed internally, so in practice this never errors; the [`Result`] keeps the
/// surface uniform with the other key/clean presets.
///
/// The name describes the mechanism (Unicode canonicalization for matching), not
/// a safety guarantee: this is **not** an output sanitizer — encode at the sink.
/// Deprecated alias for [`canonicalize`]. Renamed in 0.11 because the `*_clean`
/// name overpromised safety (see `THREAT_MODEL.md`); removed in 1.0.
///
/// # Errors
/// Propagates [`canonicalize`]'s error.
/// ML/NLP text normalization: NFKC → emoji→text → transliterate → strip accents →
/// case fold → collapse whitespace.
///
/// `lang` selects the transliteration table (`None` skips transliteration).
/// `emoji_style` is `"cldr"` (expand emoji to CLDR short names) or `"none"`
/// (leave emoji as-is). Fails ([`ErrorKind::InvalidArgument`](crate::ErrorKind))
/// on an unknown `lang` or an unsupported `emoji_style`.
/// Library catalog deduplication key: NFKC → strip bidi → case fold →
/// transliterate → confusables → strip accents → case fold → collapse whitespace.
///
/// `strict_iso9` selects the ISO 9:1995 Cyrillic scheme. Fails
/// ([`ErrorKind::InvalidArgument`](crate::ErrorKind)) on an unknown `lang`.
/// Case/accent/script-insensitive search lookup key (like [`catalog_key`] without
/// confusable folding). Fails ([`ErrorKind::InvalidArgument`](crate::ErrorKind))
/// on an unknown `lang`.
/// Collation sort key (like [`search_key`] but preserves base accented characters
/// for correct ordering). Fails ([`ErrorKind::InvalidArgument`](crate::ErrorKind))
/// on an unknown `lang`.
/// Strip bidi/format and other invisible-injection vectors from rendered user
/// content: strip bidi/format → strip invisibles (rendering policy) → collapse
/// whitespace (also stripping control + zero-width). Infallible.
///
/// Visual hygiene only — **not** markup-safe; still escape at the output layer.
/// Deprecated alias for [`strip_format`]. Renamed in 0.11 because `display_clean`
/// implied markup-safety it does not provide (see `THREAT_MODEL.md`); removed in 1.0.
/// Strip bidirectional override and formatting characters (UAX #9 §3.3.2 plus the
/// soft hyphen and deprecated/interlinear format controls). A composable primitive
/// shared by the security/key presets. Infallible.
/// Normalize user-submitted input — Unicode hygiene that **preserves the original
/// script** (no transliteration): NFKC → strip bidi/format → strip zero-width →
/// strip control → strip invisible classes (#413) → cap combining marks
/// (anti-zalgo) → confusables → collapse whitespace → NFC. (The invisibles are
/// stripped before the zalgo cap so they cannot split a mark run, and the
/// terminal NFC recomposes any base+mark left adjacent — keeping the preset
/// idempotent, #121/#416.)
///
/// Not an output sanitizer (no HTML/JS/SQL escaping). Fallible only through the
/// fixed-target confusables stage; the [`Result`] keeps the surface uniform.
/// Deprecated alias for [`canonicalize_strict`]. Renamed in 0.11 (the old name
/// conceded itself a relic in `THREAT_MODEL.md`); removed in 1.0.
///
/// # Errors
/// Propagates [`canonicalize_strict`]'s error.
/// Maximum-strength deobfuscation: NFKC → strip all combining marks → strip bidi →
/// strip zero-width → demojize → confusables → strip accents → collapse
/// whitespace. Preserves case; does not transliterate.
///
/// Fallible only through the fixed-target confusables stage; the [`Result`] keeps
/// the surface uniform.
// ── Named policy profiles ─────────────────────────────────────────────────────
/// Sorted names of the available named policy profiles (the registry that the
/// `get_pipeline` Python entrypoint builds from).
///
/// The stateful pipeline builder itself (`_TextPipeline`) stays binding-only for
/// now — exposing it as a pure crates.io type is deferred (see the module-level
/// `src/pipeline.rs` `Pipeline` core), so this read-only registry view is the
/// pipeline surface Layer 2 exposes. Infallible.
/// A reusable, read-only handle to a named policy-profile pipeline (#404).
///
/// Built with [`get_pipeline`] from one of the profiles in [`list_profiles`],
/// then applied to any number of inputs via [`Pipeline::process`]. This is the
/// pure crates.io counterpart to the binding-only stateful builder
/// (`_TextPipeline`): a precompiled, immutable handle the Node/Ruby bindings can
/// wrap as a reusable object (mirroring the `Lexicon` handle), not a mutable
/// pipeline builder.
///
/// ```
/// use disarm::api::get_pipeline;
/// let pipe = get_pipeline("search_index").unwrap();
/// assert_eq!(pipe.process("Café").unwrap(), "cafe");
/// ```
/// Build the reusable [`Pipeline`] handle for a named policy profile (#404).
///
/// `profile` is one of the names returned by [`list_profiles`]. Fails
/// ([`ErrorKind::InvalidArgument`](crate::ErrorKind)) on an unknown profile,
/// naming the offending value and the available profiles.
///
/// # Errors
/// Returns an [`ErrorKind::InvalidArgument`](crate::ErrorKind) error if
/// `profile` is not a known profile name.