1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
//! Layer 2 (part of [`crate::api`]) — text measurement (width, graphemes),
//! whitespace / zalgo / case / normalization cleanup, output encoders, slugify,
//! and emoji.
use Cow;
use crateError;
// ── Terminal width (UAX #11 / UAX #29) ───────────────────────────────────────
/// Total terminal column width of `text`, summed over UAX #29 grapheme clusters
/// (#224). Measures cells, not pixels; does not expand tabs or model wrapping.
///
/// `ambiguous_wide` selects the East-Asian Ambiguous policy (UAX #11): when
/// `true`, ambiguous-width characters count as 2 cells, otherwise 1.
/// Column width of a single grapheme cluster (see [`terminal_width`]).
///
/// `ambiguous_wide` selects the East-Asian Ambiguous policy (UAX #11): when
/// `true`, ambiguous-width characters count as 2 cells, otherwise 1.
// ── Whitespace ───────────────────────────────────────────────────────────────
/// Fold interior Unicode whitespace runs to a single ASCII space and **strip
/// leading and trailing whitespace** (#433): `" café world "` → `"café world"`.
///
/// Folds **whitespace only** — the line controls (TAB/LF/VT/FF/CR), the
/// information separators (`U+001C`–`U+001F`), NEL, the `Zs`/`Zl`/`Zp` spaces,
/// and the blank-rendering set (Braille blank, the Hangul fillers) each fold to a
/// single space; the result is then trimmed at both ends. It does **not** delete
/// control or zero-width characters — pair it with [`strip_control_chars`] /
/// [`strip_zero_width_chars`] for that. Folding (not deleting) the line controls
/// means `a\rb` → `a b`, never `ab`.
/// Remove C0/C1 control characters that are **not** whitespace (#433): NUL, DEL,
/// the C1 block, etc. are stripped, while the line controls (TAB, LF, VT, FF, CR,
/// `U+001C`–`U+001F`, NEL) are preserved for [`collapse_whitespace`] to fold. A
/// composable primitive of [`collapse_whitespace`].
/// Remove zero-width / invisible characters (ZWSP, ZWJ/ZWNJ, BOM, word joiner,
/// the invisible math operators). A composable primitive of [`collapse_whitespace`].
// ── Invisible / non-interchange code points (#413) ───────────────────────────
/// Remove the Unicode **Tags** block (`U+E0000`–`U+E007F`) — the "ASCII
/// smuggling" covert channel — **preserving** well-formed emoji subdivision flag
/// sequences (`U+1F3F4` + tag letters + `U+E007F`, e.g. the Scotland flag).
/// Remove every Unicode **variation selector** (VS1–VS16, `U+FE00`–`U+FE0F`, and
/// the Variation Selectors Supplement VS17–VS256, `U+E0100`–`U+E01EF`) — the
/// arbitrary-byte smuggling channel.
/// Remove every Unicode **noncharacter** (`U+FDD0`–`U+FDEF` and the last two
/// code points of every plane) — permanently reserved, invalid for interchange.
/// Remove every **Private Use Area** code point (BMP `U+E000`–`U+F8FF`, plane 15,
/// plane 16) — renders as arbitrary, font-defined glyphs.
// ── Zalgo (combining-mark abuse) ─────────────────────────────────────────────
/// True if any base character carries more than `threshold` consecutive
/// combining marks in NFD (zalgo-style abuse). A sane default is 3.
/// Cap combining marks at `max_marks` per base character (recomposed to NFC),
/// stripping zalgo stacking while preserving legitimate diacritics. `max_marks`
/// of 0 strips all combining marks.
// ── Case folding ─────────────────────────────────────────────────────────────
/// Full Unicode case folding per CaseFolding.txt (status C + F) — stronger than
/// `str::to_lowercase` (folds ß→ss, fi→fi, ς→σ, and ~1,500 other mappings). Use
/// for caseless matching, not display.
///
/// Returns `Cow::Borrowed` when `text` is already folded (zero allocation).
// ── Grapheme clusters (UAX #29) ──────────────────────────────────────────────
/// Number of user-perceived characters (extended grapheme clusters): `"👩👩👧👦"` → 1.
/// Split `text` into its extended grapheme clusters, one user-perceived
/// character per element. Allocates a `String` per cluster; prefer
/// [`graphemes`] when borrowed slices suffice.
/// Iterate the extended grapheme clusters of `text` as borrowed `&str` slices —
/// no `Vec`, no per-cluster `String`. Callers that only need a count or the
/// first few never pay for the rest; `.collect()` when you want owned data.
///
/// ```
/// use disarm::api;
/// assert_eq!(api::graphemes("a❤️b").count(), 3);
/// ```
/// Truncate `text` to at most `max_graphemes` clusters without ever splitting a
/// cluster (so emoji / combining sequences stay intact). Returned unchanged if
/// already within the limit. Infallible — `usize` rules out the negative count
/// the Python binding must guard against.
// ── Unicode normalization (UAX #15) ──────────────────────────────────────────
/// Unicode normalization form for [`normalize`] / [`is_normalized`].
/// Normalize `text` to the given Unicode normalization form.
///
/// Infallible: a [`NormalizationForm`] is always a valid form.
/// True if `text` is already in the given Unicode normalization form.
///
/// Infallible: a [`NormalizationForm`] is always a valid form.
// ── Output encoders (encode once, at the sink) ───────────────────────────────
/// Escape the five HTML metacharacters for element-body (PCDATA) and
/// quoted-attribute context: `&`→`&`, `<`→`<`, `>`→`>`, `"`→`"`,
/// `'`→`'`. Returns `Cow::Borrowed` (zero-copy) when nothing needs escaping.
///
/// **Not** correct inside `<script>` / `<style>`, unquoted attributes, or URL
/// attributes — there HTML-entity escaping is insufficient or corrupting. Encode
/// once at the output sink; disarm is not a context-aware auto-escaper.
/// URL component whose RFC 3986 safe-character set drives [`percent_encode`].
/// Percent-encode `text` for `component` (RFC 3986): the input is UTF-8 encoded,
/// then every byte outside the component's safe set becomes `%XX`. Output is ASCII.
///
/// Infallible: a [`UrlComponent`] always names a known component.
// ── Slugification ────────────────────────────────────────────────────────────
pub use crateSlugConfig;
/// Generate a URL-safe slug from `text` according to `config` (separator, max
/// length, case folding, stopwords, custom regex, HTML-entity handling, …).
///
/// Build a [`SlugConfig`] with [`SlugConfig::new`] and the `with_*` setters.
///
/// Infallible by design — and therefore **`config.lang` is not validated**: an
/// unknown language code is treated as "best effort" and falls back to the
/// default transliterator (the same lenient behaviour as the underlying engine),
/// rather than erroring. The Python `slugify` wrapper treats `lang` the same way
/// — it forwards the code unvalidated and silently falls back, so neither
/// binding raises on an unknown slug `lang`. If you need strict validation,
/// check the code against [`list_langs`](crate::api::list_langs) before building the config.
// ── Emoji ────────────────────────────────────────────────────────────────────
/// Expand emoji sequences in `text` to their CLDR short-name text descriptions
/// (e.g. `"😀"` → `"grinning face"`). The matching engine handles ZWJ sequences,
/// skin-tone modifiers, flag/keycap sequences, and presentation selectors;
/// `strip_modifiers` drops the modifier suffix (`": light skin tone"`, etc.) from
/// each name. Pure-ASCII input is returned unchanged.
///
/// This uses the **built-in CLDR data** (latest English). The custom Python
/// `EmojiProvider` override exposed by the `disarm` package is binding-layer-only
/// (Python-only) and is intentionally **not** part of the Rust surface.