pub trait RemoveInvisibleCharacters<'a> {
// Required method
fn remove_all_invisible_characters(self) -> Cow<'a, str>;
}alloc only.Expand description
To extend str and Cow<str> to have remove_all_invisible_characters method.
Required Methods§
Sourcefn remove_all_invisible_characters(self) -> Cow<'a, str>
fn remove_all_invisible_characters(self) -> Cow<'a, str>
Removes all invisible or non-printable characters from a given string.
This function filters out a comprehensive set of Unicode characters that are typically invisible or used for control or formatting purposes. This includes:
- Control characters:
- U+0000 to U+001F and U+007F (ASCII/C0 controls), excluding tab and newline characters
- U+0080 to U+009F (C1 controls), excluding U+0085 (Next Line)
- Soft hyphen and joining/grapheme controls:
- U+00AD (Soft Hyphen)
- U+034F (Combining Grapheme Joiner)
- U+061C (Arabic Letter Mark)
- Script-specific invisible/filler characters:
- U+115F to U+1160 (Hangul fillers)
- U+17B4 to U+17B5 (Khmer inherent vowels)
- U+3164 and U+FFA0 (Hangul fillers)
- Zero-width characters and format controls:
- U+180E (Mongolian Vowel Separator)
- U+200B to U+200D (Zero-width characters)
- U+2060 to U+2064 (Word Joiner and Invisible Math Symbols)
- U+FEFF (Byte Order Mark / Zero Width No-Break Space)
- Bidirectional formatting and isolate controls:
- U+200E to U+200F (Directional marks)
- U+202A to U+202E (Directional formatting)
- U+2066 to U+206F (Bidi isolates and deprecated formatting controls)
- Variation selectors:
- U+180B to U+180D and U+180F (Mongolian variation selectors)
- U+FE00 to U+FE0F (Variation Selectors)
- U+E0100 to U+E01EF (Variation Selectors Supplement)
- Shorthand, musical, and tag format controls:
- U+1BCA0 to U+1BCA3 (Shorthand format controls)
- U+1D173 to U+1D17A (Musical symbol format controls)
- U+FFF9 to U+FFFB (Interlinear Annotation controls)
- U+E0001 and U+E0020 to U+E007F (Language tag controls)
- Reserved default-ignorable code points:
- U+2065, U+FFF0 to U+FFF8
- U+E0000, U+E0002 to U+E001F, U+E0080 to U+E00FF, U+E01F0 to U+E0FFF
- Noncharacters treated as non-printable/internal-use code points:
- U+FDD0 to U+FDEF
- U+FFFE to U+FFFF
- The last two code points of each supplementary plane, U+1FFFE to U+10FFFF
Noncharacters are valid Unicode scalar values. This method removes them as part of destructive text sanitization, not as Unicode validity checking.
Tab and newline characters are intentionally kept. Use expand_tabs, normalize_newlines, or replace_newlines_with_space to handle them.
These characters can interfere with text rendering, parsing, and display, and are often used in text-based attacks (e.g., for spoofing).
Dyn Compatibility§
This trait is dyn compatible.
In older versions of Rust, dyn compatibility was called "object safety".