txt-cleaner

txt-cleaner is a small Rust library for smarter cleanup of text data. It goes beyond trim() to normalize whitespace, remove invisible characters, and strip stray markdown/html artifacts from the edges of a string.

Features

• Remove BOM and zero-width characters like \u{200B}
• Collapse consecutive spaces, tabs, and newlines
• Preserve paragraph structure or collapse all whitespace to a single space
• Strip common markdown/html edge artifacts such as headings, unclosed wrappers, and stray tags
• Builder-style configuration API

Usage

use txt_cleaner::{clean, CleanOptions};

let raw = "\u{feff}**\n  Hello   \n\n\n  world!  **\u{200B} ";
let cleaned = clean(raw);
assert_eq!(cleaned, "Hello\n\nworld!");

let options = CleanOptions::builder()
    .strip_invisible_chars(true)
    .strip_markdown_artifacts(true)
    .preserve_paragraphs(false)
    .collapse_all_whitespace(true)
    .build();

let cleaned_single_line = txt_cleaner::clean_with_options(raw, options);
assert_eq!(cleaned_single_line, "Hello world!");

License

MIT