rehuman 0.2.0

Unicode-safe text cleaning & typographic normalization for Rust
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98

# rehuman

Unicode-safe text cleaning & normalization for Rust.

Strip invisible characters, normalize typography, and enforce consistent formatting for text sourced from web scraping, user input, or [LLMs](https://archive.fo/PrRYl).

> This crate is a Rust rewrite and expansion of [humanize-ai-lib](https://github.com/Nordth/humanize-ai-lib) by [Nordth](https://github.com/Nordth).

## Install

Add the Rust library crate:

```toml
[dependencies]
rehuman = "0.2.0" # replace with the latest published version
```

Install CLI binaries (`rehuman`, `ishuman`):

```bash
cargo install rehuman
```

Or use it from Python: the bindings are on PyPI as
[`rehuman`](https://pypi.org/project/rehuman/):

```bash
pip install rehuman
```

See [python/README.md](python/README.md) for supported platforms and the Python API.

<details>
<summary><b>Click to Expand:</b> Build from Source</summary>

For the latest version(s), clone this repo and run `cargo install --path .`:

```bash
git clone https://github.com/pszemraj/rehuman.git
cd rehuman
cargo install --path .
```

Binaries will be installed to `~/.cargo/bin` by default.[^1]

[^1]: You may need to add `~/.cargo/bin` to your `PATH` if it is not already there; add `export PATH="$HOME/.cargo/bin:$PATH"` to your shell profile (`.bashrc`, `.zshrc`, etc.).

</details>

## Quick Start

```rust
use rehuman::{clean, humanize};

let cleaned = clean("Hello\u{200B}there"); // -> "Hellothere"
let humanized = humanize("“Quote”—and…more"); // -> "\"Quote\"-and...more"
```

```rust
use rehuman::clean;

// Default behavior removes emoji
let cleaned = clean("Thanks 👍"); // -> "Thanks"
```

By default, keyboard-only mode emits ASCII-safe output.
Non-ASCII text is normalized/transliterated when feasible: Latin diacritics
fold (`café` -> `cafe`), common symbols map to ASCII equivalents
(`→` -> `->`, `≠` -> `!=`, `•` -> `-`, `©` -> `(c)`, `✅` -> `[x]`), and Greek
letters spell out to their names (`λ` -> `lambda`); other unmappable characters
(emoji, letter scripts like CJK or Cyrillic) are removed, never romanized.
Tune this with `--non-ascii-policy`, `--extended-keyboard`, and
`--preserve-joiners` (details in [docs/api.md](docs/api.md#keyboard-only-behavior)
and [docs/cli.md](docs/cli.md#output-options)).
For docs/source files where Unicode glyphs matter (for example box-drawing diagrams),
use the CLI with `--preset code-safe`; it keeps glyphs, ellipses, and emoji while
still normalizing typographic quotes and dashes.

## Documentation

Primary docs by concern:

- Rust API semantics (defaults, options, presets, stats, errors):
  [docs/api.md](docs/api.md)
- CLI flags, modes, config, and exit behavior:
  [docs/cli.md](docs/cli.md)
- Usage recipes:
  [docs/examples.md](docs/examples.md)
- Python package (`pip install rehuman`):
  [python/README.md](python/README.md)
- Roadmap and development notes:
  [docs/development.md](docs/development.md)

For CLI help at runtime: `rehuman --help` and `ishuman --help`.

## License

MIT