UniWorld

Correct Unicode text handling for every script.

UniWorld is an open-source library, a set of language bindings, and developer tools that implement the Unicode standard's core text algorithms -- all from a single, conformance-tested Rust core. It ships as a library (Rust, Python, JavaScript/WASM, C, Go), a VS Code extension, and a PowerShell module.

uniworld.world -- Full documentation, install guides, and the complete UniWorld ecosystem.

The problem UniWorld solves

Unicode text handling is one of the most pervasive unsolved problems in everyday software. It affects everyone:

If you work in English or other Latin-script languages, you've seen emoji split apart by your cursor, combining accents orphaned by backspace, and pasted text that looks identical but doesn't match because of invisible normalization differences. Your terminal miscounts column widths when it encounters fullwidth characters. Your truncation logic cuts strings in the middle of grapheme clusters. These are Unicode problems, and they happen constantly in English-language workflows.

If you work with Arabic, Hebrew, or any right-to-left script, correct bidirectional layout is essential and routinely broken. Numbers embedded in RTL paragraphs reorder incorrectly. Cursor movement goes the wrong direction. Mixed-direction text renders as gibberish.

If you work with Thai, Lao, Khmer, or Myanmar, your text has no spaces between words. Line breaking requires dictionary-based segmentation that most tools simply don't have. Text wraps mid-word or not at all.

If you work with CJK (Chinese, Japanese, Korean), Indic scripts (Devanagari, Bengali, Tamil), or emoji, selection and editing break on complex characters. Cursors land inside ligatures, conjuncts, and ZWJ sequences. Column counts are wrong. Truncation corrupts display.

The Unicode Consortium publishes the algorithms to handle all of this correctly. Most implementations address only one or two, partially, for a subset of scripts. UniWorld implements five core standards completely and makes them available everywhere.

What UniWorld provides

Conformance

Every algorithm is tested against the official Unicode conformance test suites for UCD 17.0.0. Run cargo test --features conformance; the harness prints pass totals. Row counts below match the number of test lines in each file except BidiTest.txt, which expands each data row across paragraph directions (see printed total).

Unicode 17.0 throughout (UCD 17.0.0 data files).

Get UniWorld

Rust (core library)

cargo add uniworld

crates.io/crates/uniworld | API docs

Python

pip install uniworld

pypi.org/project/uniworld | Integration guide

JavaScript / WASM

npm install uniworld

npmjs.com/package/uniworld | Integration guide

C

cargo build --release --features cffi
cbindgen --crate uniworld --output uniworld.h

Integration guide

Go

cargo build --release --features cffi
cd bindings/go && go test

Integration guide

VS Code extension

Search "UniWorld" in the Extensions panel, or:

ext install aguywithai.uniworld

VS Code Marketplace | Extension README

Grapheme-aware cursor and delete, bidi visualization, display width, Unicode inspector, normalization commands, line break decorations, script-aware word selection. See the full feature list.

PowerShell module

Install-Module UniWorld

PowerShell Gallery | Module README

12 cmdlets: Get-GraphemeBoundaries, Get-WordBoundaries, Get-SentenceBoundaries, Get-DisplayWidth, Limit-DisplayWidth, ConvertTo-NFC, ConvertTo-NFD, ConvertTo-NFKC, ConvertTo-NFKD, Get-BidiClasses, Get-LineBreakOpportunities, Get-UnicodeInfo. See the full cmdlet reference.

Quick start

Rust

use uniworld::{grapheme_boundaries, display_width, normalize_nfc};

let clusters = grapheme_boundaries("cafe\u{0301}");  // ["c", "a", "f", "e\u{0301}"]
let nfc = normalize_nfc("cafe\u{0301}");              // "cafe" (composed e-acute)
let width = display_width("Hello");                    // 5

Python

import uniworld

uniworld.grapheme_boundaries("cafe\u0301")   # ["c", "a", "f", "e\u0301"]
uniworld.display_width("Hello")              # 10 (CJK)
uniworld.normalize_nfc("cafe\u0301")         # "cafe" (composed)

PowerShell

Import-Module UniWorld
"Hello" | Get-DisplayWidth                   # 5
"cafe`u{0301}" | ConvertTo-NFC              # composed e-acute
Get-BidiClasses "Hello" | Format-Table       # per-character bidi levels

Architecture

                         UniWorld Rust core
                        /    |    |    \    \
                      /      |    |     \     \
                 Python   JS/WASM  C    Go    cdylib
                (PyO3)  (wasm-   (FFI) (CGo)  (DLL/so/dylib)
                         bindgen)              |
                                        C# P/Invoke
                                              |
                    VS Code extension    PowerShell module
                    (WASM binding)       (native FFI)

One Rust implementation. Every binding shares the same algorithms, the same data tables, and the same conformance test results. The behavior is identical everywhere because it is the same code.

Build and test

# Core library
cargo build
cargo test

# With conformance tests (requires test data in _development/data/)
cargo test --features conformance

# C FFI (for PowerShell / C / Go)
cargo build --release --features cffi

# WASM (for VS Code / JavaScript)
wasm-pack build --release --features wasm --no-default-features

# VS Code extension
cd extensions/vscode && npm install && npm run compile

# PowerShell module
Import-Module extensions/powershell/UniWorld.psd1
Invoke-Pester -Path extensions/powershell/Tests/

Scripts covered

UniWorld correctly handles text in: Latin, Greek, Cyrillic, Arabic, Hebrew, Devanagari, Bengali, Gurmukhi, Tamil, Sinhala, Thai, Lao, Khmer, Myanmar, Chinese (Simplified/Traditional), Japanese (Kanji + Hiragana + Katakana), Korean (Hangul), Ethiopic, Tifinagh, Cherokee, Canadian Aboriginal Syllabics (Cree, Inuktitut, Ojibwe), and emoji (including ZWJ sequences, skin tones, and flag pairs).

See the Unicode Showcase for a comprehensive stress-test document demonstrating UniWorld across all supported scripts.

Documentation

Repository layout

README.md                          # This file
src/                               # Rust core (algorithms, data tables, bindings)
tests/                             # Rust integration tests
docs/                              # User-facing docs (integration guides, showcase)
extensions/vscode/                 # VS Code extension (TypeScript + WASM)
extensions/powershell/             # PowerShell module (cmdlets + native FFI)
bindings/go/                       # Go CGo wrapper
_development/                      # Dev-only: notes, scripts, working docs
_publishing/                       # Publishing: marketing, site, outreach
.github/workflows/                 # CI: cross-platform native library builds

Contributing

See CONTRIBUTING.md for build instructions, test procedures, and how to submit test cases or dictionary entries.

License

MIT. See LICENSE.

Unicode Character Database data is used under the Unicode License. ICU dictionary data is used under the ICU License. Both are permissive and compatible with commercial use.

UniWorld is an A Guy With AI project by Sean MacNutt, developed using HAIMU, the AI development methodology also originated by MacNutt. HAIMU (Human-AI Mutual Understandability) generated the insight that led to UniWorld -- when prompted for the largest-ROI neglected technical benefit projects an AI could conceive of, correct Unicode handling emerged as the clear winner. The library was largely built within 14 hours of project idea generation. "Move fast and fix things." Initial development funded by Grand Beta. Visit uniworld.world for the full ecosystem.