base-d

Turns bytes into anything. Playing cards, hieroglyphs, emoji, RFC base64 — same engine, your choice of alphabet.

Why base-d?

You probably have base64, sha256sum, crc32 as separate tools. base-d does all of it:

• Encode with 45+ dictionaries (or define your own)
• Hash with 26 algorithms — SHA-256, BLAKE3, CRC32, xxHash3
• Compress with gzip, zstd, brotli, lz4, snappy, lzma
• Stream multi-GB files with constant 4KB memory
• Detect which encoding was used automatically

One tool. SIMD-accelerated. 7.4 GiB/s decode, 500 MiB/s encode.

Pick your path

Want to mess around?

cargo install base-d
base-d neo

Wake up, Neo...

Want the CLI? → CLI Quick Start | Full CLI docs

Want the library? → Library Quick Start | Full API docs

CLI Quick Start

# Encode with playing cards (default dictionary)
echo "secret" | base-d encode cards
# 🂡🂢🂣🂤🂥🂦

# RFC base64
echo "hello" | base-d encode base64
# aGVsbG8=

# Hieroglyphics, because why not
echo "pharaoh" | base-d encode hieroglyphics

# Word-based encoding (BIP-39 seed phrases)
echo "secret" | base-d encode bip39
# abandon absorb morning...

# Hash a file
base-d hash sha256 myfile.bin

# Compress + encode in one shot
base-d encode base64 --compress zstd < bigfile.json

# Auto-detect and decode
echo "aGVsbG8=" | base-d decode --detect

More CLI examples →

Library Quick Start

use base_d::{encode, decode, Dictionary, DictionaryRegistry};

// Basic encoding
let registry = DictionaryRegistry::with_builtins();
let dict = registry.get("base64").unwrap();
let encoded = encode(b"hello world", dict);
let decoded = decode(&encoded, dict)?;

// Streaming for large data
use base_d::streaming::{StreamEncoder, StreamDecoder};
let mut encoder = StreamEncoder::new(dict);
encoder.update(chunk1)?;
encoder.update(chunk2)?;
let result = encoder.finalize()?;

More library examples →

How it works

The core idea

base-d is a universal encoder. It converts bytes into symbols using dictionaries — lookup tables that map values to characters. The dictionary is the only thing that changes between "serious RFC base64" and "playing cards."

bytes → [dictionary] → symbols

Three encoding modes

Deep dive: Encoding modes →

Dictionaries

45+ built-in dictionaries organized by encoding type:

dictionaries/
├── word/           # Whole words as symbols (BIP-39, Pokemon, NATO...)
├── chunked/        # RFC 4648 style (base64, base32, base16...)
├── byterange/      # 1:1 byte mapping (base100, weather symbols)
└── radix/          # Mathematical conversion (cards, hieroglyphs, emoji...)

Browse the dictionaries/ folder to explore — the structure teaches you what the tool can do.

Full dictionary list → | Create your own →

Performance

SIMD-accelerated with runtime detection:

• x86_64: AVX2, SSSE3 with specialized RFC dictionary paths
• ARM: NEON with equivalent optimizations
• Fallback: Portable LUT-based implementation

SIMD internals → | Benchmarks →

Schema encoding (fiche)

Structured data encoding that preserves type information:

echo '{"name": "Neo", "age": 30}' | base-d fiche encode
# Compact binary with recoverable structure

base-d fiche decode < encoded.bin
# {"name": "Neo", "age": 30}

Schema deep dive →

Documentation

Install

# From crates.io
cargo install base-d

# From source
git clone https://github.com/coryzibell/base-d
cd base-d
cargo build --release

License

MIT OR Apache-2.0