Crate base_d 

base-d

A universal, multi-dictionary encoding library for Rust.

Encode binary data using numerous dictionaries including RFC standards, ancient scripts, emoji, playing cards, and more. Supports three encoding modes: mathematical base conversion, RFC 4648 chunked encoding, and direct byte-range mapping.

Quick Start

use base_d::{DictionariesConfig, Dictionary, encode, decode};

// Load built-in dictionaries
let config = DictionariesConfig::load_default()?;
let base64_config = config.get_dictionary("base64").unwrap();

// Create dictionary
let chars: Vec<char> = base64_config.chars.chars().collect();
let padding = base64_config.padding.as_ref().and_then(|s| s.chars().next());
let dictionary = Dictionary::new_with_mode(
    chars,
    base64_config.mode.clone(),
    padding
)?;

// Encode and decode
let data = b"Hello, World!";
let encoded = encode(data, &dictionary);
let decoded = decode(&encoded, &dictionary)?;
assert_eq!(data, &decoded[..]);

Features

• 33 Built-in Alphabets: RFC standards, emoji, ancient scripts, and more
• 3 Encoding Modes: Mathematical, chunked (RFC-compliant), byte-range
• Streaming Support: Memory-efficient processing for large files
• Custom Alphabets: Define your own via TOML configuration
• User Configuration: Load dictionaries from ~/.config/base-d/dictionaries.toml

Encoding Modes

Mathematical Base Conversion

Treats data as a large number. Works with any dictionary size.

use base_d::{Dictionary, EncodingMode, encode};

let chars: Vec<char> = "😀😁😂🤣😃😄😅😆".chars().collect();
let dictionary = Dictionary::new_with_mode(
    chars,
    EncodingMode::BaseConversion,
    None
)?;

let encoded = encode(b"Hi", &dictionary);

Chunked Mode (RFC 4648)

Fixed-size bit groups, compatible with standard base64/base32.

use base_d::{Dictionary, EncodingMode, encode};

let chars: Vec<char> = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/"
    .chars().collect();
let dictionary = Dictionary::new_with_mode(
    chars,
    EncodingMode::Chunked,
    Some('=')
)?;

let encoded = encode(b"Hello", &dictionary);
assert_eq!(encoded, "SGVsbG8=");

Byte Range Mode

Direct 1:1 byte-to-emoji mapping. Zero encoding overhead.

use base_d::{Dictionary, EncodingMode, encode};

let dictionary = Dictionary::new_with_mode_and_range(
    Vec::new(),
    EncodingMode::ByteRange,
    None,
    Some(127991)  // U+1F3F7
)?;

let data = b"Hi";
let encoded = encode(data, &dictionary);
assert_eq!(encoded.chars().count(), 2);  // 1:1 mapping

Streaming

For large files, use streaming to avoid loading entire file into memory:

use base_d::{DictionariesConfig, StreamingEncoder};
use std::fs::File;

let config = DictionariesConfig::load_default()?;
let alphabet_config = config.get_dictionary("base64").unwrap();

// ... create dictionary from config

let mut input = File::open("large_file.bin")?;
let output = File::create("encoded.txt")?;

let mut encoder = StreamingEncoder::new(&dictionary, output);
encoder.encode(&mut input)?;

Structs

CompressionConfig

Configuration for a compression algorithm.

DictionariesConfig

Collection of dictionary configurations loaded from TOML files.

Dictionary

Represents an encoding dictionary with its characters and configuration.

DictionaryConfig

Configuration for a single dictionary loaded from TOML.

DictionaryDetector

Detector for automatically identifying which dictionary was used to encode data.

DictionaryMatch

A match result from dictionary detection.

Settings

Global settings for base-d.

StreamingDecoder

Streaming decoder for processing large amounts of encoded data efficiently.

StreamingEncoder

Streaming encoder for processing large amounts of data efficiently.

Enums

CompressionAlgorithm

Supported compression algorithms.

DecodeError

Errors that can occur during decoding.

EncodingMode

Encoding strategy for converting binary data to text.

HashAlgorithm

Supported hash algorithms.

Functions

compress

Compress data using the specified algorithm and level.

decode

Decodes a string back to binary data using the specified dictionary.

decompress

Decompress data using the specified algorithm.

detect_dictionary

Convenience function to detect dictionary from input.

encode

Encodes binary data using the specified dictionary.

hash

Compute hash of data using the specified algorithm.