Yore

A Rust library for decoding and encoding character sets based on OEM code pages.

Features

• Fast performance *
• Minimal memory usage with Cow and shrink_to_fit
• Easy-to-use API
• Broad range of supported code pages
• Handles code pages with redefined ASCII characters (<0x80), such as '٪' in CP864
• no_std support, with or without an allocator — down to allocation-free encode_char / decode_byte primitives for embedded use

Usage

Add yore to your Cargo.toml file.

[dependencies]
yore = "2.1.1"

Examples

Using a specific code page

use yore::code_pages::{CP857, CP850};
use yore::{DecodeError, EncodeError};

// Vec contains ASCII "text"
let bytes = vec![116, 101, 120, 116];
// Vec contains ASCII "text " and codepoint 231
let bytes_undefined = vec![116, 101, 120, 116, 32, 231]; 

// Notice that decoding CP850 can't fail because it is completely defined
assert_eq!(CP850.decode(&bytes), "text");

// However, CP857 can fail
assert_eq!(CP857.decode(&bytes).unwrap(), "text");

// "text " + codepoint 231 
assert!(matches!(CP857.decode(&bytes_undefined), DecodeError));

// Lossy decoding won't fail due to fallback
assert_eq!(CP857.decode_lossy(&bytes_undefined), "text �");

// Encoding
assert_eq!(CP850.encode("text").unwrap(), bytes);
assert!(matches!(CP850.encode("text 🦀"), EncodeError));
assert_eq!(CP850.encode_lossy("text 🦀", 231), bytes_undefined);

Using a trait object

use yore::CodePage;
fn do_something(code_page: &dyn CodePage, bytes: &[u8]) {
    println!("{}", code_page.decode(bytes).unwrap());
}

Supported code pages

* Benchmarks

encoding_rs supports only a few of the encodings that oem_cp and yore support. Additionally, encoding_rs focuses on streaming use cases.

Refer to the bench crate for more details.

no_std

yore has three feature tiers, so it scales from std down to bare-metal targets with no allocator:

The allocating encode/decode family returns owned Cow buffers and so requires the alloc feature. Without it, only the allocation-free encode_char / decode_byte primitives are available:

[dependencies]
# no_std with an allocator: keep the full Cow-returning API
yore = { version = "2.1.1", default-features = false, features = ["alloc"] }

# no_std without an allocator: char primitives only
yore = { version = "2.1.1", default-features = false }

use yore::code_pages::CP850;

// Encode/decode one character at a time, no allocation required.
assert_eq!(CP850.encode_char('A'), Some(b'A'));
assert_eq!(CP850.decode_byte(b'A'), 'A');

CP437G (VGA text-mode glyphs)

The optional cp437g feature adds the CP437G code page: CP437 overlaid with the IBM-Graphics glyphs (☺ ♥ ♪ → ⌂ ...) at the C0 control byte range, as the VGA BIOS glyph ROM renders them. It pairs well with encode_char for driving a text buffer from a no_std kernel.

[dependencies]
yore = { version = "2.1.1", features = ["cp437g"] }

use yore::code_pages::CP437G;

assert_eq!(CP437G.encode_char('☺'), Some(0x01));
assert_eq!(CP437G.decode_byte(0x01), '☺');

Some glyphs share a byte with an ASCII control character (0x09 ○, 0x0A ◙, 0x0D ♪). The ASCII fast-path still encodes '\t'/'\n'/'\r' to those bytes, so intercept the source char first if you need newline semantics.

Contributing

See CONTRIBUTING.md for development setup, benchmarking, and fuzzing.