yore 2.0.0

Rust library for decoding/encoding character sets according to OEM code pages
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134

# Yore

A Rust library for decoding and encoding character sets based on OEM code pages.

[![yore at crates.io](https://img.shields.io/badge/crates.io-2.0.0-blue)](https://crates.io/crates/yore)
[![yore at docs.rs](https://docs.rs/yore/badge.svg)](https://docs.rs/yore)

# Features
* [Fast performance](https://bonega.github.io/yore-criterion/report/index.html) [*](#*-benchmarks)
* Minimal memory usage with `Cow` and `shrink_to_fit`
* Easy-to-use API
* Broad range of [supported code pages](#supported-code-pages)
* Handles code pages with redefined ASCII characters (<0x80), such as '٪' in CP864
* `no_std` support, with or without an allocator, via cargo features
* Allocation-free `encode_char` / `decode_byte` primitives for embedded use

# Usage

Add `yore` to your `Cargo.toml` file.

```toml
[dependencies]
yore = "2.0.0"
```

## `no_std`

`yore` has three feature tiers, so it scales from std down to bare-metal targets
with no allocator:

| Cargo features | Environment | API |
| --- | --- | --- |
| `std` (default) | `std` | Full API + `std::error::Error` impls |
| `alloc` | `no_std` + allocator | Full API; `Error` impls omitted (`Display` stays) |
| *(none)* | `no_std`, no allocator | Allocation-free char primitives only |

The allocating `encode`/`decode` family returns owned `Cow` buffers and so
requires the `alloc` feature. Without it, only the allocation-free
`encode_char` / `decode_byte` primitives are available:

```toml
[dependencies]
# no_std with an allocator: keep the full Cow-returning API
yore = { version = "2.0.0", default-features = false, features = ["alloc"] }

# no_std without an allocator: char primitives only
yore = { version = "2.0.0", default-features = false }
```

```rust
use yore::code_pages::CP850;

// Encode/decode one character at a time, no allocation required.
assert_eq!(CP850.encode_char('A'), Some(b'A'));
assert_eq!(CP850.decode_byte(b'A'), 'A');
```

# Examples

## Using a specific code page
```rust
use yore::code_pages::{CP857, CP850};
use yore::{DecodeError, EncodeError};

// Vec contains ASCII "text"
let bytes = vec![116, 101, 120, 116];
// Vec contains ASCII "text " and codepoint 231
let bytes_undefined = vec![116, 101, 120, 116, 32, 231]; 

// Notice that decoding CP850 can't fail because it is completely defined
assert_eq!(CP850.decode(&bytes), "text");

// However, CP857 can fail
assert_eq!(CP857.decode(&bytes).unwrap(), "text");

// "text " + codepoint 231 
assert!(matches!(CP857.decode(&bytes_undefined), DecodeError));

// Lossy decoding won't fail due to fallback
assert_eq!(CP857.decode_lossy(&bytes_undefined), "text �");

// Encoding
assert_eq!(CP850.encode("text").unwrap(), bytes);
assert!(matches!(CP850.encode("text 🦀"), EncodeError));
assert_eq!(CP850.encode_lossy("text 🦀", 231), bytes_undefined);
```

### Using a trait object
```rust
use yore::CodePage;
fn do_something(code_page: &dyn CodePage, bytes: &[u8]) {
    println!("{}", code_page.decode(bytes).unwrap());
}
```

# Supported code pages

| Identifier | Name        | Description                                                                                 |
|------------|-------------|---------------------------------------------------------------------------------------------|
| 437        | ibm437      | OEM United States                                                                           |
| 737        | ibm737      | OEM Greek (formerly 437G); Greek (DOS)                                                      |
| 775        | ibm775      | OEM Baltic; Baltic (DOS)                                                                    |
| 850        | ibm850      | OEM Multilingual Latin 1; Western European (DOS)                                            |
| 852        | ibm852      | OEM Latin 2; Central European (DOS)                                                         |
| 855        | ibm855      | OEM Cyrillic (primarily Russian)                                                            |
| 857        | ibm857      | OEM Turkish; Turkish (DOS)                                                                  |
| 860        | ibm860      | OEM Portuguese; Portuguese (DOS)                                                            |
| 861        | ibm861      | OEM Icelandic; Icelandic (DOS)                                                              |
| 862        | dos-862     | OEM Hebrew; Hebrew (DOS)                                                                    |
| 863        | ibm863      | OEM French Canadian; French Canadian (DOS)                                                  |
| 864        | ibm864      | OEM Arabic; Arabic (864)                                                                    |
| 865        | ibm865      | OEM Nordic; Nordic (DOS)                                                                    |
| 866        | cp866       | OEM Russian; Cyrillic (DOS)                                                                 |
| 869        | ibm869      | OEM Modern Greek; Greek, Modern (DOS)                                                       |
| 874        | windows-874 | Thai (Windows)                                                                              |
| 910        | ibm910      | IBM-PC APL2
| 1250       | windows-1250| ANSI Central European; Central European (Windows)                                           |
| 1251       | windows-1251| ANSI Cyrillic; Cyrillic (Windows)                                                           |
| 1252       | windows-1252| ANSI Latin 1; Western European (Windows)                                                    |
| 1253       | windows-1253| ANSI Greek; Greek (Windows)                                                                 |
| 1254       | windows-1254| ANSI Turkish; Turkish (Windows)                                                             |
| 1255       | windows-1255| ANSI Hebrew; Hebrew (Windows)                                                               |
| 1256       | windows-1256| ANSI Arabic; Arabic (Windows)                                                               |
| 1257       | windows-1257| ANSI Baltic; Baltic (Windows)                                                               |
| 1258       | windows-1258| ANSI/OEM Vietnamese; Vietnamese (Windows)                                                   |

# * Benchmarks
`encoding_rs` supports only a few of the encodings that `oem_cp` and `yore` support. Additionally, `encoding_rs` focuses on streaming use cases.

Refer to the [bench crate](https://github.com/bonega/yore/blob/28198ff8d4e487a8f7e6a477fe7cbc19313618c0/benchmark/README.md) for more details.

# Contributing

See [CONTRIBUTING.md](CONTRIBUTING.md) for development setup, benchmarking, and fuzzing.