Crate base256u 

base256u

Just a simple Rust crate to map between bytes and unicode glyphs. Includes reference printable-ascii-preserved Unicode (papu) encoder and decoder functions, as well as emoji ones. The papu encoding will preserve all text that is already only printable ascii characters and all the other bytes map to single-codepoint non-combining printable glyphs, skipping odd things like NBSP and SHY.

Creating your own custom encodings is trivial as well.

Using this crate is as simple as use base256u::{Decode, Encode}; and then calling the base256u() method or base256u_papu() to get the default papu encoding.

Encoding

use base256u::Encode as _;

let encoded: String = (u8::MIN..=u8::MAX).base256u_papu().collect();
assert_eq!(encoded, r##"°±²³´µ¶·¸¹º»¼½¾¿ÀÁÂÃÄÅÆÇÈÉÊËÌÍÎÏ !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~§ĀāĂ㥹ĆćĈĉĊċČčĎďĐđĒēĔĕĖėĘęĚěĜĝĞğĠġĢģĤĥĦħĨĩĪīĬĭĮįİıIJijĴĵĶķĸĹĺĻļĽľĿŀŁłŃńŅņŇň¤ŊŋŌōŎŏŐőŒœŔŕŖŗŘřŚśŜŝŞşŠšŢţŤťŦŧŨũŪūŬŭŮůŰűŲųŴŵŶŷŸŹźŻżŽžſ"##);
let encoded: String = b"Pack my box with five dozen liquor jugs."
    .into_iter()
    .copied()
    .base256u_papu()
    .collect();
assert_eq!(encoded, "Pack my box with five dozen liquor jugs.");

Decoding

use base256u::Decode as _;

let decoded: Vec<Result<u8, char>> = r##"°±²³´µ¶·¸¹º»¼½¾¿ÀÁÂÃÄÅÆÇÈÉÊËÌÍÎÏ !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~§ĀāĂ㥹ĆćĈĉĊċČčĎďĐđĒēĔĕĖėĘęĚěĜĝĞğĠġĢģĤĥĦħĨĩĪīĬĭĮįİıIJijĴĵĶķĸĹĺĻļĽľĿŀŁłŃńŅņŇň¤ŊŋŌōŎŏŐőŒœŔŕŖŗŘřŚśŜŝŞşŠšŢţŤťŦŧŨũŪūŬŭŮůŰűŲųŴŵŶŷŸŹźŻżŽžſƝʼn"##.chars().base256u_papu().collect();
let mut matcher: Vec<Result<u8, char>> = (u8::MIN..=u8::MAX).map(Ok).collect();
matcher.push(Err('Ɲ'));
matcher.push(Err('ʼn'));
assert_eq!(decoded, matcher);
let decoded: Vec<u8> = "Pack my box with five dozen liquor jugs."
    .chars()
    .base256u_papu()
    .map(|c| c.unwrap())
    .collect();
assert_eq!(
    String::from_utf8(decoded).unwrap(),
    "Pack my box with five dozen liquor jugs."
);

Structs

Decoder

Decoding iterator, converting unicode chars into bytes based on the contained decoding callable. Must be fallible because not all unicode codepoints are valid bytes.

Encoder

Encoder iterator, converting bytes into unicode chars based on the contained encoding callable.

Traits

Decode

Iterator adapter trait, adding methods to iterators that already yield chars (or optional or result chars).

DecodeItem

A trait of a decodable item. This is used to enable Option and Result with passthrough.

Encode

Iterator adapter trait, adding methods to iterators that already yield bytes (or optional or result bytes).

EncodeItem

A trait of an encodable item. This is used to enable Option and Result with passthrough.

Functions

decode_basic

Decoding function for encoding to a straight 1-to-1 byte-to-codepoint conversion. Does the reverse of encode_basic.

decode_emoji

Decode function for emoji. Does the reverse of encode_emoji.

decode_papu

Decode function for printable-ascii-preserving Unicode. Does the reverse of encode_papu.

decode_tpapu

Decode function for tight-printable-ascii-preserving Unicode. Does the reverse of encode_tpapu.

encode_basic

Encode function for encoding to a straight 1-to-1 byte-to-codepoint conversion.

encode_emoji

Encode function for encoding to emoji.

encode_papu

Encode function for encoding to printable-ascii-preserving Unicode.

encode_tpapu

Encode function for encoding to tight-printable-ascii-preserving Unicode. Similar to papu, but packed as tightly as possible while mapping to all printable characters, skipping deprecated and invisible characters (SHY and NSBP, primarily).

Type Aliases

DecodeOutput