Crate ender

Expand description

A rust ENcoding and DEcoding library for writing custom protocols and file formats.

It aims to be intuitive, expandable and correct.

§Example

let mut the_matrix = vec![0u8; 256];

// Summon John into existence!
let john = Person {
    name: String::from("John"),
    age: 35,
	height: 1.75,
	eye_color: EyeColor::Brown
};

// Encode John into the matrix
encode_with(SliceMut::new(&mut the_matrix), Context::default(), &john)?;

// Bring him back
let john_2: Person = decode_with(Slice::new(&the_matrix), Context::default())?;

// But is he really the same John?
assert_eq!(john, john_2);

§Encoding format

The encoding process aims at being correct and unsurprising.

Ende supports a series of options, which can be changed during the encoding/decoding process to get certain parts of a binary format to be represented exactly how you want them.

This can be done in a manual implementation as well as with the derive macro, using the custom attributes provided.

Certain types also support “flattening”, which means omitting information known from the context. For instance, you can omit writing whether an Option is present if that information is already stored somewhere else in the file format.

For integer primitives, usize, and enum variants you can customize the endianness, the numerical encoding (read: var-ints), the bit-width (how many bytes does a usize or enum variant take up in your encoding format?), the max-size (to prevent maliciously crafted binary formats to cause unlimited-size allocations).

§Var-int format

Fixed - not var-int, simply encode the number as-is
Leb128
Protobuf - both its zigzag and “wasteful” variants

§String formats

As for strings, currently length-prefixed, null-terminated (with and without a maximum length) strings are supported, as well as the following encoding formats.

Ascii
Utf8
Utf16
Utf32
Windows1252

If you need a new var-int encoding or string encoding added, feel free to open a PR!

§Motivation

One of the main reasons I made this library is because I found myself needing more sophisticate macros and runtime flexibility for existing binary formats.

While for example bincode is perfectly ok for many applications, ender was made with compatibility with existing data formats in mind.

For this very purpose, many internal details of the encoder are exposed through settings or the derive macros themselves, for the purpose of fine-tuning the data format exactly how you want it, while providing an easy-to-understand interface.

§Deriving

A big selling point of ender are its macros, which allow you to heavily customize the codegen through a series of attributes. To learn more about those, check out DERIVE.md in this crate’s repository root.

§MSRV

This crate will always target the latest version of rust, in order to get access to new features as soon as they are released and update the code accordingly if beneficial. Of course, breaking API changes will be accompanied by a major version bump.

§Future plans

I plan on adding support for async io through a feature gate.

Modules§

facade: Contains some fake functions for encryption/decryption, compression/decompression to test the derive macros.
io: An abstraction over the underlying IO implementation, removing unneeded elements and functions, while improving the interoperability with the library.
Custom Write, Read, BorrowRead traits are provided, as well as a compatibility layer with std::io (see Std)

Macros§

val_error

Structs§

BinSettings: An aggregation of NumRepr, SizeRepr, VariantRepr, StringRepr
Context: The state of the encoder, including its options and a flatten state variable
Encoder: The base type for encoding/decoding. Wraps a stream, and a Context.
It’s recommended to wrap the stream in a std::io::BufReader or std::io::BufWriter, because many small write and read calls will be made.
NumRepr: Controls the binary representation of numbers (different from sizes and enum variants). Specifically, controls the Endianness and NumEncoding.
Opaque: An opaque integer type, used to represent enum variants, usize and isize.
SizeRepr: Controls the binary representation of sizes. Specifically, controls the Endianness, the NumEncoding, the BitWidth, and the greatest encodable/decodable size before an error is thrown
StringRepr: Controls the binary representation of strings. Specifically, controls the StrEncoding of strings and chars and the Endianness in which the encoded bytes are ordered.
TaggedError: An EncodingError which also displays all the error stack. This is useful for debugging, because the entire structure tree is displayed.
VariantRepr: Controls the binary representation of enum variants. Specifically, controls the Endianness, the NumEncoding, and the BitWidth.

Enums§

BitWidth: How many bits a size or enum variant will occupy in the binary format. If the value contains more bits, they will be trimmed (lost), so change this value with care
BorrowError
EncodingError: Represents any kind of error that can happen during encoding and decoding
Endianness: Controls the endianness of a numerical value. Endianness is just the order in which the value’s bytes are written.
FlattenError: Represents an error related to the “flatten” functionality, with potentially useful diagnostics
NumEncoding: Controls the encoding of a numerical value. For instance, controls whether the numbers are compressed through a var-int format or if the entire length of their value is encoded.
SeekError
Signedness: The signedness of an integer value - whether it can store negative numbers.
StrEncoding: The encoding method used for strings and chars.
StrLen: The encoding method use for the length of a string.
StringError: Represents an error occurred while encoding or decoding a string, including intermediate conversion errors and the presence of null bytes in unexpected scenarios.

Traits§

Decode: A binary data structure specification which can be decoded from its binary representation.
Encode: A binary data structure specification which can be encoded into its binary representation.
IntoRead: Something that can be turned into a reader compatible with Encoder
IntoWrite: Something that can be turned into a writer compatible with Encoder

Functions§

decode: Decodes the given value by constructing an encoder on the fly and using it to wrap the reader, with the default context.
decode_bytes: Decodes the given value by constructing an encoder on the fly and using it to wrap a byte slice.
decode_bytes_with: Decodes the given value by constructing an encoder on the fly and using it to wrap a byte slice.
decode_with: Decodes the given value by constructing an encoder on the fly and using it to wrap the reader, with the given context.
encode: Encodes the given value by constructing an encoder on the fly and using it to wrap the writer, with the default context.
encode_bytesalloc: Encodes the given value by constructing an encoder on the fly backed by a VecStream, then returning the wrapped vector of bytes.
encode_bytes_withalloc: Encodes the given value by constructing an encoder on the fly backed by a VecStream, then returning the wrapped vector of bytes
encode_with: Encodes the given value by constructing an encoder on the fly and using it to wrap the writer, with the given context.

Type Aliases§

EncodingResult: A convenience alias to Result<T, EncodingError>

Derive Macros§

Decodederive: The Ender Derivonomicon
Encodederive: The Ender Derivonomicon

Crate enderCopy item path