musli

Müsli

Müsli is a flexible and generic binary serialization framework.

Müsli currently depends on GATs and is nightly-only

We make the following assumptions:

• Anything being deserialized must be fully held in memory and able to hand out contiguous slices of it. This allows users of musli to perform zero-copy deserialization for certain types.

• Decoding is biased to assume strings are encoded verbatim in the format used so that references to strings can always be used.

I've chosen to internally use the term "encoding", "encode", and "decode" because it's common terminology when talking about binary formats. It's also distinct from serde's use of "serialization" allowing for the ease of using both libraries side by side if desired.

Formats

Formats are currently distinguish by supporting various degrees of upgrade stability. A fully upgrade stable serialization format must tolerate that one model can add fields that an older version of the model should be capable of ignoring.

use musli::{Encode, Decode};

#[derive(Debug, PartialEq, Encode, Decode)]
struct Version1 {
    name: String,
}

#[derive(Debug, PartialEq, Encode, Decode)]
struct Version2 {
    name: String,
    #[musli(default)]
    age: Option<u32>,
}

let version2 = musli_wire::to_vec(&Version2 {
    name: String::from("Aristotle"),
    age: Some(62),
})?;

let version1: Version1 = musli_wire::decode(&version2[..])?;

assert_eq!(version1, Version1 {
    name: String::from("Aristotle"),
});

Partial upgrade stability can still be useful as is the case of the musli-storage format below, because reading from storage only requires decoding to be upgrade stable. So if correctly managed with #[musli(default)] this will never result in any readers seeing unknown fields.

use musli::{Encode, Decode};

let version2 = musli_storage::to_vec(&Version2 {
    name: String::from("Aristotle"),
    age: Some(62),
})?;

assert!(musli_storage::decode::<_, Version1>(&version2[..]).is_err());

let version1 = musli_storage::to_vec(&Version1 {
    name: String::from("Aristotle"),
})?;

let version2: Version2 = musli_storage::decode(&version1[..])?;

assert_eq!(version2, Version2 {
    name: String::from("Aristotle"),
    age: None,
});

The available formats and their capabilities are:

recorder? determines whether fields must occur in exactly the order in which they are specified. So reordering fields in such a struct would cause an error. This is only suitable for byte-oriented IPC where data models are strictly synchronized.

missing? determines if the reader can handle missing fields, as exemplified above. This is suitable for on-disk storage.

unknown? determines if the format can skip over unknown fields. This is suitable for network communication.

For every feature you drop, the format becomes more compact and efficient. musli-storage #[musli(packed)] for example is as compact and efficient as bincode while musli-wire is comparable to something like protobuf*.

Usage

Add it to your Cargo.toml:

musli = "0.0.1"
musli-wire = "0.0.1"

The Encode and Decode derives

See the derives module for documentation on how to use the Encode and Decode derives.

Examples

Basic example which uses the [default encoding format]:

use musli::{Encode, Decode};

#[derive(Debug, PartialEq, Encode, Decode)]
struct Struct<'a> {
    name: &'a str,
    age: u32,
}

let mut out = Vec::new();

let expected = Struct {
    name: "Aristotle",
    age: 61,
};

musli_wire::encode(&mut out, &expected)?;
let actual = musli_wire::decode(&out[..])?;

assert_eq!(expected, actual);

License: MIT/Apache-2.0