# MessagePacker - a no-std msgpack implementation
[](https://crates.io/crates/msgpacker)
[](https://docs.rs/msgpacker/)
[]()
The protocol specification can be found [here](https://github.com/msgpack/msgpack/blob/master/spec.md).
This crate targets simplicity and performance. It extends [bytes](https://crates.io/crates/bytes) `BufMut`, for increased interoperability.
The library is designed with production in mind, and will return auxiliary, useful runtime data such as buffer consumption bytes count.
## Example
```rust
use msgpacker::{MsgPacker, Packable as _, Unpackable as _};
use std::collections::HashMap;
// boilerplate derives - those aren't required
#[derive(Debug, PartialEq, Eq)]
// this convenience derive macro will implement `Packable` and `Unpackable`
#[derive(MsgPacker)]
pub struct City {
name: String,
// The traits are implemented for stdlib collections. If you have a custom map, you can use the
// directive `#[msgpacker(map)]` so the traits will be automatically implemented through the
// iterators of the map.
inhabitants_per_street: HashMap<String, u64>,
// This is also automatically implemented. The manual implementation is via `#[msgpacker(array)]`.
zones: Vec<String>,
// Binary serialization is specialized in MessagePack. We can use the directive here.
#[msgpacker(binary)]
metadata: Vec<u8>,
}
// create an instance of a city.
let city = City {
name: "Kuala Lumpur".to_string(),
inhabitants_per_street: HashMap::from([
("Street 1".to_string(), 10),
("Street 2".to_string(), 20),
]),
zones: vec!["Zone 1".to_string(), "Zone 2".to_string()],
metadata: b"foo".to_vec(),
};
// serialize the city into bytes
let buf = city.pack_to_vec();
// deserialize the city and assert correctness
let deserialized = City::unpack(&buf).unwrap();
assert_eq!(city, deserialized);
```
We also support borrowed deserialization:
```rust
use msgpacker::{MsgPackerBorrowed, Packable as _, UnpackableBorrowed as _};
#[derive(Debug, MsgPackerBorrowed, PartialEq, Eq)]
pub struct City<'a> {
name: &'a str,
}
// create an instance of a city.
let city = City {
name: "Kuala Lumpur"
};
let buf = city.pack_to_vec();
let deserialized = City::unpack(&buf).unwrap();
assert_eq!(city, deserialized);
```
## Features
- derive: Enables `MsgPacker` derive convenience macro.
- strict: Will panic if there is a protocol violation of the size of a buffer; the maximum allowed size is `u32::MAX`.
- assertions: Will panic on runtime if an invalid buffer is provided for prealloc serialization.
- std: Will implement the `Packable` and `Unpackable` for `std` collections.
- serde: Adds support for [serde](https://crates.io/crates/serde)
## Serde
Version `0.5.0` introduces [serde](https://crates.io/crates/serde) support.
```rust
use msgpacker::serde;
use serde_json::{json, Value};
let val = serde_json::json!({"foo": "bar"});
let ser = serde::to_vec(&val);
let des: Value = serde::from_slice(&ser).unwrap();
assert_eq!(val, des);
```
Serde performance is excelling for deserialization, but is slower for serialization.
For more information, refer to `Benchmarks`.
## Benchmarks
Results obtained with `AMD EPYC 7402P 24-Core Processor`.
The results have their correctness asserted for each benchmark item.
- pack alloc: creates a `Vec` for each serialization. Useful if the user doesn't know the size of his serialized instances. It's the most common case.
- pack prealloc: preallocates a `Vec` with the known serialization length, and executes the packing with a mutable, no-alloc slice.
- unpack owned: unpacks the structure to an owned type. Useful for complex collections such as `HashMap`.
- unpack borrowed: unpacks the structure as a borrowed type from the original bytes. Useful for very simple structures with mostly atomic types suck as `Vec<_>` and `String` (not supported by rmp-serde).
| pack alloc 1 | 67.629 ns | 251.75 ns | 322.38 ns | 277.77 ns |
| pack alloc 10 | 333.36 ns | 764.16 ns | 1.0981 µs | 799.47 ns |
| pack alloc 100 | 3.6478 µs | 4.2132 µs | 6.9683 µs | 4.4160 µs |
| pack alloc 1000 | 66.081 µs | 67.055 µs | 105.28 µs | 66.911 µs |
| pack prealloc 1 | 49.170 ns | 54.865 ns | 94.714 ns | 46.801 ns |
| pack prealloc 10 | 293.30 ns | 367.28 ns | 647.95 ns | 264.44 ns |
| pack prealloc 100 | 2.9884 µs | 3.5127 µs | 6.2381 µs | 2.8508 µs |
| pack prealloc 1000 | 56.887 µs | 65.492 µs | 102.09 µs | 52.176 µs |
| unpack owned 1 | 137.10 ns | 92.482 ns | 251.77 ns | 138.14 ns |
| unpack owned 10 | 1.2414 µs | 1.0115 µs | 2.2232 µs | 992.32 ns |
| unpack owned 100 | 11.820 µs | 9.2322 µs | 22.659 µs | 9.9914 µs |
| unpack owned 1000 | 231.64 µs | 205.06 µs | 333.20 µs | 221.33 µs |
| unpack borrowed 1 | 96.822 ns | 57.465 ns | † | 67.606 ns |
| unpack borrowed 10 | 868.06 ns | 662.06 ns | † | 572.57 ns |
| unpack borrowed 100 | 7.9413 µs | 5.8026 µs | † | 5.4400 µs |
| unpack borrowed 1000 | 127.29 µs | 101.97 µs | † | 102.96 µs |
To run the benchmarks:
```sh
RUSTFLAGS="-C target-cpu=native" cargo bench --profile bench
```
## Non-uniform collections
MessagePack is a language-agnostic format. Dynamically typed languages like Python, JavaScript, and Ruby naturally allow mixed-type collections — for instance, a Python list `[0, 1694166331209.0]` containing both an integer and a float is perfectly valid. When these values are serialized into MessagePack, the resulting byte stream encodes each element with its own type tag (`u64`, `f64`, etc.), producing an array whose elements have heterogeneous types.
Rust's type system does not directly support such collections: a `Vec<T>` requires a single concrete `T`. As noted in [#18](https://github.com/codx-dev/msgpacker/issues/18), the native `Packable`/`Unpackable` traits cannot deserialize these non-uniform arrays because they rely on a statically known element type at compile time.
The `serde` feature provides a workaround: deserialize the MessagePack bytes into `serde_json::Value`, which is a dynamically typed enum that can represent any JSON-compatible value. **This will incur performance overhead** compared to the native traits, since serde uses a visitor pattern that involves runtime type dispatch and heap allocations for every element.
```rust
use msgpacker::serde;
use serde_json::Value;
// MessagePack bytes encoding a 2-element array: [0_u64, 1694166331209.0_f64]
// This kind of payload is common when receiving data from Python, JS, or other
// dynamically typed languages that don't distinguish collection element types.
let bytes: &[u8] = &[146, 0, 203, 66, 120, 167, 66, 234, 244, 144, 0];
// Deserialize into a dynamic Value — works for any valid MessagePack payload
let value: Value = serde::from_slice(bytes).unwrap();
let items = value.as_array().unwrap();
// Each element retains its original type
assert!(items[0].is_u64());
assert!(items[1].is_f64());
```
If your use case involves only uniform collections (e.g. `Vec<u64>`), prefer the native `Packable`/`Unpackable` traits for zero-overhead deserialization.
# Note on Github
Although GitHub offers exceptional CI and hosting services virtually for free, its questionable approach towards user sovereignty and privacy is notable. Consequently, I chose to disengage from their infrastructure.
For more information, check [Give Up GitHub!](https://sfconservancy.org/GiveUpGitHub/)