pack-io 0.3.0

Compact binary wire format with schema evolution and zero-copy deserialization for Rust. The serialization substrate under network-protocol and Hive DB.
Documentation
# pack-io v0.2.0 — Foundation

**The codec is live.** v0.2.0 ships the Tier-1 / Tier-2 / Tier-3 public surface, primitive `Serialize` / `Deserialize` implementations for every type in the `0.2` scope, and the property-based test harnesses that lock the defining invariants — round-trip integrity, determinism, and safe decode under untrusted input — in place from the first release that has anything to defend.

## What is pack-io?

A compact binary wire format for Rust. It exists because no Rust crate today owns **speed + schema evolution + zero-copy deserialization** as a single coherent contract — `bincode` lacks evolution, `rkyv` needs alignment discipline at every use site, `postcard` is embedded-focused. `pack-io` combines them under a small, deterministic wire format that a reader can implement from a normative spec instead of from the source.

## What's new in 0.2.0

### The Tier-1 API

Two free functions, no setup, no type parameters the caller has to name beyond the target type.

```rust
use pack_io::{encode, decode};

let v = (7_u64, true, String::from("hello"));
let bytes = encode(&v).unwrap();
let back: (u64, bool, String) = decode(&bytes).unwrap();
assert_eq!(back, v);
```

[`decode`](https://github.com/jamesgober/pack-io/blob/main/src/codec.rs) is **strict**: it rejects trailing bytes with `SerialError::TrailingBytes`. Callers that want to read several values from a single buffer use the Tier-2 [`Decoder`](https://github.com/jamesgober/pack-io/blob/main/src/codec.rs) directly.

### The Tier-2 `Encoder` / `Decoder`

In-memory, allocation-aware, validated at construction. Re-use one `Vec<u8>` across many encodes; read several values from one buffer; refuse hostile length prefixes before allocating a single byte.

```rust
use pack_io::{Encoder, Decoder, Config};

// One encoder, many writes, no per-call allocation.
let mut buf = Vec::with_capacity(256);
let mut enc = Encoder::into_buffer(buf);
enc.write(&7_u64).unwrap();
enc.write(&"hello").unwrap();
buf = enc.into_inner();

// One decoder, many reads, with a tight allocation cap for untrusted input.
let cfg = Config::new().with_max_alloc(16 * 1024);
let mut dec = Decoder::with_config(&buf, cfg).unwrap();
let n: u64 = dec.read().unwrap();
let s: String = dec.read().unwrap();
assert_eq!((n, s.as_str()), (7, "hello"));
```

`Config` is `#[non_exhaustive]` and validated once at `Decoder::with_config`, not on every operation. The default cap is 1 GiB — large enough to be irrelevant for well-formed inputs and small enough to refuse the obvious `length = u64::MAX` DoS.

### The Tier-3 `Serialize` / `Deserialize` traits

The seam between a Rust value and its wire-format bytes. Built-in primitive implementations live in [`src/impls.rs`](https://github.com/jamesgober/pack-io/blob/main/src/impls.rs); user types implement these traits directly today. The `derive` macro arrives in `0.4`.

```rust
use pack_io::{Encoder, Decoder, Serialize, Deserialize, SerialError};

struct Point { x: i32, y: i32 }

impl Serialize for Point {
    fn serialize(&self, enc: &mut Encoder) -> Result<(), SerialError> {
        self.x.serialize(enc)?;
        self.y.serialize(enc)
    }
}

impl Deserialize for Point {
    fn deserialize(dec: &mut Decoder<'_>) -> Result<Self, SerialError> {
        Ok(Point {
            x: i32::deserialize(dec)?,
            y: i32::deserialize(dec)?,
        })
    }
}
```

### Primitive support — everything in the `0.2` scope

`u8` through `u128`, `i8` through `i128`, `usize` / `isize`, `bool`, `f32`, `f64`, `String`, `&str` (encode), `Vec<u8>`, `&[u8]` (encode), fixed-size arrays `[T; N]` for arbitrary `N`, tuples of arity 1 through 12, `Option<T>`, `Result<T, E>`, `()`, and `&T` (encode). Collections (`Vec<T>`, `HashMap`, `BTreeMap`, sets) land at `0.3` alongside the wire-format freeze.

### Wire format

Documented in detail in [`docs/API.md`](https://github.com/jamesgober/pack-io/blob/main/docs/API.md#wire-format). Highlights:

- **LEB128 varint** for multi-byte unsigned integers, **ZigZag → LEB128** for signed. Same shape as `protobuf`, `postcard`, and `bincode`'s varint mode — a third-party implementer can read pack-io integers without reading our source.
- **1 byte fixed** for `u8` / `i8`. No varint overhead for the common case of a standalone byte outside a `Vec<u8>`.
- **IEEE 754 bit pattern, little-endian** for `f32` / `f64`. NaN, ±Inf, subnormals, and signed zeros all round-trip bit-for-bit. Consumers that care about IEEE equality should compare via `to_bits()` — `NaN != NaN` by spec.
- **Strict** `Option` / `Result` / `bool` tag bytes. Any byte outside the legal set is rejected with `SerialError::InvalidTag` / `SerialError::InvalidBool` — no implicit "treat anything non-zero as true" C-isms.

The encoding is **unstable** through the `0.x` series. The normative spec and freeze land at `0.3`. Wire-format breaks inside the `0.x` line are called out prominently in [`CHANGELOG.md`](https://github.com/jamesgober/pack-io/blob/main/CHANGELOG.md).

### `SerialError` — the single failure type

`#[non_exhaustive]` so MINOR releases can add variants without breaking downstream `match` arms. Variants name a single failure mode each and carry the smallest context needed to act on it. Error messages **never** echo the offending bytes back to the caller — safe to log without further sanitisation.

| Variant | Meaning |
|---------|---------|
| `UnexpectedEof { needed, remaining }` | Decoder needed more bytes than were available. |
| `InvalidLength { declared, remaining }` | Length prefix exceeded the buffer or the configured `max_alloc`. |
| `VarintOverflow` | LEB128 varint exceeded its target width. |
| `IntegerOutOfRange` | Decoded `u64` did not fit in the requested narrower target. |
| `InvalidBool { byte }` | A boolean byte was neither `0x00` nor `0x01`. |
| `InvalidUtf8` | A length-prefixed byte run was not valid UTF-8 when decoding a `String`. |
| `InvalidTag { kind, tag }` | An `Option` / `Result` tag byte was outside `0x00` / `0x01`. |
| `TrailingBytes { remaining }` | Strict `decode` left bytes unread. |

`SerialError` implements `Display`, `Debug`, `Clone`, `PartialEq`, `Eq`, and (under the default `std` feature) `std::error::Error`.

### Test suite — invariants enforced, not aspired to

The defining contracts are property-tested from the first release that has anything to test:

- **Round-trip** ([`tests/roundtrip.rs`](https://github.com/jamesgober/pack-io/blob/main/tests/roundtrip.rs)) — 25 `proptest` properties asserting `decode(encode(v)) == v` for every primitive. Floats compare by `to_bits()` to handle NaN correctly.
- **Determinism** ([`tests/determinism.rs`](https://github.com/jamesgober/pack-io/blob/main/tests/determinism.rs)) — 21 `proptest` properties asserting that encoding the same value twice produces identical bytes. The contract that makes hashing, signing, and content-addressing safe — locked in now, before collections arrive in `0.3` where deterministic key ordering becomes load-bearing.
- **Adversarial decode** ([`tests/adversarial.rs`](https://github.com/jamesgober/pack-io/blob/main/tests/adversarial.rs)) — 20 properties covering random-bytes safety on every public decode entry point (`u64`, `i64`, `u128`, `String`, `Vec<u8>`, `Option<String>`, tuples, arrays), truncation behaviour, hostile length prefixes (`length = u64::MAX`, `length > max_alloc`), and the precise error variants returned for the obvious malformed-input cases (overlong varint, invalid bool, invalid UTF-8, invalid `Option` / `Result` tag, trailing bytes).

The decoder **never** panics, **never** reads past the input, and **never** allocates above `Config::max_alloc`. The `cargo-fuzz` continuous fuzz harness arrives in `0.7`.

### Examples

Three self-contained programs, runnable from a fresh clone:

```bash
cargo run --example basic_roundtrip --release   # Tier-1 round-trip of a tuple
cargo run --example primitive_tour --release    # one encoded value per primitive, with byte counts
cargo run --example reuse_buffer --release      # Tier-2 Encoder + multi-value Decoder, zero per-round alloc
```

## Breaking changes

- The `0.1.0` scaffold release exposed only `pack_io::VERSION`. `0.2.0` adds the codec surface — that is purely additive, so no `0.1.0` code breaks. The version remains in the `0.x` series; the wire format is **not** stable until `1.0`.

## Verification

Run on Windows x86_64, Rust stable + 1.85 (MSRV); identical commands pass on Linux (WSL2 Ubuntu) and via the configured CI matrix on macOS:

```bash
cargo fmt --all -- --check
cargo clippy --all-targets --all-features -- -D warnings
cargo test --all-features
cargo build --no-default-features              # no_std build
RUSTDOCFLAGS="-D warnings" cargo doc --no-deps --all-features
cargo audit
cargo deny check

cargo +1.85 clippy --all-targets --all-features -- -D warnings
cargo +1.85 test --all-features
RUSTDOCFLAGS="-D warnings" cargo +1.85 doc --no-deps --all-features
```

All green. Test counts at this tag (stable, `--all-features`):

- **63** unit tests (in-source `#[cfg(test)]` modules).
- **25** round-trip property tests.
- **21** determinism property tests.
- **20** adversarial decode tests.
- **15** doctests.
- **144** total, every one passing.

All three example programs run end-to-end and round-trip their values.

## What's next

- **0.3.0 — Wire-format freeze + collections + streaming `Encoder` / `Decoder`.** The hard part of the roadmap, deliberately not deferred: lock the wire format down with a normative `docs/WIRE_FORMAT.md` spec a third party can implement from, add `Vec<T>` / `HashMap` / `BTreeMap` / `HashSet` / `BTreeSet` with deterministic key-sorted encoding for hash-based collections, and grow the Tier-2 `Encoder` / `Decoder` to wrap anything that implements `std::io::Write` / `Read`.

## Installation

```toml
[dependencies]
pack-io = "0.2"

# no_std build:
pack-io = { version = "0.2", default-features = false }
```

MSRV: Rust 1.85 (2024 edition).

## Documentation

- [README](https://github.com/jamesgober/pack-io/blob/main/README.md)
- [API Reference](https://github.com/jamesgober/pack-io/blob/main/docs/API.md)
- [CHANGELOG](https://github.com/jamesgober/pack-io/blob/main/CHANGELOG.md)

---

**Full diff:** [`v0.1.0...v0.2.0`](https://github.com/jamesgober/pack-io/compare/v0.1.0...v0.2.0).
**Changelog:** [`CHANGELOG.md`](https://github.com/jamesgober/pack-io/blob/main/CHANGELOG.md#020---2026-05-28).