Module bin_codec

Expand description

#[bin] — the unified whole-message codec.

#[bin] folds three things over one struct: the read codec, the write codec, and a required-by-default builder. Fields are read and written at arbitrary bit offsets, so the same attribute handles byte-aligned headers and sub-byte frames, and any Bits type — including a #[bitfield], #[derive(BitEnum)], or #[bitflags] — drops in as one field with no glue.

§A DNS header, end to end

The 12-byte DNS message header (RFC 1035 §4.1.1) is a good tour: a 16-bit flags word (itself a bitfield of enums and bools) between five u16 counts/ids.

use bnb::{bin, bitfield, BitEnum, u3, u4};

#[derive(BitEnum, Clone, Copy, Debug, PartialEq, Eq)]
#[bit_enum(u4)]
enum OpCode { Query, IQuery, Status, #[catch_all] Other(u4) }

#[derive(BitEnum, Clone, Copy, Debug, PartialEq, Eq)]
#[bit_enum(u4)]
enum RCode { NoError, FormErr, ServFail, NxDomain, #[catch_all] Other(u4) }

// The 16-bit flags word, MSB-first (RFC diagram order), big-endian.
#[bitfield(u16, bits = msb, bytes = big)]
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
struct Flags {
    qr: bool, opcode: OpCode, aa: bool, tc: bool,
    rd: bool, ra: bool, z: u3, rcode: RCode,   // 1+4+1+1+1+1+3+4 = 16
}

#[bin(big)]
#[derive(Debug, Clone, PartialEq)]
struct Header {
    id: u16,
    flags: Flags,        // a 16-bit bitfield, nested as one field
    qdcount: u16,
    ancount: u16,
    nscount: u16,
    arcount: u16,
}

// Build with the required-by-default builder.
let flags = Flags::new().with_qr(true).with_rd(true).with_ra(true);
let h = Header::builder()
    .id(0x1234).flags(flags)
    .qdcount(1).ancount(1).nscount(0).arcount(0)
    .build().unwrap();

// Encode -> 12 bytes; decode is the exact inverse.
let bytes = h.to_bytes().unwrap();
assert_eq!(bytes.len(), 12);
assert_eq!(&bytes[..4], &[0x12, 0x34, 0x81, 0x80]); // id, then flags 0x8180
assert_eq!(Header::decode_exact(&bytes).unwrap(), h);

§The generated API

Every #[bin] type gets a consistent surface:

Direction	Method	Use
decode	`decode(&mut S)`	one message from a `Source` cursor (`BitReader`/socket/file/`BitBuf`), advancing it
decode	`decode_exact(&[u8])`	one message that consumes every whole byte
decode	`decode_all(&[u8]) -> Vec<_>`	every message in the buffer (bit-aware, layout baked in)
decode	`decode_iter(&[u8])`	a lazy iterator over the buffer’s messages
decode	`peek(&[u8])`	one message, tail-tolerant, no buffer mutation
encode	`to_bytes() -> Vec<u8>`	encode to a fresh buffer (verbatim)
encode	`to_canonical_bytes()`	encode the spec-normalized form (canonical) †
encode	`encode(&mut W)`	encode to any `std::io::Write` (verbatim)
encode	`bit_encode(&mut K)`	write into an explicit `Sink` — a `BitEncode` trait method (`canonical_bit_encode` for the canonical form)
build	`builder()`	the required-by-default builder
build	`Name { … }`	a struct literal names every stored field directly

decode_exact/decode_all/peek/to_bytes are the everyday slice/Vec path; decode/encode(&mut W)/bit_encode open the door to the I/O ladder. († to_canonical_bytes and the canonical helpers exist only when the message has a reserved or calc field — see Two encode forms.)

§Decoding a buffer of messages

decode_all / decode_iter walk a &[u8] of back-to-back messages — bit-aware (a message needn’t end on a byte boundary) and decoding in the message’s own byte/bit order:

use bnb::bin;
#[bin(big)]
#[derive(Debug, PartialEq, Eq)]
struct Ping { seq: u16 }

let wire = [0x00, 0x01, 0x00, 0x02, 0x00, 0x03]; // three Pings, back to back
assert_eq!(Ping::decode_all(&wire).unwrap(),
           vec![Ping { seq: 1 }, Ping { seq: 2 }, Ping { seq: 3 }]);
// decode_iter is the lazy form (yields Result per message):
assert_eq!(Ping::decode_iter(&wire).filter_map(Result::ok).count(), 3);

For a cursor — a stream, socket, or BitBuf — reach for decode(&mut S) instead (see the I/O ladder).

§Struct-level options

Inside #[bin(...)]:

big / little — byte order (default big).
bits = msb | lsb — bit order (default msb).
magic = <expr> — a leading constant verified on read, emitted on write.
read_only / write_only — generate only one direction.
no_builder — skip the builder.
forward_only — bound decoding to a forward Source (a seek directive is then a compile error).
ctx(name: Ty, …) — declare context the message needs from its parent.
validate = <path> — a soundness check run by build().

§Each option, by example

Byte and bit order:

use bnb::{bin, u4};

#[bin(little)] // little-endian byte order
#[derive(Debug, PartialEq)]
struct Le { v: u32 }
assert_eq!(Le { v: 0x1234_5678 }.to_bytes().unwrap(), [0x78, 0x56, 0x34, 0x12]);

#[bin(big, bits = lsb)] // the first field lands in the LOW bits of the byte
#[derive(Debug, PartialEq)]
struct Lsb { a: u4, b: u4 }
assert_eq!(Lsb { a: u4::new(0xA), b: u4::new(0xB) }.to_bytes().unwrap(), [0xBA]);

§Byte order × bit order — the natural-layout rule

The two knobs aren’t fully independent — they compose by one rule. Each bit order has a natural byte layout: MSB-first emits a value’s high bits first, so its bytes land big-endian; LSB-first emits low bits first, so they land little-endian (value bit k → stream bit k — exactly the CAN/DBC “Intel” layout, raw |= v << start; raw.to_le_bytes()). The declared byte order swaps a byte-multiple value only when it differs from that natural layout; sub-byte widths are never byte-swapped. So the two identity corners are the real-world conventions — big+msb is network order, little+lsb is DBC-Intel/SMB — and the mixed corners (little+msb, big+lsb) are the deliberate byte swaps.

use bnb::bin;

// `little`+`lsb` is an identity corner: a byte-multiple value needs no swap, so it
// lands in `to_le_bytes` order — the DBC-Intel convention.
#[bin(little, bits = lsb)]
#[derive(Debug, PartialEq)]
struct Intel { v: u32 }
assert_eq!(Intel { v: 0x1234_5678 }.to_bytes().unwrap(), 0x1234_5678u32.to_le_bytes());
assert_eq!(Intel { v: 0x1234_5678 }.to_bytes().unwrap(), [0x78, 0x56, 0x34, 0x12]);

// The `big`+`msb` identity corner is the mirror image: network byte order.
#[bin(big, bits = msb)]
#[derive(Debug, PartialEq)]
struct Net { v: u32 }
assert_eq!(Net { v: 0x1234_5678 }.to_bytes().unwrap(), [0x12, 0x34, 0x56, 0x78]);

Directional codecs and the builder:

use bnb::bin;

#[bin(big, read_only)] // only decodes — no `to_bytes`/`encode`
#[derive(Debug, PartialEq)]
struct Ro { v: u16 }
assert_eq!(Ro::decode_exact(&[0x12, 0x34]).unwrap(), Ro { v: 0x1234 });

#[bin(big, write_only)] // only encodes — no `decode`/`peek`
struct Wo { v: u16 }
assert_eq!(Wo { v: 0x1234 }.to_bytes().unwrap(), [0x12, 0x34]);

#[bin(big, no_builder)] // no `Nb::builder()` — construct directly
#[derive(Debug, PartialEq)]
struct Nb { v: u16 }
assert_eq!(Nb { v: 5 }.to_bytes().unwrap(), [0x00, 0x05]);

forward_only — decode from a non-seekable stream, with a compile-time no-seek guarantee (a #[br(restore_position)] field would then be a compile error):

use bnb::{bin, StreamBitReader};

#[bin(big, forward_only)]
#[derive(Debug, PartialEq)]
struct Hdr { magic: u16, len: u16 }

let data: &[u8] = &[0xCA, 0xFE, 0x00, 0x10]; // `&[u8]` is `Read` but not `Seek`
let mut s = StreamBitReader::new(data);
assert_eq!(Hdr::decode(&mut s).unwrap(), Hdr { magic: 0xCAFE, len: 16 });

ctx(...) — context a message needs from its parent to decode. The parent passes it with #[br(ctx { … })]; standalone, decode_with/decode_with_exact take a …Ctx (built positionally with …Ctx::new). ctx is decode-only: encode stays a plain to_bytes() unless the write side actually reads a ctx param (a keyed bw(map), calc, or write_with), in which case the type gets to_bytes_with/encode_with:

use bnb::bin;

#[bin(big, ctx(len: u16))]
#[derive(Debug, PartialEq)]
struct Body {
    #[br(count = len)]
    data: Vec<u8>,
}

#[bin(big)]
#[derive(Debug, PartialEq)]
struct Packet {
    len: u16,
    #[br(ctx { len })] // pass `len` to `Body`
    body: Body,
}

let p = Packet { len: 3, body: Body { data: vec![1, 2, 3] } };
assert_eq!(p.to_bytes().unwrap(), [0x00, 0x03, 1, 2, 3]);
assert_eq!(Packet::decode_exact(&[0x00, 0x03, 1, 2, 3]).unwrap(), p);

// Standalone: decode needs the context (build it with `BodyCtx::new`); encode is
// plain — `ctx` is decode-only, and `Body`'s write side doesn't read `len`.
let b = Body::decode_with_exact(&[0xAA, 0xBB], BodyCtx::new(2)).unwrap();
assert_eq!(b.data, vec![0xAA, 0xBB]);
assert_eq!(b.to_bytes().unwrap(), vec![0xAA, 0xBB]);

(magic and validate are shown below.)

§Dual-use: `validate` gates the builder, not the parser

validate = path runs a fn(&Self) -> Result<(), impl Display> in build() only. The parser stays permissive — it never rejects representable input — so a deliberately malformed message is still decodable (for fuzzing / interop), even though it can’t be built. It’s also exposed as re-runnable methods: build() checks once, but a value can be mutated before you send it, so validate() / is_valid() re-check the current value on demand (computed, never a stale flag). By convention validate expresses semantic soundness — not calc/reserved fields, which are representational (normalized by to_canonical_bytes), so it’s a property of the message’s meaning that holds for the canonical form too.

use bnb::bin;

#[bin(big, validate = check)]
#[derive(Debug, PartialEq)]
struct Msg { kind: u8, len: u8 }

fn check(m: &Msg) -> Result<(), String> {
    if m.kind == 0 { return Err("kind 0 is reserved".into()); }
    Ok(())
}

assert!(Msg::builder().kind(0).len(4).build().is_err());      // builder: rejected
assert!(Msg::decode_exact(&[0x00, 0x04]).is_ok());            // parser: permissive

let mut m = Msg::builder().kind(1).len(4).build().unwrap();
assert!(m.is_valid());                                        // sound as built
m.kind = 0;                                                   // mutated before sending…
assert!(!m.is_valid());                                       // …re-check catches it

§Derived, never-drifting fields

A length or count you don’t want to store can be read into a temp local and recomputed on write, so it can never disagree with the data. Here len drives a count-bound Vec on read and is recomputed from payload.len() on write:

use bnb::bin;

#[bin(big, magic = 0xCAFEu16)]
#[derive(Debug, PartialEq)]
struct Frame {
    #[br(temp)]
    #[bw(calc = self.payload.len() as u8)]
    len: u8,
    #[br(count = len)]
    payload: Vec<u8>,
}

let f = Frame::builder().payload(vec![0xDE, 0xAD, 0xBE, 0xEF]).build().unwrap();
assert_eq!(f.to_bytes().unwrap(), [0xCA, 0xFE, 0x04, 0xDE, 0xAD, 0xBE, 0xEF]);
assert_eq!(Frame::decode_exact(&[0xCA, 0xFE, 0x02, 0x01, 0x02]).unwrap().payload, vec![1, 2]);

When the count sits immediately before its Vec (as here), the whole triad collapses to one directive on the Vec — #[brw(count_prefix = u8)] payload: Vec<u8> — same wire bytes, plus a checked (never-truncating) length on encode. See directives § count_prefix.

§`temp` + `calc` vs a stored `calc`

Whether a calc field is also temp is the choice between never keep it and keep it but be able to recompute it:

temp + calc (above) — the field is not stored: it’s read into a local, written from the expression, and absent from the struct. Use it for purely-derived prefixes you never need to read back (a len/count). It always recomputes, so it creates no verbatim/canonical gap — there’s nothing to drift.
calc without temp — the field is stored (you can read the decoded value, e.g. an as-received checksum), and to_bytes writes it verbatim while to_canonical_bytes recomputes it. Use it when you want to inspect or preserve the on-wire value (dual-use: send a deliberately-wrong checksum) yet still be able to emit the correct one. This is what makes a message canonical-bearing.

§Two encode forms: verbatim vs canonical

A dual-use codec needs to do two opposite things: reproduce a message exactly (even a malformed one you parsed off the wire), and emit a spec-clean one. So #[bin] gives you both, and never silently picks for you:

to_bytes() is verbatim — it writes exactly what’s stored. Retained reserved bits stay, a stored calc value is written as-is. This is the faithful inverse of decode: decode then to_bytes is byte-for-byte identical.
to_canonical_bytes() is canonical — reserved fields are written as their spec value and calc fields are recomputed, so the result is always spec-compliant.

The two differ only when a message has a reserved or non-temp calc field — so to_canonical_bytes (and the three helpers below) are generated only then; otherwise verbatim is canonical and only to_bytes exists. (temp + calc fields are never stored, so they always recompute — they don’t create a verbatim/canonical gap.)

Three helpers inspect or normalize the value in memory, without encoding: is_canonical(), canonical_diff() (the names of the fields that differ from canonical), and to_canonical(self) -> Self.

§Choosing a form

Every encoder is explicit — nothing silently rewrites your data. to_bytes / bit_encode / the std-writer encode are always verbatim; to_canonical_bytes / canonical_bit_encode are always canonical. To stream the canonical form over a std::io::Write, encode the canonical copy: value.to_canonical().encode(&mut w). A reserved/calc field is an ordinary stored field, so these messages construct via a struct literal, the builder, or decode, and coexist with serde derives fine — one type can carry both codecs (see tests/serde_compat.rs; bnb is deliberately not a serde data format, whose model has no bit widths or byte order).

use bnb::{bin, EncodeExt};

#[bin(big)]
#[derive(Debug, Clone, PartialEq)]
struct Packet {
    tag: u8,
    #[reserved]
    rsv: u8,                       // spec value 0
    #[bw(calc = self.tag ^ 0x5A)]
    #[builder(default)]
    check: u8,                     // canonical value: tag ^ 0x5A
}

// A value a peer sent us with non-spec reserved bits and a stale checksum:
let p = Packet::builder().tag(0x10).rsv(0xFF).check(0x99).build().unwrap();

// VERBATIM — exactly what's stored (so decode -> to_bytes round-trips):
assert_eq!(p.to_bytes().unwrap(), [0x10, 0xFF, 0x99]);

// CANONICAL — reserved -> 0, checksum recomputed (0x10 ^ 0x5A = 0x4A):
assert_eq!(p.to_canonical_bytes().unwrap(), [0x10, 0x00, 0x4A]);

// Inspect the gap without encoding:
assert!(!p.is_canonical());
assert_eq!(p.canonical_diff(), ["rsv", "check"]);

// Stream the canonical form over a writer: encode the canonical copy.
let mut out: Vec<u8> = Vec::new();
p.clone().to_canonical().encode(&mut out).unwrap();
assert_eq!(out, [0x10, 0x00, 0x4A]);

See directives for every field directive, and io for decoding from a socket or file rather than a slice. The examples/bin_message.rs example in the repository is a runnable version of the header + frame above.

Module bin_codec

Module bin_codec Copy item path

§A DNS header, end to end

§The generated API

§Decoding a buffer of messages

§Struct-level options

§Each option, by example

§Byte order × bit order — the natural-layout rule

§Dual-use: validate gates the builder, not the parser

§Derived, never-drifting fields

§temp + calc vs a stored calc

§Two encode forms: verbatim vs canonical

§Choosing a form

Module bin_codec

§Dual-use: `validate` gates the builder, not the parser

§`temp` + `calc` vs a stored `calc`