ergo-sbe

ergo-sbe generates binary-compatible Rust SBE codecs with compile-time wire-order enforcement, closure-based groups, exact buffer sizing, and zero heap allocation on hot paths.

	ergo-sbe
Wire-order safety	Compile-time type-state stages — calling `asks` before `bids` is a type error, not a runtime bug
Exact buffer sizing	`compute_length_with_header(…)` gives the exact byte count before you encode — no oversize scratch buffers, works directly with Aeron `try_claim`
Closure-based groups	`bids(n, \|g\| g.add(\|e\| { … }))` — nests like the schema, no `.parent()` hopscotch
Trust boundary	`try_from` / `try_wrap` for untrusted input; `wrap` for trusted — explicit in the type system
Composite wire images	`#[repr(transparent)] Engine([u8; N])` — the value IS the on-wire bytes, zero-copy with portable LE/BE accessors
Domain types	Map wire `Decimal` to `rust_decimal::Decimal` at the codec boundary — one line of config, no hand-rolled converters
Bulk group ops	`bulk_add(&[Entry])` / `bulk_decode()` — measured about 22-23% lower encode latency than `add()` for 1,000-entry flat groups on the audited Apple M4 profiles; eligible DTO groups use an allocation-free domain bulk writer automatically
Zero dependencies at runtime	Generated codecs embed their own `sbe_rt` — no `ergo-sbe` on your critical path

// build.rs — one call, no template files
ergo_sbe::generate_to_out_dir("schemas/market-data.xml", "market_data")?;

let expected_len = QuoteEncoder::compute_length_with_header(1, 13);
let mut buf = vec![0u8; expected_len];
let len = QuoteEncoder::wrap_and_apply_header(&mut buf, 0)
    .fixed(&QuoteFixedFields {
        seq: 1,
        some_numbers: [10, 20, 30, 40],
        vehicle_code: *b"ABC012",
        qty: 500,
    })
    .legs(1, |legs| {
        legs.add(|leg| { leg.value(42u32); Ok(()) })?;
        Ok(())
    })?
    .note(b"optional note")?
    .encoded_length_with_header();
assert_eq!(len, expected_len);
let encoded = &buf[..len];

let dec = QuoteDecoder::try_wrap_and_apply_header(encoded, 0)?;
assert_eq!(dec.seq(), 1);
assert_eq!(dec.some_numbers(), [10, 20, 30, 40]);
let mut legs = dec.into_legs()?;
while let Some(entry) = legs.next() {
    assert_eq!(entry.value(), 42u32);
}
let (note, _) = legs.finish()?.into_note()?;
assert_eq!(note, b"optional note");

AI assistance. Large parts of this project were written with heavy AI assistance. Humans directed the work, approved designs, and ran verification. Details: AI-ASSISTANCE.md.

ergo-sbe parses Simple Binary Encoding (SBE) schemas and generates Rust codecs that are binary-compatible with the official SBE wire format (header, field layout, groups, var-data, byte order).

It is not a line-for-line port of the java/rust sbe-tool stubs. The goals for the generated API are:

Easier to use — especially nested groups and var-data under Rust’s borrow checker
Safer — wire order and trust boundaries enforced by types / Result
Easier to read — nested structure looks like the schema, not a pile of temporary handles

Still built for low-latency and binary-compatible SBE. The style uses named stage structs (not Encoder<State> generics), closures + method chaining for groups, checked entry points, version-aware accessors, and optional domain/conversion helpers.

Why not “Java-style” parent hopping?

In sbe-tool you often juggle flyweights and call something like .parent() to hand ownership back up the tree. In Rust that fight becomes borrow-checker pain: move a group encoder in, get stuck returning the parent, lose the thread of the code.

ergo-sbe leans on scoped closures and chaining so nested schemas stay readable and you rarely pass encoder ownership field-to-field by hand:

// Nested shape mirrors the schema — no .parent() hopscotch.
enc.fixed(&fields)
    .bids(n, |bids| {
        bids.add(|level| {
            level.price(p).size(s);
            level.orders(m, |ords| {
                ords.add(|o| { o.order_id(id); Ok(()) })?;
                Ok(())
            })?;
            Ok(())
        })?;
        Ok(())
    })?
    .ask_var_data(bytes)?;

Wire parity is exercised three ways: official Java .sbe fixtures, live dual-encode suites that require ergo-sbe and sbe-tool Rust bytes to be identical (sbe_tool_wire_parity_test for deep Car matrices; sbe_tool_multi_schema_wire_parity_test across example/unit schemas with checked-in sbe-tool reference crates under sbe/tests/sbe_tool_reference/), and a maintained benchmark gate versus sbe-tool-generated codecs (see BENCHMARKS.md).

Early release (0.x). This is the first published line of the crate. The experimental banner stays until the project has been battle-tested in enough real production environments — not merely until unit tests pass.

Binary compatibility is covered by a large automated suite (golden bytes, schema edge cases, parity benches). That is necessary, not sufficient, for removing this warning.

If you use ergo-sbe in production, please say so (GitHub issue or discussion). Hearing from heavy production users is how this banner goes away. Until then, expect possible API and generated-surface churn on the 0.x series, and pin versions deliberately.

What we most want reports on (open an issue titled e.g. production-use: <your domain>):

Live multi-schema / multi-template streams (not only unit fixtures)

Domain DTOs (enable_domain_objects) in a real app path — especially DomainVarData::LossyStrings re-encode behaviour

Exact buffer sizing + Aeron/IPC try_claim (no oversize scratch buffers)

Nested/ragged books or similar twin groups (bids/asks order safety)

Schema evolution (sinceVersion) under mixed acting versions

Quick start — generate_to_out_dir + sbe_mod! → first encode/decode
Compile-checked feature tour — fixed/dynamic messages, arrays, stages, DTOs, dispatch
Core ideas — trust boundary, wire order, buffer sizing, flyweight vs whole struct, composite LE layout
Feature matrix — full capability scan
Recipes — encode known/unknown groups, Display, DTO, conversion
Configuration — wire vs app types, with_conversion / with_domain_type
Samples
Rust version
Verify
Package scope

Names in snippets use Heartbeat, Quote, or FixedString from the docs_codec fixture — your types follow your schema names.

Every bare rust fence in this README is extracted and compiled by docs_validation_test. Schematic fragments (build scripts, config, generated API illustrations) use rust,no_run or xml fences. The Heartbeat and Quote snippets below compile against a small schema fixture generated by the current ergo-sbe code generator, so an API change cannot silently leave these examples stale.

Quick start

1. Depend on the generator

Minimal product path — codegen only; generated codecs embed their own sbe_rt and do not link ergo-sbe into the application. Schema parse errors render a source snippet (line + span) by default:

[build-dependencies]
ergo-sbe = "0.1"
# no [dependencies] ergo-sbe

Convenience path — also pull ergo-sbe as a normal dependency when you use sbe_mod! / include_sbe! (macros expand in the app crate):

[build-dependencies]
ergo-sbe = "0.1"

[dependencies]
ergo-sbe = "0.1"   # only needed for sbe_mod! / include_sbe!

See Samples for monorepo crates that use each pattern.

2. Generate in `build.rs` (short form)

generate_to_out_dir parses the schema file, generates codecs, writes $OUT_DIR/{module}.rs, and emits cargo::rerun-if-changed for you:

// build.rs
//
// Use `ergo_sbe::miette::Result`, not `Box<dyn std::error::Error>` — on a
// malformed schema, `Box<dyn Error>` prints a raw `Debug` dump (unreadable
// struct fields). `miette::Result` renders the actual XML snippet with a span
// pointing at the bad element. No separate `miette` dependency needed — it is
// re-exported and enabled by default.
fn main() -> ergo_sbe::miette::Result<()> {
    ergo_sbe::generate_to_out_dir(
        "schemas/messages.xml",
        ergo_sbe::GenerationConfig::new("messages"),
        // .enable_domain_objects(DomainVarData::LossyStrings)  // String var-data on DTOs
        // .with_domain_type(…)?
    )?;
    Ok(())
}

Schema from a string / include_str!: generate_str_to_out_dir (add your own cargo::rerun-if-changed for the file you included).

Need multi-schema or custom output paths? Use the lower-level parse_file + Generator API (same steps the helper runs). For shared types across schemas, see Multi-schema patterns below.

3. Include generated code

Prefer build-dep only for product crates (no runtime ergo-sbe link). Generated codecs embed sbe_rt; plain include! is enough:

// Module name must match GenerationConfig::new("messages") → messages.rs
#[allow(dead_code, unused_imports, non_camel_case_types, non_snake_case, clippy::all)]
mod messages {
    include!(concat!(env!("OUT_DIR"), "/messages.rs"));
}
use messages::*;

Optional convenience — sbe_mod! needs ergo-sbe as a normal dependency (macro expansion only; not required for encode/decode):

// Cargo.toml: [dependencies] ergo-sbe = "0.1"
ergo_sbe::sbe_mod!(messages);
use messages::*;
// Or only the include: ergo_sbe::include_sbe!("messages");

See Samples · samples README for which crates use which pattern.

4. Encode and decode (fixed message)

Two styles — pick whichever fits:

Individual setters (chainable, set only what you need):

let mut buf = [0u8; HeartbeatEncoder::compute_length_with_header()];
let len = HeartbeatEncoder::try_wrap_and_apply_header(&mut buf, 0)?
    .seq(7)
    .encoded_length_with_header();
let dec = HeartbeatDecoder::try_from(&buf[..len])?;
assert_eq!(dec.seq(), 7);

fixed() struct (fill every field at once — compile error if a field is missing):

let mut buf = [0u8; HeartbeatEncoder::compute_length_with_header()];
let len = HeartbeatEncoder::try_wrap_and_apply_header(&mut buf, 0)?
    .fixed(&HeartbeatFixedFields { seq: 7 })
    .encoded_length_with_header();
let dec = HeartbeatDecoder::try_from(&buf[..len])?;
assert_eq!(dec.seq(), 7);

C-style fixed-width strings: pass a shorter &str — auto-padded with NULs. On decode, copy_* copies the raw bytes into your buffer:

let mut buf = [0u8; FixedStringEncoder::compute_length_with_header()];
let len = FixedStringEncoder::try_wrap_and_apply_header(&mut buf, 0)?
    .code_str("ABC")?
    .encoded_length_with_header();
let dec = FixedStringDecoder::try_from(&buf[..len])?;
let mut code = [0u8; 6];
dec.copy_code(&mut code);
assert_eq!(&code, b"ABC\0\0\0");

Start here for a full runnable map of features: sbe-feature-tour (cargo run --manifest-path samples/sbe-feature-tour/Cargo.toml).
More recipes: Recipes.

5. Method chaining — the preferred encode style

ergo-sbe encoders are designed so the entire encode reads as one expression, from try_wrap_and_apply_header through .fixed(...) and every dynamic tail, ending in .encoded_length_with_header() (or .as_bytes() on a complete stage when you need the raw slice). Bind only the resulting length; do not retain intermediate encoder variables.

Prefer (one chain, one let):

let mut buf = [0u8; QuoteEncoder::compute_length_with_header(1, 2)];
let len = QuoteEncoder::try_wrap_and_apply_header(&mut buf, 0)?
    .fixed(&QuoteFixedFields {
        seq: 1,
        some_numbers: [10, 20, 30, 40],
        vehicle_code: *b"ABCDEF",
        qty: 25,
    })
    .legs(1, |legs| {
        legs.add(|leg| {
            leg.value(99);
            Ok(())
        })?;
        Ok(())
    })?
    .note(b"ok")?
    .encoded_length_with_header();

Avoid (interrupted chain, rebinding):

// Each `let` breaks the chain and splays the pipeline across the screen.
// The `.unwrap()` calls are a code smell — the fallible chain should use `?`.
let enc = QuoteEncoder::try_wrap_and_apply_header(&mut buf, 0)?.fixed(&fields);
let enc = enc.legs(1, |legs| {
    legs.add(|leg| { leg.value(99); Ok(()) })?;
    Ok(())
}).unwrap();
let enc = enc.note(b"ok").unwrap();
let len = enc.encoded_length_with_header();

Every encoder stage is chainable — fixed setters such as price() and qty() return &mut Self; fallible group/var-data transitions return Result<NextStage, _> and compose with ? in the same expression. Intermediate encoder rebinding and manual .unwrap() defeat this design.

Multi-schema patterns

SBE schemas often share types (messageHeader, groupSizeEncoding, composites, enums, sets). ergo-sbe supports two approaches:

Approach	When	Method
`xi:include` (standard)	Schema files live together; official SBE portability matters	`<include href="common-types.xml"/>` — `parse_file` resolves includes relative to the base dir
Shared `Ir` (programmatic)	Schemas are parsed from strings, generated, or live in separate repos; no filesystem dependency	`parse_with_shared` / `parse_file_with_shared` — seed one parse from another's resolved types

The <include> path is what the SBE spec expects. The shared-Ir path is a convenience for tooling, build scripts, and any workflow where you already have the shared schema parsed in memory.

Shared `Ir` — parse, then share

// 1. Parse the shared schema once (composites, enums, sets).
let common = ergo_sbe::parse_file("schemas/common-types.xml")?;

// 2. Parse a consumer schema — no <include> needed.
let orders = ergo_sbe::parse_file_with_shared("schemas/orders.xml", &common)?;

// 3. Each schema gets its own module.
let generator = ergo_sbe::Generator::new(
    ergo_sbe::GenerationConfig::new("common_types").with_shared_module("common_types"),
);
let modules = generator.generate_multi(&[
    (&ergo_sbe::Schema::from_ir(common), "common_types"),
    (&ergo_sbe::Schema::from_ir(orders), "orders"),
])?;

With with_shared_module("common_types"), the first entry owns the shared enums/sets/composites; later entries pub use super::common_types::* and skip duplicate type generation.

`parse_with_shared` from in-memory strings

let common = ergo_sbe::parse(
    r#"<?xml version="1.0"?>
<messageSchema package="common" id="0" version="1" byteOrder="littleEndian">
  <types>
    <composite name="messageHeader">
      <type name="blockLength" primitiveType="uint16"/>
      <type name="templateId" primitiveType="uint16"/>
      <type name="schemaId" primitiveType="uint16"/>
      <type name="version" primitiveType="uint16"/>
    </composite>
    <composite name="Price">
      <type name="mantissa" primitiveType="int64"/>
      <type name="exponent" primitiveType="int8"/>
    </composite>
  </types>
</messageSchema>"#,
)?;

// No <types> / <include> — Price resolves from `common`.
let orders = ergo_sbe::parse_with_shared(
    r#"<?xml version="1.0"?>
<messageSchema package="orders" id="1" version="1" byteOrder="littleEndian"
               headerType="messageHeader">
  <message name="NewOrder" id="1">
    <field name="price" id="1" type="Price"/>
  </message>
</messageSchema>"#,
    &common,
)?;

The shared Ir path does not recover bare top-level <type> typedefs (those are inlined during parsing and dropped from the token stream). Reference them through a <composite> / <enum> / <set> in the shared schema instead.

Full `build.rs` — parse → share → generate

// build.rs
fn main() -> miette::Result<()> {
    let common = ergo_sbe::parse_file("schemas/common-types.xml")?;
    let orders = ergo_sbe::parse_file_with_shared("schemas/orders.xml", &common)?;
    let fills  = ergo_sbe::parse_file_with_shared("schemas/fills.xml", &common)?;

    let generator = ergo_sbe::Generator::new(
        ergo_sbe::GenerationConfig::new("common_types")
            .with_shared_module("common_types"),
    );
    let modules = generator.generate_multi(&[
        (&ergo_sbe::Schema::from_ir(common), "common_types"),
        (&ergo_sbe::Schema::from_ir(orders), "orders"),
        (&ergo_sbe::Schema::from_ir(fills),  "fills"),
    ])?;

    for m in modules {
        let out = std::path::Path::new(&std::env::var("OUT_DIR")?).join(&m.path);
        std::fs::write(&out, m.source)?;
        println!("cargo::rerun-if-changed=schemas/{}", m.path.replace(".rs", ".xml"));
    }
    Ok(())
}

Consumer modules import the shared module for cross-schema type resolution:

mod common_types { include!(concat!(env!("OUT_DIR"), "/common_types.rs")); }
mod orders {
    use super::common_types::*;       // shared types + header composite
    include!(concat!(env!("OUT_DIR"), "/orders.rs"));
}
mod fills {
    use super::common_types::*;
    include!(concat!(env!("OUT_DIR"), "/fills.rs"));
}

See also: sbe-codegen-examples (reusable generator setup), multi_schema_versioning_test (versioned schemas with shared types), exchange-example (multi-schema exchange feed with IPC).

Compile-checked feature tour

These examples use two generated fixture messages:

Heartbeat: one fixed seq: uint32 field.
Quote: fixed fields, a four-element array, a fixed ASCII code, a repeating legs group, and a length-prefixed note.

Exact size, then staged encode

Dynamic messages expose schema-aware size APIs. Flat shapes get a direct checked helper; nested or ragged shapes get a staged *EncodedLength builder. Allocate or claim exactly that many bytes, then write groups and var-data in wire order:

let mut buf = [0u8; QuoteEncoder::compute_length_with_header(1, 2)];
let len = QuoteEncoder::try_wrap_and_apply_header(&mut buf, 0)?
    .fixed(&QuoteFixedFields {
        seq: 1,
        some_numbers: [10, 20, 30, 40],
        vehicle_code: *b"ABCDEF",
        qty: 25,
    })
    .legs(1, |legs| {
        legs.add(|leg| {
            leg.value(99);
            Ok(())
        })?;
        Ok(())
    })?
    .note(b"ok")?
    .encoded_length_with_header();
assert_eq!(len, buf.len());

Bulk arrays and metadata

Generated bulk helpers avoid per-element boilerplate, while constants and MetaAttribute expose schema metadata:

let mut buf = [0u8; QuoteEncoder::compute_length_with_header(0, 0)];
let len = QuoteEncoder::try_wrap_and_apply_header(&mut buf, 0)?
    .fixed(&QuoteFixedFields {
        seq: 7,
        some_numbers: [1, 2, 3, 4],
        vehicle_code: *b"EURUSD",
        qty: 10,
    })
    .legs(0, |_| Ok(()))?
    .note(b"")?
    .encoded_length_with_header();

let quote = QuoteDecoder::try_from(&buf[..len])?;
assert_eq!(quote.some_numbers(), [1, 2, 3, 4]);
let mut code = [0u8; 6];
assert_eq!(quote.copy_vehicle_code(&mut code), code.len());
assert_eq!(&code, b"EURUSD");
assert_eq!(QuoteDecoder::SEQ_ID, 1);
assert_eq!(
    QuoteDecoder::seq_meta_attribute(sbe_rt::MetaAttribute::Presence),
    Some("required")
);

Consuming decode stages

Groups and var-data are consumed in schema order. finish() hands the next named stage back to you:

let mut buf = [0u8; QuoteEncoder::compute_length_with_header(1, 5)];
let len = QuoteEncoder::try_wrap_and_apply_header(&mut buf, 0)?
    .fixed(&QuoteFixedFields {
        seq: 1,
        some_numbers: [1, 2, 3, 4],
        vehicle_code: *b"ABCDEF",
        qty: 10,
    })
    .legs(1, |legs| {
        legs.add(|leg| {
            leg.value(99);
            Ok(())
        })
    })?
    .note(b"hello")?
    .encoded_length_with_header();

let quote = QuoteDecoder::try_from(&buf[..len])?;
let mut legs = quote.into_legs()?;
let leg = legs.next().expect("one leg was encoded");
assert_eq!(leg.value(), 99);
let after_legs = legs.finish()?;
let (note, _) = after_legs.into_note()?;
assert_eq!(note, b"hello");

Validate untrusted input

Checked entry points validate the message header and fixed block. verify walks the complete dynamic tail before trusted access:

let mut buf = [0u8; HeartbeatEncoder::compute_length_with_header()];
let len = HeartbeatEncoder::try_wrap_and_apply_header(&mut buf, 0)?
    .fixed(&HeartbeatFixedFields { seq: 42 })
    .encoded_length_with_header();

HeartbeatDecoder::verify(&buf[..len])?;
assert_eq!(HeartbeatDecoder::try_from(&buf[..len])?.seq(), 42);
assert!(HeartbeatDecoder::try_from(&buf[..4]).is_err());
assert!(HeartbeatDecoder::verify(&buf[..4]).is_err());

Owned domain objects

Latency-sensitive paths: never use DTOs. Domain objects allocate (Vec, String) and copy every field out of the wire buffer. If latency matters, use the zero-copy flyweight decoder instead. DTOs are for tooling, logging, and offline processing — not the hot path.

DTO construction still owns and allocates its Vec/String fields. Re-encode does not add another allocation: wire-compatible flat groups automatically use the generated bulk_add_domain(&[EntryDomain]) path, which validates one complete output region and writes directly from the DTO slice. Groups with nested tails, var-data, optional/versioned fields, domain conversions, or bool domain remapping retain the general per-entry path.

Enable domain objects during generation when an owned application value is more convenient than a zero-copy flyweight. This fixture uses DomainVarData::Bytes, so re-encoding preserves arbitrary bytes:

let mut buf = [0u8; QuoteEncoder::compute_length_with_header(0, 2)];
let len = QuoteEncoder::try_wrap_and_apply_header(&mut buf, 0)?
    .fixed(&QuoteFixedFields {
        seq: 3,
        some_numbers: [1, 2, 3, 4],
        vehicle_code: *b"ABCDEF",
        qty: 10,
    })
    .legs(0, |_| Ok(()))?
    .note(b"\x00\xff")?
    .encoded_length_with_header();

let dto = QuoteDomain::try_from_decoder(QuoteDecoder::try_from(&buf[..len])?)?;
assert_eq!(dto.note, b"\x00\xff");
let expected = dto.encoded_length_with_header()?;
let mut output = [0u8; 256];
let written = dto.encode(&mut output[..expected])?;
assert_eq!(&output[..written], &buf[..len]);

Multi-template dispatch

AnyMessage reads the generated header layout and dispatches on the template ID:

let mut buf = [0u8; HeartbeatEncoder::compute_length_with_header()];
let len = HeartbeatEncoder::try_wrap_and_apply_header(&mut buf, 0)?
    .fixed(&HeartbeatFixedFields { seq: 11 })
    .encoded_length_with_header();

match AnyMessage::decode(&buf[..len], 0)? {
    AnyMessage::Heartbeat(heartbeat) => assert_eq!(heartbeat.seq(), 11),
    AnyMessage::Quote(_) | AnyMessage::FixedString(_) | AnyMessage::Unknown { .. } => {
        return Err("expected Heartbeat".into());
    }
}

The complete runnable version is sbe-feature-tour; focused nested/ragged group sizing is in l3-book.

Core ideas

Design rationale and internal details — skip this section on first read. The Quick start and Recipes cover everyday usage.

Trust boundary

API	When
`try_from` / `try_wrap_and_apply_header` / `try_*`	Untrusted buffers (network, file, other process)
`wrap` / trusted companions	Buffer already validated (or built by you this turn)
`verify`	Walk the full dynamic tail before trusting accessors

Wire order via named stage structs

SBE is a positional wire format: groups and var-data appear in a fixed schema order with no per-field tags on the wire. That matters a lot in financial markets, where it is common to have two nearly identical repeating groups back-to-back — e.g. bids then asks (same entry layout, different meaning). If you encode or decode them in the wrong order, the bytes still look like a valid message: prices and sizes land in the opposite book side. You only discover the disaster at runtime (wrong trades, inverted books, silent corruption). Compile-time order exists so that mistake becomes a type error while you still have the schema in front of you, not a production incident.

Order is enforced with the same idea as the classic type-state pattern (Encoder<State> / PhantomData), but not that implementation.

Implementation note: an early design did use generic type-state stages. On some encode paths that was about ~17% slower than comparable free-order flyweights. Profiling pointed at LLVM failing to optimise through the type-parameter stage chain the way it does for plain monomorphic code. The API was switched to named stage structs — same compile-time “you can only call the next legal method” behaviour, without the generic tax on the hot path.

Generated code emits separate types for each stage, same fields, different methods:

// Approximate generated shape — not Encoder<AfterBids>:
pub struct BookEncoder<'a> { /* buf, pos, … */ }
pub struct BookAfterBids<'a> { /* same layout */ }
pub struct BookAfterAsks<'a> { /* same layout */ }
// …

impl BookEncoder<'a> {
    pub fn bids(self, …) -> Result<BookAfterBids<'a>, …> { … }
    // no asks() here — bids first on the wire
}
impl BookAfterBids<'a> {
    pub fn asks(self, …) -> Result<BookAfterAsks<'a>, …> { … }
    // no bids() here — already done
}

So after fixed fields you may only call the next group/var-data in schema order. Calling asks before bids is a type error (BookEncoder has no asks method). Decoders use the same idea: consuming stages (BookDecoder → BookDecoderAfterBids → …).

Group bodies use |g| { g.add(|e| { … }) } so the outer encoder is not left half-borrowed while you fill nested levels — the closure ends, then chaining continues. That is intentional API ergonomics for Rust (avoids .parent() style ownership hand-offs that fight the borrow checker on deep books).

Buffer sizing

Why this exists: true zero-copy publish on Aeron (and similar systems) uses try_claim / a pre-sized slot. The transport hands you a buffer of a known length; you must know the full encoded message size before you write. Guessing with an oversized scratch Vec and copying later defeats that model and is easy to get wrong for groups and var-data.

ergo-sbe therefore generates schema-aware length APIs so you describe the shape you are about to encode (counts, nested groups, var-data byte lengths) and get an exact size first — safer and easier than hand-computing header + block + Σ(groups) + Σ(var-data).

Message shape	Generated sizing	Prefer
Fixed only	`{Msg}Encoder::compute_length_with_header()` (const)	stack / claim of that length
Groups / nested / ragged	`{Msg}EncodedLength` staged builder	`len` then encode into a claim/slot of `len`

// Exact size first (Car example), then encode into a slot of that length —
// e.g. Aeron try_claim, or any &mut [u8] with len == claim.
let len = CarEncodedLength::new()
    .fuel_figures(2)
    .usage_description(5)?
    .performance_figures(1)
    .acceleration(2)?
    .manufacturer(5)?
    .model(9)?
    .activation_code(6)?
    .encoded_length_with_header();

// claim_or_slot.len() == len — no oversize guess buffer.
let actual_len = CarEncoder::try_wrap_and_apply_header(&mut claim_or_slot, 0)?
    .fixed(&fields)
    .fuel_figures(2, |g| { /* … */ Ok(()) })?
    .performance_figures(1, |g| { /* … */ Ok(()) })?
    .manufacturer(b"Honda")?
    .model(b"Civic")?
    .activation_code(b"active")?
    .encoded_length_with_header();
assert_eq!(actual_len, len);

Nested books: book_encoded_length. API matrix: encoded_length_api_test.

Flyweight (per-field) vs whole struct

You can work field-by-field (classic flyweight) or fill / materialise a whole struct. Use the style that matches how much of the message you touch.

Style	Best when	Cost	Schema evolution
Flyweight (per-field)	You only read one or a few fields; hot path	Zero-copy; no heap	New fields are optional at call sites (you simply don’t read them)
*`FixedFields` + `.fixed(...)`**	You always write the entire fixed block	One struct write, still flyweight buffer	Adding a required fixed field to the schema → compile error until you set it in the struct
*`Domain` DTO** (`.enable_domain_objects(DomainVarData::…)`)	Whole message as owned data; enum picks `String` vs `Vec<u8>` var-data	Allocates — never use on the hot path. Easier app code for tooling, logging, offline processing	Same idea: regenerating after a schema change forces you to fill new struct fields

Decode — individual fields (flyweight)

// Only read what you need; no owned DTO or whole-message materialisation.
let car = CarDecoder::try_from(buf)?;
let serial_number = car.serial_number();
let model_year = car.model_year();

Encode — whole fixed block as a struct

When you always populate every fixed field, a struct is clearer and schema additions break at compile time:

// Generated (simplified):
// pub struct CarFixedFields {
//     pub serial_number: u64,
//     pub model_year: u16,
//     pub available: BooleanType,
//     pub code: Model,
//     pub some_numbers: [u32; 4],
//     pub vehicle_code: [u8; 6],
//     pub extras: OptionalExtras,
//     pub engine: Engine,
// }

let len = CarEncoder::try_wrap_and_apply_header(&mut buf, 0)?
    .fixed(&CarFixedFields {
        serial_number: 1234,
        model_year: 2013,
        available: true.into(),
        code: Model::A,
        some_numbers: [10, 20, 30, 40],
        vehicle_code: *b"ABCDEF",
        extras,
        engine,
    })
    .fuel_figures(0, |_| Ok(()))?
    .performance_figures(0, |_| Ok(()))?
    .manufacturer(b"Honda")?
    .model(b"Civic")?
    .activation_code(b"active")?
    .encoded_length_with_header();
let frame = &buf[..len];

// If the schema later adds `paint_code` to the fixed block, this stops compiling
// until you add `paint_code: …` to the struct literal — you cannot silently omit it.

Decode — flyweight (prefer for single-field reads)

let car = CarDecoder::try_from(buf)?;
// Only touch what you need — no allocation, no materialising the rest of the car.
let year = car.model_year();

Decode — whole message as a DTO

Do not use on the latency-sensitive path. DTO decode allocates Vec/String and copies every field. For the hot path, use the flyweight decoder instead.

When you always need (almost) everything, or want to pass a value across threads / into non-SBE code:

// build.rs: .enable_domain_objects(DomainVarData::LossyStrings)
let dto = CarDomain::try_from_decoder(CarDecoder::try_from(buf)?)?;
// dto is a plain Rust struct: Vecs for groups/strings, owned fields.
process_order(&dto);
let n = dto.encode(&mut out)?; // round-trip back to wire when needed

Rule of thumb: one field on the hot path → flyweight. Always fill or always consume the whole message → FixedFields / Domain for clarity and compile-time breakage on schema growth. More on DTOs in Recipes.

Composite layout & little-endian

A common question: on a little-endian host, can a composite just be a #[repr(C)] / #[repr(C, packed)] struct overlaid on the buffer so field access is a free load?

Almost — but not via repr(C) transmute. ergo-sbe does something safer that is still effectively free on LE hosts:

Approach	What ergo-sbe does	Why not the other thing
Wire image	`#[repr(transparent)] pub struct Engine(pub [u8; 10])` — the value is the on-wire bytes	`#[repr(C)]` native fields would insert alignment padding; SBE is packed and may have unaligned fields
Accessors	`u16::from_le_bytes` / `to_le_bytes` at schema offsets	Native loads without endian conversion break big-endian schemas and unaligned safety
Flyweight	`EngineDecoder { buf, pos }` reads in place — zero copy	Default decode path for composites
Eager value	`engine_value()` copies the `N`-byte image once	Still not field-by-field re-pack; `.0` is the wire block
Encode	Writer copies `engine.0` bulk into the frame	Same image the decoder reads back

On little-endian hosts, from_le_bytes lowers to a plain load (aligned or unaligned as needed) — so member access is “super fast” without casting the buffer to a padded Rust struct. The generator also emits

const _: () = assert!(core::mem::size_of::<Engine>() == 10);

so the Rust type size is locked to the wire size at compile time.

*FixedFields (e.g. CarFixedFields) is a different beast: an application struct with typed fields used to fill the fixed block in one call. It is not a zero-copy overlay of the message buffer; .fixed(&…) writes each field with endian conversion into the flyweight buffer.

Conclusion — why not `repr(C, packed)`?

Single-field access is already one load. Head-to-head Criterion arms on a 256-byte composite (mid-block field f15), field-only, no alloc on the timed path (layout_access_bench):

Arm	What is timed	Median (order of)
Flyweight	`dec.block().f15()`	~0.4 ns
Wire-image value (preheld)	`BigBlock([u8; 256]).f15()`	~0.4 ns
`#[repr(C, packed)]` overlay	unaligned load of `f15`	~0.4 ns
Copy then field	`block_value()` (256 B) then `.f15()`	~24 ns (~60×)

So:

Flyweight ≈ preheld wire-image ≈ packed for one field — all one load on LE.
repr(C, packed) does not unlock free access beyond what [u8; N] + from_le_bytes already gives. Hand-rolling packed overlays is extra UB/layout risk for no speed win.
The expensive mistake is materialising a large composite just to touch one field. Prefer flyweight when you only need a few members; use *_value() when you need the whole wire blob (or pass it around) and pay the N-byte copy once.
We still do not generate repr(C) / packed field structs: packing + unaligned references, big-endian schemas, enums/sets/nested composites. The transparent wire image is the portable form that already optimizes to the packed load on LE.

You need…	Use
One or a few fields on the hot path	Flyweight — no composite copy
Whole composite as an owned wire blob	Value `Engine([u8; N])` / `*_value()` — pay `N` once
Hand-rolled `repr(C, packed)` for speed	Skip it — same cost as wire-image field access

Layout contracts: composite_layout_test. Decode microbench: layout_access_bench.

Encode — FixedFields vs setters, composite write, LE vs BE

Confirmed by encode_style_bench (Apple M4, LE host; values prebuilt / seeded so LLVM cannot delete the work):

Comparison	Result
`.fixed(&CarFixedFields{…})` vs all setters	~equal (~2.6 ns both) — `.fixed` is the same setter sequence after inlining
Composite `Engine::new` + write vs preheld `engine(e)`	~equal when the rest of the fixed block is also written (10-byte image is noise next to the other stores)
256 B block build+write LE vs BE	BE ~5% slower on LE host (`to_be_bytes` / bswap on 32×`u64`) — 26.1 ns LE vs 27.5 ns BE
Preheld wire image memcpy LE vs BE	~equal (~77 ns) — endian already in `.0`; only bulk copy remains

So on encode:

Prefer .fixed for clarity / schema completeness — not for speed.
Prefer a prebuilt composite wire image on the hot path when you can; for small N the win is tiny next to other field stores.
LE body on LE host is free endian; BE body costs a bswap per multi-byte field when building the image. Once the image exists, write cost matches LE.

Feature matrix

Scannable map of capabilities. Use the More links for samples and tests.

Feature	What it does	How to use / more
`build.rs` codegen	Compile-time schema → Rust module in `OUT_DIR`	`generate_to_out_dir("schemas/….xml", config)?` · plain `include!` or `sbe_mod!(name)` · Quick start · codegen examples
Wire compatibility	Same on-wire layout as official SBE	Dual encode ergo vs sbe-tool · sbe_tool_wire_parity_test · golden fixtures · BENCHMARKS.md · baseline_test
Flyweight decode	Zero-copy over `&[u8]`	`CarDecoder::try_from(buf)?; car.serial_number()` · feature-tour
Composite wire image	`#[repr(transparent)] Engine([u8; N])` + LE accessors; flyweight default	Not a `repr(C)` overlay · Core ideas · composite_layout_test
Per-field vs whole struct	Flyweight or `FixedFields` / `Domain`	Single field: flyweight · always fill fixed block: `.fixed(&CarFixedFields { … })` · whole message owned: `CarDomain` · Core ideas · feature-tour
Stage-struct encode + closures	Wire order as named monomorphic stages; groups via nested closures	`bids(n, \|g\| g.add(\|e\| …))?` · wrong order = missing method · Core ideas · Recipes · BENCHMARKS.md
Consuming decode stages	Distinct after-stage decoder types	`into_bids()?` → next named stage · ordered_decoder_stages_test · l3_consuming_stages_test
Checked vs trusted	Explicit trust boundary	`try_*` untrusted · `wrap` trusted · `verify` full tail · demo_try_vs_trusted
Exact buffer sizing	Schema-aware length for nested/ragged msgs — no hand-calculated sizes	`compute_length_with_header()` (fixed) · `compute_length_with_header(…)` (flat) · `*EncodedLength` (nested) · Core ideas · l3-book · encoded_length_api_test
Schema docs → rustdoc	XML descriptions become item docs	`description="…"` / `<description>` / `<comment>` / `<!-- -->` · schema_docs_provenance_test
`Display` / `Debug`	Diagnostic print (not wire format)	`println!("{car}");` · Recipes · demo_display_debug
NULL / MIN / MAX	Schema sentinels as consts	`MODEL_YEAR_NULL` · baseline_test
Version-aware fields	`sinceVersion` / acting version	`Option` or skip on older wire · baseline_test · multi_schema_versioning_test
Groups / nested groups	Repeating dimensions	`bids(n, \|g\| g.add(…))?` · l3-book · l3_orderbook_test
Bulk group encode / decode	`bulk_add(&[Entry])` / `bulk_add_domain(&[EntryDomain])` / `bulk_decode() -> Vec<Entry>` for eligible flat groups	Wire `bulk_add`: about 22-23% lower encode latency than per-entry `add()` for the audited 1,000-entry cases. DTO re-encode selects the domain bulk path automatically when wire and domain fields match; remeasure for your schema · group_encode_bench
Var-data / text	Length-prefix; optional UTF-8/ASCII	`manufacturer(b"Honda")?` · `*_as_str` when encoding set · feature-tour
Fixed arrays + bulk helpers	Arrays, put, pad string, copy-out	`put_some_numbers(…)` · `vehicle_code_str` · `copy_vehicle_code` · java_parity_features_test
Enums / sets / bool	Wire enums, bitsets, `_bool`	`available()` / `available_bool(true)` · comprehensive_test
`with_conversion`	Wire type → any app type you impl	`price_from(&Cents)?` / `price_as::<Cents>()?` · Configuration · exchange-example
`with_domain_type`	Wire type → one fixed Rust path	`enc.price(d); let d = dec.price()` · l3-book · Configuration
Domain DTOs	Owned structs + re-encode; allocation-free automatic bulk write for eligible flat groups; var-data via [`DomainVarData`]	`.enable_domain_objects(DomainVarData::LossyStrings)` · Recipes · domain_objects_test
`AnyMessage` + frames	Multi-template + framed streams	`AnyMessage::decode` · `FrameCursor` · demo_any_message
`verify`	Full tail bounds check	`car.verify()?` · feature-tour try/trusted demos
Schema identity	Id / version / hashes	`SCHEMA_ID`, `SCHEMA_HASH`, `SCHEMA_SHA256_HEX` · generated module header
Multi-schema shared types	Dedup across packages	`.with_shared_module` + `generate_multi` · exchange-example · multi_schema_versioning_test
Keyword-safe names	`type` → `type_`	`.with_keyword_append_token("_")` · java_parity_features_test
XSD-shaped validation	Structural check before parse	`validate_against_sbe_xsd` / `parse_with_xsd_validation` · xsd.rs
Zero-alloc hot path	Flyweights + caller buffers	allocation_count_test · BENCHMARKS.md
Property round-trip	Random messages encode→decode	`cargo test -p ergo-sbe --test proptest_roundtrip` · proptest_roundtrip

Recipes

Runnable, tested code for every pattern lives in sbe-feature-tour. See its src/lib.rs for the full API map.

Pattern	Demo function
Fixed message encode/decode	`demo_fixed_heartbeat`
Sized encode with `EncodedLength` builder	`demo_car_size_and_encode`
Known vs unknown group counts	`demo_car_size_and_encode`
`Display` / `Debug` diagnostic output	`demo_display_debug`
Domain objects (DTOs)	`demo_domain_dto`
Multi-template `AnyMessage` dispatch	`demo_any_message`
`with_conversion` generic adapters	`demo_conversion_only`
Trust boundary (`try_` vs `wrap`)	`demo_try_vs_trusted`
Bulk group encode (`bulk_add`)	`group_encode_bench`
Timestamp multi-precision converters	Configuration below

Quick reference

Known vs unknown group count:

// Known count (must add() exactly `count` times):
let known_len = CarEncoder::try_wrap_and_apply_header(&mut buf, 0)?
    .fixed(&fields)
    .fuel_figures(2, |g| {
        g.add(|e| { e.speed(30).mpg(35.5); Ok(()) })?;
        g.add(|e| { e.speed(55).mpg(49.0); Ok(()) })?;
        Ok(())
    })?
    .performance_figures(0, |_| Ok(()))?
    .manufacturer(b"Honda")?
    .model(b"Civic")?
    .activation_code(b"active")?
    .encoded_length_with_header();

// Unknown size: count back-patched after the closure (streaming producers).
let unknown_len = CarEncoder::try_wrap_and_apply_header(&mut buf, 0)?
    .fixed(&fields)
    .fuel_figures_unknown_size(|g| {
        for row in rows {
            g.add(|e| {
                e.speed(row.speed).mpg(row.mpg);
                Ok(())
            })?;
        }
        Ok(())
    })?
    .performance_figures(0, |_| Ok(()))?
    .manufacturer(b"Honda")?
    .model(b"Civic")?
    .activation_code(b"active")?
    .encoded_length_with_header();

println!("known={known_len} unknown={unknown_len}");

Display / Debug

Diagnostic only — not a stable wire or log schema. Do not treat either format as a protocol or long-term log contract.

Display currently equals Debug for generated decoders ({car} and {car:?} print the same text). Prefer Debug in logs if you want that intent to stay obvious when/if the two diverge later.

Real output from the feature-tour Car (demo_car_size_and_encode → CarDecoder):

let car = CarDecoder::try_from(buf.as_slice())?;
println!("{car}");
println!("{car:?}"); // same text as Display today

CarDecoder { serialNumber: 1234, modelYear: 2013, available: true, code: A, fuelFigures: ["{ speed: 30, mpg: 35.9, usageDescription: Urban }", "{ speed: 60, mpg: 25.0, usageDescription: Highway }"], performanceFigures: ["{ octaneRating: 95, acceleration: [{ mph: 30, seconds: 4.0 }, { mph: 60, seconds: 7.5 }] }"], manufacturer: "Honda", model: "Civic VTi", activationCode: "abcdef" }

Truncated / incomplete buffers omit missing tails rather than panicking. See demo_display_debug.

Schema description → rustdoc

<field name="serialNumber" id="1" type="uint64" description="VIN-style serial"/>

// Generated (approx):
/// VIN-style serial
pub fn serial_number(&self) -> u64 { … }

Provenance of all four XML doc sources: schema_docs_provenance_test.

Domain DTO (ease of use)

Use when you want owned values (Vec groups, owned tails) and simple structs — not the zero-copy hot path. Flyweights stay faster for low-latency applications.

For re-encode, eligible flat groups are bulk-written directly from &[EntryDomain]: no temporary Vec<Entry> and no encode-time allocation. Eligibility requires fixed-size entries whose domain fields have the same wire representation; nested groups, var-data, optional/versioned fields, configured domain conversions, and bool remapping use the general add path. Integer min/max checks are preserved in both paths.

On the audited Apple M4 1,000-entry fixture, automatic DTO bulk encode measured 509 ns versus 1.336 µs for the exact previous per-entry path with LTO, and 509 ns versus 1.998 µs without LTO. This is a DTO-to-DTO diagnostic, not an ergon/sbe-tool fairness ratio.

// build.rs — DomainVarData is a big deal (DTO var-data type):
.enable_domain_objects(DomainVarData::LossyStrings) // manufacturer: String (invalid UTF-8 → "")
// .enable_domain_objects(DomainVarData::Bytes)      // manufacturer: Vec<u8> (byte-exact)

// --- generated shape with DomainVarData::LossyStrings ---
// pub struct CarDomain {
//     pub serial_number: u64,
//     pub model_year: u16,
//     pub fuel_figures: Vec<CarFuelFiguresEntryDomain>,
//     pub manufacturer: String,
//     // …
// }
// impl CarDomain {
//     pub fn try_from_decoder(dec: CarDecoder<'_>) -> Result<Self, DecodeError>;
//     pub fn encode(&self, buf: &mut [u8]) -> Result<usize, EncodeError>;
//     pub fn encoded_length_with_header(&self) -> Result<usize, EncodeError>;
// }

// Wire → DTO (prefer try_from_decoder; From can panic on bad tails)
let dto = CarDomain::try_from_decoder(CarDecoder::try_from(buf)?)?;
assert_eq!(dto.manufacturer, "Honda");

// Edit / build like normal Rust
dto.model_year = 2014;
dto.manufacturer = "Toyota".into();

// DTO → wire (re-encodes; integer min/max checked).
// Prefer stack when the message is fixed-size; otherwise size then write into
// a claim / slot of that exact length (avoid oversize scratch Vecs).
let len = dto.encoded_length_with_header()?;
// e.g. let mut out = [0u8; CarEncoder::compute_length_with_header()];  // fixed
// or encode into a transport claim of `len` bytes
let n = dto.encode(&mut out[..len])?;
println!("re-encoded {n} bytes");

`enable_domain_objects(DomainVarData)`

SBE <data> is length-prefixed bytes. The enum picks the DTO field type:

Call	Field type	Invalid UTF-8	When to use
`.enable_domain_objects(DomainVarData::LossyStrings)`	`String`	silent empty `""` (not U+FFFD, not an error)	Text schemas; easiest app API
`.enable_domain_objects(DomainVarData::Bytes)`	`Vec<u8>`	n/a (raw copy)	Binary tails or byte-exact re-encode

LossyStrings is not lossless on re-encode. Materialise clears invalid UTF-8 to ""; dto.encode then writes empty var-data, so the bad bytes are not preserved. Use Bytes (or stay on flyweights) when you need audit / replay fidelity of non-UTF-8 tails.

Runnable demo (text path): sbe-feature-tour uses DomainVarData::LossyStrings. Flyweight path is unchanged: with schema characterEncoding="UTF-8" you still get into_manufacturer_as_str() without a DTO.

demo_car_domain_dto · domain_objects_test.

App types on top of wire composites

See Configuration — start with wire type vs app type, then Option A (Cents + with_conversion) or Option B (rust_decimal + with_domain_type).

Multiple timestamp precisions → one Rust type

Three wire uint64 fields, each a different timestamp precision on the wire, but all mapping to chrono::DateTime<Utc> in Rust. The generated u64↔DateTime converter always uses nanosecond precision, so millis and micros need their own TryFromSbe/TryToSbe impls. Distinguish them with FieldPath selectors:

<!-- schema fragment — three uint64 fields, same wire type, three precisions -->
<field name="created_at"  id="1" type="uint64" semanticType="UTCTimestamp"/>
<field name="updated_at"  id="2" type="uint64" semanticType="UTCTimestampMicros"/>
<field name="received_at" id="3" type="uint64" semanticType="UTCTimestampMillis"/>

// build.rs — register converters for all three
let config = GenerationConfig::new("msgs")
    .with_conversion(ConversionSelector::field_path("Event.created_at"))   // nanos, built-in
    .with_conversion(ConversionSelector::field_path("Event.updated_at"))   // micros, custom
    .with_conversion(ConversionSelector::field_path("Event.received_at")); // millis, custom

Writing TryFromSbe<u64> for micros would clash with the built-in nano converter — TryFromSbe<u64> can only exist once. Resolve this by naming the wire fields unique types — the idiomatic pattern when three uint64 columns mean three different things:

<!-- Distinguish wire types by name — all are uint64 under the hood -->
<composite name="TimestampNanos">  <type name="ts" primitiveType="uint64"/>  </composite>
<composite name="TimestampMicros"> <type name="ts" primitiveType="uint64"/>  </composite>
<composite name="TimestampMillis"> <type name="ts" primitiveType="uint64"/>  </composite>

<message name="Event" id="1">
  <field name="created_at"  id="1" type="TimestampNanos"/>
  <field name="updated_at"  id="2" type="TimestampMicros"/>
  <field name="received_at" id="3" type="TimestampMillis"/>
</message>

Now each wire type generates a distinct Rust newtype (TimestampNanos, TimestampMicros, TimestampMillis — all #[repr(transparent)] wrappers around u64). Implement the converters per-type, no blanket-clash:

// Nanos — trivially delegates to the built-in logic
impl TryFromSbe<TimestampNanos> for chrono::DateTime<chrono::Utc> {
    type Error = &'static str;
    fn try_from_sbe(wire: TimestampNanos) -> Result<Self, Self::Error> {
        chrono::DateTime::from_timestamp(
            (wire.0 / 1_000_000_000) as i64,
            (wire.0 % 1_000_000_000) as u32,
        )
        .ok_or("timestamp out of range")
    }
}
// … TryToSbe, etc.

// Micros
impl TryFromSbe<TimestampMicros> for chrono::DateTime<chrono::Utc> {
    type Error = &'static str;
    fn try_from_sbe(wire: TimestampMicros) -> Result<Self, Self::Error> {
        chrono::DateTime::from_timestamp(
            (wire.0 / 1_000_000) as i64,
            ((wire.0 % 1_000_000) * 1_000) as u32,
        )
        .ok_or("timestamp out of range")
    }
}

// Millis
impl TryFromSbe<TimestampMillis> for chrono::DateTime<chrono::Utc> {
    type Error = &'static str;
    fn try_from_sbe(wire: TimestampMillis) -> Result<Self, Self::Error> {
        chrono::DateTime::from_timestamp(
            (wire.0 / 1_000) as i64,
            ((wire.0 % 1_000) * 1_000_000) as u32,
        )
        .ok_or("timestamp out of range")
    }
}

// build.rs — three selectors, each naming a distinct named type
let config = GenerationConfig::new("msgs")
    .with_conversion(ConversionSelector::named_type("TimestampNanos"))
    .with_conversion(ConversionSelector::named_type("TimestampMicros"))
    .with_conversion(ConversionSelector::named_type("TimestampMillis"));

// Encode — all use the same Rust type, with the wire precision implicit
let now = chrono::Utc::now();
enc.created_at_from(&now)?;      // → TimestampNanos  (wire: u64 nanos)
enc.updated_at_from(&now)?;      // → TimestampMicros (wire: u64 micros)
enc.received_at_from(&now)?;     // → TimestampMillis (wire: u64 millis)

// Decode — all return chrono::DateTime<Utc>, precision transparent
let created:  chrono::DateTime<chrono::Utc> = dec.created_at_as()?;
let updated:  chrono::DateTime<chrono::Utc> = dec.updated_at_as()?;
let received: chrono::DateTime<chrono::Utc> = dec.received_at_as()?;

The pattern generalises: one Rust type, N wire representations → one single-field composite per representation, each with its own TryFromSbe / TryToSbe impl, all distinguished by ConversionSelector::named_type.

Configuration

Wire type vs app type

Name	Role
`Decimal` (schema composite)	Wire — generated type / `price_value()` — what is in the buffer
`Cents`, `rust_decimal::Decimal`, …	App — what your code wants to use

  app  ──price_from / price()──►  wire Decimal on the buffer
  buf  ──price_as / price()──►   app value

`with_conversion` vs `with_domain_type` (one per field)

Do not call both for the same selector — domain type already enables conversion.

	A `with_conversion`	B `with_domain_type`
Idea	Generic convert API; you plug any app type	Always use this Rust path
build.rs	`.with_conversion(named_type("Decimal"))`	`.with_domain_type(…, "rust_decimal::Decimal")`
You write	`TryFromSbe<Decimal>` / `TryToSbe<Decimal>` for your type	Usually nothing for bool / rust_decimal / chrono
Decode	`let p: Cents = dec.price_as()?`	`let p: rust_decimal::Decimal = dec.price()`
Encode	`enc.price_from(&cents)?`	`enc.price(rust_decimal::Decimal::new(12345, 2))`
Raw wire	`price_value()` / `price_wire(...)`	same when conversion is active
Sample	exchange-example · demo_conversion_only	l3-book

Option A — you choose the app type (`Cents`)

use ergo_sbe::{ConversionSelector, GenerationConfig};

// build.rs — names the *wire* schema type only
let config = GenerationConfig::new("msgs")
    .with_conversion(ConversionSelector::named_type("Decimal"));
let _ = config;

// app — YOU adapt wire Decimal ↔ Cents
// `Decimal` below is the *generated SBE composite*, not rust_decimal.

struct Cents(i64);

impl TryFromSbe<Decimal> for Cents {
    type Error = &'static str;
    fn try_from_sbe(wire: Decimal) -> Result<Self, Self::Error> {
        Ok(Cents(wire.mantissa()))
    }
}
impl TryToSbe<Decimal> for Cents {
    type Error = &'static str;
    fn try_to_sbe(&self) -> Result<Decimal, Self::Error> {
        Ok(Decimal::new(self.0, -2))
    }
}

enc.price_from(&Cents(12345))?;
let cents: Cents = dec.price_as()?;
let wire = dec.price_value();
println!("mantissa={} exponent={}", wire.mantissa(), wire.exponent());

// Same buffer, another app type if you impl TryFromSbe for it too:
// let also: rust_decimal::Decimal = dec.price_as()?;

Option B — one fixed app type

use ergo_sbe::{ConversionSelector, GenerationConfig};

let config = GenerationConfig::new("msgs")
    .with_domain_type(
        ConversionSelector::named_type("Decimal"),
        "rust_decimal::Decimal",
    );
let _ = config;

enc.price(rust_decimal::Decimal::new(12345, 2));
let p: rust_decimal::Decimal = dec.price();

Both styles on different fields: sbe-feature-tour.

Other `GenerationConfig` options

Option	Purpose
`enable_domain_objects(DomainVarData::…)`	Owned `Domain` + `encode`; `LossyStrings`* → `String` (bad UTF-8 → `""`), `Bytes` → `Vec<u8>`
`with_shared_module` / `generate_multi`	Multi-schema shared types
`with_external_sbe_rt`	Share one `sbe_rt` runtime module
`enable_error_from_impls`	`From<EncodeError/DecodeError>` for your error type
`with_unchecked_companions`	Bench-only fast accessors
`with_keyword_append_token`	Schema `type` → Rust `type_` (default `"_"`)
`enable_bool_domain_type`	Syntax sugar: auto-registers `bool` converters for every boolean enum. Equivalent to calling `.with_domain_type(ConversionSelector::named_type("BooleanType"), "bool")` for each — detects by name, `semanticType="Boolean"`, or True/False value pairs
`with_deprecated_attrs`	`#[deprecated]` on schema-deprecated items

Text fields stay bytes unless the schema declares a supported character encoding (then strict UTF-8/ASCII helpers apply). Display/Debug are diagnostic only (Display currently equals Debug on generated decoders).

Samples

Monorepo only (not on crates.io). Absolute links for docs.rs.

Start here (product teaching path)

sbe-feature-tour — golden path. Runnable map of encode/decode stages, EncodedLength, trust boundary, Display, DTO (DomainVarData::LossyStrings), both conversion styles.
cargo run --manifest-path samples/sbe-feature-tour/Cargo.toml
Conversion choice — pick one sample that matches your app style:
- l3-book → with_domain_type only (+ build-dep only include)
- exchange-example → with_conversion only (+ IPC)
Optional — sbe-codegen-examples (generator as library); cluster samples for Aeron integration.

Full table and dependency patterns: samples/README.md.

Where `ergo-sbe` sits in `Cargo.toml`

Generated codecs embed their own sbe_rt runtime. You do not need ergo-sbe as an application dependency just to encode/decode — only as a build dependency to run codegen. Prefer build-only for published products; add a runtime dep only for sbe_mod! convenience or library-API use.

Pattern	`build-dependencies`	`dependencies`	What for	Sample
Build only (product default)	yes	no	`generate_to_out_dir` + plain `include!`	l3-book · cluster-rfq
Build only (generated module)	yes	no	Same, with a generated source module committed under `src/`	sbe-feature-tour
Build + runtime (macros)	yes	yes	Same + `sbe_mod!` / `include_sbe!`	Add this only when your application uses the macros
Runtime only (library API)	no	yes	`parse` / `Generator` in-process; no `build.rs`	sbe-codegen-examples

git clone https://github.com/mimran1980/ergon.git && cd ergon
cargo run  --manifest-path samples/sbe-feature-tour/Cargo.toml   # start here
cargo run  --manifest-path samples/l3-book/Cargo.toml            # domain_type + build-only
cargo test --manifest-path samples/exchange-example/Cargo.toml   # conversion only

Rust version and edition


Edition	2024
MSRV	1.88

If edition 2021 (and an older MSRV) would unblock you, open an issue — happy to maintain a 2021 path if there is real demand. Say what toolchain you need (e.g. 1.75 / 1.80).

Verify the crate

cargo test -p ergo-sbe --all-features -- --test-threads=1
cargo test -p ergo-sbe --doc --all-features
RUSTDOCFLAGS="-D warnings" cargo doc -p ergo-sbe --all-features --no-deps
cargo clippy -p ergo-sbe --all-targets --all-features -- -D warnings

just test in the monorepo also runs doctests, docs_validation_test (README fences + generated-API smoke), and rustdoc with -D warnings.

Performance method: BENCHMARKS.md (not in the crates.io package).

Package scope

crates.io ships generator source, manifest, and this README. Tests, fixtures, samples, and benches live on GitHub only — use the links above.

License

Apache-2.0 · mimran1980/ergon

ergo-sbe 0.1.5