Crate ergo_sbe 

Opinionated, idiomatic Rust code generation for Simple Binary Encoding (SBE).

Simple Binary Encoding describes messages in XML; ergo-sbe parses those schemas and emits safe, version-aware Rust codecs for low-latency trading.

Documentation

• ergo-sbe book — getting started, feature tour, core concepts, configuration, recipes, design notes
• Getting started · Feature tour · Coming from sbe-tool · Type-state design
• Crate README

In a few words

• Compile-time wire order — calling asks before bids is a type error
• Closure-based groups — nested shape mirrors the schema, no .parent() hopscotch
• Exact buffer sizing — no oversize scratch buffers; works directly with Aeron try_claim
• Three-tier trust boundary — try_* constructors validate the buffer extent and return Result; bare wrap / wrap_and_apply_header / decode prove the same extent and panic if short; unsafe fn *_unchecked skips checks (caller proves the extent in # Safety)
• Zero heap allocation on generated hot paths; zero runtime dependencies
• Domain types — map wire Decimal to rust_decimal::Decimal with one line of config

Architecture

Layer	Module	Responsibility
Schema input	xml, xsd, schema	Parse SBE XML, optional XSD shape check, Schema
Intermediate	[ir], [resolve]	Token stream + offsets / block lengths
Options	config	Module name, conversions, domain objects, …
Codegen	codegen	Rust source modules

Quick-start (build.rs) — doctest pins generated idioms

Generated application types do not exist in this crate at doctest time. The example below runs the generator on an inline schema and asserts on the emitted source so chained encode / length-builder names cannot drift unnoticed. For end-to-end encode/decode of real types, see samples/sbe-feature-tour and the golden file sbe/tests/golden/car_example.rs.

use ergo_sbe::{parse, Generator, GenerationConfig, Schema};

let schema_xml = r#"<?xml version="1.0" encoding="UTF-8"?>
<messageSchema package="example.sbe" id="1" version="0"
               byteOrder="littleEndian">
  <types>
    <composite name="messageHeader">
      <type name="blockLength" primitiveType="uint16"/>
      <type name="templateId"   primitiveType="uint16"/>
      <type name="schemaId"     primitiveType="uint16"/>
      <type name="version"      primitiveType="uint16"/>
    </composite>
  </types>
  <message name="Car" id="1">
    <field name="serialNumber" id="1" type="uint64" offset="0"/>
    <field name="modelYear"    id="2" type="uint16" offset="8"/>
  </message>
</messageSchema>"#;

let ir = parse(schema_xml).expect("parse schema");
let schema = Schema::from_ir(ir);
let output = Generator::new(GenerationConfig::new("my_messages"))
    .generate(&schema)
    .expect("generate");
let src = &output.modules().next().expect("one module").source;
assert!(src.contains("CarDecoder"));
assert!(src.contains("CarEncoder"));
assert!(src.contains("CarFixedFields"));
assert!(src.contains("wrap_and_apply_header"));
assert!(src.contains("compute_length_with_header") || src.contains("ENCODED_LENGTH"));
// write module.source to OUT_DIR and `include!` it from your crate

What gets generated (how to use it)

Names depend on your schema (Car below is illustrative). Prefer the feature-tour sample and golden file over prose-only snippets.

Composites = wire images (not repr(C) overlays)

Generated composites are #[repr(transparent)] struct Engine(pub [u8; N]): the value is the on-wire byte block. Accessors use explicit from_le_bytes / to_le_bytes at schema offsets (portable; free on LE). Default decode is a flyweight (EngineDecoder { buf, pos }) — zero-copy into the message. Do not transmute the buffer to a padded #[repr(C)] field struct: SBE is packed and may be unaligned. See the crate README section Composite layout & little-endian.

Decode flyweight

→ samples/sbe-feature-tour

Encode + type-state tails (buffer sizing)

Size the buffer first with the staged *EncodedLength builder — never guess with a large vec![0u8; 4096]. For fixed-only messages use the const ENCODED_LENGTH.

→ samples/sbe-feature-tour

Buffer sizing

Schema-aware helpers size the buffer before you write — including nested groups and ragged var-data — so you do not hand-calculate wire length for complicated messages. Prefer a stack array when the length is const (ENCODED_LENGTH / compute_encoded_length_*); for runtime lengths, size first then claim or encode into an exact-length slot.

Shape	API	Prefer
Fixed-only	HeartbeatEncoder::ENCODED_LENGTH (const)	[0u8; N] stack
Flat / known tails	compute_encoded_length_with_message_header(...)	stack when const
Groups / nested / ragged	CarEncodedLength::new()…encoded_length_with_header()	exact claim / slot

Why wire order is compile-time

SBE is positional. Two adjacent identical groups (e.g. bids then asks) are common in market data; swapping them still produces “valid” bytes and only fails in production. Named stage structs make the wrong order a type error.

Field metadata (Java parity)

→ sbe/tests/java_parity_features_test.rs

Conversion: with_conversion vs with_domain_type

Pick one style per selector — with_domain_type already enables conversion.

Config	Generated decode	Generated encode
GenerationConfig::with_conversion	dec.price_as::<T>()?	enc.price_from(&t)?
GenerationConfig::with_domain_type	dec.try_price()? -> path::Type	enc.try_price(value)?

use ergo_sbe::{ConversionSelector, GenerationConfig};

// A — pluggable: you implement TryFromSbe / TryToSbe
let _a = GenerationConfig::new("msgs")
    .with_conversion(ConversionSelector::named_type("Decimal"));

// B — concrete Rust type (implies conversion for the same selector)
let _b = GenerationConfig::new("msgs")
    .with_domain_type(
        ConversionSelector::named_type("Decimal"),
        "rust_decimal::Decimal",
    );

See sbe/tests/comprehensive_test.rs (conversion and domain-object coverage).

Domain objects

GenerationConfig::with_domain_objects with a DomainVarData mode emits owned structs. Use DomainVarData::Strings for text (String; invalid UTF-8 → InvalidUtf8 error) or DomainVarData::Bytes for Vec<u8>.

See sbe/tests/domain_objects_test.rs

Multi-message dispatch

See sbe/fuzz/fuzz_targets/any_message_frame_cursor.rs

Fixed arrays / char fields

See sbe/tests/java_parity_features_test.rs

Keywords in schema names

Field type becomes type_ (default append "_"). Override with GenerationConfig::with_keyword_append_token.

XSD structural check

Optional CI gate: validate_against_sbe_xsd or parse_with_xsd_validation. Official XSD text is embedded as SBE_XSD.

Note this is a shape check, not a full W3C engine, and it is opt-in. parse itself always rejects malformed XML, a bad root, unexpected elements, and unknown attributes.

Design

• Wire-compatible with official SBE / sbe-tool baselines where tested
• Idiomatic Rust (type-state tails, borrow flyweights) — not a Java port
• Zero allocation on decode hot paths by default
• Version-aware accessors (sinceVersion / acting version)
• Unsafe only on explicit _unchecked / documented paths

Benchmarks: benchmarks chapter.

Re-exports

pub use build::BuildError;

pub use build::SchemaFile;

pub use build::generate_multi_to_dir;

pub use build::generate_multi_to_out_dir;

pub use build::generate_str_to_dir;

pub use build::generate_str_to_out_dir;

pub use build::generate_to_dir;

pub use build::generate_to_out_dir;

pub use build::out_dir;

pub use codegen::GenerateError;

pub use codegen::GeneratedModule;

pub use codegen::GeneratedModuleSet;

pub use codegen::Generator;

pub use config::ConversionSelector;

pub use config::DomainImpl;

pub use config::DomainVarData;

pub use config::EnumVariantInfo;

pub use config::FieldInfo;

pub use config::GenerationConfig;

pub use config::GenerationProfile;

pub use config::ItemContext;

pub use config::ItemKind;

pub use config::SetChoiceInfo;

pub use schema::Schema;

pub use xml::IncludeCause;

pub use xml::ParseError;

pub use xml::parse;

pub use xml::parse_file;

pub use xml::parse_file_with_shared;

pub use xml::parse_with_shared;

pub use xml::parse_with_xsd_validation;

pub use xsd::SBE_XSD;

pub use xsd::XsdValidationError;

pub use xsd::validate_against_sbe_xsd;

pub use miette;

pub use bytes;

pub use compact_str;

pub use smol_str;

Modules

build

Cargo build.rs helpers (generate_to_out_dir, sbe_mod!). Helpers for Cargo build.rs scripts.

chrono_converters

Chrono timestamp converters — feature-gated behind chrono.

codegen

Codec generation (Generator). Rust code generation from a resolved crate::Schema.

config

GenerationConfig — conversions, domain objects, keywords, etc. Code generation configuration (GenerationConfig).

schema

Schema handle for codegen. Normalised schema handle for codegen: package identity + resolved Ir.

xml

XML parse (parse, parse_file). SBE XML → token Ir.

xsd

Optional XSD-shaped validation (validate_against_sbe_xsd, SBE_XSD). Optional SBE XSD-shaped structural validation (not a full W3C engine).

Macros

include_sbe

Include a module written by generate_to_out_dir / generate_str_to_out_dir.

sbe_mod

Declare a module that includes generated SBE codecs from OUT_DIR.

Structs

Encoding

How a token is encoded on the wire.

Ir

The parsed schema IR: schema-level metadata plus the token stream.

Token

One token in the flat IR stream.

Enums

ByteOrder

Byte order declared by the schema; applies to every primitive encoding.

Presence

Field presence semantics.

PrimitiveType

SBE primitive wire types.

ResolveError

Errors raised during schema resolution/validation.

Signal

Structural role of a token in the IR stream.

Functions

resolve_schema

Resolve offsets, block lengths, and default null/min/max on ir in place.