Exavian Veritate

One serialization format that is actually all three things at once — zero-copy, self-describing, and schema-evolvable — with no unsafe, and one wire spec implemented byte-for-byte across six independent implementations.

verit is a binary serialization format that refuses the usual pick-two trilemma. Reads are bounds-checked indexed loads straight out of the buffer (no parse step, no allocation); every message carries the 128-bit content hash of its writer schema and can embed the schema inline; and fields are matched by stable numeric IDs so schemas evolve without breaking old readers. The indirection that evolution needs is paid once per (writer, reader) schema pair — compiled into a cached access plan — not per message and not at build time.

The load-bearing claim is verifiable, not aspirational. The format has five independent implementations (Rust, Python, C++, Go, TypeScript), each written from the wire specification alone. For a shared corpus they produce byte-for-byte identical messages and schema ids, each reads and re-encodes the others' output, and the same cases are frozen as committed golden vectors. A sixth implementation — a GnuCOBOL reader and writer for mainframe integration — is held to those same vectors: it reproduces every message byte-for-byte and re-derives each 128-bit schema id from scratch (its own from-scratch SHA-256). See VERIT - Introduction and Vision.

Repository layout

One public umbrella crate; docs as build evidence.

veritate/
├── workspace/    # DEV ONLY — roadmap, checklists, ADRs. NOT git-tracked.
├── docs/         # PUBLIC docs — architecture (incl. the wire spec), governance, benchmarks.
├── engine/       # The umbrella crate `verit` + internal crates: verit-core (engine), verit-cli (the `verit` tool).
├── ports/        # The five non-Rust implementations: python, cpp, go, ts, cobol (the byte-identity proof).
├── settings/     # .toml knobs (benchmark params, defaults) — change behavior without code.
├── deploy/       # Dockerfile, CI, the run_all cross-language proof harness, file(1)/MIME registration.
├── fuzz/         # cargo-fuzz targets (standalone workspace).
├── Cargo.toml    # Root virtual workspace.
├── rust-toolchain.toml
└── rustfmt.toml

One public crate. Consumers depend only on the umbrella verit (engine), which re-exports the internal verit-core engine. All crate-boundary re-exports live in engine/src/lib.rs, so the engine can be split or rewritten from one edit point. Detail in VERIT - System Architecture.

Build & use

cargo add verit               # zero runtime dependencies; --features derive for #[derive(Verit)]

Working in this repo:

cargo test --workspace        # engine tests, golden vectors, fuzz, codegen conformance
cargo run -p verit --release --example bench    # vs protobuf, Cap'n Proto, FlatBuffers, JSON

use verit::prelude::*;

let schema = SchemaBuilder::new()
    .add_struct("Person", vec![(1, "name", Dt::Str), (2, "age", Dt::U8)])
    .build("Person")?;
let bytes = encode(&schema, &Value::Struct(vec![
    (1, Value::str("Ada")), (2, Value::U8(36)),
]), SchemaMode::Inline)?;
assert_eq!(dump_json(&bytes)?, r#"{"name":"Ada","age":36}"#);   // bytes alone, self-describing
# Ok::<(), verit::Error>(())

The verit command-line tool (cargo install verit-cli, or from this repo cargo install --path engine/crates/verit-cli):

verit ls      events.verit   # generation, records, ids, schemas, live/dead space
verit dump    events.verit   # JSON Lines — one record per line, pipes into jq
verit dump    events.verit --id 42     # one record, addressed by its stable id
verit verify  events.verit   # structural check + decode every record; reports a rolled-back commit
verit compact events.verit   # reclaim dead space — and ERASE removed records
verit pack    a.bin b.bin -o out.verit # build a file from self-describing messages
verit unpack  events.verit -o dir/     # records named by id, plus the schema bundle

verit dump   msg.bin    # a bare message works too — dispatch is on magic, not filename
verit id     msg.bin    # print the 128-bit schema id
verit gen    msg.bin    # emit a typed Rust reader/writer for the message's schema
verit build  order.vsc                 # compile a .vsc schema IDL: print its schema id
verit build  order.vsc --lang ts       # emit typed bindings (rust|python|ts|go|cpp) to stdout

dump, id, and verify accept a .verit file or a bare message and do the right thing for what they were handed — the tool identifies artifacts by their magic, the same way the format does.

A shared schema for a polyglot backend. Write the contract once as a .vsc schema file; every language compiles it to the same 128-bit id, so their wire bytes are identical by construction:

dense struct Point { 1: x f64  2: y f64 }
struct Order { 1: id u64  2: item string  3: origin Point }
root Order

verit build order.vsc --lang <rust|python|ts|go|cpp> emits typed bindings for any of the five languages — the Rust generator produces the codegen fast-path reader/writer; the others produce classes/structs with to_verit/from_verit over their dynamic runtime. All five embed the same canonical schema and 128-bit id, so their bytes are identical by construction (proven by deploy/ci/emitter_conformance.sh). When one language owns the type and no shared file is needed, annotate the type in place instead: Python's @verit decorator, or Rust's #[derive(Verit)] (behind the optional derive feature, so the default build stays zero-dependency). See VERIT - System Architecture Part V.

The implementations and their interop matrix live in ports/ — the five-language emit matrix (pwsh deploy/ci/run_all.ps1) plus the GnuCOBOL reader/writer, whose read + write conformance runs via ports/cobol/conformance.sh.

A reference integration. cargo run -p verit --features derive --example eventstore runs a small append-only event store that dogfoods the whole surface end to end — #[derive(Verit)] event types written to a .verit file, a consumer opening that file cold (no registry, no schema: the file carries its own), a tailing consumer resuming from a stable record id, a legacy event resolved across schema versions at rest, a simulated crash rolling back to the previous generation, and purge_ids erasing a record for real.

One file, any language. A .verit file (magic VRTF) holds many messages plus the schemas to read them, mmap-able and read in place. Appends are crash-safe without a journal — a commit is atomic and a torn one rolls back — and every record carries a stable id that survives removal and compaction. See the File Format Specification.

Benchmarks at a glance

One workload (a log event: strings, ints, a list of Point{f64,f64} structs, codes) at three sizes, vs the safe read APIs of protobuf (prost 0.13), Cap'n Proto (capnp 0.20), and FlatBuffers (24.3.25, verified root). ns/op, best of 4 calibrated loops; checksums verified identical across formats each run. Captured 2026-07-21, AMD EPYC (Genoa), Linux x86-64, rustc 1.97.1, --release. Full method, variance notes, the untrusted/DoS-bounded rows, and the losses: docs/Benchmarks.

Three things to read off the table:

• Partial access is flat at ~15 ns from 164 B to 5 KB. Reading 2 fields costs the same whatever the message size, because a read is a bounds-checked indexed load and nothing else is touched. That flat line is the zero-copy claim. (Cap'n Proto is likewise flat at ~60 ns; protobuf and FlatBuffers must decode the record, so their cost climbs with size.)
• Fastest safe encode and scan at every shape — with no unsafe anywhere and the DoS traversal budget available on the untrusted path (~3–10% cost; see the benchmarks doc for those rows).
• Where Veritate loses, plainly: wire size. protobuf is smaller at every shape (varints beat fixed slots; 268 vs 364 B on the medium record) — Veritate spends those bytes on absolute offsets plus a 32-byte self-describing envelope to buy O(1) access and carried schema identity. For sparse data, packed structs recover most of it (70% smaller blocks, still zero-copy).

Documentation

• Architecture — Introduction and Vision, System Architecture, Wire Specification (normative)
• Governance — Contributing (+ Releasing, Reporting Issues, Security Policy)
• Benchmarks — Benchmarks (method, results, honest losses)

License

Apache License 2.0 — see LICENSE and NOTICE. A permissive, OSI-approved open source license with an explicit patent grant: use it freely, including in closed commercial products, no strings attached.