# Exavian Veritate
[](https://github.com/ExavianLabs/veritate/actions/workflows/ci.yml)
[](https://crates.io/crates/verit)
[](https://docs.rs/verit)
[](LICENSE)
> One serialization format that is actually all three things at once —
> **zero-copy**, **self-describing**, and **schema-evolvable** — with no
> `unsafe`, and one wire spec implemented byte-for-byte across six independent
> implementations.
`verit` is a binary serialization format that refuses the usual pick-two
trilemma. Reads are bounds-checked indexed loads straight out of the buffer
(no parse step, no allocation); every message carries the 128-bit content hash
of its writer schema and can embed the schema inline; and fields are matched by
stable numeric IDs so schemas evolve without breaking old readers. The
indirection that evolution needs is paid **once per (writer, reader) schema
pair** — compiled into a cached access plan — not per message and not at build
time.
**The load-bearing claim is verifiable, not aspirational.** The format has five
independent implementations (Rust, Python, C++, Go, TypeScript), each written
from the [wire specification](docs/Architecture/VERIT%20-%20Wire%20Specification.md)
alone. For a shared corpus they produce **byte-for-byte identical** messages and
schema ids, each reads and re-encodes the others' output, and the same cases are
frozen as committed golden vectors. A **sixth** implementation — a GnuCOBOL
reader **and** writer for mainframe integration — is held to those same vectors:
it reproduces every message byte-for-byte and re-derives each 128-bit schema id
from scratch (its own from-scratch SHA-256). See
[VERIT - Introduction and Vision](docs/Architecture/VERIT%20-%20Introduction%20and%20Vision.md).
---
## Repository layout
One public umbrella crate; docs as build evidence.
```
veritate/
├── workspace/ # DEV ONLY — roadmap, checklists, ADRs. NOT git-tracked.
├── docs/ # PUBLIC docs — architecture (incl. the wire spec), governance, benchmarks.
├── engine/ # The umbrella crate `verit` + internal crates: verit-core (engine), verit-cli (the `verit` tool).
├── ports/ # The five non-Rust implementations: python, cpp, go, ts, cobol (the byte-identity proof).
├── settings/ # .toml knobs (benchmark params, defaults) — change behavior without code.
├── deploy/ # Dockerfile, CI, the run_all cross-language proof harness, file(1)/MIME registration.
├── fuzz/ # cargo-fuzz targets (standalone workspace).
├── Cargo.toml # Root virtual workspace.
├── rust-toolchain.toml
└── rustfmt.toml
```
**One public crate.** Consumers depend only on the umbrella `verit`
(`engine`), which re-exports the internal `verit-core` engine. All
crate-boundary re-exports live in `engine/src/lib.rs`, so the engine can be
split or rewritten from one edit point. Detail in
[VERIT - System Architecture](docs/Architecture/VERIT%20-%20System%20Architecture.md).
## Build & use
```bash
cargo add verit # zero runtime dependencies; --features derive for #[derive(Verit)]
```
Working in this repo:
```bash
cargo test --workspace # engine tests, golden vectors, fuzz, codegen conformance
cargo run -p verit --release --example bench # vs protobuf, Cap'n Proto, FlatBuffers, JSON
```
```rust
use verit::prelude::*;
let schema = SchemaBuilder::new()
.add_struct("Person", vec![(1, "name", Dt::Str), (2, "age", Dt::U8)])
.build("Person")?;
let bytes = encode(&schema, &Value::Struct(vec![
(1, Value::str("Ada")), (2, Value::U8(36)),
]), SchemaMode::Inline)?;
assert_eq!(dump_json(&bytes)?, r#"{"name":"Ada","age":36}"#); // bytes alone, self-describing
# Ok::<(), verit::Error>(())
```
The `verit` command-line tool (`cargo install verit-cli`, or from this repo
`cargo install --path engine/crates/verit-cli`):
```bash
verit ls events.verit # generation, records, ids, schemas, live/dead space
verit dump events.verit # JSON Lines — one record per line, pipes into jq
verit dump events.verit --id 42 # one record, addressed by its stable id
verit verify events.verit # structural check + decode every record; reports a rolled-back commit
verit compact events.verit # reclaim dead space — and ERASE removed records
verit pack a.bin b.bin -o out.verit # build a file from self-describing messages
verit unpack events.verit -o dir/ # records named by id, plus the schema bundle
verit dump msg.bin # a bare message works too — dispatch is on magic, not filename
verit id msg.bin # print the 128-bit schema id
verit gen msg.bin # emit a typed Rust reader/writer for the message's schema
verit build order.vsc # compile a .vsc schema IDL: print its schema id
`dump`, `id`, and `verify` accept a `.verit` file or a bare message and do the
right thing for what they were handed — the tool identifies artifacts by their
magic, the same way the format does.
**A shared schema for a polyglot backend.** Write the contract once as a
`.vsc` schema file; every language compiles it to the *same* 128-bit id, so their wire
bytes are identical by construction:
```text
dense struct Point { 1: x f64 2: y f64 }
struct Order { 1: id u64 2: item string 3: origin Point }
root Order
```
`verit build order.vsc --lang <rust|python|ts|go|cpp>` emits typed bindings for
any of the five languages — the Rust generator produces the codegen fast-path
reader/writer; the others produce classes/structs with `to_verit`/`from_verit`
over their dynamic runtime. All five embed the same canonical schema and 128-bit
id, so their bytes are identical by construction (proven by
`deploy/ci/emitter_conformance.sh`). When one language owns the type and no
shared file is needed, annotate the type in place instead: Python's `@verit`
decorator, or Rust's `#[derive(Verit)]` (behind the optional `derive` feature, so
the default build stays zero-dependency). See
[VERIT - System Architecture](docs/Architecture/VERIT%20-%20System%20Architecture.md) Part V.
The implementations and their interop matrix live in [ports/](ports/) — the
five-language emit matrix (`pwsh deploy/ci/run_all.ps1`) plus the GnuCOBOL
reader/writer, whose read + write conformance runs via
`ports/cobol/conformance.sh`.
**A reference integration.** `cargo run -p verit --features derive --example
eventstore` runs a small append-only event store that dogfoods the whole surface
end to end — `#[derive(Verit)]` event types written to a **`.verit` file**, a
consumer opening that file cold (no registry, no schema: the file carries its
own), a tailing consumer resuming from a stable record id, a legacy event
resolved across schema versions *at rest*, a simulated crash rolling back to the
previous generation, and `purge_ids` erasing a record for real.
**One file, any language.** A `.verit` file (magic `VRTF`) holds many messages
plus the schemas to read them, `mmap`-able and read in place. Appends are
crash-safe without a journal — a commit is atomic and a torn one rolls back —
and every record carries a stable id that survives removal and compaction. See
the [File Format Specification](docs/Architecture/VERIT%20-%20File%20Format%20Specification.md).
## Benchmarks at a glance
One workload (a log event: strings, ints, a list of `Point{f64,f64}` structs,
codes) at three sizes, vs the safe read APIs of protobuf (prost 0.13), Cap'n
Proto (capnp 0.20), and FlatBuffers (24.3.25, verified `root`). ns/op, best of
4 calibrated loops; checksums verified identical across formats each run.
Captured 2026-07-21, AMD EPYC (Genoa), Linux x86-64, rustc 1.97.1, `--release`.
Full method, variance notes, the untrusted/DoS-bounded rows, and the losses:
[docs/Benchmarks](docs/Benchmarks/VERIT%20-%20Benchmarks.md).
| **small** (164–74 B) | encode | **59** | 100 | 224 | 393 |
| | full scan | **32** | 166 | 131 | 190 |
| | partial (2 fields) | **16** | 136 | 60 | 152 |
| **medium** (364–268 B) | encode | **73** | 237 | 247 | 953 |
| | full scan | **35** | 427 | 128 | 592 |
| | partial (2 fields) | **15** | 420 | 58 | 470 |
| **large** (~5–8 KB) | encode | **351** | 4052 | 546 | 14324 |
| | full scan | **292** | 5067 | 444 | 8389 |
| | partial (2 fields) | **15** | 5207 | 62 | 8016 |
Three things to read off the table:
- **Partial access is flat at ~15 ns from 164 B to 5 KB.** Reading 2 fields
costs the same whatever the message size, because a read is a bounds-checked
indexed load and nothing else is touched. That flat line *is* the zero-copy
claim. (Cap'n Proto is likewise flat at ~60 ns; protobuf and FlatBuffers
must decode the record, so their cost climbs with size.)
- **Fastest safe encode and scan at every shape** — with no `unsafe` anywhere
and the DoS traversal budget available on the untrusted path (~3–10% cost;
see the benchmarks doc for those rows).
- **Where Veritate loses, plainly: wire size.** protobuf is smaller at every
shape (varints beat fixed slots; 268 vs 364 B on the medium record) —
Veritate spends those bytes on absolute offsets plus a 32-byte
self-describing envelope to buy O(1) access and carried schema identity.
For sparse data, packed structs recover most of it (70% smaller blocks,
still zero-copy).
## Documentation
- **Architecture** — [Introduction and Vision](docs/Architecture/VERIT%20-%20Introduction%20and%20Vision.md),
[System Architecture](docs/Architecture/VERIT%20-%20System%20Architecture.md),
[Wire Specification](docs/Architecture/VERIT%20-%20Wire%20Specification.md) (normative)
- **Governance** — [Contributing](docs/Governance/VERIT%20-%20Contributing.md) (+ Releasing, Reporting Issues, Security Policy)
- **Benchmarks** — [Benchmarks](docs/Benchmarks/VERIT%20-%20Benchmarks.md) (method, results, honest losses)
## License
Apache License 2.0 — see [LICENSE](LICENSE) and [NOTICE](NOTICE). A permissive,
OSI-approved open source license with an explicit patent grant: use it freely,
including in closed commercial products, no strings attached.