Expand description
§Veritate
One serialization format that is actually all three things at once:
- Zero-copy: every field read is a bounds-checked indexed load straight
from the message buffer. Strings and bytes come back as
&str/&[u8]borrowing the buffer. No parse step, no allocation on the read path. - Self-describing: every message carries the 128-bit content hash of its
writer schema, and can carry the full schema inline. A message plus
nothing else is fully interpretable — see
dump_json. - Schema-evolvable: fields are identified by stable numeric IDs.
Readers resolve (writer schema, reader schema) once into a cached
Resolveraccess plan; added fields read asNonefor old readers’ data, unknown fields cost nothing, renames are free, integer widening is converted on load.
The trick that resolves the classic pick-two trilemma: the indirection that evolution needs is paid once per schema pair, not per message (protobuf tags, JSON keys) and not at build time (FlatBuffers codegen). See the architecture docs for the wire format.
Re-exports§
pub use container::Container;Deprecated pub use container::ContainerWriter;Deprecated pub use derive::VeritType;pub use dump::dump_json;pub use dump::dump_json_with;pub use encode::encode;pub use encode::SchemaMode;pub use error::Error;pub use error::Result;pub use file::FileBuilder;pub use file::FileReader;pub use file::FileView;pub use file::FileWriter;pub use file::Record;pub use file::Resolvers;pub use file::FIRST_RECORD_ID;pub use file::OPT_RECORD_CRC;pub use message::Budget;pub use message::ListReader;pub use message::Message;pub use message::Ref;pub use message::StructReader;pub use registry::SchemaRegistry;pub use resolve::Resolver;pub use schema::Default;pub use schema::Dt;pub use schema::EnumDef;pub use schema::FieldDef;pub use schema::Schema;pub use schema::SchemaBuilder;pub use schema::StructDef;pub use schema::StructMode;pub use schema::Type;pub use schema::TypeDef;pub use value::Scalars;pub use value::Value;
Modules§
- codegen
- Rust code generation: compile a
Schemainto typed readers and writers. - container
.vertc— the Veritate container: an at-rest file holding many messages for zero-copy random access, designed to bemmap-ped and read in place.- derive
- Runtime support for
#[derive(Verit)]— the Rust peer of Python’s@veritdecorator. - dump
- The self-description proof: given message bytes and nothing else, recover the writer schema from the inline region and render every present field — with its human-readable name — as JSON.
- encode
- Encoder:
Schema+Valuetree → message bytes. - error
- file
.verit— the Veritate file: many messages in one self-contained,mmap-able, appendable file.- hash
- Content hashing for schema ids.
- idl
- The
.vscschema IDL — a small text front-end that compiles to aSchema. It invents no wire semantics: it drivesSchemaBuilder, so the result is exactly the same canonicalVSC1bytes (and 128-bit id) any other definition of the same schema produces. The IDL therefore cannot drift from the wire format — the id is the contract, and it is computed the same way regardless of how the schema was written. - layout
- Deterministic struct layout. Because the layout is a pure function of the schema, a schema id fully determines every byte offset — the schema acts as one shared “vtable” for every message that uses it, which is what makes per-message zero-copy access possible without per-object tables.
- message
- Zero-copy message access.
Message::parseonly reads the 24-byte header; nothing else is touched until a field is asked for, and every field read is a bounds-checked load straight out of the buffer through a precompiledResolverplan. Strings and byte fields are returned as&str/&[u8]borrowing the message buffer — no allocation, no copy. - registry
- Schema registry + distribution bundle for multi-service deployments.
- resolve
- Schema resolution: the piece that buys evolution without giving up
zero-copy. A
Resolveris built once per (writer schema, reader schema) pair and compiles, for every corresponding struct type, an access plan mapping each reader field ID to either a concrete byte slot in the writer’s layout (with an optional lossless widening) orAbsent. After that, reading any number of messages costs no per-message resolution work. - schema
- Schema model, builder, canonical binary encoding (“VSC1”), and content hashing. The canonical form sorts types by name, fields by ID, and enum variants by value, so one logical schema has exactly one byte encoding and therefore exactly one id.
- value
- Dynamic values for the write path. The prototype has no codegen; you build
a
Valuetree against a runtimecrate::Schemaand encode it. A struct value lists (field id, value) pairs — omitted fields are absent (their presence bit stays 0 and readers seeNone). - wire
- Public low-level wire primitives for generated code (see
crate::codegen). Everything here is bounds-checked andunsafe-free; generated readers/writers compose these with offsets computed at generation time from the deterministic layout algorithm.