Skip to main content

Crate verit_core

Crate verit_core 

Source
Expand description

§Veritate

One serialization format that is actually all three things at once:

  • Zero-copy: every field read is a bounds-checked indexed load straight from the message buffer. Strings and bytes come back as &str / &[u8] borrowing the buffer. No parse step, no allocation on the read path.
  • Self-describing: every message carries the 128-bit content hash of its writer schema, and can carry the full schema inline. A message plus nothing else is fully interpretable — see dump_json.
  • Schema-evolvable: fields are identified by stable numeric IDs. Readers resolve (writer schema, reader schema) once into a cached Resolver access plan; added fields read as None for old readers’ data, unknown fields cost nothing, renames are free, integer widening is converted on load.

The trick that resolves the classic pick-two trilemma: the indirection that evolution needs is paid once per schema pair, not per message (protobuf tags, JSON keys) and not at build time (FlatBuffers codegen). See the architecture docs for the wire format.

Re-exports§

pub use container::Container;Deprecated
pub use container::ContainerWriter;Deprecated
pub use derive::VeritType;
pub use dump::dump_json;
pub use dump::dump_json_with;
pub use encode::encode;
pub use encode::SchemaMode;
pub use error::Error;
pub use error::Result;
pub use file::FileBuilder;
pub use file::FileReader;
pub use file::FileView;
pub use file::FileWriter;
pub use file::Footer;
pub use file::Record;
pub use file::Resolvers;
pub use file::FIRST_RECORD_ID;
pub use file::OPT_RECORD_CRC;
pub use message::Budget;
pub use message::ListReader;
pub use message::Message;
pub use message::Ref;
pub use message::StructReader;
pub use registry::SchemaRegistry;
pub use resolve::Resolver;
pub use schema::Default;
pub use schema::Dt;
pub use schema::EnumDef;
pub use schema::FieldDef;
pub use schema::Schema;
pub use schema::SchemaBuilder;
pub use schema::StructDef;
pub use schema::StructMode;
pub use schema::Type;
pub use schema::TypeDef;
pub use value::Scalars;
pub use value::Value;

Modules§

codegen
Rust code generation: compile a Schema into typed readers and writers.
container
.vertc — the Veritate container: an at-rest file holding many messages for zero-copy random access, designed to be mmap-ped and read in place.
derive
Runtime support for #[derive(Verit)] — the Rust peer of Python’s @verit decorator.
dump
The self-description proof: given message bytes and nothing else, recover the writer schema from the inline region and render every present field — with its human-readable name — as JSON.
encode
Encoder: Schema + Value tree → message bytes.
error
file
.verit — the Veritate file: many messages in one self-contained, mmap-able, appendable file.
hash
Content hashing for schema ids.
idl
The .vsc schema IDL — a small text front-end that compiles to a Schema. It invents no wire semantics: it drives SchemaBuilder, so the result is exactly the same canonical VSC1 bytes (and 128-bit id) any other definition of the same schema produces. The IDL therefore cannot drift from the wire format — the id is the contract, and it is computed the same way regardless of how the schema was written.
layout
Deterministic struct layout. Because the layout is a pure function of the schema, a schema id fully determines every byte offset — the schema acts as one shared “vtable” for every message that uses it, which is what makes per-message zero-copy access possible without per-object tables.
message
Zero-copy message access. Message::parse only reads the 24-byte header; nothing else is touched until a field is asked for, and every field read is a bounds-checked load straight out of the buffer through a precompiled Resolver plan. Strings and byte fields are returned as &str / &[u8] borrowing the message buffer — no allocation, no copy.
registry
Schema registry + distribution bundle for multi-service deployments.
resolve
Schema resolution: the piece that buys evolution without giving up zero-copy. A Resolver is built once per (writer schema, reader schema) pair and compiles, for every corresponding struct type, an access plan mapping each reader field ID to either a concrete byte slot in the writer’s layout (with an optional lossless widening) or Absent. After that, reading any number of messages costs no per-message resolution work.
schema
Schema model, builder, canonical binary encoding (“VSC1”), and content hashing. The canonical form sorts types by name, fields by ID, and enum variants by value, so one logical schema has exactly one byte encoding and therefore exactly one id.
value
Dynamic values for the write path. The prototype has no codegen; you build a Value tree against a runtime crate::Schema and encode it. A struct value lists (field id, value) pairs — omitted fields are absent (their presence bit stays 0 and readers see None).
wire
Public low-level wire primitives for generated code (see crate::codegen). Everything here is bounds-checked and unsafe-free; generated readers/writers compose these with offsets computed at generation time from the deterministic layout algorithm.