Crate substrate_data 

Substrate Data: one data model, text syntax, binary form, packed frames and schema system — for a program’s files, saves, caches and wire messages alike. See docs/format.md.

• Value: the model — JSON’s values plus typed ones (references @kind:id, bytes);
• text: JSON with comments, trailing commas and unquoted keys, parsed with the position of every value; patch edits a document in place, keeping its comments and layout;
• typed: the one grammar of the typed values’ string forms (@file:images/logo.png, @bytes:…);
• binary: the compact, self-describing tagged form;
• frame: the packed form for messages whose schema both sides know;
• Schema (#[derive(Schema)]): a format written once — read with the path of every problem, written, encoded, packed, and described (Shape) for editors and docs.

Re-exports

pub use diagnostic::Code;

pub use diagnostic::Diagnostic;

pub use diagnostic::Errors;

pub use diagnostic::Problem;

pub use diagnostic::Severity;

pub use schema::with;

pub use schema::Expected;

pub use schema::Id;

pub use schema::Kind;

pub use schema::Kinds;

pub use schema::NamedType;

pub use schema::OneOrList;

pub use schema::Ordered;

pub use schema::Read;

pub use schema::Reader;

pub use schema::RefId;

pub use schema::Resolve;

pub use schema::Schema;

pub use schema::Shape;

pub use schema::ShapeDoc;

pub use schema::Uid;

pub use schema::Unresolved;

pub use schema::from_doc_with;

pub use schema::from_text;

pub use schema::from_value;

pub use schema::from_value_with;

pub use schema::placed;

pub use schema::read_bytes;

pub use schema::read_ref_value;

pub use schema::read_value;

pub use schema::to_bytes;

pub use schema::to_text;

Modules

atom

Atoms: names that repeat across messages (core:open, a live record’s state) kept once a process and sent in frames as their number. An Atom is a u32: copied, compared and packed as one; its text is the table’s, alive as long as the process. A frame carries the number, every other form (text, the tagged binary form, files) the text — the numbers mean nothing outside the process.

binary

The binary form — self-describing, compact: what a build compiles content to, what saves and caches are kept in, what a reader that doesn’t know the schema can still walk. docs/format.md (“Binary form”) has the layout: a header (SDAT, a version byte), then the root value, a tag byte and its payload. A string is written where it’s first used and referred to after that, so repeated keys and ids cost one varint each and the writer never goes back over what it wrote.

diagnostic

What a tool finds wrong — reading a document, building a mod, serving a request — each as one record: what it is (a code), how bad, what’s wrong and where (a file, its line and column, a value’s path in it). The one kind of problem every tool says and every tool reads.

frame

Frames: data packed for messages — both sides know the schema, so nothing says what a value is or what it’s called. A record’s fields go in the order they’re declared, after the record’s length (so a reader that knows fewer fields skips the rest; one that knows more takes their defaults: fields are only ever appended). Strings go in a table at the end, each once; the body refers to them by index. docs/format.md (“Frames”) has the layout of every value.

io

Messages over a byte stream — a socket, a pipe: each message a frame, its length (a little-endian u32) and then its binary form. What programs say to each other (a program and its editor, a tool’s service).

names

The words the formats share, defined once: the typed value grammar’s mark and the keys a format’s own structure uses. Problem codes are crate::diagnostic::Problem; the words a shape is described by are crate::schema::ShapeNode’s variants.

patch

Editing text in place: patch changes a document to hold a new value and keeps everything that didn’t change as it was — comments, key order, number spelling, layout. What changed is written by format(), the way editors write a value.

schema

Schemas: a format is a Rust type implementing Schema (usually #[derive(Schema)]). One schema reads and writes the format, checks it with the path of every problem, collects its references for the build to check, and describes itself (Shape) for editors (property grids, completion) and the docs.

source

A source file as it’s kept and as it’s read: on disk the binary form (what an editor may save), to people its text — formatted as editors write it, so a line in an error is the line the editor shows. A file still in text is read as it is.

text

The text syntax: JSON, plus // and /* */ comments, trailing commas and unquoted keys (the part of JSON5 editors already understand).

typed

The typed values’ string forms — one grammar for every format:

Macros

field

Field ident of struct T as files write it: field!(UserProfile, display_name) → "display_name". A field T hasn’t got doesn’t compile.

kind

Declares kinds, each with its id type (String when none is said): substrate_data::kind!(User = "user": u64, Doc = "doc");.

named_schema

Implements Schema for a NamedType: every form is its stored form’s, so a reader that only knows the shape reads it as stored.

Structs

Atom

A name kept once a process; see the module.

Bump

An arena to bump allocate into.

Doc

A parsed document: the value and where each of its parts is.

KindName

A reference’s kind: lower case ASCII letters, digits and _, starting with a letter (user, doc, file). Checked when it’s made, then a number: copied and compared as one.

LineCol

A 1-based line and column (in characters).

Map

Keys are shared: a decoded document allocates each distinct key once, however many maps have it.

MapRef

A borrowed map: entries in file order.

Path

Where a value is in a document: tasks[2].parts[0].owner.

Ref

A reference: its kind (user, doc, file, or any other the format names) and id (images/logo.png, team:alice). The id is opaque here; what it names and whether it exists is the format’s business (a crate::Resolve checks it), and a typed field reads it as its kind’s id type (crate::Id).

RefKey

A reference’s id as written, untyped: anything but empty, whitespace or control characters. What it names is the format’s business; a typed field reads it as its kind’s id type (crate::RefId).

Span

A byte range of the source.

Spans

Where each value of a parsed document is: the value’s span, its key’s (map entries), and its children’s, in the same order as the value’s.

Stream

A list of T kept written: its body in the binary form, its strings its own.

StreamWriter

Writes a Stream’s items as they come.

SyntaxError

A syntax error.

Enums

Seg

One step of a Path.

Value

The text form has no syntax of its own for the typed values (Ref, Bytes): they’re written as strings ("@file:images/logo.png"), so every file stays JSON, and the schema of a field says what its string means. The binary form keeps the type.

ValueKind

Which of a Value’s cases a value is — what messages name ("a string", "a list") when they say what was found.

ValueRef

A borrowed Value: the same cases, strings &'a str, lists and maps slices.

Derive Macros

Schema

#[derive(Schema)]: see the crate’s docs.