Expand description
state_root_v1 — one hash that commits to what the database currently says.
state_root_v1
├── namespace_root which collections exist
└── records_root what is in them§What a state root is NOT
It is not a commitment to history. History already has one: the running
Merkle head (Db::head), which chains every write by seq and object hash,
and answers “did this database’s past change?”. The state root answers a
different question — “do these two databases say the same thing right now?”
— and it has to be computable without replaying anything.
Keeping them separate is deliberate. A root that folded in history could not be compared across two databases that reached the same state by different routes, and that comparison is most of what a root is for: replica agreement, drift detection, anchoring, and the “before” and “after” sides of a diff.
§Why the leaves are logical, not object hashes
The obvious construction is a tree over node.hash. It is wrong here, and
the reason is worth stating because it is not obvious from reading the
struct: with encryption on, a node’s hash is not a function of its content.
ObjectStore::write hashes the CIPHERTEXT, and encrypt draws a fresh
random AES-GCM nonce per call. Measured on the running engine:
PLAINTEXT same=true a=144eb088e2f5 b=144eb088e2f5
ENCRYPTED same=false a=d501c7798dcf b=9af86e8fafecSame logical node, written twice under one DEK, two different hashes. A root built on object hashes would therefore differ between an encrypted replica and a plaintext one holding identical data — which destroys the only property anybody wants from it. So the leaves are built from the logical content, and the encryption layer never touches the root.
§Decided questions
Every one of these is a place where two reasonable implementations would
disagree, which is exactly the set that has to be pinned before the format
locks. The cross-language vectors in vectors/state_root_v1.json pin them
as data, so a second implementation can be checked without reading this.
Hash. BLAKE2b-512 truncated to the first 32 bytes, matching the rest of the engine.
Domain separation. Every hash input begins with a distinct tag. Leaves, internal nodes, subtree roots and the final composition cannot be confused for one another, so no leaf can be presented as an internal node.
Length prefixing. Every variable-length field is preceded by its length
as a u64 little-endian. Concatenation is therefore unambiguous: ("ab", "c")
and ("a", "bc") do not collide.
Ordering. Leaves are sorted by their key bytes, comparing raw UTF-8. Not by locale, not by code point after normalisation — by bytes, because that is the one ordering every language agrees on without a library.
Unicode. None applied. Names are committed as the exact UTF-8 bytes they were created with. Normalising inside the encoder would make two distinct collections collide in the root; the canonicalisation belongs at creation time, not at hashing time.
Odd leaves. Promoted unchanged to the next level. NOT duplicated — leaf duplication is the Bitcoin CVE-2012-2459 construction, where two different leaf sets produce one root.
Leaf count. Committed alongside the tree in the subtree root. Promotion alone leaves the tree shape ambiguous for some leaf counts; the count removes the question entirely rather than requiring an argument that it cannot arise.
Empty. A distinct constant H(tag), never zero. Zero is what an
uninitialised field looks like, and “no collections” must not be confusable
with “nobody computed this”.
Tombstones. Absent. A record leaf exists for each currently-live document; a deleted document contributes nothing. The root commits to current visible state, and history carries the tombstone.
Dropped collections. Absent from the namespace, present in history. An emptied-but-live collection IS in the namespace with no records under it — which is the whole reason durable collection identity had to land first.
Document field order. Preserved as written, not sorted. NEDB treats
document order as meaningful (serde_json preserve_order is on crate-wide
so SELECT * returns columns in document order), so two documents whose
keys are ordered differently are two databases that answer differently, and
a root that could not tell them apart would not be committing to state.
Structs§
- Record
Ref - One live document, in the form
records_rootconsumes. - Root
Record - A root, plus the sequence it describes.
- Root
Verification - The result of checking one persisted root.
- State
Root - The three roots together — what
root inspectreports and what a root record stores.
Enums§
- Recomputation
- Could the root be recomputed, and did it agree?
- Record
Status - Is the stored record itself intact and readable?
- Unavailable
Reason - Why a recomputation could not be performed.
Functions§
- compute
- Compose the three roots from already-gathered material.
- encode_
value - Encode a JSON value canonically.
- hex
- Hex, because every boundary this crosses is textual.
- namespace_
leaf - One live collection.
- namespace_
root - Commit to which collections exist. Input need not be sorted.
- record_
leaf - One live document, as logical content.
- records_
root - Commit to document content. Input need not be sorted.
- state_
root - The public root.
Type Aliases§
- Digest32
- A 32-byte digest, hex-encoded at the boundary and kept as bytes inside.