Module document 

What a document record holds, which is the document and nothing around it.

06 section 2.1 gives kind 2 to a document and says nothing about what is inside it. The answer is that a document record’s value is the YOJB value byte for byte, with no header of its own, and this module is where that decision is written down and checked.

+---------+----------------------------------------+
|  head   |            the rest of the value       |
|    4    |                                        |
+---------+----------------------------------------+

Why there is no framing

A vector record needs a header because a run of f32 says nothing about itself: the dimension and the element type have to come from somewhere, and the record is the only place that a reader with no catalogue can get them. YOJB is the opposite. Every value begins with a four byte header that carries its kind, its flags and its count, offsets inside a container are relative to that header, and the last entry is enough to work out the whole length. A frame around it would be four to eight bytes on every document that repeat what the first word already says, and it would be a second length to disagree with the first one.

So the record’s value is the document, DocumentBody::decode is the check that the first word is one this version understands, and the length a reader gets back from the log is the length of the document.

What this checks and what it does not

The record layer owns the framing and yo-doc owns the value. This checks the head: that it is there, that the tag is one of the seven this version defines, and, for a scalar, that the payload is exactly as long as the head says and the right length for its type. It does not walk a container, because walking a container means knowing where the entry tables are and how deep the nesting is allowed to go, and there is one copy of that in yo-doc on purpose. Value::validate is the deep check and yodb check is what calls both.

Getting this split wrong in the other direction would be worse than the duplication it saves. A reader that has to understand documents to skip a document record cannot skip a kind it does not know, and skipping is what 07 section 9 requires of it.

Why the numbering is here as well as in yo-doc

These are the bytes on disk, so they belong with the other frozen shapes, and a reader that only wants to know whether a record is an object or an array should not have to pull in the document model to find out. yo-doc has the same numbers because it is the one that reads them, and a test in that crate holds the two together, which is what yo-kv already does with crate::ValueType.

What is not in here

The key table. An interned object stores two byte ids instead of key bytes, and the names those ids stand for live in the collection rather than in any one document. That is a collection chunk under a checkpoint, not a record kind, and it is not written yet.

Interning needs no generation number alongside it, which is worth saying because it looks like it should. An id is the row a name sits at in a table that never removes anything, so an id handed out at any point stays the same name for the life of the collection, and a document interned against an early state of the table reads correctly against every later one.

Modules

doc_flags

The flag bits of a value header.

Structs

DocumentBody

A document record’s value, borrowed.

Enums

ValueTag

What the low three bits of a value header say the value is.

Constants

DOC_COUNT_MAX

The largest count a header can hold, which caps a container at 16.7 M elements and a scalar at 16 MiB of payload.

DOC_COUNT_SHIFT

Where the count starts in the header.

DOC_HEADER_LEN

The header every YOJB value begins with.