Expand description
Collection identity — what it means for a collection to EXIST.
§Why this module exists
Before this, “which collections exist” was not a fact about the database. It was a fact about the storage substrate, and the two substrates disagreed:
disk, flush between PUT and DELETE : ["orders"]
disk, both inside one flush tick : []
memory : []All three are the same logical history — create a collection, then empty it.
Disk mode answered by listing directories (crate::index::IdIndex::collections
did a read_dir), and the WAL write buffer is keyed by (coll, id), so a PUT
followed by a DELETE before the 1-second flush ticker fires overwrites the
buffered entry with its own tombstone. No directory is ever created. PUT,
flush, DELETE leaves the directory behind forever, because the flush path only
ever calls remove_file — it has no remove_dir in it at all.
So the namespace was decided by a background timer. That is survivable for a
LIST COLLECTIONS convenience call, and fatal for a state root: a root
commits to a namespace, which is only meaningful if two replicas of the same
history agree on what the namespace IS.
§The rule
A collection exists because a record says so, not because a directory is lying around. Creation is an event, the event is a node, and the node lives in the DAG like everything else. Emptying a collection does not destroy it; only an explicit drop does, and a drop is a tombstone rather than an absence.
Putting the registry in the DAG rather than in a sidecar file is the boring
choice and it pays three times: since() replicates collection creation to
followers for free, AS OF answers “which collections existed at seq N”
for free, and verify() covers the registry for free. A COLLECTIONS file
would have needed all three written by hand.
§Reserved names
The registry has to live somewhere, and wherever it lives must not be user-writable — otherwise a client can forge the namespace by writing to it directly. The same reservation is what will later keep state-root records from being part of the state they describe, which is a decent sign it is the right primitive: one rule, used twice.
Constants§
- COLLECTIONS
- The collection registry. Ids are collection names; the latest version of each says whether that collection is currently live.
- META
- Engine metadata that is neither a collection record nor a root: the history
floor lives here. Kept out of
ROOTSso that collection stays homogeneous andlist_rootsnever has to skip an entry it cannot parse – an entry skipped silently is indistinguishable from one that failed to parse. - RESERVED_
PREFIX - Everything under this prefix belongs to the engine. User writes are refused.
- ROOTS
- Persisted state roots. Ids are zero-padded sequence numbers so that the index’s lexicographic id ordering is also numeric ordering.
Functions§
- is_
reserved - Is this name part of the engine’s own namespace?
- refuse_
reserved - Refuse a write the caller is not allowed to make.
- seq_id
- Zero-padded so lexicographic ordering is numeric ordering.
u64::MAXis 20 digits. - validate_
name - Is this a name a collection can durably HAVE?
- validate_
writable - A name that may be written to: valid AND not engine-owned.