# kcode-kweb-db 1.0.1
`kcode-kweb-db` is a blocking, filesystem-backed convergent store for canonical Ed25519-signed Kweb transactions. Version 1 is a new root and wire format; it does not open a 0.1 root.
## Public API
```rust
KwebDb::open(path, Config) -> Result<KwebDb>
KwebDb::start_transaction(&self, Provenance) -> Result<Transaction<'_>>
KwebDb::accept_transaction(&self, TransactionPackage) -> Result<bool>
KwebDb::accept_gossip_transaction(&self, TransactionPackage, &dyn TransactionSource) -> Result<bool>
KwebDb::get_node(&self, NodeId) -> Result<Node>
KwebDb::get_node_history(&self, NodeId) -> Result<NodeHistory>
KwebDb::get_object(&self, ObjectId) -> Result<Vec<u8>>
Transaction::create_object(&mut self, Vec<u8>) -> Result<ObjectId>
Transaction::reserve_node_id(&mut self) -> Result<NodeId>
Transaction::create_reserved_node(&mut self, NodeId, NodeData) -> Result<()>
Transaction::create_node(&mut self, NodeData) -> Result<NodeId>
Transaction::update_node(&mut self, NodeId, NodeData) -> Result<()>
Transaction::merge(&mut self, TransactionId, TransactionId) -> Result<()>
Transaction::finalize(self) -> Result<TransactionId>
```
`NodeData` contains `short_name`, `short_description`, `long_description`, `owner`, ordered `fixed_connections: Vec<NodeId>`, ordered `recent_connections: Vec<NodeId>`, and `objects`. Create and update sign and replace the complete value. Empty vectors clear connections. Reordering is meaningful. Each vector is unique internally, but one target may occur once in each vector. The database preserves order and imposes no fixed/recent count policy. It does not assign special meaning to any position.
For mutually referential new nodes, call `reserve_node_id` for every node first and then materialize each reservation with `create_reserved_node`. Reservations belong only to their transaction. An unreserved or already materialized ID is rejected, and `finalize` rejects any reservation that was not materialized. `create_node` remains the convenience path for ordinary creation and internally reserves and materializes one ID. Same-transaction node references, including circular references, are validated and committed atomically.
A visible node is one whole signed revision. Concurrent text, owner, connection, and object-reference changes compete together; fields are never combined across revisions. Causal descendants supersede ancestors. For incomparable candidates, the configured lower writer-priority index wins, followed by the lexicographically higher transaction ID for the same writer. `merge` names an exact conflict pair and a complete update in the same transaction. `NodeHistory` exposes provenance, complete candidate data, the current frontier, and current committed visibility.
`Config` contains a 32-byte Ed25519 signing key, a unique strict `writers_by_priority` list containing the local writer, and an `Arc<dyn Gossip>`. `Gossip::announce` receives one owned transaction package after durable commit and returns `true` only when handoff succeeded. Failure leaves the durable outbox item queued. `accept_transaction` is strict admission for callers that are expected to submit a causally complete package: any uncommitted head is an `InvalidTransaction` error. `accept_gossip_transaction` is the only waiting admission path and calls its supplied `TransactionSource` with missing IDs, allowing the transport to request dependencies from the exact peer.
An exact duplicate of either a committed or currently waiting transaction is a successful no-op and returns `Ok(false)`; a newly accepted transaction returns `Ok(true)`. Duplicate admission never rewrites state or advances gossip. Payload-bearing `ObjectPayload` and `TransactionPackage` deliberately do not implement `Clone`. The crate has no Serde or JSON code: Kennedy-owned adapters should use separate transport DTOs and should frame canonical signed transactions and raw object payloads directly.
The application-facing mutation and read API is typed Rust data: callers pass `NodeData`, `Provenance`, IDs, and object buffers and never encode or decode database files. All state/node/history/index/WAL/object envelopes are private implementation formats. Replication is the deliberate exception: `TransactionPackage.transaction` is an opaque canonical signed binary transaction and its objects are raw byte buffers, because peers must exchange and authenticate exactly the same signed bytes. Kennedy should forward that package through its transport rather than reinterpret its fields.
`NodeId` and `ObjectId` are six-byte locators rendered as exactly eight case-sensitive URL-safe unpadded Base64 characters. The high bit is clear for nodes and set for objects, making their raw domains disjoint. Use `from_bytes` for validated raw construction and `to_bytes` for extraction; the tuple field is not public. Paths use the first two characters as a shard and the final six as a stem: `nodes/AA/BBBBBB.kwn` and `objects/AA/BBBBBB.kwo`. Transaction and writer IDs remain 64 lowercase hexadecimal characters; raw construction uses `TransactionId::from_bytes` or the validating `WriterId::from_verifying_key`, and both expose `to_bytes`. Locators are not capabilities.
A transaction guard excludes another local builder or inbound package. `start_transaction` snapshots every current database head and `finalize` includes them in the signed transaction, so application callers do not track or supply a single latest transaction ID. Reads continue to see the old complete files while a transaction is built and while its durable intent is prepared. Dropping the guard commits nothing. One transaction may create or update many different nodes, but may not create and update one node or update one node twice.
Objects are immutable owned buffers. `MAX_OBJECT_BYTES` and `MAX_TRANSACTION_OBJECT_BYTES` are both exactly 34,359,738,368 bytes (32 GiB); the aggregate applies to all new payloads in one transaction. The crate supports only 64-bit targets. Object payloads move through builders and inbound admission without payload-sized clones. Waiting gossip admission writes each final checksummed object envelope once beneath `incoming/`. When the transaction becomes ready, the WAL and final object path are hard links to that same inode, so promotion performs metadata operations and checksum reads but never rewrites the payload. Local finalization releases its buffers before the durable gossip package is loaded. Reads and gossip load only one requested object or one current package.
## Disk and recovery formats
The root-format-3 marker is the exact 16-byte binary value `KWDBROOT`, seven zero bytes, and a final byte `0x03`. A missing marker in a nonempty root, including any earlier layout, returns `OfflineUpgradeRequired`; unknown markers fail closed. The root also contains `LOCK`, `transactions.kwl`, `state.kws`, and the `wal`, `nodes`, `objects`, `incoming`, `history`, `transactions`, and `gossip-outbox` directories. `incoming` is the uncommitted object-staging area for memory-resident gossip waiters, not authoritative database state.
State, node, history, transaction-index, outbox, and WAL records use one canonical binary envelope: an eight-byte kind magic, big-endian `u64` payload length, the exact binary payload, then SHA-256 over the magic, length, and payload. Payload fields have one encoding: unsigned and signed integers are fixed-width big-endian; booleans and enum/optional tags are one byte; strings are `u32` byte length plus exact UTF-8; vectors are `u64` item count plus canonical items; IDs are their raw fixed-width bytes; and UTC timestamps are big-endian `i64` seconds plus `u32` nanoseconds. Decoders reject unknown tags, invalid IDs, impossible lengths, trailing bytes, noncanonical collection state, and checksum mismatches.
Transaction metadata and signed bytes are sharded separately under `transactions/<first-2>/<remaining-62>.kwi|kwt`. A `.kwt` file is binary: `KWTBYTE3`, transaction ID, big-endian `u64` signed-byte length, SHA-256, and exact canonical signed bytes. Per-node history uses `history/<first-2>/<remaining-6>/index.kwh` plus one immutable `entries/<transaction>.khe` record per revision, so adding history does not rewrite the node's complete history. Unknown versions, non-regular addressed files, identity mismatches, noncanonical payloads, or checksum failures are corruption.
The canonical record kinds and payload fields are:
| Magic | Role | Payload fields |
| --- | --- | --- |
| `KWSTATE3` | `state.kws` | `format`, `generation`, `log_offset`, sorted committed `heads`, and exact ordered `writers_by_priority` |
| `KWNODE03` | current node | `format`, `id`, `visible_transaction`, and complete frontier candidates; the public visible node is derived from the named candidate rather than stored twice |
| `KWHIDX03` | history index | `format`, `node_id`, current `frontier`, optional `visible`, and effective creation transaction IDs |
| `KWHENT03` | immutable history entry | transaction/writer IDs, timestamp, provenance, one creation/update tag, complete node data, and merge pairs; public `active` and `updated` fields are derived |
| `KWTMETA3` | transaction DAG metadata | `format`, `id`, sorted parents, and DAG generation |
| `KWQUEUE3` | durable gossip marker | `format` and transaction ID |
| `KWWAL003` | prepared intent | `format`, expected generation/log offset, optional staged-log metadata, and ordered staged-file operations |
Offline tools should treat these envelopes and payloads as read-only versioned data; only this library publishes live root mutations.
Object files use `KWOBJ\0\x03\0`, the six-byte object ID, 32-byte creating transaction ID, big-endian `u64` payload length, 32-byte payload SHA-256, a 32-byte SHA-256 over those preceding header fields, then exact payload bytes. They are installed with create-new/no-overwrite semantics. An existing different envelope is a collision or corruption, never an overwrite.
`transactions.kwl` is append-only. Each frame is big-endian `u64` signed-byte length, 32-byte transaction ID, 32-byte SHA-256 of the signed bytes, then those bytes. The current expected offset is in `state.kws`; ordinary startup never scans this log.
Canonical signed bytes are, in order: the eight-byte `KWTX\0\x02\0\0` magic; 32-byte writer key; big-endian commit seconds (`i64`) and nanoseconds (`u32`); a big-endian `u64` count followed by 32-byte head IDs; provenance author and source as `u32`-length UTF-8, provenance source seconds/nanoseconds, and `u32`-length provenance data; a `u64` count of ordered 32-byte merge-ID pairs; a `u64` count of object declarations containing six-byte ID, `u64` length, and 32-byte SHA-256; then `u64`-counted create and update operations. Each node operation is a six-byte ID plus complete `NodeData`: three `u32`-length UTF-8 strings, a one-byte owner tag (`0` unowned, `1` self, `2` followed by a six-byte owner ID), and three `u64`-counted six-byte-ID vectors for fixed connections, recent connections, and objects. A 64-byte Ed25519 signature over all preceding bytes ends the record. The transaction ID is SHA-256 over `b"kcode-kweb-db transaction v2\0"` followed by the complete signed bytes.
Each commit stages files and one binary log frame in a transaction-specific WAL directory. Its canonical checksummed intent records the staged log-frame identity, length, and hash; expected generation and log offset; and every staged-file hash and create/replace mode. Publishing the prepared directory is the durable commit decision. Roll-forward accepts only the exact frame already at its expected offset or appends at the expected end, verifies and applies staged operations idempotently, applies `state.kws` last, syncs affected files/directories, and durably removes the WAL entry. A crash before preparation leaves only abandoned staging; a crash after preparation rolls the complete transaction forward before reads are served. If roll-forward cannot complete while a handle is open, that handle is poisoned and serves no further reads or writes; closing and reopening runs recovery again.
Normal open reads the small format/state records, removes abandoned WAL staging, rolls forward only prepared WAL entries, clears abandoned uncommitted incoming-object staging, and resumes one unacknowledged outbox item. It never scans or rebuilds the transaction log, node store, object store, or complete transaction index. `get_node`, `get_node_history`, and `get_object` validate only the addressed authoritative record. Missing or corrupt authoritative state is an error for separately versioned offline recovery tooling.
A submitted transaction is not processed into committed state until every transaction named by its `heads` has already been committed. Strict `accept_transaction` returns `InvalidTransaction` immediately if any head is missing and writes no staging state. For `accept_gossip_transaction`, the canonical signed transaction instead remains in an in-memory dependency graph, its validated final object envelopes are written once beneath `incoming/<transaction>/` to release the payload buffers, and the missing IDs are requested from the supplied `TransactionSource`. A newly supplied gossip dependency is subject to the same rule recursively. Committing a dependency releases its direct waiters; newly ready transactions are then committed one at a time in topological order through the ordinary WAL path. Waiting transactions never enter the transaction log, transaction index, node/history state, authoritative object store, heads, or gossip outbox.
Waiting state intentionally does not survive process shutdown because the transaction itself is memory-resident. Graceful close and the next open remove its object spool; the sender must resubmit after a restart. The process retains only waiting canonical transactions and small dependency/object-spool metadata, not their object buffers.
Gossip consists only of canonical signed transaction bytes plus objects created by that transaction. Projected nodes, checkpoints, root mappings, and the complete store are never announced. Delivery atomically renames one queue marker to `gossip-outbox/INFLIGHT` before loading and invoking the callback. Successful callback handoff removes the claim; failure or an interrupted callback requeues it for at-least-once delivery. The callback runs without a database mutex held and may reenter the database. Startup recovers only an existing claim and does not reconstruct announcements from history.
The canonical transaction magic/version is `KWTX\0\x02\0\0`. V2 operations contain complete `NodeData` and no additive connection section. V1 signed bytes are rejected, not reinterpreted.
Root roles, user/group identity, authorization beyond configured writers, connection-position policy, Ktools/routes/frontend identifiers, provenance filenames and media types, chunking, migration, backup, whole-database validation, enumeration/statistics/transaction-inspection APIs, application queues, and cross-database journals remain outside this generic storage crate.