# Kweb DB Core Library Specification
## 1. Scope
`kweb-db-core` is a Rust library that owns the durable knowledge-web data model,
its SQLite representation, and its sibling provenance-artifact store. It is
deliberately independent of HTTP, frontend assets, system prompts, users,
groups, root roles, context-window policy, and model tool orchestration. Its
Rust crate path is `kweb_db_core`.
The crate is a standalone package: its manifest contains explicit package and
dependency metadata so the directory can be moved into its own repository
without inheriting a parent workspace.
## 2. Deliberate limitations
The library is intended to be serialized by its caller.
- It provides no internal concurrency policy.
- Every explicit mutation is idempotent when the caller reuses its required
`IdempotencyId`. The library does not invent a logical-operation boundary
for the caller or recover an identifier that the caller discarded.
- One SQLite create or update is internally atomic, but multiple library calls
cannot share a transaction and may leave a partially completed caller-level
operation when a later call fails.
- Artifact bytes are durably written before their SQLite metadata transaction.
A failure between those steps may leave an unreferenced immutable file, but
cannot commit a database reference to missing bytes. An exact retry uses the
same filename and verifies its bytes before completing.
- The library provides no optimistic revision or lost-update protection.
- It gives no meaning to connection order and provides no connection limits,
promotion, demotion, consolidation, or activation operations.
An HTTP server normally wraps one `Kmap` in a mutex and owns all transport and
caller-level policy.
## 3. Identifiers
`NodeId` contains exactly 20 bytes. `NodeId::random` uses the operating
system's secure random source. Its display, Serde, and HTTP representation is
exactly 40 lowercase hexadecimal characters.
`ProvenanceId` is an alias for `NodeId` and has the same representation.
`IdempotencyId` contains exactly 16 bytes. Callers generate one cryptographically
random identifier for each logical mutation and retain it for every retry.
`IdempotencyId::random` uses the operating system's secure random source. Its
display and Serde representation is exactly 32 lowercase hexadecimal
characters.
Successful mutations store a permanent receipt in the same SQLite transaction
as the requested write. The receipt covers the operation kind and normalized
semantic request. Replaying the same identifier with the same operation and
request performs no write and succeeds; reusing it for any different operation
or request is a conflict. Validation or transaction failures do not reserve the
identifier. Receipts must remain present for as long as retries may arrive.
## 4. Data model
### 4.1 Knowledge nodes
A returned `Node` contains:
| `id` | Durable node identifier |
| `short_name` | Trimmed text containing 4–50 Unicode characters |
| `short_description` | Trimmed text containing at most 200 Unicode characters |
| `long_description` | Caller text containing at most 1,000 whitespace-delimited words |
| `last_modified_by` | Trimmed opaque caller attribution containing 1–200 characters |
| `last_modified_at` | Library-generated RFC 3339 UTC timestamp |
| `owner_node_id` | Another node, the node itself, or null for an unowned node |
| `fixed_connections` | Stored array of node identifiers |
| `recent_connections` | Stored array of node identifiers |
The library preserves the order supplied for both connection arrays but does
not interpret that order. It rejects duplicates within one array and requires
every referenced node to exist. The same target may appear in both arrays.
`Owner::SelfNode` resolves to the node being created or updated.
`Owner::Node(id)` selects another existing node. `Owner::Unowned` exists for
caller operations that deliberately retain that state. The library does not
know whether an owner is a user, Kennedy, a group, or a root.
### 4.2 Provenance
Provenance is immutable and contains:
- `id`
- arbitrary `data`
- a trimmed 1–200 character `source`
- an RFC 3339 `source_created_at` supplied by the caller
- an ordered array of artifact metadata: relative path, preserved original
filename, media type, semantic role, byte length, and SHA-256
`create_provenance` creates it once. Any number of node creates and updates may
then reuse its provenance identifier without duplicating the provenance data;
each such mutation still requires its own idempotency identifier.
### 4.3 History
Every node create and update appends one internal history row referring to the
supplied provenance identifier. History rows form a newest-to-oldest linked
list. `get_node_history` follows the entire list, detects cycles, and returns
only the ordered provenance identifiers, newest first. Internal history-row
identifiers are not part of the public API.
## 5. Public API
### `Kmap::open(path)`
Opens or creates the SQLite file, enables foreign keys and WAL journaling,
sets a five-second busy timeout, and initializes the current schema for a new
database before returning. Its artifact directory is the sibling
`<database-stem>-provenance-artifacts`; therefore `kweb.sqlite3` uses
`kweb-provenance-artifacts`.
### `Kmap::open_with_artifacts(database_path, artifact_path)`
Opens the same storage while letting an embedding application explicitly
provide the artifact directory.
### `create_provenance(IdempotencyId, NewProvenance) -> ProvenanceId`
Validates and inserts immutable provenance. An exact replay returns the
original provenance identifier. UTF-8 provenance data larger than 256 KiB is
stored as an artifact and loaded transparently by `get_provenance`.
### `create_provenance_with_storage(IdempotencyId, NewProvenance, ProvenanceStorage) -> ProvenanceId`
Creates the same immutable provenance while accepting attached byte artifacts
and a required filename for the main provenance data if it crosses the
external-storage threshold. This is a separate method, so ordinary
`create_provenance` has no optional arguments. Attached artifacts participate
in the idempotency request hash.
### `create_node(IdempotencyId, CreateNode) -> Node`
The input supplies an explicit node ID, an existing provenance ID, owner,
descriptive fields, attribution, and both complete connection arrays. In one
transaction the library validates references, inserts the node and first
history row, stores its connection arrays, and advances its history head.
An exact replay appends no history and returns the node's current state.
### `update_node(IdempotencyId, NodeId, UpdateNode) -> Node`
The input supplies the complete replacement state and an existing provenance
ID. In one transaction the library appends history, replaces the node fields
and both connection arrays, and advances the history head.
An exact replay appends no history and returns the node's current state.
### `get_node(NodeId) -> Node`
Returns the complete node.
### `get_provenance(ProvenanceId) -> Provenance`
Returns immutable provenance. Main provenance data is read transparently from
its artifact when externalized. Attached bytes are not copied into the response;
ordered metadata contains their relative filenames.
### `get_node_history(NodeId) -> Vec<ProvenanceId>`
Returns every provenance identifier from newest to oldest.
### `stats() -> Stats`
Returns current-node text statistics: node count; full-text and
long-description character and word counts; and approximate token counts at
one token per four Unicode characters. History, provenance, and connections
are excluded.
`Stats` has private fields, stable typed getters, and a Serde `Serialize`
implementation. New fields and getters may be added compatibly. JSON clients
must ignore unknown fields; existing field names and meanings remain stable.
## 6. SQLite schema
The library owns these tables:
- `knowledge_nodes`
- `data_provenance_nodes`
- `provenance_artifacts`
- `provenance_artifact_links`
- `data_history_nodes`
- `fixed_connections`
- `recent_connections`
- `idempotency_receipts`
Connections are normalized internally to preserve array order and foreign-key
integrity; the tables do not imply graph policy. Foreign-key deletion is
restrictive and the library exposes no deletion methods.
Artifact filenames preserve the caller's safe basename and insert exactly 12
URL-safe unpadded Base64 characters before its final extension, for example
`telegram-vnote.aaaa1111jjjj.wav`. The random material comes from the
mutation's 16-byte idempotency identifier, making retries stable. The first two
suffix characters form a shard directory. Files and directories use private
permissions on Unix. SQLite stores each relative path, original basename,
media type, size, SHA-256, creation time, provenance link, role, and position.
## 7. Verification
The crate tests creation, updates, provenance history, self ownership,
policy-free connection storage, and additive stats serialization. They also
prove exact mutation replays make no additional history/provenance rows and
reject identifier reuse with changed content. Artifact tests cover transparent
large-data reads, retry-stable storage, preserved filenames, 12-character
suffixes, and two-character sharding.