Skip to main content

Module events_split

Module events_split 

Source
Expand description

Events-daemon split (ADR-170): the audit lane leaves the domain store.

The ADR-133 idempotent audit batch is the measured bulk of event write volume, and in a single-store deployment its rows queue on the same SQLite writer lane as domain mutations. This module moves that lane into a dedicated events daemon that owns the events database, reachable over a Unix socket with the same length-prefixed framing and peer-uid admission the main daemon socket uses. Plain event appends stay on the domain store — the legacy events table has raw-SQL consumers (schedule provenance, kg projection guards, graph-query substrate unions) whose correctness depends on finding those rows there.

Cooperating pieces:

  • run_events_daemon — the server loop the events-daemon subcommand runs: binds the events socket, owns the only resident writer of the events database, and serves append/read requests through the ordinary SqlEventStore.
  • EventsSplitClient — one per domain process. Plain appends ride a bounded in-memory queue drained by a background forwarder (fire-and-forget; overflow or a dead daemon drops the batch, counts it, and logs — the loss-tolerant durability class made concrete; unused by the default routing until telemetry producers opt in). Idempotent audit-batch appends and reads are synchronous framed round-trips with bounded timeouts, because their callers are background flushers or query paths, never the dispatch hot path.
  • ForwardingEventStore — the lane-side EventStore over the socket. Preflight validation delegates to an in-memory SqlEventStore, so the ADR-133 audit-batch seam keeps its pre-enqueue shape check without any I/O on the dispatch path.
  • SplitEventStore — the per-namespace handle crate::runtime::KhiveRuntime::events returns when the split is configured: routes the idempotent lane to the events store, plain appends to the legacy store, and merges reads across both.

Domain availability never depends on events-daemon liveness: every failure path here degrades (drop + count + log, or a typed storage error for the synchronous lanes) instead of blocking the caller.

Structs§

EventsDaemonGuard
Advisory lock guaranteeing at most one events daemon per socket path. Held for the daemon’s lifetime; a second daemon exits instead of stealing the socket path from the live one.
EventsForwardingMetrics
Counters describing the fire-and-forget lane’s degradation. Zero drops is the healthy state; any non-zero dropped_batches means the loss-tolerant contract was exercised and says so.
EventsSplitClient
One per domain process: the connection to the events daemon plus the bounded fire-and-forget append queue.
EventsSplitConfig
How a runtime reaches event storage when the split is configured.
ForwardingEventStore
EventStore implementation the runtime hands out when the events split runs in daemon mode. Appends are fire-and-forget through the client’s bounded queue; the ADR-133 idempotent lane and all reads are synchronous round-trips to the events daemon.
SplitEventStore
The store the runtime hands out when the events split is configured: routes by APPEND CLASS rather than moving the whole event plane.

Enums§

EventsRequest
Request frame sent from a domain process to the events daemon.
EventsResponse
Response frame from the events daemon.
WireWriterTaskFailure
Whether a writer-state error belongs to one failed request on a still-live seam or to a permanently retired writer task.
WireWriterTaskState
Wire mirror of khive_storage::WriterTaskRequestState. A separate type because the storage enum is not serializable and the wire shape must stay under this module’s protocol-version control, not the storage crate’s.

Constants§

DEFAULT_APPEND_QUEUE_BATCHES
Default bound on the fire-and-forget append queue, in batches. The byte bound below also applies, so a large batch cannot multiply this depth into unbounded retained event memory.
DEFAULT_APPEND_QUEUE_BYTES
Maximum serialized append-request bytes retained by the fire-and-forget queue and its one in-flight delivery. A single request must also fit the daemon’s per-frame cap; the queue budget covers several such requests.
EVENTS_PROTOCOL_VERSION
Bump whenever the request or response frame shape changes incompatibly. The server rejects frames whose version it does not speak, so a skewed client gets a typed refusal instead of a deserialization panic.
MAX_QUERY_EVENTS_PAGE_ROWS
Cap on QueryEvents page size, in rows. The wire PageRequest.limit is client-supplied u32, and the daemon materializes the full page as a Vec<Event> and serializes it into one response frame — so an unbounded limit is attacker-controlled memory and serialization work in a process that lives for months. Over-cap requests get a typed refusal naming the cap, never a silently clamped page: a caller that asked for more rows than it got would otherwise read the short page as the end of the data. The split client’s merged read requests a prefix of offset + limit rows, so this cap also bounds the deep-offset window a socket client can demand in one request.

Functions§

client_for
The process-wide client for socket_path, created (and its forwarder spawned) on first use. Requires a tokio runtime context on first call.
direct_backend_for
The process-wide direct (embedded-mode) backend for db_path, opened read-write on first use.
direct_backend_read_only_for
The process-wide direct backend for db_path, opened READ-ONLY on first use. The file must already exist — this constructor never creates or schema-initializes an events database, which is what a read-only runtime’s no-DB-creation contract requires of its event lane.
events_db_path_beside
Default events database file, beside the main database file.
events_socket_path_beside
Events daemon socket path, beside the events database it serves.
forwarding_metrics
Forwarding metrics for the process-wide client at socket_path, if one exists. None means the split never initialized in this process.
run_events_daemon
Serve the events daemon loop on socket_path, owning db_path.
run_events_daemon_with_policies
Serve using the exact policies inherited from the opened main backend.
run_events_daemon_with_wal_ceiling
Serve the events daemon with a WAL ceiling already resolved by its host; the disk policy and lock directory come from this process’s environment.
supervise_events_daemon
Supervise the events daemon from the main daemon process: probe the socket periodically and (re)spawn the daemon subcommand when unreachable.
supervise_events_daemon_with_policies
Supervise using the main pool’s captured writer policies and lock directory.
supervise_events_daemon_with_wal_ceiling
Supervise every events-daemon child with the main backend’s resolved WAL ceiling; the disk policy and lock directory come from this process’s environment.
try_acquire_events_daemon_guard
Try to become the events daemon for socket_path, returning None for contention or any refusal. Callers must validate the parent with ensure_socket_dir_is_trusted before lock-path operations in that directory.
validate_events_socket_path
Refuse an events socket pathname that cannot fit the platform address field. The daemon anchors relative paths before binding, so preflight counts that same absolute spelling, including the terminating NUL.