Module audit_chain

Expand description

Forward Ed25519-signed hash chain for the audit log.

Each transactions row stores chain_hash and prev_chain_hash columns. On insert, the daemon computes

chain_hash = ed25519_sign(ROW_DOMAIN || canonical(immutable_fields) || prev_chain_hash, signing_key)

The ROW_DOMAIN prefix is part of the signed message, not decoration: it is what stops a row signature from ever verifying as a checkpoint or approval-receipt signature. An independent verifier that omits it will never reproduce a valid chain_hash.

and stores both (the signature is hex-encoded; the first row in a chain has prev_chain_hash = ""). Ed25519 signatures are deterministic (RFC 8032), so the same content always produces the same chain_hash.

Verification (sysknife audit verify) walks rows in seq order and checks each row’s signature with the public key, reporting the first broken link. A row this build cannot check (a chain_version it does not know, or a key_id it does not hold whose signature the loaded key does not reproduce) is recorded and walked past, with its prev_chain_hash link still checked. The walk returns Broken if any later row is provably broken, and CannotVerify only when nothing is. Through 0.26.1 the walk stopped at such a row, so editing one chain_version or key_id column hid every forged or deleted row after it behind exit 2.

Because verification needs only the public key, an auditor or a central log aggregator can verify the chain without holding the private key — they cannot forge entries. This is the property a symmetric MAC (the previous HMAC-SHA256 design) could not provide: with an HMAC, the verifier and the forger are the same principal.

§Threat model

Non-repudiation (asymmetric). The daemon signs with the private key; anyone with the exported public key (<key>.pub) can verify but not forge. A compromise of the verifier does not enable forgery.
Compromised host / root. An attacker who reads the private key file can forge future entries. Mitigation is to anchor signed checkpoints to an append-only external sink (see checkpoint_sink) so that later tampering with past entries becomes detectable and the tamper window is bounded to “after compromise”.
Tail truncation is undetectable by the chain walk alone: an attacker who deletes the last K rows leaves a still-consistent chain. It is caught by anchoring signed checkpoints to an independent append-only sink, since a truncated chain can no longer reproduce a previously anchored (seq, chain_tip) (see verify_checkpoints and checkpoint_sink). The best-effort audit_watermark journald forward is a lighter complement.
In-flight modification between insert and read is mitigated by computing the signature before INSERT and writing it in the same SQL statement.
Status mutations are not in the chain. The mutable status field is intentionally excluded — the chain protects the authorisation decision (immutable fields captured at insert time), not the live execution state. Status transitions are chained separately, in the append-only audit_events table (created by migration v2 in transactions.rs), and verify_event_chain / verify_event_binding below verify that table.

Until 0.19.0 this paragraph said the same thing while TransactionStore::update_status called append_event zero times, so audit_events held the approval lifecycle and nothing else. No event recorded Running -> Succeeded, Failed, RolledBack or NeedsReboot. An action that ran to completion could be rewritten in the status column as Canceled, sysknife history reported it canceled, and sysknife audit verify still reported the chain Intact, because nothing in the chain had an opinion about the outcome. An investigator reading this paragraph would have believed the outcome was protected.

update_status now appends a status_* event in the same sqlite transaction as the column write, so the two cannot diverge through a crash, and TransactionStore::status_matches_chain compares each row’s status against the newest status event chained for it. Rewriting the column no longer agrees with a signed record.

That fix reached SQLite only. Through 0.26.1 the Postgres backend, the one meant for centralised deployments, appended no status event on any path and its verify functions left the status verdict unset, so the original gap stayed open there. Postgres now appends the same events in the same sqlx transactions and runs the same comparison (status_outcome). Its rows written before migration 5 carry no status event, so a terminal status on one of them is reported as unproven rather than as a rewrite. The marker that says so (predates_status_events) is an unsigned column: flipping it can turn a detected rewrite on an older row into “cannot verify” (exit 2), never into a pass.

Exploiting the original gap needed write access to a database that is 0600 inside a 0700 directory owned by the daemon account, so on SQLite the attacker was already root or the sysknife user. On Postgres it needs UPDATE on transactions, held by the daemon’s role and by any other role the deployment grants it. Either way the cost was to the investigator afterwards, not to the boundary.

§Key management

The Ed25519 private key (a 32-byte seed) lives in a file. By default the path is <db_dir>/audit-key (sibling of the SQLite database, or of whatever directory sysknife_core::default_database_path resolves to in production), and the env var SYSKNIFE_AUDIT_KEY_PATH overrides it for systemd unit drop-ins (typically /etc/sysknife/audit-key). The file is created with mode 0o600 on first daemon start if it does not exist; subsequent runs refuse to start if it is world-readable. The public key is written alongside as <key>.pub (hex) for auditors and aggregators.

Future epochs (key rotation): each row already carries a key_id column. A planned rotation flow appends a checkpoint row signed with the outgoing key whose payload references both public-key fingerprints; verification walks the chain through epoch boundaries by looking up each row’s key_id in a directory of retired public keys. For now, all rows use key_id = "v1" and rotation is manual (delete the chain, regenerate). With one epoch, a row whose key_id differs is still tried against the loaded key before it is conceded as unverifiable: key_id is inside the signed message, so a signature the loaded key reproduces is authentic.

Structs§

AttributionCensus: Attribution census over every transaction row read.
AuditKey: Loaded Ed25519 signing key + its identifier. Construct via AuditKey::load_or_generate.
AuditVerification: Everything sysknife audit verify checks, in one value.
ChainContent
ChainRow: One row’s worth of chain data, as fetched from the store.
Checkpoint: A signed commitment to the chain tip at a point in time. Periodically emitted and anchored to an independent, append-only sink (a separate database, a WORM store, or an RFC 3161 timestamp) so that a later attempt to rewrite or truncate the local chain is detectable: the anchored (seq, chain_tip) can no longer be reproduced from the shortened chain.
EventContent: Immutable content of one approval event, signed into the event chain.
EventRow: One approval event as fetched from the store.
RequestHashKey: HMAC-SHA256 key for the request_hash column (#268).

Enums§

AuditEventKind: A lifecycle event on an approval receipt.
AuditKeyError
BindingOutcome: Result of checking that the transaction chain’s committed event_tip values are still reproducible from the event chain.
ChainExportError: Invalid input or stored data encountered while selecting rows for export.
ChainIdentity: Immutable fields hashed into the chain. Status is intentionally absent — see module docs.
CheckpointOutcome: Result of checking anchored checkpoints against the current chain.
EventIdentity: Encoding generation of an approval-event row, mirroring what ChainIdentity does for transaction rows: the audit_events.chain_version column selects, per row, which message its signature was made over, so rows written by an older binary keep verifying (#249).
StatusOutcome: Whether each transaction’s mutable status column still agrees with the newest status_* event chained for it.
VerifyOutcome: Result of verify_chain.

Constants§

CHAIN_VERSION_CURRENT: The encoding this binary writes. Always an alias for the newest versioned constant, never used to encode or dispatch a specific generation.
CHAIN_VERSION_LEGACY: Row encoding written before the caller-identity migration: no caller_role, no event_tip. Still verifiable — see ChainIdentity.
CHAIN_VERSION_V2: Row encoding that added caller_role and event_tip.
CHAIN_VERSION_V3: Row encoding that added caller_principal on top of v2, so a row names the account that asked, not only its role class.
CHAIN_VERSION_V4: Row encoding that changed what request_hash is, without changing which fields are signed.
CURRENT_KEY_ID: Stable identifier for the current key generation. Stored in every row. Tied to the schema, not the key bytes — rotation will introduce "v2" etc.
EVENT_VERSION_CURRENT: The event encoding this binary writes for approval events. Always an alias for the newest versioned constant, never used to dispatch a specific generation.
EVENT_VERSION_LEGACY: Event-row encoding written before the approver-identity migration: the six-field message with no account among them. Still verifiable — see EventIdentity.
EVENT_VERSION_V2: Event-row encoding that appends caller_principal, so a verified approval event names the account that granted, consumed or revoked the receipt instead of leaving the investigator to infer it from the daemon’s authorization rules (#249).
HASH_HEX_LEN: Hex-encoded length of an Ed25519 signature (64 raw bytes → 128 hex chars).

Functions§

approval_receipt_digest
binding_outcome_to_exit_code: Exit code for the binding check, on the same scale as outcome_to_exit_code: a missing event is a detected tamper (1), while a check that could not run is inconclusive (2).
checkpoint_outcome_to_exit_code: Process exit code for sysknife audit checkpoint verification, mirroring outcome_to_exit_code: 0 consistent, 1 a detected tamper (bad signature / truncation / rewrite), 2 cannot-verify (e.g. malformed public key). The 1-vs-2 split matters for CI exactly as it does for chain verify.
outcome_to_exit_code: Process exit code for sysknife audit verify.
resolve_audit_key_path: Resolve the audit key file path: $SYSKNIFE_AUDIT_KEY_PATH if set, otherwise <db_dir>/audit-key (sibling of the database). This is the single definition of the key-location precedence documented in the module header; the CLI verify/checkpoint paths resolve through it so they always agree with the daemon on where the key lives.
select_chain_rows_for_export: Select stored chain rows for export while preserving their seq order.
status_outcome_to_exit_code: Exit code for the status cross-check, on the same scale as outcome_to_exit_code. A disagreement is a detected tamper, not an inconclusive read: both sides were present and they contradict each other.
validate_export_since: Validate the timestamp accepted by sysknife audit export --since.
verify_all: Run all three checks with the daemon’s key.
verify_all_with_pubkey: Run all three checks with only the exported public key (the auditor path).
verify_chain: Verify a chain using the daemon’s key. Verification uses the public key (so it proves, but cannot forge) and also asserts every row was written under this key generation (key_id).
verify_chain_with_pubkey: Verify a chain with only the hex-encoded Ed25519 public key. This is the auditor / aggregator path: it proves the chain without the private key and cannot be used to forge entries. key_id is not checked — the public key itself identifies the signer.
verify_checkpoints: Verify anchored checkpoints against rows (the current chain) with the hex public key. Detects truncation (a checkpoint seq no longer in the chain) and rewrite (tip mismatch at a checkpoint seq). rows must be in seq order. This is the anti-root guarantee: a host attacker who shortens or edits the local chain cannot reproduce a previously anchored signed tip.
verify_event_binding: Check the cross-chain binding: each chain_version = 2 transaction row signs the event-chain tip as of its insert, so a later deletion of trailing approval events leaves a tip that can no longer be found.
verify_event_chain: Verify the approval-event chain with the daemon’s key.
verify_event_chain_with_pubkey: Verify the approval-event chain with only the exported public key.