agentplane 0.45.0

//! Envelope encryption, and the erasure it makes provable.
//!
//! # Why deleting is not erasing
//!
//! [`BlobStore::expire`](crate::blob::BlobStore::expire) drops a payload's bytes
//! and leaves a tombstone, and the hash chain still verifies because it only
//! ever committed to a digest. That is the right shape, and it is not enough:
//! it erases the bytes *in the live store*. Every backup taken before the
//! request, every replica, every snapshot an operator forgot about still holds
//! them, and an erasure obligation is not discharged by deleting one copy of
//! something that exists in six places.
//!
//! Chasing the copies does not work either. Backups are the point of backups —
//! they are offline, offsite, and frequently immutable by design, because that
//! is what makes them survive the incident they exist for. A retention story
//! that requires rewriting immutable backups has two guarantees in direct
//! conflict, and whichever one loses, loses silently.
//!
//! # What this does instead
//!
//! Payload bytes are sealed under a **data key**, and the data key is wrapped by
//! a key this crate never holds — a KMS, an HSM, whatever the deployment trusts.
//! Erasure destroys the data key. Every copy of the ciphertext becomes
//! unreadable at the same instant, including the ones nobody can reach, because
//! the thing that was destroyed was never in them.
//!
//! This is the standard answer to the immutable-log-versus-erasure problem, and
//! it is the only one that survives contact with a backup regime.
//!
//! Two operations fall out of the same structure, which is the argument for it:
//!
//! * **Erasure** — destroy a scope's wrapping key. Every payload ever sealed
//!   under that scope is unreadable, everywhere, including the copies nobody
//!   can reach.
//! * **Revocation** — the same act for a different reason. The blast radius a
//!   compromised key should have is exactly the scope it wrapped.
//!
//! # Sealed bytes are rotation-immutable
//!
//! A third operation is conspicuously absent, and its absence is a decision
//! rather than unfinished work. Envelope encryption's usual selling point is
//! that a wrapping key rotates by *re-wrapping* data keys, leaving bulk data
//! alone. That operation cannot exist here, and offering it would be a control
//! that quietly does nothing:
//!
//! An envelope carries its wrapped data key **inline**, and the journal's hash
//! chain commits to the envelope bytes — which is what lets an auditor holding
//! no keys verify a run whose payloads have been erased. Re-wrapping a journal
//! payload therefore rewrites a record the chain covers, so it breaks the chain
//! it sits inside. Nor could re-wrapping the *other* stores buy the operational
//! thing rotation is wanted for: an erasure scope's journal payloads and its
//! case state share one wrapping key, so a scope stays pinned to the oldest
//! version any of its journal envelopes names, and no amount of re-wrapping
//! case rows moves that floor.
//!
//! So the rule is the other half of the choice, stated rather than discovered:
//! **sealed payload bytes never change, and the erasure scope is the rotation
//! unit.** A scope is already narrow — one case, one run, one memory version —
//! so a compromised wrapping key exposes that unit and nothing else, which is
//! the blast radius rotation is bought for. Adding a key version is safe and
//! needs nothing from this crate: envelopes sealed before a rotation keep
//! opening, which is what AWS KMS does by construction — it retains every prior
//! version of a key's material in perpetuity, resolves the right one from the
//! ciphertext, and lets you delete only the whole key, which is erasure.
//!
//! *Retiring* a version is the exposure, and only some services offer it.
//! Vault's transit engine does: `min_decryption_version` refuses ciphertext
//! below a floor, so raising that floor past a live envelope makes un-erased
//! history unreadable — an erasure nobody requested and no retention record
//! explains. That is why [`KeyError::Retired`] exists as its own answer; see
//! its documentation for why it must not be reported as loss.
//!
//! # What is deliberately absent
//!
//! No key material is generated by, or stored in, this crate beyond the process
//! that is using it. [`KeyRing`] is a seam: a deployment points it at the thing
//! that already holds its keys. There is a `MemoryKeyRing` for tests, and it
//! lives behind the `testkit` feature rather than here: a key ring in the same
//! process as the data it protects protects it from nobody, so the gate is the
//! guarantee instead of a warning in a doc comment.

use std::fmt::Debug;

use async_trait::async_trait;
use serde::{Deserialize, Serialize};
use zeroize::{Zeroize, ZeroizeOnDrop};

use crate::core::{KeyId, Timestamp};

/// A key that seals payload bytes.
///
/// Zeroized on drop. Not `Clone`, not `Debug`-printable, and it does not
/// serialize: a data key that reaches a log line or a journal record is a data
/// key whose destruction no longer erases anything.
#[derive(Zeroize, ZeroizeOnDrop)]
pub struct DataKey([u8; 32]);

impl DataKey {
    /// Take ownership of raw key material.
    #[must_use]
    pub fn new(bytes: [u8; 32]) -> Self {
        Self(bytes)
    }

    /// The raw bytes, for a cipher that needs them.
    #[must_use]
    pub fn expose(&self) -> &[u8; 32] {
        &self.0
    }
}

// Written by hand so the key cannot reach a log through a derived `Debug`.
impl Debug for DataKey {
    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
        f.write_str("DataKey(<redacted>)")
    }
}

/// A data key as it may safely be stored: sealed by a key this crate never has.
///
/// Safe to journal, back up, and replicate — which is the point, because it
/// travels *with* the payload it sealed. A backup therefore contains everything
/// needed to restore and nothing needed to read: the wrapping key stayed in the
/// service, and destroying it is what makes the backup unreadable.
///
/// **Unknown members are refused**, the rule every durable format here is held
/// to. This one travels with the payload it sealed, so a reader that skipped a
/// member it did not know would unwrap under parameters somebody else wrote
/// down and this build never saw — and the wrapping key is the one thing an
/// erasure destroys, so guessing at its header is guessing about whether data
/// is still readable.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct WrappedKey {
    /// What this key protects: a case, a tenant, whatever the deployment erases
    /// as a unit. Erasure destroys a scope, so the scope *is* the erasure unit.
    pub scope: String,
    /// Which wrapping key, at which version, sealed it.
    ///
    /// Written once and never rewritten: sealed bytes are rotation-immutable,
    /// so this is the version the key service must still admit for as long as
    /// this envelope is retained. It is what [`KeyError::Retired`] names when
    /// that stops being true.
    pub wrapped_by: KeyId,
    /// The sealed data key.
    pub sealed: Vec<u8>,
}

/// Why a key operation did not happen.
#[derive(Debug, Clone, PartialEq, Eq, thiserror::Error)]
pub enum KeyError {
    /// The key is gone, on purpose. The bytes it sealed are unrecoverable.
    ///
    /// Distinct from every other failure, and deliberately so: this is a
    /// completed erasure reporting itself, not an outage. A caller that retries
    /// this forever is waiting for something that will never come back, and a
    /// caller that reports it as corruption sends somebody to look for a fault
    /// that does not exist.
    #[error(
        "the data key for scope '{scope}' was destroyed at {at} ({reason}) — the bytes it \
         sealed are unrecoverable by design, in every copy including backups"
    )]
    Destroyed {
        scope: String,
        at: Timestamp,
        reason: String,
    },

    /// The wrapping key still exists, but this envelope names a version the
    /// key service has been told to stop decrypting.
    ///
    /// A third answer, because the two that already existed are both wrong for
    /// it and wrong in opposite directions. It is not [`Destroyed`]: nobody
    /// requested an erasure, no retention record explains it, and the bytes
    /// come back the moment the floor is lowered. It is not [`Refused`] either
    /// — reported that way it reaches an operator as *this data neither opens
    /// nor was erased*, which is the signature of loss or tampering, and sends
    /// somebody to hunt a fault that does not exist while the real remedy is a
    /// one-line configuration change.
    ///
    /// It is reachable only by operator action, and only in one direction:
    /// sealed bytes are rotation-immutable (see the module documentation), so
    /// an envelope names its wrapping-key version for as long as it exists, and
    /// raising a version floor past it is an erasure that no erasure record
    /// accounts for.
    ///
    /// [`Destroyed`]: Self::Destroyed
    /// [`Refused`]: Self::Refused
    #[error(
        "the wrapping key version '{key_id}' for scope '{scope}' has been retired by policy — \
         this is not an erasure and not a loss: the sealed bytes are intact and become readable \
         again if the key service's minimum decryption version is lowered to admit '{key_id}'"
    )]
    Retired { scope: String, key_id: KeyId },

    /// The bytes are a sealed envelope written to a construction this build
    /// does not read.
    ///
    /// A fourth answer for the same reason [`Retired`] is a third: the two
    /// obvious classifications are both wrong. It is not [`Destroyed`] —
    /// nothing was erased and the wrapping key is untouched. It is not
    /// [`Refused`] either, and this is the direction that matters: without a
    /// version in the header a reader walks its own layout over somebody
    /// else's and reaches the AEAD, which reports *this payload did not
    /// authenticate* — the signature of tampering, for a build skew whose
    /// remedy is deploying a build that reads version `version`.
    ///
    /// Sealed bytes are rotation-immutable (see the module documentation), so
    /// an envelope outlives the build that wrote it by design. A mixed-version
    /// fleet, a rollback, or a restore from a backup taken by a newer plane
    /// all produce this, and all of them are ordinary operations rather than
    /// incidents.
    ///
    /// [`Destroyed`]: Self::Destroyed
    /// [`Refused`]: Self::Refused
    /// [`Retired`]: Self::Retired
    #[error(
        "this sealed envelope is format version {version} and this build reads {supported} — \
         the bytes are intact and not erased; they open under a build that reads version \
         {version}"
    )]
    UnknownFormat { version: u8, supported: u8 },

    /// The envelope's version is one this build reads, and its header is not.
    ///
    /// **Two causes, and this build cannot separate them.** The header is
    /// parsed before anything authenticates — the tag that would establish the
    /// bytes is inside the payload, and reaching it needs the key this header
    /// names. So these bytes are either damaged in place or written by a build
    /// whose header shape differs, which before the format freeze is what a
    /// hard cut produces.
    ///
    /// Distinct from [`Refused`](Self::Refused) so that a report can say both
    /// rather than pick one. Naming it tampering sends somebody to hunt a fault
    /// that may not exist; naming it a skew tells them to change binaries when
    /// the data may be gone. The honest answer is the one that fits in neither
    /// arm, so it has its own.
    #[error(
        "this sealed envelope is a format version this build reads and its header is not one          it parses ({detail}) — either the bytes were damaged or another build wrote them,          and nothing here can tell which"
    )]
    UnreadableHeader { detail: String },

    /// The key ring could not be reached. May succeed later.
    #[error("the key ring is unavailable: {0}")]
    Unavailable(String),

    /// The key ring declined. Will not succeed on retry.
    #[error("the key ring refused: {0}")]
    Refused(String),
}

/// Where data keys are made, wrapped, and destroyed.
///
/// The wrapping key never leaves the implementation. That is the whole seam: a
/// deployment can put it in a KMS and this crate is still only ever holding a
/// data key it was handed, for as long as it takes to seal or open one payload.
#[async_trait]
pub trait KeyRing: Send + Sync + Debug {
    /// Mint a **fresh** data key, wrapped under the scope's key.
    ///
    /// Every call returns a new key. That is not a simplification — it is what
    /// a key-management service does: Vault's `transit/datakey` and AWS KMS's
    /// `GenerateDataKey` both mint one per call and wrap it under a *named*
    /// key. A ring that returned a stable per-scope key could not be
    /// implemented against either.
    ///
    /// It is also the better shape. The erasure unit is the **wrapping key**,
    /// not the data key: destroying a scope's wrapping key makes every data key
    /// ever wrapped under it unopenable at once, however many payloads there
    /// were and wherever their copies ended up.
    ///
    /// # Errors
    ///
    /// [`KeyError::Destroyed`] if the scope was already erased — a scope does
    /// not come back, because a scope that can be re-created is one where a late
    /// write silently lands in an erased unit.
    async fn data_key(&self, scope: &str) -> Result<(DataKey, WrappedKey), KeyError>;

    /// Open a wrapped data key.
    ///
    /// The wrapped form travels **with the payload it sealed**, because that is
    /// the only thing that makes a restore work: a backup holds ciphertext and
    /// its wrapped key, and neither is readable without the wrapping key the
    /// service still holds.
    ///
    /// Erasure is enforced by the service, not by this crate refusing to look:
    /// once the scope's wrapping key is destroyed, whoever kept a copy of the
    /// envelope holds bytes nobody can open — including the operator.
    ///
    /// # Errors
    ///
    /// [`KeyError::Destroyed`] once the scope has been erased, and
    /// [`KeyError::Retired`] when the scope is alive but the key service has
    /// been configured to stop decrypting the version this envelope names. An
    /// implementation that collapses the second into [`KeyError::Refused`]
    /// leaves an operator reading a configuration change as data loss.
    async fn open(&self, wrapped: &WrappedKey) -> Result<DataKey, KeyError>;

    /// Destroy a scope's data key. This is the erasure.
    ///
    /// Idempotent: the first destruction stands, so a retry cannot rewrite when
    /// or why the data went. The reason is kept because "erased" without one is
    /// not an answer anybody can give a regulator.
    ///
    /// # Errors
    ///
    /// If the key ring cannot be reached.
    async fn destroy(&self, scope: &str, at: Timestamp, reason: &str) -> Result<(), KeyError>;
}

/// The erasure scope for a unit within a tenant.
///
/// Derived here and nowhere else. The write path and the erasure path must
/// agree byte-for-byte about which key seals a payload; deriving the string
/// separately in each is how writes come to seal under `tenant/case` while
/// erasure destroys `case`, so every erasure reports success and destroys
/// nothing. Two places deriving one fact is the defect, so there is one place.
///
/// [`TenantId`](crate::core::TenantId) refuses `/`, which is what stops a tenant
/// named `acme/prod` from colliding with tenant `acme`, unit `prod`.
///
/// The implementation is [`crate::core::erasure_scope`], because the blob
/// layer consumes the same string as the unit that leads a storage address —
/// and the key a scope destroys and the addresses it expires must name one
/// unit, not two spellings of one.
#[must_use]
pub fn scope(tenant: &crate::core::TenantId, unit: &str) -> String {
    crate::core::erasure_scope(tenant, unit)
}

/// The erasure scope of one buffered event, named by its `(source, id)` pair.
///
/// `(source, id)` is the pair `CloudEvents` defines uniqueness by and the pair
/// the buffer deduplicates on, so the erasure unit is exactly the message an
/// erasure request names. The pair is joined by
/// [`origin_key`](crate::core::origin_key), which puts the length of `source`
/// in front: a `source` is a URI and routinely contains `/`, so a plain join
/// would let `("bus/x", "1")` and `("bus", "x/1")` share one key, and erasing
/// either would destroy the other's.
#[must_use]
pub fn event_scope(tenant: &crate::core::TenantId, source: &str, id: &str) -> String {
    scope(
        tenant,
        &format!("event/{}", crate::core::origin_key(source, id)),
    )
}

/// Refuse a sealing scope that is not the scope the wrapped store writes under.
///
/// Every wrapper here takes the store it seals and the tenant to seal for. Those
/// are two spellings of one fact, and when they differ nothing fails: both
/// scopes are real, the rows are written, the state is sealed, and the tenant's
/// erasure destroys a key that does not reach them. The failure surfaces only as
/// data that survived a deletion request, long after anyone could connect it to
/// the wiring.
///
/// So the pair is checked where both halves are in hand. The plane's own
/// `try_build` asks the same question of every store it is given; this covers
/// the wrapper an embedder builds and hands over already sealed, which `build`
/// sees only from the outside.
///
/// # Panics
///
/// If the two disagree. A startup wiring mistake, not a runtime condition.
pub(crate) fn assert_serves(store: &str, tenant: &crate::core::TenantId, kind: &str) {
    assert!(
        store == tenant.as_str(),
        "this {kind} store serves tenant '{store}' but is being sealed for \
         '{tenant}'. Both scopes are real, so nothing would fail — an erasure \
         for either tenant would destroy a key that does not reach these rows, \
         and report success"
    );
}

/// The AEAD's own nonce type, over bytes read back from a store.
///
/// `None` where the slice is not exactly [`chacha20poly1305::XNonce`]'s width.
/// This is the conversion, not the check — every caller has already refused a
/// wrong-length nonce with an error of its own vocabulary — but it is fallible
/// rather than panicking, because the bytes come from a store and a reader of
/// stored bytes that aborts the process is a denial of service.
pub(crate) fn xnonce(bytes: &[u8]) -> Option<&chacha20poly1305::XNonce> {
    <&chacha20poly1305::XNonce>::try_from(bytes).ok()
}

mod envelope;

/// The sealed-envelope construction this build writes, and the only one it
/// reads.
///
/// Public for the reason [`canon::VERSION`](crate::core::canon::VERSION) and
/// [`export::FORMAT_VERSION`](crate::export::FORMAT_VERSION) are: an operator
/// planning a restore, or diagnosing a
/// [`KeyError::UnknownFormat`], needs to name what this build speaks without
/// reading the source.
///
/// It stays `1` until the durable-format freeze. A number counting the
/// pre-release cuts would advertise that older envelopes are readable, and the
/// whole point of a hard cut is that they are not — here more sharply than
/// elsewhere, because sealed bytes cannot be rewritten into the new shape.
pub const ENVELOPE_FORMAT_VERSION: u8 = envelope::FORMAT_VERSION;

/// What a cryptographic erasure did: the key, then the live ciphertext.
///
/// The key's destruction is the erasure — every copy, backups included, stops
/// opening at that instant — so a failure *after* it is not a failed erasure
/// and must not be reported as one. It is not a clean one either: the live
/// store still holds ciphertext for the rows it names, which a retry of the
/// same call removes. Returned rather than logged so the caller answering an
/// erasure request can say which of the two it got.
#[derive(Debug, Clone, PartialEq, Eq)]
#[must_use]
pub struct Erasure {
    /// How many stored units the erasure reached: rows the cleanup removed, or
    /// — when the cleanup failed — the units enumerated before the key went.
    pub reached: usize,
    /// Why removing the live ciphertext failed after the key was destroyed,
    /// or `None` when it was removed.
    pub cleanup_failed: Option<String>,
}

impl Erasure {
    /// Whether the key is destroyed **and** the live ciphertext is gone.
    #[must_use]
    pub fn is_complete(&self) -> bool {
        self.cleanup_failed.is_none()
    }
}

mod cases;
pub use cases::{SealedCases, probe_sealed_case_state};

mod events;
pub use events::SealedEvents;

mod tasks;
pub use tasks::SealedTasks;

mod journal;
pub use journal::SealedJournal;
pub(crate) use journal::open_payloads;

#[cfg(feature = "push")]
mod push;
#[cfg(feature = "push")]
pub use push::SealedPush;

mod sealed;
pub use sealed::EncryptedBlobs;

pub mod coordinator;
pub use coordinator::{ErasureCoordinator, Lease, LocalCoordinator, UnderLock, under_lock};
mod memory;
pub use memory::EncryptedMemoryStore;

#[cfg(feature = "keyring-vault")]
mod vault;
#[cfg(feature = "keyring-vault")]
pub use vault::VaultTransit;