Crate batpak 

batpak is an embedded, sync-first event store for Rust: an append-only, hash-chained journal with typed events, verifiable receipts, deterministic replay, and derived projections — no server, no async runtime.

The store keeps immutable events in append-only segment files, tracks causation metadata, and evaluates caller-defined gates before commit; state is rebuilt by replaying the log into typed projections, all through a synchronous API.

Use it when you need a tamper-evident, replayable record of what happened: every event is hash-bound to its per-entity ancestor with Blake3, every accepted write returns a verifiable (optionally Ed25519-signed) receipt, and projections are derived views rebuilt from the log by construction.

Most callers start with the eight-job path: open a Store, append typed events, page commit order with Store::query_entries_after, point-read with Store::get, walk bounded hash-chain ancestry with Store::walk_ancestors, verify receipts with Store::verify_append_receipt, project derived state with Store::project, then close the store. Gates and Pipeline are advanced batteries for caller-owned evaluation before commit.

use batpak::prelude::*;

#[derive(serde::Serialize, serde::Deserialize, EventPayload)]
#[batpak(category = 0xF, type_id = 1)]
struct ThingHappened {
    value: i64,
}

let dir = tempfile::tempdir()?;
let store = Store::open(StoreConfig::new(dir.path()))?;
let coord = Coordinate::new("entity:a", "scope:1")?;

let receipt = store.append_typed(&coord, &ThingHappened { value: 42 })?;
let stored = store.get(receipt.event_id)?;

assert_eq!(stored.coordinate.entity(), "entity:a");
assert_eq!(stored.event.header.event_id, receipt.event_id);

Reading order:

1. coordinate: Identify entities and scopes.
2. event: Structure typed payloads and projection inputs.
3. store: Persist, page, point-read, walk, verify, and project.
4. guard and pipeline: Add caller-defined write evaluation.
5. artifact, registry, transition, reservation, and schema: Advanced substrate batteries for envelopes, ledgers, transition evidence, reservation mechanics, and drift reports.

Fail-closed defaults (verifiability). A store refuses to open on an ambiguous or undecodable payload registry: EventPayloadValidation defaults to FailFast, so a duplicate-kind collision or an incomplete upcast chain is rejected at Store::open (opt out explicitly with Warn/Silent). A binary that registers EventPayload types but may never open a store should call verify_registry once at startup (or enable the non-default startup-registry-check feature for automatic enforcement), since the derive’s own collision test is #[cfg(test)]-only and a release binary would otherwise see no check. Receipt signing is governed by SigningPolicy: the default Optional permits a keyless store, while Required refuses to open without a signing key so an unsigned receipt is never accepted. A configured signer fails the append closed rather than silently emitting an unsigned receipt unless StoreConfig::with_signing_downgrade_allowed opts in.

On-demand integrity. Store::verify_chain recomputes the full blake3 hash chain over every committed event and returns a ChainVerificationReport; opt into ChainVerification::Recompute to run that pass automatically at open and fail closed on tamper. For ancestry, Store::walk_ancestors_outcome returns an AncestorWalk whose AncestryBoundary makes a truncated lineage (for example, a retention-dropped mid-chain parent) observable instead of indistinguishable from a complete walk to genesis.

Payload encryption & crypto-shred (opt-in). Off by default: a default build writes plaintext payloads and pulls no crypto dependency. Enabling the non-default payload-encryption cargo feature and calling StoreConfig::with_payload_encryption seals every payload at rest under a per-scope 256-bit XChaCha20-Poly1305 key (a pure-Rust AEAD; key and nonce bytes come from the OS CSPRNG, and key material zeroizes on drop and never appears in Debug/Display output). KeyScopeGranularity chooses which events share a key — and therefore what a single erasure destroys: PerEntity (default, one key per entity, across all kinds), PerCategory (one key per event-kind category), PerTypeId (one key per full kind), or PerEvent (one key per individual event, the finest). Store::shred_scope then crypto-shreds a scope: it destroys that scope’s KEY and flushes the keyset durable, making every payload sealed under it permanently unrecoverable. A later read of a shredded payload reports StoreError::PayloadShredded (or a ReadDisposition::Shredded value via Store::get_shreddable) — never corruption and never the raw ciphertext.

Shredding destroys only the key, never any event frame: the ciphertext and its Blake3 chain identity survive on disk, so verify_chain, receipts, and signatures stay intact — identity is taken over the STORED CIPHERTEXT, not the plaintext. Erasure is EXACTLY this explicit op; tombstone/retention compaction never auto-destroys a key, so a coarse scope (the default PerEntity) is erased only when the caller names that entity by selector.

Threat model — keys at rest. The keyset lives inside the store’s own data directory, next to the ciphertext it protects. What crypto-shred DOES buy: once a scope’s key is destroyed AND that destruction is flushed, the scope’s payloads are unrecoverable even to an operator with full disk access — deletion becomes cryptographically effective rather than a best-effort overwrite. What it does NOT buy: it does not protect a disk image captured before the shred (the key was still present then), and a stolen live data directory yields both key and ciphertext. Holding the keyset OUT of the data directory — a separate volume, an OS keyring, or an external KMS — is a deployment concern, outside the core mechanism. batpak only ever observes “the key for scope X was destroyed”; the layer above maps that erasure to its own policy.

Snapshot/fork portability. Because the keyset never travels with the ciphertext it opens, a Store::snapshot_with_evidence / Store::fork of an encryption-active store FAILS CLOSED by default (StoreError::KeysetNotPortable): a keyless copy is silently unrestorable, and a copy carrying the keyset could resurrect crypto-shredded data. Opt into a keys-excluded copy with KeysetPolicy::ExcludeKeys (managing the keyset out-of-band); restoring one without its keyset reports StoreError::KeysetMissing, never a Shredded lookalike.

use batpak::prelude::*;
use batpak::store::{KeyScopeGranularity, ShredScope};

#[derive(serde::Serialize, serde::Deserialize, EventPayload)]
#[batpak(category = 0xF, type_id = 1)]
struct ThingHappened {
    value: i64,
}

let dir = tempfile::tempdir()?;
// PerEntity (the default): one key covers every payload of an entity, so a
// single shred erases all of that entity's payloads at once.
let store = Store::open(
    StoreConfig::new(dir.path())
        .with_payload_encryption(KeyScopeGranularity::PerEntity),
)?;
let coord = Coordinate::new("entity:a", "scope:1")?;
let receipt = store.append_typed(&coord, &ThingHappened { value: 42 })?;

// While the key is live the payload reads back transparently.
assert_eq!(
    store.get(receipt.event_id)?.event.header.event_id,
    receipt.event_id,
);

// Destroy this entity's key: its plaintext is now permanently gone...
store.shred_scope(ShredScope::Entity(&coord))?;
assert!(matches!(
    store.get(receipt.event_id),
    Err(StoreError::PayloadShredded { .. })
));

// ...yet identity survives: the chain still verifies over the ciphertext.
assert!(store.verify_chain()?.is_intact());

Cargo features (all non-default). A default build enables none of these and pulls none of their dependencies. payload-encryption adds the crypto-shred surface above (with_payload_encryption / shred_scope, XChaCha20-Poly1305 + an OS CSPRNG). startup-registry-check runs verify_registry automatically before main via one process-wide constructor, so a release binary that registers EventPayload types but never opens a store still aborts on a kind collision; the always-on, portable path is the explicit verify_registry() call described above, which needs no constructor.

Re-exports

pub use crate::encoding as canonical;

pub use crate::event::EventPayload;

pub use crate::event::EventSourced;

pub use crate::event::MultiReactive;

Modules

artifact

Crate-level substrate: canonical artifact body digest vs envelope digest. Canonical artifact envelope: digest of the serializable body vs digest of the envelope (signatures, attestations, diagnostics).

coordinate

Entity and scope addressing for events.

encoding

Stable named-field MessagePack encoding helpers. Stable batpak encoding helpers.

event

Event types, headers, and sourcing traits.

guard

Caller-defined gate evaluation before event commitment.

id

UUID v7 identifier generation.

outcome

Result-like type for pipeline operations.

pipeline

Propose-evaluate-commit workflow.

prelude

Common re-exports for convenient use. Beginner-oriented imports for the canonical BatPAK store path.

registry

Crate-level substrate: generic signed registry rows composing artifact envelopes. Batpak Substrate Closure attested registry row: stable row identity, canonical row body digest, lifecycle and supersession pointers, drift evidence, and verification reports that compose crate::artifact::CanonicalArtifactEnvelope without importing crate::store.

reservation

Crate-level substrate: generic reservation ledger mechanics. Batpak Substrate Closure reservation ledger: dimensionless units, opaque subject_ref, closed structural states, explicit transition operations, deterministic findings, and reconciliation buckets. This module does not import crate::store and encodes no payment, inventory, capability, or workflow policy.

schema

Deterministic schema/fixture snapshot drift evidence. Deterministic Batpak Substrate Closure schema/fixture snapshot drift evidence.

store

Persistent event storage and querying.

transition

Crate-level substrate: generic state transition events and reports. Batpak Substrate Closure state transition evidence: opaque machine/subject identifiers, prior and successor state discriminants, transition identity, sorted cause references, optional ordering metadata, and deterministic reports with structural findings only.

typestate

Compile-time state machine transitions.

wire

Module declarations in DEPENDENCY ORDER: wire → coordinate → outcome → event → guard → pipeline → store → typestate → id → prelude Serde serialization helpers.

Macros

define_entity_id

define_entity_id!: Layer 1+ macro. Uses generate_v7_id() helper. Downstream crates do NOT need uuid as a direct dependency.

define_state_machine

define_state_machine!: generates a sealed marker trait + zero-sized state structs.

define_typestate

define_typestate!: generates a PhantomData wrapper for typed state machines.

register_upcast

Register an Upcast implementation so the decode seam can find it.

Derive Macros

EventPayload

Derives batpak::event::EventPayload for a named-field struct.

EventSourced

Derives batpak::event::EventSourced for a named-field struct.

MultiEventReactor

Derives batpak::event::MultiReactive<Input> for a named-field struct, for use with Store::react_loop_multi (JSON) or Store::react_loop_multi_raw (msgpack).