nautilus-event-store

license

Embedded event store and authoritative log of state-affecting messages for NautilusTrader.

[!WARNING] Early alpha. The API is not stable and may change between versions. Event-store capture, replay, and verification workflows are the current focus.

The nautilus-event-store crate captures commands, generated events, raw venue reports, reconciliation outputs, and request/response traffic at the message bus boundary. It persists those messages as a durable per-run log and exposes read-only verification and replay surfaces.

For the design model, capture boundaries, replay modes, and operational examples, see the Event Sourcing concept guide.

Design contract

The event store is the durable boundary for deterministic engine history.

The synchronous core is deterministic.
The cache is a write-through projection, not the source of truth.
Replay derives state from captured history and named deterministic replay rules.
The event store records ordered inputs and generated state-affecting messages.
Anything not recorded is non-state-affecting or named as a deterministic replay rule.
External I/O becomes replayable when captured as raw reports or commands.

What this crate provides

nautilus-event-store is a single-node, embedded event store for one trading instance. It defines:

BusCaptureAdapter: the seam between message bus dispatch and the writer.
EventStoreLifecycleOptions: runtime-only lifecycle policy for custom encoder registries and backend openers.
EventStoreWriter: the append path, batching, high-watermark advancement, and fail-stop signaling.
EventStoreReader: the read-only range scan, point lookup, and replay-facing surface.
RedbBackend: the default on-disk backend, with one redb file per run.
MemoryBackend: the in-process backend used by focused tests and simulation-style capture.
Verifier: the library surface for integrity checks over a single run.
verify: the standalone binary for process-isolated verification of sealed run files.
plan_redb_retention: a non-destructive planner for sealed run-file reclaim candidates.
ReplayInputPlan: planned event-store entries in seq order, optionally with typed catalog slice plans.
ReplayInputs: loaded replay entries in seq order, optionally with catalog records grouped under selected slices.

The crate does not replace the data catalog, provide OLAP queries, or aggregate multiple trader instances into a consensus log.

Operational use today

Verify a sealed run file:

cargo run -p nautilus-event-store --bin verify -- /path/to/run.redb

Verifier exit codes:

0: the run is clean.
1: the run has corrupt findings, or the verifier worker aborted or timed out.
2: the verifier could not open or run against the requested file.

The verifier opens the run through a read-only redb handle and reports quarantine=not-performed. A supervisor or operator process owns quarantine policy.

Read a sealed run from Rust:

use nautilus_event_store::{EventStoreReader, RedbBackend, ScanDirection};

fn inspect_run() -> Result<(), Box<dyn std::error::Error>> {
    let backend =
        RedbBackend::open_sealed_file("./event_store/trader-001/1700000000-cafe0001.redb")?;
    let reader = EventStoreReader::new(backend);
    let high_watermark = reader.high_watermark()?;

    for entry in reader.scan_range(1, high_watermark, ScanDirection::Forward) {
        let entry = entry?;
        println!("{} {}", entry.seq, entry.topic);
    }

    Ok(())
}

Lifecycle options

EventStoreLifecycle::boot(...) keeps the default behavior: it opens RedbBackend and installs default_registry(). Advanced callers can pass runtime-only options through EventStoreLifecycle::boot_with_options(...) without changing serialized EventStoreConfig.

use bytes::Bytes;
use nautilus_event_store::{
    EncodedPayload, EncoderRegistry, EventStoreLifecycleOptions,
};
use ustr::Ustr;

#[derive(Debug)]
struct AuditRecord {
    payload: Bytes,
}

let mut registry = EncoderRegistry::new();
registry.register::<AuditRecord, _>(Ustr::from("AuditRecord"), |record| {
    Ok(EncodedPayload::without_indices(record.payload.clone()))
});

let options = EventStoreLifecycleOptions::new().with_encoder_registry(registry);
// Pass `options` to EventStoreLifecycle::boot_with_options(...).

The same options type accepts a backend opener. The default opener remains RedbBackend; tests or simulation harnesses can supply an opener that returns MemoryBackend or another EventStore implementation.

Storage model

The default backend is redb, a pure-Rust ACID key-value store. The backend uses one file per run:

<base>/<instance_id>/<run_id>.redb

Each file stores:

Entries keyed by monotonic seq.
Secondary indices for client_order_id and venue_order_id.
A manifest written at run start and sealed at run end.
An optional snapshot anchor for cache restore.

Every commit uses Durability::Immediate. The high-watermark advances only after the backend acknowledges durability.

Failure and verification model

The verifier is intentionally process-isolated. Some corrupted redb files can panic on open or first read, and the release profile builds with panic = "abort". The verify binary delegates the scan to a worker subprocess so a bad run file aborts the worker, not the caller.

Integrity checks include:

Recomputing every entry hash.
Detecting gaps inside the high-watermark.
Checking table-key and embedded-seq agreement.
Cross-checking secondary indices against entries.
Validating manifest status and high-watermark fields.

Documentation

Event Sourcing concept guide: design, replay, recovery, and operational examples.
docs.rs: generated Rust API reference.

NautilusTrader

NautilusTrader is an open-source, production-grade, Rust-native engine for multi-asset, multi-venue trading systems.

The system spans research, deterministic simulation, and live execution within a single event-driven architecture, providing research-to-live semantic parity.

License

The source code for NautilusTrader is available on GitHub under the GNU Lesser General Public License v3.0.

NautilusTrader™ is developed and maintained by Nautech Systems, a technology company specializing in the development of high-performance trading systems. For more information, visit https://nautilustrader.io.

Use of this software is subject to the Disclaimer.

nautilus-event-store 0.58.0