datawal 0.1.0-alpha.1

Core record store for datawal: append-only framed records (CRC32C), valid-prefix recovery, bytes-based KV projection with tombstones, manual compaction, and JSONL export. v0.1-pre.
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63

//! `datawal` — append-only framed record log and bytes-based KV (v0.1-pre).
//!
//! ## What this crate is
//!
//! - [`RecordLog`]: append-only segmented record log with per-record CRC,
//!   valid-prefix recovery, and explicit `fsync`/`rotate`. The substrate.
//! - [`DataWal`]: last-write-wins bytes-based key/value projection on top of
//!   [`RecordLog`], with `put` / `get` / `delete` / tombstone semantics and
//!   manual `compact_to` plus JSONL export.
//!
//! ## What this crate is not
//!
//! v0.1-pre intentionally has **no**:
//!
//! - content-addressed storage / blob CAS
//! - Python bindings (PyO3)
//! - compression
//! - server / RPC / network exposure
//! - query / analytics engine
//! - multi-writer or multi-process safety
//! - JSON awareness — payloads are opaque bytes
//! - schema validation
//! - encryption
//! - application-specific event kinds, manifest schemas, or hashing semantics
//!
//! ## Layout on disk
//!
//! ```text
//! dir/
//!   .lock                   # cooperative sentinel file
//!   00000001.dwal           # segment 1
//!   00000002.dwal           # segment 2 (created by `rotate`)
//!   ...
//! ```
//!
//! There is no MANIFEST file in v0.1-pre: the set of segments is whatever
//! matches `[0-9]{8}\.dwal` on disk. Adding a MANIFEST is a v0.2 concern.
//!
//! ## Wire format
//!
//! See [`mod@format`] for the byte-exact specification. Each record is a 24-byte
//! header + key + payload + 4-byte CRC trailer. Multi-byte integers are
//! little-endian.
//!
//! ## Filesystem primitives
//!
//! Atomic FS operations (`write_atomic`, `fsync_dir`, `rename_atomic`, etc.)
//! live in the sibling crate [`safeatomic_rs`]. `datawal` consumes them;
//! it does not redefine them.

#![forbid(unsafe_code)]
#![deny(rust_2018_idioms)]

pub mod format;
pub mod lock;
pub mod segment;

mod datawal;
mod record_log;

pub use datawal::{CompactionStats, DataWal};
pub use format::{RecordType, MAX_KEY_LEN, MAX_PAYLOAD_LEN, WIRE_VERSION};
pub use record_log::{Record, RecordLog, RecordRef, RecoveryReport};