linkrs_storage/edge/edge_table/checkpoint/
snapshot.rs

1//! Frozen group mmap snapshot file.
2//!
3//! Derived read-only cache beside the authoritative checkpoint: while the
4//! checkpoint stays the source of truth, a frozen group can also serve
5//! queries straight from a memory-mapped flat column file, paging entries in
6//! on demand instead of decoding the whole group into the heap at open.
7//!
8//! File layout, everything little-endian:
9//! - magic (u32)
10//! - rows (u64), entries (u64), live edge count (u64), flags (u64)
11//! - seven `(offset u64, length u64)` column descriptors: degrees, endpoints,
12//!   ranks, edge ids, delete stamps, bundled values, validity bytes
13//! - degrees: `rows` u32 values
14//! - endpoints: `entries` u32 values
15//! - ranks: `entries` i64 values
16//! - edge ids, delete stamps, bundled values: `entries` u64 values each
17//! - validity: `ceil(entries / 8)` bytes, one bit per slot
18//!
19//! The bundled value columns only occupy bytes when the flags word marks a
20//! valued snapshot; otherwise both descriptors are empty ranges. Column
21//! widths are fixed, so any slot is addressable by index with one
22//! little-endian decode and no full-file decode. Row offsets are rebuilt in
23//! memory on open and never persisted, mirroring the heap frozen form.
24//!
25//! The snapshot payload carries a trailing CRC32 covering every preceding
26//! byte, using the same checksum pattern as the heap checkpoint dumps. Open
27//! rejects structural mismatches (magic, out-of-range descriptors,
28//! length disagreements, trailing bytes) and CRC mismatches alike, and the
29//! caller falls back to the authoritative checkpoint, optionally
30//! regenerating the snapshot file. A bad cache is discarded and rebuilt. Writers use a sibling temp file
31//! plus atomic rename, so readers only ever observe complete files.
32//!
33//! The mapped handle is reference-counted (`MappedFrozen` clones share one
34//! mapping), so readers holding a view keep the file alive across snapshot
35//! file replacement. Single-writer discipline applies: the checkpoint flush
36//! syncs base file and snapshot file together, and flushing a non-frozen
37//! group removes its snapshot file. On Linux the mapping carries a transparent
38//! huge page hint; a rejected hint falls back to base pages without failing
39//! the open.
40//!
41//! # Snapshot cache state machine
42//!
43//! The snapshot file is a derived cache, never the truth. States and
44//! transitions, all covered by the checkpoint tests:
45//!
46//! - Absent: no sidecar beside the base. Frozen or mapped bases backfill one
47//!   on flush when missing; mutable bases stay absent.
48//! - Valid: sidecar parses (magic, descriptors, lengths, CRC) and
49//!   loads validate it against the base. Loads map it directly and skip the
50//!   authoritative decode.
51//! - Stale: the base was rewritten as mutable, or the group is gone. The
52//!   flush removes the sidecar; readers never see it.
53//! - Expired: the sidecar fails validation (corrupt bytes or
54//!   a pending append delta the read-only view cannot absorb). Loads fall
55//!   back to the authoritative base and regenerate the cache; the failure is
56//!   counted in debug logs, never silent, and never fails the load.
57//!
58//! Generation, deletion, expiry and fallback are all decided in the
59//! checkpoint snapshot helpers and the group load path; no other module
60//! creates or removes sidecars.
61
62use std::sync::Arc;
63
64pub(crate) mod columns;
65pub(crate) mod format;
66pub(crate) mod iter;
67pub(crate) mod open;
68pub(crate) mod persistence;
69pub(crate) mod read;
70pub(crate) mod trait_impl;
71
72#[cfg(test)]
73mod tests;
74
75use format::SnapshotColumns;
76pub use format::{snapshot_path_for, write_snapshot_file};
77pub use iter::{MappedFrozenIterator, MappedFrozenRowIter};
78
79/// Memory-mapped read view of one frozen group snapshot file.
80///
81/// Clones share one mapping through the reference count, so snapshotting the
82/// handle for a reader is cheap and the mapping outlives snapshot file
83/// replacement while readers hold it. The query surface mirrors
84/// [`ImmutableCsr`] so variant dispatch treats both identically; writes are
85/// rejected the same way.
86#[derive(Debug, Clone)]
87pub struct MappedFrozen {
88    map: Arc<memmap2::Mmap>,
89    columns: SnapshotColumns,
90    offsets: Arc<Vec<u32>>,
91    rows: usize,
92    entries: usize,
93    edge_count: u64,
94}
linkrs_storage/edge/edge_table/checkpoint/snapshot.rs

linkrs_storage/edge/edge_table/checkpoint/
snapshot.rs
