Expand description
Immutable ordered-record segment files — build once, binary-search forever. The storage stone every cold tier shares: row segments, scalar-index cold segments and text bucket segments differ only in what their records MEAN; how records are laid out, located, checksummed and retired is identical, so it lives here once.
use kevy_seg::{Manifest, ManifestEntry, Seg, SegBuilder};
// keys go in strictly ascending; the footer seals the file
let mut b = SegBuilder::create(&dir.join("s1.seg"))?;
b.push(b"a", b"1")?;
b.push(b"b", b"2")?;
let meta = b.finish()?;
// the manifest makes the sealed segment part of the live set
Manifest::open(&dir)?.add(ManifestEntry::new("s1.seg", meta))?;
let seg = Seg::open(&dir.join("s1.seg"))?;
assert_eq!(seg.get(b"b")?, Some(b"2".to_vec()));
assert_eq!(seg.count_range(b"a", b"z")?, 2);§Model
SegBuilder appends records in strictly ascending key order,
pages them, checksums every page, and seals the file with a footer
(record count, min/max key, and a fence table — the first key of
every page). Seg opens the sealed file, keeps the fence table
in memory, and answers get / range / count_range with one
fence binary search plus one page read. Nothing ever mutates a
sealed segment; deletion is the caller’s directory concern
(tombstones live above this crate), and removal is unlink.
§Layout (v1)
Data pages are 4 KiB: a small header, cells packed forward, a slot directory (u16 cell offsets) packed backward from the tail, and a CRC32C over the page in the last 4 bytes. A record too large for one page spills its payload into a run of dedicated overflow pages (the cell keeps the key and points at the run — SQLite’s overflow idea, shorn of its freelist because nothing here is ever freed). The footer is written last: fence entries, min/max keys, counts, magic, and its own CRC; the final 16 bytes locate and size it.
§References
The read-only half of SQLite’s page format (slot directory growing backward, overflow chains) is the reference for the data pages. Its WAL, freelist, cursors and varint cell headers are deliberately absent: an immutable segment needs none of them.
Structs§
- Manifest
- The append-only ledger. One per segment directory.
- Manifest
Entry - One live segment as the manifest knows it.
- Range
Iter - Ascending
(key, payload)iterator over a closed range. - Seg
- An open segment. Cheap to clone-by-Arc above this crate; internally one file handle plus the in-memory fence table.
- SegBuilder
- Streaming builder. Records must arrive in strictly ascending key order (the reader’s binary search is the reason); a violation is a refusal, not a sort.
- SegMeta
- Sealed-segment summary, from the footer.
Enums§
- SegError
- Why a segment file was refused at open. Corruption is a refusal, never a silent partial read.