Expand description
§store-io
Durable storage I/O for Rust databases.
store-io is the layer between a storage engine and the files and devices it writes to. It moves caller bytes to media and proves when they are durable. It does not frame, parse or validate the bytes it moves, and it never decides what a caller keeps after a crash.
The design rests on four rules:
- Durability is decided per device, with evidence. Each store is probed at open and classed as power-safe, flush-required, unverified or unsafe from what the device and operating system report. Operating-system cache toggles and vendor tables never promote a device.
- A durable write produces an unforgeable receipt. Positions, write tickets and receipts are distinct opaque types, so a sequence number can never be passed where a byte position is expected, and a barrier can never be a silent no-op.
- Failure is fail-stop. The first failed write or flush poisons the device’s barrier domain. A flush is never retried and success is never reported after a failed one.
- Nothing hides latency. There are no timers, no group-commit windows and no background threads that wake an idle process.
§Quick start
use store_io::{Store, StoreOptions};
// First run: create. Every later run: `Store::open`.
let store = Store::create("data/orders", StoreOptions::default())?;
// Regions are provisioned once; looking one up never creates it.
let wal = store.provision_append_region("wal", 256 << 20)?;
// Append any number of bytes; durable when this returns.
let (pos, receipt) = wal.append_durable(b"order 1001: 3 widgets")?;
println!("durable through {} ({})", receipt.durable_through(), receipt.class());
// Many records, one write and one flush.
let mut batch = wal.batch();
batch.append(b"order 1002")?;
batch.append(b"order 1003")?;
let (start, receipt) = batch.commit()?;
// Small objects: old or new after any crash.
let manifest = store.provision_slot("manifest")?;
let _receipt = manifest.commit(b"wal@0")?;docs/GUIDE.md in the repository walks through every part: append and
page regions, batches, many writers, small objects, recovery with
AppendRegion::scan and resume_at, the device report, errors and
space.
§Container format
Stores are created in container format 2. A store made before 0.5
(format 1) opens read-only (Store::open_readonly); a writable open
refuses it with Unsupported(FormatVersion) before writing anything.
copy_store (or store-io copy <old> <new>) copies it into a new
format-2 store, all or nothing, the source untouched.
§Layers
- Simple:
Store,AppendRegion,PageRegion,SlotandDirectory; every call blocks until its I/O is done and copies the caller’s bytes once into aligned buffers. - Batch:
AppendBatchandPageBatch: many writes, one barrier, with in-place encoding and no copy. - Submitter:
Submitter(submit): many reads and writes in flight from one thread without blocking, from pooled buffers with no copy; tickets come with the completions. - Engine: the generic
enginecrate under all of it, which also runs on the deterministic simulator.
§Platforms
Windows (unbuffered overlapped I/O on IOCP), Linux (direct I/O, io_uring
or pwritev2) and macOS (pwrite and F_FULLFSYNC, uncached where the
kernel can) have native backends (NativePlatform). On other targets the
types compile but there is no native backend.
Re-exports§
pub use store_io_engine as engine;
Modules§
- observe
- The observer’s types:
Store::observe()returns anobserve::Snapshot. The observer: per-stage latency histograms and event counters (D-094). - submit
- The
Submitter’s types: its configuration, buffers, completions and refusals, and the waker that ends itswaitfrom another thread.
Structs§
- Byte
Range - Byte range
[start, end)within a region or file. - CertId
- Opaque identity of a certificate (exact model, firmware, capacity and LBA format, backed by a power-cut rig log) or of an operator attestation.
- Class
Decision - The outcome of
decide. - Copy
Options - How
copy_storebuilds its target. - Copy
Report - What
copy_storecopied. - Directory
Linux or macOS or Windows - A directory of small files, each replaced atomically and durably (see
store_io_engine::Directory). - Domain
Stats - Counters a domain exposes for observability.
- Durable
Receipt - Unforgeable proof that bytes are durable.
- Error
Context - Where an error happened. The volume is the store the call was made on.
- Evidence
- Everything a backend could learn about one path.
- First
Cause - What the domain was poisoned by.
- Generation
- A generation: how many times an object or region has been rewritten, recycled or re-owned. Starts at 1; 0 is never written.
- IoPath
- The kernel interface a platform’s queues use, and why (reported in
StoreReport). - OsError
- Raw operating-system error code, kept exactly as the OS reported it.
- Outstanding
- The oldest operation in flight on the store’s queues.
- Probe
- What
probefound out about a directory’s device. - Region
Id - Identity of a region within a volume. Never reused within a volume.
64-bit: container format 2 ids are
page << 32 | seq; format 1’s fit in the low 32 bits. - Region
Options - A region’s size, and the most it may ever grow to.
- Region
Pos - A byte position inside one generation of one region.
- Scan
Summary - How a scan ended.
- Space
Report - Where a store’s space goes.
- Store
Linux or macOS or Windows - A store: one container file in a directory of its own, on one device.
- Store
Options - Explicit, reported choices for a store. Every durability-affecting field
appears in
Store::report. - Store
Report - What the store learned about its device, and what it decided.
- TagSpace
- Usage of one tag.
- Trust
- Caller-supplied trust inputs. Every field is an explicit, reported choice.
- Unavailable
Region - A region the region table names that could not be opened (D-090).
- Volume
Id - 128-bit identity of a volume (one container or raw namespace).
- Write
Ticket - Proof that a write was submitted and completed. Barrier it to make it
durable. Not
Clone: one ticket per write.
Enums§
- Capability
- A capability the platform, filesystem or device does not offer.
- Corruption
Kind - Corruption store-io detected in its own metadata or on media.
- Durability
Class - What a completed durable operation means on this device.
- Durable
Open - Whether a durable open may proceed.
- Error
- Every error store-io returns.
- IoInterface
- A kernel interface for I/O (
IoPath). - Linux
Tier - Which kernel interface the Linux backend’s queues use (D-082). Other backends have one interface and ignore it.
- Missing
- Evidence the probe could not obtain.
- Named
- What a lookup did not find, or found already existing.
- NotWritten
Cause - Why an operation was rejected before reaching the device.
- Op
- The operation that failed.
- OsError
Source - Numbering space of an
OsErrorcode. - Reason
- Why a device was classed below power-safe, or refused.
- Receipt
Label - Why the holder of a receipt may trust it.
- Region
Kind - What a region is for.
- Repair
- What one repair of a store did (v0.5 E6), as
Store::repairsreturns it: for the caller’s own log (the library prints nothing). - Reserve
Tag - What ran out: a space-accounting tag (tenant, class), store-io’s own table, or a region’s own limit.
- Scan
Item - One step of a scan, in region order.
- Unavailable
- Why a region the region table names could not be opened (D-090).
Constants§
- CONTAINER
- The container file name inside the store directory.
- COPYING
- The temporary name a copy’s target is built under (no live store ever has it: one left by a crashed copy is removed by the next).
- COPY_
LOCK - The file in a copy’s target directory whose lock a copy holds from before it looks for a store there until after its rename (created by the first copy and kept: removing a lock file races its next user).
Functions§
- copy_
store Linux or macOS or Windows - Copies the store in
frominto a new store (container format 2) into, whose directory must hold no store: the way a format-1 store moves to format 2, and the way to compact a format-2 one. The source is opened read-only and never written; the copy is built under a temporary name and renamed into place only when whole and durable, so after a crashtoholds nothing or the whole copy (run the copy again). - io_
uring_ fallbacks Linux - Operations, process-wide, that the io_uring tier finished with a blocking system call because the kernel would have blocked or cut them short (never a failure; how often the asynchronous path was not taken). Operations, in this process so far, that the io_uring tier finished with one blocking system call because the kernel would have blocked (or could not honour NOWAIT for the file) or transferred short. A measure of how often the asynchronous path was not taken; never a failure.
- probe
Linux or macOS or Windows - Probes the device under the existing directory
dir: the evidence store-io reads and the durability class it would decide with default trust. Creates, probes and removes a small temporary file there.
Type Aliases§
- Append
Batch Linux or macOS or Windows - Many appends, one barrier (see
store_io_engine::AppendBatch). - Append
Region Linux or macOS or Windows - A region store-io appends to (see
store_io_engine::AppendRegion). - Missing
Set - Set of
Missingevidence items. - Native
Platform Linux or macOS - The operating system’s native backend.
- Page
Batch Linux or macOS or Windows - Many page writes, one barrier (see
store_io_engine::PageBatch). - Page
Region Linux or macOS or Windows - A region of caller-addressed blocks (see
store_io_engine::PageRegion). - Reason
Set - Set of
Reasons. - Reservation
Linux or macOS or Windows - Space set aside for provisioning (see
store_io_engine::Reservation). - Result
ResultwithError.- Slot
Linux or macOS or Windows - A small object committed atomically (see
store_io_engine::Slot). - Submitter
Linux or macOS or Windows - Many reads and writes in flight from one thread, without blocking (see
store_io_engine::Submitter); fromStore::submitter.