fsys 1.1.0

fsys is a foundation-tier filesystem IO crate for Rust storage engines, embedded databases, and durable services. It pairs an explicit durability model with a journal substrate, io_uring on Linux, NVMe passthrough, and atomic-replace writes — sitting one layer below your data structures and one layer above std::fs.

It is not trying to replace std::fs for ordinary application code.

Quickstart

use std::sync::Arc;
use fsys::{builder, JournalOptions};

fn main() -> fsys::Result<()> {
    // Build a handle once, share via Arc.
    let fs = Arc::new(builder().build()?);

    // Open an append-only journal &mdash; the WAL primitive.
    let log = fs.journal("/var/lib/myapp/log.wal")?;

    // Append many records without per-call fsync.
    let _ = log.append(b"txn 1: insert")?;
    let _ = log.append(b"txn 2: update")?;
    let lsn = log.append(b"txn 3: commit")?;

    // One fsync covers every prior append &mdash; group-commit.
    log.sync_through(lsn)?;

    Ok(())
}

For one-shot file IO (atomic-replace, durable), fsys::quick::write / read skip the handle:

fsys::quick::write("/etc/myapp/config.toml", b"value = 42")?;
let data = fsys::quick::read("/etc/myapp/config.toml")?;

See examples/ (33 runnable patterns) and docs/EXAMPLES.md for the full catalogue.

At a glance

• Five durability methods — Sync, Data, Mmap, Direct, and hardware-aware Auto. Every method is platform-honest: the actual primitive in use is observable via Handle::active_durability_primitive().
• Journal substrate — open-once append-only log with atomic LSN reservation, group-commit fsync, and a CRC-32C-protected frame format. Three throughput tiers (sync, lock-free concurrent, native io_uring async on Linux). The HiveDB-class WAL primitive.
• Atomic-replace writes — every write / write_copy / Batch::commit uses temp-file + atomic rename. The target is either entirely the old payload or entirely the new payload — never torn.
• Linux io_uring on the hot path — Method::Direct and the journal Direct-IO path submit through io_uring with IORING_OP_WRITE_FIXED against pre-registered buffer slots. Falls back to O_DIRECT + pwrite + fdatasync cleanly when io_uring is unavailable.
• NVMe passthrough flush — on Linux (NVME_IOCTL_IO_CMD) and Windows (IOCTL_STORAGE_PROTOCOL_COMMAND) when the hardware supports it. Transparent fallback to fdatasync / WRITE_THROUGH otherwise.
• Cross-platform reflinks — macOS clonefile(2) + Windows FSCTL_DUPLICATE_EXTENTS_TO_FILE give APFS / ReFS instant copy-on-write semantics. Multi-GiB checkpoint clones drop from seconds to microseconds.
• Optional async layer (async feature) — every sync method gets an _async sibling. On Linux + Method::Direct, async ops submit directly to the per-handle io_uring ring (no spawn_blocking thread-pool hop).
• Hardware-aware tuning — PLP detection, NAWUN/NAWUPF probe (atomic-write unit), Builder::tune_for(Workload::Database) preset, runtime CPU-feature detection for hardware CRC-32C.
• Capability cache + SPDK gating (1.1.0) — fsys::capability::capabilities() probes the system once (50–200 ms), caches the result to disk, and returns sub-millisecond loads thereafter. Method::Spdk is wired through the public API; the kernel-bypass backend lives in the companion fsys-spdk crate.
• Journal backend observability (1.1.0) — every JournalHandle exposes backend_kind() / backend_health() / backend_info() so ops teams can verify which IO path is live without ambiguity.

When to use fsys

The "fair comparison" for durable writes is fsys::Sync versus std::fs plus a manual temp-file + sync_all + rename dance — the latter is what most application code gets wrong. fsys provides this as a single public API call.

Performance

Numbers below were captured on windows-ntfs-nvme (Windows 11 Pro, x86_64, local NVMe SSD; std::env::temp_dir() resolves to NTFS) with 100 timed iterations after 10 warmup. Run-to-run noise is roughly ±5% on this host class. Full methodology, additional payload sizes, and Linux numbers live in docs/BENCH.md; reproduce locally with cargo bench.

Journal substrate vs atomic-replace

The headline result. Atomic-replace pays 5–7 syscalls per durable write; the journal opens once, appends without per-call fsync, and amortises durability across a sync_through call — the canonical WAL pattern.

At an intermediate cadence (sync every 100 appends), the journal still delivers 109–255× the atomic-replace throughput. See docs/BENCH.md for the full table including per-append sync cadence.

Atomic-replace write vs std::fs::write

fsys::Auto pays a deterministic durability cost; std::fs::write defers that cost to OS scheduling and pays it at p99 instead.

At 1 MiB, fsys::Auto is 3.3× faster than std::fs::write at p99 — durability paid up-front rather than at unpredictable points.

Read parity

The read path is essentially std::fs::read plus handle bookkeeping — no durability cost on reads.

tokio::fs::read is 1.5–4.4× slower than fsys::Auto because tokio's own fs module routes through spawn_blocking. On Linux + Method::Direct + the async feature, fsys's native io_uring substrate bypasses that thread-pool hop entirely.

Installation

[dependencies]
fsys = "1.1"

With the async layer:

[dependencies]
fsys = { version = "1.1", features = ["async"] }

Cargo features

Minimum supported Rust version

1.75. Through the 1.x line, MSRV bumps are allowed only in 1.x.0 minor releases (within the 12 most recent stable Rust versions at release time). Patch releases never bump MSRV. See docs/STABILITY-1.0.md for the full policy.

Highlights by release

The full per-version delta lives in CHANGELOG.md. Headline capabilities by release:

Documentation

• API reference: https://docs.rs/fsys
• 33 runnable examples: docs/EXAMPLES.md — catalogues every example in examples/ with a "when to use this pattern" guide.
• Architecture overview: docs/ARCHITECTURE.md
• Method matrix + Auto decision ladder: docs/METHODS.md
• Performance targets + tuning: docs/PERFORMANCE.md
• Crash-safety contract per method: docs/CRASH-SAFETY.md
• Per-platform behavior + capability requirements: docs/PLATFORM-NOTES.md
• SPDK setup guide (1.1.0): docs/SPDK.md — hardware requirements, system setup, capability probe, and per-SpdkSkipReason remediation steps.
• Benchmark methodology + results: docs/BENCH.md
• Public-API reference: docs/API.md
• Per-version migration deltas: CHANGELOG.md

LICENSE

Licensed under the Apache License version 2.0 [ LICENSE-APACHE ], or the MIT License [ LICENSE-MIT ]; otherwise known as the ("License Agreement"); you are permitted to use this software, its source code, documentation, concepts, and any of the associated contents, within the limitations defined by the "License Agreement".