fsys 0.9.8

fsys is a foundation-tier filesystem IO crate for Rust storage engines, embedded databases, and durable services. It pairs an explicit durability model with a journal substrate, io_uring on Linux, NVMe passthrough, and atomic-replace writes — sitting one layer below your data structures and one layer above std::fs.

It is not trying to replace std::fs for ordinary application code.

Quickstart

use std::sync::Arc;
use fsys::{builder, JournalOptions};

fn main() -> fsys::Result<()> {
    // Build a handle once, share via Arc.
    let fs = Arc::new(builder().build()?);

    // Open an append-only journal &mdash; the WAL primitive.
    let log = fs.journal("/var/lib/myapp/log.wal")?;

    // Append many records without per-call fsync.
    let _ = log.append(b"txn 1: insert")?;
    let _ = log.append(b"txn 2: update")?;
    let lsn = log.append(b"txn 3: commit")?;

    // One fsync covers every prior append &mdash; group-commit.
    log.sync_through(lsn)?;

    Ok(())
}

For one-shot file IO (atomic-replace, durable), fsys::quick::write / read skip the handle:

fsys::quick::write("/etc/myapp/config.toml", b"value = 42")?;
let data = fsys::quick::read("/etc/myapp/config.toml")?;

See examples/ (17 runnable patterns) and docs/EXAMPLES.md for the full catalogue.

At a glance

• Five durability methods — Sync, Data, Mmap, Direct, and hardware-aware Auto. Every method is platform-honest: the actual primitive in use is observable via Handle::active_durability_primitive().
• Journal substrate — open-once append-only log with atomic LSN reservation, group-commit fsync, and a CRC-32C-protected frame format. Three throughput tiers (sync, lock-free concurrent, native io_uring async on Linux). The HiveDB-class WAL primitive.
• Atomic-replace writes — every write / write_copy / Batch::commit uses temp-file + atomic rename. The target is either entirely the old payload or entirely the new payload — never torn.
• Linux io_uring on the hot path — Method::Direct and the journal Direct-IO path submit through io_uring with IORING_OP_WRITE_FIXED against pre-registered buffer slots. Falls back to O_DIRECT + pwrite + fdatasync cleanly when io_uring is unavailable.
• NVMe passthrough flush — on Linux (NVME_IOCTL_IO_CMD) and Windows (IOCTL_STORAGE_PROTOCOL_COMMAND) when the hardware supports it. Transparent fallback to fdatasync / WRITE_THROUGH otherwise.
• Cross-platform reflinks — macOS clonefile(2) + Windows FSCTL_DUPLICATE_EXTENTS_TO_FILE give APFS / ReFS instant copy-on-write semantics. Multi-GiB checkpoint clones drop from seconds to microseconds.
• Optional async layer (async feature) — every sync method gets an _async sibling. On Linux + Method::Direct, async ops submit directly to the per-handle io_uring ring (no spawn_blocking thread-pool hop).
• Hardware-aware tuning — PLP detection, NAWUN/NAWUPF probe (atomic-write unit), Builder::tune_for(Workload::Database) preset, runtime CPU-feature detection for hardware CRC-32C.

When to use fsys

The "fair comparison" for durable writes is fsys::Sync versus std::fs plus a manual temp-file + sync_all + rename dance — the latter is what most application code gets wrong. fsys provides this as a single public API call.

Performance

Numbers below were captured on windows-ntfs-nvme (Windows 11 Pro, x86_64, local NVMe SSD; std::env::temp_dir() resolves to NTFS) with 100 timed iterations after 10 warmup. Run-to-run noise is roughly ±5% on this host class. Full methodology, additional payload sizes, and Linux numbers live in docs/BENCH.md; reproduce locally with cargo bench.

Journal substrate vs atomic-replace

The headline result. Atomic-replace pays 5–7 syscalls per durable write; the journal opens once, appends without per-call fsync, and amortises durability across a sync_through call — the canonical WAL pattern.

At an intermediate cadence (sync every 100 appends), the journal still delivers 109–255× the atomic-replace throughput. See docs/BENCH.md for the full table including per-append sync cadence.

Atomic-replace write vs std::fs::write

fsys::Auto pays a deterministic durability cost; std::fs::write defers that cost to OS scheduling and pays it at p99 instead.

At 1 MiB, fsys::Auto is 3.3× faster than std::fs::write at p99 — durability paid up-front rather than at unpredictable points.

Read parity

The read path is essentially std::fs::read plus handle bookkeeping — no durability cost on reads.

tokio::fs::read is 1.5–4.4× slower than fsys::Auto because tokio's own fs module routes through spawn_blocking. On Linux + Method::Direct + the async feature, fsys's native io_uring substrate bypasses that thread-pool hop entirely.

Installation

[dependencies]
fsys = "0.9.8"

With the async layer:

[dependencies]
fsys = { version = "0.9.8", features = ["async"] }

Cargo features

Minimum supported Rust version

1.75. MSRV may be raised in any minor version before 1.0.0. After 1.0.0, MSRV bumps require a minor version bump.

Highlights by release

The full per-version delta lives in CHANGELOG.md. Headline capabilities by release:

Documentation

• API reference: https://docs.rs/fsys
• 17 runnable examples: docs/EXAMPLES.md — catalogues every example in examples/ with a "when to use this pattern" guide.
• Architecture overview: docs/ARCHITECTURE.md
• Method matrix + Auto decision ladder: docs/METHODS.md
• Performance targets + tuning: docs/PERFORMANCE.md
• Crash-safety contract per method: docs/CRASH-SAFETY.md
• Per-platform behavior + capability requirements: docs/PLATFORM-NOTES.md
• Benchmark methodology + results: docs/BENCH.md
• Public-API reference: docs/API.md
• Per-version migration deltas: CHANGELOG.md

LICENSE

Licensed under the Apache License version 2.0 [ LICENSE-APACHE ], or the MIT License [ LICENSE-MIT ]; otherwise known as the ("License Agreement"); you are permitted to use this software, its source code, documentation, concepts, and any of the associated contents, within the limitations defined by the "License Agreement".