am-fs-ext4 0.5.0

Pure-Rust ext4 filesystem driver. Exposes a C ABI (fs_ext4_*) suitable for FFI from C/C++/Go/etc.
Documentation

am-fs-ext4 — pure-Rust ext2/3/4 driver

Pure-Rust read/write driver for the ext2, ext3, and ext4 on-disk formats. Mounts disk images and block devices, journals writes through JBD2, replays the journal on dirty mounts, and exposes a stable C ABI (fs_ext4_*) so any FFI host (Swift/C/C++/Go/…) can link libfs_ext4.a and #include "fs_ext4.h". MIT-licensed. Zero kernel calls; zero non-MIT/BSD/Apache dependencies. Runtime crates are crc32c, bitflags, unicode-normalization and caseless (casefold), plus the sister am-fs-core block-device crate.

Designed for FFI: the C ABI is stable and the static library has no host-specific assumptions, so the same libfs_ext4.a works equally well in a macOS FSKit extension, a Linux FUSE binary, a Windows tool, or any embedded environment with a BlockDevice shim.

Status

Read/write driver for the common case across ext2, ext3, and ext4. Mount + read is exhaustive against the ext4 feature matrix we test against; write is journaled through JBD2 and crash-safe under fault-injection sweeps for every multi-block op the driver exposes. Specific gaps are listed under "What doesn't work" below.

  • 700+ automated tests — 200+ lib unit tests and 450+ integration tests across ~100 test binaries (cargo test --release).
  • All 15 multi-block write ops (5.2.1–5.2.15 of the write-support plan) committed atomically through the JBD2 writer with explicit crash-safety sweeps.
  • ext2 and ext3 RW are unlocked (Phase 9.1/9.2/Phase B) via the same flavor-aware extent/indirect dispatcher.

Features

Read

  • Mount ext2 / ext3 / ext4 images and block devices (block sizes 1 KiB, 2 KiB, 4 KiB).
  • Inode parse with extra-fields support; stat, readdir, readlink, read, getxattr, listxattr.
  • Extent trees (depth 0, depth 1, depth ≥ 2 read; uninitialized extents present as zero on read).
  • Indirect-block trees (legacy ext2/3 — direct, indirect, double-indirect, triple-indirect).
  • HTree directory traversal (legacy / half_md4 / tea hash).
  • Inline data (≤ 60 bytes inline file body, plus inline xattr overflow).
  • External xattr blocks; ACL attributes (system.posix_acl_*).
  • metadata_csum and csum_seed verification on every block that has an on-disk checksum (superblock, BGD, inode table, dir tail, extent tail, htree tail, xattr block, journal commit/descriptor blocks).
  • JBD2 journal replay on mount: descriptor / commit / revoke blocks all honoured, including v2 (csum-tail) format.
  • Read-only audit pass (Filesystem::audit) reconciles link counts and reports dangling directory entries.
  • Optional LRU block cache (CachingDevice) for callback-mode hosts where every read hits a remote/RPC boundary.

Write

All mutating ops route through the same multi-block transaction buffer and commit atomically through JournalWriter:

  • chmod, chown, utimens, in-inode setxattr / removexattr.
  • create, unlink, mkdir, rmdir.
  • link (in-place), symlink (inline + slow path), rename (POSIX semantics: EEXIST on no-clobber, EINVAL into own subtree, cross-parent .. updates).
  • truncate shrink and grow (sparse-grow leaves holes; reads return zeros).
  • setxattr / removexattr on the external xattr block (alloc on first overflow, free when the block becomes empty).
  • File replace (full-content overwrite via fresh extent allocation).
  • fallocate(KEEP_SIZE), fallocate(PUNCH_HOLE), fallocate(ZERO_RANGE).
  • Extent-tree mutation: depth 0 → 1 promotion and depth-1 inserts.

Crash-safety guarantees: the four-fence JBD2 protocol (journal-write → dirty-flag → final-write → clean-flag) is enforced per transaction with an explicit flush between each fence. A post-remount state is always either pre-op or post-op — never torn. Pinned by parameterized fault-injection sweeps (see "Test contract" below).

Filesystem variants

Variant Read Write
ext2 done done (indirect-block extents, no journal)
ext3 done done (indirect-block extents, JBD2)
ext4 done done (extent trees, JBD2)

ext3 RW landed via Phase B's flavor-aware journal dispatch (jbd2::journal_block_to_physical and JournalWriter::open both branch on EXTENTS_FL). ext2 has no journal but otherwise shares the same write helpers.

What works

Per-operation, on a clean image:

  • mount / mount_rw / mount_with_callbacks / mount_rw_with_callbacks / mount_rw_with_callbacks_lazy (lazy variant defers journal replay).
  • stat, readdir, readlink, read.
  • getxattr, listxattr, setxattr (in-inode + external block), removexattr (in-inode + external block).
  • chmod, chown, utimens.
  • create, unlink, mkdir, rmdir, link, symlink, rename.
  • truncate (shrink and grow).
  • write (replace-content; allocates fresh extents and frees old ones in one journaled transaction).
  • fallocate(KEEP_SIZE), fallocate(PUNCH_HOLE), fallocate(ZERO_RANGE).
  • audit (read-only fsck — link counts, dangling entries).
  • fsck_run (callback-mode audit exposed through the C ABI).
  • format_filesystem (in-process mkfs; the C-ABI entry FFI hosts call to format a fresh volume is fs_ext4_mkfs).

What doesn't work

Honest gap list — pulled from docs/ext4-full-write-support.md and the in-tree TODOs:

  • Extent-tree depth ≥ 2 mutation. Read works; write refuses with a structured error in extent_mut.rs. Phase 4 of the write-support plan covers the design (docs/extent-tree-depth2-design.md).
  • HTree internal split. Leaf split is implemented; once a directory grows past the depth-1 leaf-block capacity (~340 extents on 4 KiB blocks), further inserts return ENOSPC-shaped errors.
  • extend_dir_and_add_entry buffer-twin. The fallback path used when an in-place dir-entry insert can't fit still does direct disk writes rather than going through BlockBuffer. Refactor pending (~150 LoC of plan_promote_leaf adaptation).
  • JBD2 journal modes. The writer effectively runs data=ordered semantics today; data=writeback and data=journal are not selectable.
  • Orphan-list inserts on still-open unlink. The driver doesn't see open-fd state today; the host (FSKit / FUSE) would need to plumb that through. Reads + replay of pre-existing orphan chains is implemented.
  • Casefold lookups. The hash function is implemented in casefold.rs; HTree wiring is not.
  • fs-verity, fscrypt, project quota, user/group quota, online-resize, mmap shared writes — not implemented.
  • EA refcount sharing on external xattr blocks — single-owner only; never shares (deferred until a consumer needs it).
  • Indirect-path file replace is un-journaled. The indirect-block replace helper writes data blocks directly. Only applies to ext2 mounts (which have no journal anyway) and ext3 mounts that disable the journal.

Architecture

The driver is split into read modules, write modules, and the JBD2 / transaction layer that ties them together.

  • block_io — BlockDevice trait. File-backed and callback-backed implementations both live here; CachingDevice wraps either for LRU read caching.
  • superblock, bgd, inode, features — on-disk parse + validate. All reject malformed geometry with structured errors; cargo fuzz-style harness in tests/fuzz_smoke.rs proves no panic on truncated / zeroed / bit-flipped inputs.
  • extent.rs vs indirect.rs — the two ways an inode can map logical blocks. Read paths dispatch on EXTENTS_FL (indirect::map_logical_any / extent::lookup).
  • extent_mut.rs, indirect_mut.rs — mutation planners. Compute "what the new tree would look like" without touching the device, then hand the planned writes to a BlockBuffer.
  • BlockBuffer (in fs.rs) — accumulates bitmap, BGD, SB, inode-table, dir-block, xattr-block, and payload-data mutations for a single high-level op. commit_block_buffer routes the whole buffer through JournalWriter when a journal is present, falling back to direct writes when not.
  • JournalWriter (journal_writer.rs) — owns the JBD2 inode's extent map, allocates a fresh transaction slot, emits descriptor + data + commit blocks with the four-fence protocol, and rewrites the super-state to clean once the final-write phase completes.
  • DeepReader — read-side helper that consults the in-flight transaction's pending writes before going to disk, so post-write reads inside the same transaction return the new data.
  • verify.rs — structural verifier reconciling bitmap vs. inode-tree-claimed blocks; used both in tests and in the read-only audit FFI surface.
  • capi.rs — C ABI. Thread-local error state with errno inference (fs_ext4_last_error(), fs_ext4_last_errno()); catch_unwind fence at every entry so a panic in the Rust interior never crosses the FFI boundary.

Deeper write-up of each phase lives in docs/:

  • docs/IMPROVEMENT-PLAN.md — triaged stability / hardening backlog.
  • docs/ext4-full-write-support.md — the 9-phase write-feature plan this README's roadmap mirrors.
  • docs/extent-tree-depth2-design.md — the index-block split design for Phase 4.
  • docs/TEST-DISKS.md — fixture / feature coverage matrix.

Spec sources: the on-disk format documentation on kernel.org's ext4 wiki, plus Carrier, File System Forensic Analysis (Addison-Wesley, 2005). Research-reference Rust/C code is yuoo655/ext4_rs (MIT) and lwext4 (BSD-2-Clause); both credited in the License section.

Test contract

  • Lib unit tests: across most modules (alloc, bgd, block_cache, block_io, casefold, checksum, dir, extent, extent_mut, features, fs, hash, htree, htree_mut, indirect, indirect_mut, inline_data, inode, jbd2, journal_apply, journal_writer, mkfs, path, superblock, transaction, verify, xattr, …).
  • Integration tests: ~100 test binaries in tests/, covering every C ABI entry point, end-to-end mutation round-trips, journaled-path regressions, htree large-dir growth, ACL parse, large-dir HTree, deep-extent reads, multi-tier indirect maps, ext2/3 RW round-trips, fuzz smoke, fsck audit, orphan recovery, sparse-grow, fallocate variants, and the cross-validators below.
  • Crash-safety sweeps (parameterized over 0..=N "drop the next write" budgets, asserting post-remount state is always either pre-op or post-op):
    • journal_writer_crash_safety.rs — chmod (0..=20 budget).
    • journal_writer_crash_dir_ops.rs — create / mkdir / link / symlink.
    • journal_writer_crash_rename_write.rs — rename, file-replace, external xattr.
    • journal_writer_truncate_shrink.rs — truncate_shrink (0..=30 budget).
    • journal_writer_unlink_rmdir.rs — unlink / rmdir (0..=40 budget).
    • fallocate_crash_safety.rs — KEEP_SIZE / PUNCH_HOLE / ZERO_RANGE.
    • orphan_recovery_crash_safety.rs — orphan replay budget sweep.
  • Cross-validators: tests/lwext4_cross_validate.rs (BSD-2-Clause reference, opt-in via env flag — built as a library, never linked by default); a FreeBSD-VM cross-validator (tests/vagrant/freebsd/, tests/qemu/) is in flight.
  • Structural verifier: crate::verify::verify reconciles the on-disk bitmap against every block claimed by the inode tree; pinned by tests/verify_basic.rs, also runnable as a post-mutation check from any integration test.

Roadmap

Pulled from docs/ext4-full-write-support.md — items not yet ticked. Numbering follows the plan doc.

  • 3.5 EA refcount sharing on external xattr blocks.
  • 4.1–4.6 Extent-tree depth ≥ 2 mutation (index-block split, recursive descent, merge-on-shrink, index csum).
  • 5.3.1–5.3.3 JBD2 journal modes (data=ordered explicit / data=writeback / data=journal).
  • 6.3 Orphan-list insert on still-open unlink (gated on host fd-tracking).
  • 6.4 Link-count audit auto-repair under a recovery transaction.
  • 7.2 Checked arithmetic in remaining hot sites (rare in practice; cosmetic).
  • 7.3 Full FFI input-validation sweep.
  • 7.4 Richer Error variants (cosmetic; Corrupt(&str) carries enough context today).
  • 8.2 Cross-call extent-lookup memoization.
  • 8.4 Coalesce adjacent dirty blocks inside the BlockBuffer.
  • 9.3 Casefold HTree wiring.
  • 9.4 Project quota.
  • 9.5 User/group disk quota.
  • 9.6 fs-verity.
  • 9.7 fscrypt v2.
  • 9.8 Online resize.
  • 9.9 mmap shared writes (host-integration dependent).
  • extend_dir_and_add_entry buffer-twin (the last direct-disk-write helper inside the otherwise-buffered write path).

Changelog

Highlights from the last 50 commits, grouped by date.

2026-09-04 — 0.5.0 — the formatter ships as a binary, and one type moves out

  • Breaking: CachingDevice is no longer part of this crate's public API. It was removed while deleting what a dead-code allow was hiding; the type lives in am-fs-core, where the other drivers already got it. The functionality moved rather than disappearing, but fs_ext4::block_io::CachingDevice no longer resolves — which for a 0.x crate is a minor bump, since cargo treats the minor as the compatibility boundary. Nothing outside this repo referenced it.
  • mkfs.ext4 is now published as a downloadable binary. A release job builds it, renames the cargo target mkfs_ext4 to the dotted name the format's convention uses — cargo refuses a dot in a target name — and attaches a per-platform archive, so a package manager can install it without a Rust toolchain.
  • A truncated inode read now fails closed. The checksum verifier had accepted a short read as a pass; a read that did not return the bytes is not a checksum that matched.
  • Four format-conformance fixes: a bigalloc filesystem is refused rather than misread (its cluster arithmetic is not implemented, so every block-group offset would be wrong); timestamps apply the epoch-extension bits and are read as signed, so dates past 2038 and before 1970 are no longer 136 years out; a missing inline-data spill xattr is corruption rather than a silently truncated file; and a writable mount of an MMP filesystem is refused, read-only being unaffected.
  • The C attribute struct's timestamps widen to int64_t, moving every field after them. Consumers must recompile, not relink.
  • Dependencies move to am-fs-core 0.2.4.

2026-08-29 — 0.4.1 — the multi-group formatter in front of e2fsck

  • mkfs output is now checked against e2fsck at three and five block groups, including a short final group, and both sizes run in CI. No bug was found: the arithmetic was right, it had simply never been checked.
  • Three mkfs.ext4 CLI bugs: -c consumed the device path and then failed about the path it had just eaten; -b reported a bad block size only after opening the device; -q suppressed warnings only when it came first.
  • A rebuilt block bitmap reserves what it must not hand out, and a group's uninitialised flags are cleared when it is first allocated from.

2026-06-30 — 0.4.0 — multiple block groups + write-path/mkfs correctness

  • Multiple block groups for ext4 (full GDT + per-group bitmaps/inode tables; sparse-super backups for e2fsck -b recovery).
  • Correctness: on-write metadata checksums + SB free-count coherency; dir/xattr/pwrite write-path hardening; 1 KiB-block & ext3 mkfs fixes (validated by the new in-process + e2fsck oracle harnesses).
  • (The deep-dir-extents / casefold / fs_ext4_mknod / fs_ext4_set_flags / extended-fs_ext4_attr_t features shipped in 0.3.0 — see CHANGELOG.md.)

2026-05-03 — Phase 1+3+5+6 rollup, ext2/3 RW, crash sweeps

  • 64d6e6b crash-safety sweeps for the remaining multi-block ops.
  • c800223 regression tests pinning rename / write / external-xattr to the journaled path.
  • a9fe115 Phase 5.2.5 / 5.2.10 / 5.2.15 — finish 5.2.x multi-block journal coverage.
  • 7f51ece Phase 5.2.9 + 5.2.14 — unlink + rmdir multi-block journaled.
  • 455438d Phase 5.2.8 / 5.2.11 / 5.2.12 / 5.2.13 — create / link / symlink / mkdir journaled.
  • 165391c Phase 5.2.6 truncate_shrink + multi-block tx (introduces BlockBuffer).
  • df526b5 Phase 1+3+5 write-path foundation — journaled writes, external xattr blocks.
  • 5c4a2f8 Phase 2.3 + 2.4 — fallocate punch-hole + zero-range.
  • 77a9ef3 Phase 2.2 — fallocate(FALLOC_FL_KEEP_SIZE).
  • 8edbd98 Phase 6.2 — orphan replay.
  • 92fdcb9 Phase 6.1 — orphan list parser.
  • 487eebd Phase 6.2 crash-safety probe sweep.
  • 47dfc33 ext3 RW unlocked via flavor-aware journal dispatch (Phase B).
  • 76b36d8 write s_journal_inum at 0xE0 (not 0xD8) so ext3 images mount.
  • 58558ed Phase 8.1 + 8.3 — LRU block cache + vectorized bitmap scan.
  • 8140470 Phase 7.5 write-path fuzz coverage + clippy fix in verify.rs.
  • 5a0bfdf Phase 1.2 + Phase 4 design doc.
  • 1baa872 license sweep — scrub references to copyleft-licensed Linux/NTFS tooling.
  • 9d2f2ff 72bbad4 direct-qemu FreeBSD cross-validator scaffolding.

2026-05-01 — mkfs_ext4 binary, lazy replay, fsck FFI

  • b788435 54ec273 standalone mkfs_ext4 binary (--create-size for one-shot create+format).
  • 7f64a38 gate Unix-only FileTypeExt behind cfg(unix) for Windows builds.
  • 987f8d5 fs_ext4_mount_rw_with_callbacks_lazy + fs_ext4_replay_journal_if_dirty for deferred journal replay.
  • 2679e60 expose fs_ext4_fsck_run + audit_with_callbacks.
  • 60974c2 CI validation of the binary's output.

2026-04-30 — Sandboxed RW callbacks

  • f08ec2e fs_ext4_mount_rw_with_callbacks for sandboxed FSKit consumers.

2026-04-22 — 0.1.3, pre-commit hook

  • b182836 release 0.1.3.
  • 4352d37 pre-commit hook (cargo fmt --check + clippy).

2026-04-20 — 0.1.1 / 0.1.2 releases

  • 8248ff4 0.1.2: expose s_state via fs_ext4_volume_info_t.mounted_dirty.
  • 47dfbb5 0.1.1: docs rewrite, neutral framing, plain-English disclaimer.

2026-04-19 — Stability hardening, perf, fsck audit

  • f8603e9 hard-cap extent tree depth; refuse cycles + spec overflow.
  • ed8223c checked arithmetic on physical block offsets.
  • b7f805f reject malformed geometry; never panic.
  • e1d2c82 CachingDevice — optional LRU read cache.
  • c684481 word-at-a-time scan in find_first_free.
  • c5ea3ba memoize last extent across a single read call.
  • 1e9bda4 read-only link-count audit (Filesystem::audit).
  • 708dcff fuzz-smoke harness (dir-walk, parser-fuzz, exhaustive bit-flip).
  • e1c18e6 IMPROVEMENT-PLAN.md — triaged gaps, phased roadmap.
  • d731e71 pin rust-toolchain to 1.94.1.

2026-04-18 — Crate rename, test-disk matrix

  • a8bcaa4 rename crate ext4rs → fs-ext4, C ABI ext4rs_* → fs_ext4_*.
  • 2af0108 add TEST-DISKS.md — feature coverage matrix.

Full changelog in CHANGELOG.md.

License

MIT — see LICENSE. Copyright (c) 2026 Chris Thomas.

Research references credited under their own licenses:

  • yuoo655/ext4_rs — MIT.
  • lwext4 — BSD-2-Clause. Used opt-in as a cross-validator (tests/lwext4_cross_validate.rs), never linked into the shipping binary.

Spec sources: kernel.org ext4 wiki documentation; Carrier, File System Forensic Analysis (Addison-Wesley, 2005). The driver does not derive from any GPL/LGPL/AGPL source.

Building

Standard cargo:

cargo build --release
# produces target/release/libfs_ext4.a (static lib for FFI consumers)
#         + target/release/libfs_ext4.rlib
#         + target/release/mkfs_ext4 (the standalone formatter binary)

Cross-compile to a specific target the usual way:

cargo build --release --target aarch64-unknown-linux-gnu
cargo build --release --target x86_64-pc-windows-gnu

Platform-specific packaging (macOS lipo for a universal static archive, an Xcode .xcframework, deb/rpm/Homebrew formulae) belongs in the consuming project. fs-ext4 itself stays portable cargo — no platform-specific build scripts.

Using from C

Link libfs_ext4.a and include fs_ext4.h:

#include "fs_ext4.h"

fs_ext4_fs_t *fs = fs_ext4_mount("/path/to/disk.img");
if (!fs) {
    fprintf(stderr, "%s\n", fs_ext4_last_error());
    return 1;
}

fs_ext4_attr_t attr;
if (fs_ext4_stat(fs, "/hello.txt", &attr) == 0) {
    printf("size=%llu mode=%o\n", attr.size, attr.mode);
}

fs_ext4_umount(fs);

See examples/capi_demo.rs for the Rust-side equivalent.

Using from Rust

[dependencies]
am-fs-ext4 = "0.3"

use fs_ext4::Filesystem;

let fs = Filesystem::mount("/path/to/disk.img")?;
let attrs = fs.stat("/hello.txt")?;

Testing

cargo test --release

Integration tests use ext4 image fixtures under test-disks/. Fixtures are gitignored — regenerate them with:

bash test-disks/build-ext4-feature-images.sh

The generator runs standard formatter tools inside a short-lived Alpine Linux VM booted under qemu-system-x86_64, so the same script works on macOS, Linux, and in CI (no Docker required). First run downloads the Alpine virt ISO + kernel (~75 MB, cached under test-disks/.vm-cache/).

Git hooks

One-time setup per clone, so every commit runs the same cargo fmt --check + cargo clippy checks CI does:

./scripts/install-hooks.sh

Bypass a single commit with git commit --no-verify.

Disclaimer — use at your own risk

Read this before pointing the crate at anything you care about.

This is experimental filesystem code that reads and writes the on-disk structures of live filesystems. Bugs in this class of code can — and sooner or later will — corrupt or destroy data. The MIT license above already contains the standard no-warranty and limitation-of-liability clauses; this section restates them in plain English so there is no ambiguity about what you are agreeing to when you use the software.

By using this software you accept that:

  • The author(s) and contributors provide this crate as is, with no warranty of any kind, express or implied — including but not limited to warranties of merchantability, fitness for a particular purpose, correctness, data integrity, durability, security, or non-infringement.
  • The author(s) and contributors are not liable for any loss, damage, or expense of any kind arising out of or related to your use of the software. This explicitly includes (non-exhaustively) lost or corrupted data, corrupted filesystems, volumes that will no longer mount, hardware damage, downtime, lost revenue, missed deadlines, support costs, or any direct, indirect, incidental, special, consequential, or punitive damages — regardless of the legal theory under which such damages might be sought.
  • You are solely responsible for backing up any data that could be touched by this software before running it. The only safe workflow when experimenting with an unofficial filesystem driver is: work on disk images or on copies, never on your only copy of anything irreplaceable.
  • If that is not acceptable to you, do not use this software.

This disclaimer is a plain-English restatement of the license terms above, not a separate license. The license terms apply in full.