EntropyFS

de Beer, R. (2026). EntropyFS: Entropy-Native Configurational Storage as a Filesystem Substrate - Broad Prior-Art Technical Disclosure and Research Architecture (Version v1.0). Zenodo. https://doi.org/10.5281/zenodo.22092869

Persist irreducible state. Materialize structure. Preserve exact bytes. Measure everything.

EntropyFS is a native-Rust, mountable Linux filesystem whose central research premise is:

Logical bytes are an interface presented to applications. They do not have to be the primary persisted representation. EntropyFS persists the minimum exact reversible entropy/configuration state necessary to reproduce those bytes.

The defining equation:

X = Materialize(D)

X is the exact logical byte sequence; D is the persisted representation descriptor. A more general family:

X = T(E(U, S, P)) ⊕ R

U = versioned entropy universe · S = seed/state · P = rank/coordinate · T = bounded reversible transform · R = exact residual · E = deterministic materialization.

EntropyFS does not claim to evade information theory: the SSD still stores physical bits. Its innovation is to make entropy state, mathematical rank/configuration, immutable references, deterministic generators, transforms, and irreducible residuals first-class storage representations instead of assuming logical byte blocks must themselves exist at rest. For random/encrypted/incompressible data it converges gracefully toward ordinary physical storage (RAW fallback) — that is a success condition, not a failure.

Current status

Measured results

All performance and storage-density claims are governed by docs/performance/methodology.md; admitted results live in evidence/performance/ (see evidence/performance/INDEX.md).

FUSE-frontend before/after pair (Phase 6, same workloads, same machine):

The fsync regression is the measured cost of deferred durability and is reported honestly; the before half reproduces the crashes the oversized-descriptor fix (Phase 6) eliminated.

What EntropyFS does and does not claim:

• The synthetic ablation fixture (evidence/ablation-2026-08-25.json) is an ablation fixture, never a headline: on a corpus containing four unique 64 KiB chunks its 16.876× is dominated by content-addressed dedup. The campaign's structured-corpus ratios (up to 1,328×) are structural + EXACT_REF aliasing + CAS object sharing — attribution is now measured, not labeled: per-run accounting separates cas_shared_bytes_saved (a store invariant) from exact_ref_bytes_saved (the gated alias representation), and A1 is pure byte rANS with SequenceRans as the post-registration E1 step (campaign-1787671040-923df7b/). The earlier “dedup = 0” and “dedup-dominated” statements conflated the two layers and are amended in evidence/performance/INDEX.md, never rewritten.
• The campaign's ablation evidence is two tables, both kept forever: the strict cumulative ladder A0–A8 (each step adds one mechanism) and the leave-one-out table (one mechanism disabled at a time). The first campaign's nine-row table is the leave-one-out table; it predates the two-table rule and is amended as such in evidence/performance/INDEX.md (protocol note, never rewritten).
• The campaign's DSFB investigation is a sealed three-era series, all with byte-identical physical representations: RANS-era 765.4 vs 334.7 MiB/s (2.29×, user CPU halved — 67d977a), SequenceRans-era 773.9 vs 717.1 MiB/s (~8%, b165d60), and CAS-era 1,120.8 vs 1,106.1 MiB/s (~1.3%, 923df7b). DSFB's marginal benefit collapsed as the SequenceRans floor simplified the search landscape — evidence for its assigned role as candidate-search budget intelligence, not compression; its marginal value now is small on this tiny synthetic corpus and its proper counters are deferred. Historical numbers are preserved in evidence/performance/INDEX.md; the 2.29× is not a current claim.
• The source-corpus progression is now sealed across eras: 923df7b pure byte rANS 1.633× / standalone SequenceRans 3.556× (with zstd-per-64KiB -1 at 3.739× — the per-extent floor was within 5%, and the gap to whole-file zstd was cross-chunk context), 8250f6b EntropyFS full 4.070× with SequenceDict on the packed stream, and the Phase-9C tree court answering the open mount-level question: on a real tree of separately-written files (279/282 single-chunk), per-file zstd -1 is 3.541× and EntropyFS per-file writes are 2.182×, rising to 2.328× after the shared-dict pass — so most of the packed-stream density was indeed cross-FILE structure, and the shared dictionary recovers a measured part of it. The rest of the gap to per-file zstd is the fixed single-anchor-per-directory v1 policy plus the greedy chain-16 matcher; both are recorded as the current state, not claims.
• The campaign's H2 experiment (synthetic drift corpus) is now a sealed controlled series: 67d977a +7.2% (RANS-era floor), a6641d1 −24% (SequenceRans floor, positional residuals only), 43bf17e +35.2% (SequenceRans floor + BASE_SEQUENCE shift-aware deltas), and 923df7b +40.6% (marginal costing). The shuffled control grows in the delta campaign because copy/literal deltas also exploit structural similarity between unrelated-history chunks — the control no longer isolates pure temporal causality, and that confounding is itself recorded as the finding.
• Random/encrypted/already-compressed data falls back toward RAW (urandom 0.997×, zstd -19 pack 0.993×) — the honest negative control.

Honesty rules

• A 128-bit seed does not "store" a gigabyte. Descriptor bits select at most 2^k states; every persisted bit is accounted (docs/theory/information-accounting.md).
• No hidden corpus, no network, no RNG in materialization, no CPU-dependent floating point. The universe specification is part of the format version.
• No arbitrary generator programs: the descriptor language is bounded and not Turing-complete (docs/adr/0005-representation-set.md).
• DSFB has zero decoding authority (docs/adr/0004-dsfb-observer.md).
• statfs reports physical capacity; effective ratio is an observation, never a promise (docs/adr/0018-statfs.md).
• Every optimization claim requires reproducible evidence (docs/performance/methodology.md).

Building

rustup toolchain install stable --profile minimal --component rustfmt,clippy
cargo build --release
cargo test
cargo clippy --all-targets -- -D warnings
cargo fmt --check

Target platform

CachyOS/Arch Linux, x86-64, FUSE (/dev/fuse + fusermount3; kernel CONFIG_FUSE_FS=y). No custom kernel, no out-of-tree module, no reboot.

Reading order

• docs/architecture/overview.md — architecture map
• docs/theory/entropy-medium.md — the information-theory boundary statement
• docs/format/ondisk-v1.md — the on-disk format
• docs/recovery/crash-consistency.md — the crash protocol
• docs/adr/ — all architecture decision records

License

MIT OR Apache-2.0, at your option.