# Changelog
All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
> Pre-1.0 caveat: while we are still on `0.x`, every minor version (`0.x.0`)
> may contain breaking API or on-disk changes. Read the **Breaking changes**
> section before upgrading.
## [0.5.0] - 2026-08-27
Adopts `btree-ondisk 0.20.0` and exposes the reachability enumeration a
directory needs, so a reclaimer can be told which storage a directory still
occupies instead of guessing. Also fixes a latent alignment hazard on the bmap
open path.
Everything here is relative to `0.4.0`.
### Breaking changes
- **`btree-ondisk` `0.18.1` → `0.20.0`, `hyperfile` `0.6.10` → `0.6.11`.**
These move together and consumers must move with them. This crate implements
`hyperfile` traits whose signatures mention `btree-ondisk` types
(`BtreeNodeDirty`, `BMap`) and passes `hyperfile`'s block loader into its own
bmap, so `hyperdir` and `hyperfile` must resolve to **one** `btree-ondisk`
version. Two copies in one dependency graph do not compile ("found a different
`BtreeNodeDirty`"; `DirFileEntryRaw: NodeValue` and `L: BlockLoader<u64>`
unsatisfied). `hyperfile 0.6.11` pins the same `0.20.0`. No on-disk change.
### Added
- **Directory reachability**: `HyperDir::fs_reachable_meta_ptrs()` returns every
external storage location a directory occupies, and
`HyperDir::fs_reachable_segments()` returns the same answer as a set of
staging segment ids (decoded here, since a bare `BlockPtr` cannot be decoded
by a caller). Also on `HyperDirFile` as `reachable_meta_ptrs` /
`reachable_segments`.
A directory's B-tree leaves hold `DirFileEntryRaw` *inline payload*, not
storage locations, so the bmap's metadata nodes are the only external storage
it occupies — that set is the complete reachable set, and the walk stops one
level above the leaves, fetching no leaf node. Backed by `btree-ondisk`'s
`collect_meta_ptrs`; the sibling `collect_ptrs` is not usable here and returns
`InvalidInput`, because this crate's `V` differs from its `P`.
Both entry points are **fail-closed**: an `Err` means reachability is unknown
and a caller must reclaim nothing, never a short result. A generic reclaimer
pointed at a `DIR/<uuid>` container must use these rather than `collect_ptrs`.
### Fixed
- **The bmap root buffer could be rejected for misalignment.** `BMap::read`
parses the root header through `BtreeNode::from_slice_ref`, which requires an
8-byte-aligned buffer and returns `InvalidInput` otherwise. Two call sites
(opening a directory, and reloading to the latest bmap) copied `i_bmap` into a
local `[u8; N]` first — alignment 1, so they were correctly aligned only by
luck of stack layout. Both now pass `&raw_inode.i_bmap`, which sits at an
8-aligned offset inside `repr(C, align(8))` `InodeRaw`. (`BMap::read` copies
the bytes into its own aligned buffer, so passing the field reference retains
no borrow.) Pre-existing since the alignment check predates `0.20.0`.
### Tests
- `tests/e2e_reachability.rs`: a populated, compacted, reopened directory
reports a non-empty duplicate-free set of metadata-node locations resolving to
at least one staging segment (40 children → 7 nodes; 80 → 14); the answer is
stable across repeated calls; and growth never reports *fewer* nodes than
before while every child stays readable — the regression class that motivated
the upstream API.
- Unit tests pin the two properties the implementation depends on:
`collect_ptrs` is refused with `InvalidInput` for this crate's `V != P` shape
while `collect_meta_ptrs` is served, and `InodeRaw` is 8-aligned with `i_bmap`
at an 8-aligned offset.
## [0.4.0] - 2026-08-27
The release that turns this crate from a directory container into a POSIX-shaped
namespace: hard links, cross-directory rename, two-phase delete, advisory locks,
extended attributes, inline small files, and the crash-recovery machinery that
makes those safe when several mounts race on the same tree. Most of the work
landed as fixes to concurrent-access bugs found by multi-client tests against
real S3.
Everything here is relative to `0.3.0`, which is the last published release.
### Breaking changes
- **On-disk layout is not compatible with `0.3.0`.** Directories and files are
addressed by UUID, scatter objects are named suffix-first
(`.inode` / `.tombstone` / `.link` / …) and keyed by name rather than by a
CRC64 hash of the name. There is no migration; this is intended for fresh
trees.
- **`hyperfile` requirement moves from `^0.3` to `0.6.10`.** `hyperfile 0.4.0`
reordered `InodeRaw`'s on-disk layout — `i_size`, `i_blocks`, `i_last_seq`,
`i_last_cno` and `i_bmap` are now a contiguous tail so that tail can double as
the payload of an inlined small file. The struct is still 160 bytes but the
serialized offsets differ, so an inode written by a `0.3.x`-based tool is
misread by `0.4.0` and later, and vice versa. `InodeRaw` is embedded in the
dirents this crate reads and writes, so both sides must be built against the
same `hyperfile`.
- **`fs_unlink` / `fs_unlink_with_interceptor` changed meaning.** They used to
delete a directory container identified by a URI. They now unlink a *name*
within a parent directory:
`fs_unlink(client, layout, bucket, parent_dir_uuid, name, …)`.
- **`read_entry` / `fs_read_entry` take a name, not a hash.** The parameter went
from `hash: &EntryNameHash` to `name: &str`, following the name-first scatter
layout. `read_entry_raw` is the `DirFileEntryRaw` variant.
- **`read_dir` / `fs_read_dir` take `&self`, not `&mut self`.** Reading a
directory no longer needs exclusive access; compaction is the only `&mut`
operation.
- **`path_encode` gained a `uuid: &Uuid` parameter**, since a scatter key is now
scoped to the directory's UUID.
- **`DEFAULT_DIR_INODE_MARKER` is gone.** Scatter objects are named by suffix
now: `DEFAULT_DIR_INODE_SUFFIX` (`.inode`) and
`DEFAULT_DIR_TOMBSTONE_SUFFIX` (`.tombstone`), plus `.link` for a link intent.
### Added
- **Namespace operations.** `fs_create_file` (namespace entry returning a
byte-I/O handle), `fs_create_file_at`, hard links with authoritative `nlink`
kept in the file inode, same-directory and cross-directory rename, rename that
replaces an existing destination, `fs_reclaim`, `fs_exchange` (atomic
same-directory swap, `RENAME_EXCHANGE`), root bootstrap and `rmdir`.
- **Inline small files, two levels.** A tiny file lives entirely inside its
inode, with no separate data object. Second-level inline ("blob") files extend
this to the directory namespace, with `fs_update_inline` and
`fs_update_blob_meta` for metadata-only dirent updates.
- **Two-phase delete (`PreDelete`).** `unlink` publishes an intent and defers the
tombstone work to compaction, so the expensive part is not on the caller's
path.
- **S3-backed advisory file locks** (`lock` module, `_lock` sidecar object,
global lock mode), with a shared clock derived from S3 `Date` response headers
so lease expiry does not depend on client clocks. The lock is an unfenced TTL
lease — documented, with a lease-expiry test.
- **Extended attributes**, stored as sidecar objects with last-write-wins
semantics.
- **Open leases for open-but-unlinked files across mounts.** Layout keys for the
lease, `fs_gc` defers keep-open files that hold a live lease, and the GC TTL is
a settable process-wide value.
- **Tree-wide maintenance.** Directory enumeration, UUID-level orphan GC
(`fs_gc_orphans`), crash-safe reclaim of rename-displaced children via a
reclaim intent, crash-safe `fs_link` via a `.link` intent, and per-key intent
recovery with a key classifier and the `.unref` orphan scheme.
- **`HyperDirLayout::heartbeat_key`** for mount liveness heartbeats.
- **Handle-less name resolution** (`fs_resolve_entry_fast`), which drops the
lookup's redundant inode GET: zero inode GETs while a name still has pending
scatter, one on the compacted fallback, versus two for open-then-resolve.
- **Opt-in S3 fault injection** behind the `fault-injection` feature
(`S3FaultInterceptor`), matching by operation name and firing by request count
so an injected run is reproducible. Absent from a normal build.
### Fixed
- **A rename dropped the blob version pointer** — data loss for blob files.
- **A partial scatter listing was returned as if complete.** A failed
`ListObjectsV2` page was silently skipped, so a name's newest
`PreDelete`/`Delete` could be missed and an unlinked file would reappear.
SQLite's rollback-journal pattern hit this and failed transactions with
"disk I/O error". Page errors now propagate.
- **`read_dir` order was arbitrary and unstable.** Entries were merged into a
`HashMap` and returned via `.into_values()`, so two calls could see different
orders. Callers that paginate by index (a FUSE `readdir` cookie) then re-sent
some entries and skipped others. The merged list is sorted by name, which is
invariant under compaction since folding never renames.
- **`unlink` nlink accounting is now exactly-once**, and **hard link is safe
under concurrent same-name races** (previously an nlink leak). `ctime` advances
on any link/unlink `nlink` change.
- **Cross-directory rename is move-once**, source-scoped — concurrent renames of
the same source could previously lose data.
- **Same-name identity races.** Scatter resolution is identity-aware, so a
delete tombstone cannot shadow a same-name recreate, and displaced same-name
identities are reclaimed so create/rename races never orphan.
- **Staleness against the concurrent compactor** in `readdir`, `lookup` and
`rename`; scatter fetch and `fs_link` tolerate compaction landing underneath
them.
- **Fail-closed recovery.** Orphan GC and reclaim recovery now error out when a
directory's or destination parent's references cannot be read, instead of
proceeding on a partial view.
- **Per-object `DeleteObjects` failures are detected** in
`remove_scatter_inodes`. `DeleteObjects` is a partial-success API, so an `Ok`
response can still carry per-key errors.
- **CRC64 directory-key collisions** are resolved by open addressing.
- **S3 SDK errors map to precise `io::ErrorKind`s** rather than collapsing into
`Other`.
### Changed
- A same-directory rename commits its scatter fold and the rename in **one**
flush instead of two. Measured against real S3 on the write-temp-then-rename
pattern that git, rsync and editors use: median 842ms → 676ms (~20%). A rename
over a directory with no pending scatter is unchanged, since that case never
flushed twice.
- `tokio` features narrowed; indentation normalized; clippy clean with
`-D warnings`.
### Tests
- 15 unit tests plus four end-to-end suites against real S3, all `#[ignore]`d and
run explicitly: `e2e_s3` (21 cases, scatter and crash-recovery heavy),
`e2e_concurrent` (18 cases, 2/4/8-client races over create, link, unlink,
rename, rmdir, compaction, orphan GC, reclaim, xattr and locks), `e2e_blob`,
and `e2e_fault` (needs `fault-injection`).
### Known limitations
- The `wal` feature's three `HyperTrait` methods (`wal_set_mem_segment`,
`wal_clear_mem_segment`, `wal_spawn_delete_segment`) are still `todo!()`, so
enabling `wal` panics on the first flush. It compiles — which is why a consumer
that enables `hyperfile/wal` must also enable `hyperdir/wal` to satisfy the
trait — but it must not be exercised on a write path.
- The advisory lock is an unfenced TTL lease: a holder that stalls past its lease
is not prevented from acting on stale state.
## [0.3.0] - 2026-05-27
### Changed
- `hyperfile` `0.2` → `0.3`, `btree-ondisk` `0.16` → `0.18`.
- Narrowed `tokio` features, typo and indentation cleanups, clippy fixes.
## [0.2.0] - 2025-09-23
First packaged release: directory contents held in a hyperfile B+tree, `stat`
updates, a flush lock, the `wal` feature gate, and crate metadata (README,
license, manifest).