# Changelog
All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
> Pre-1.0 caveat: while we are still on `0.x`, every minor version (`0.x.0`)
> may contain breaking API or on-disk changes. Read the **Breaking changes**
> section before upgrading.
## [0.10.4] - 2026-10-05
A compaction pass is no longer bounded by memory on a parted container.
### Changed
- **`relocate` writes partial segments past the dirty-data limits.** It used
to mark its blocks and return, so a pass held every block it moved in memory
until the caller's flush. After marking it now checks what a write checks —
a `PartedSegment` container, `parted_segment_enabled`, a limit or the
interval crossed, at least `data_cache_dirty_min_bytes_to_part` dirty — and
writes a partial when all hold. A pass of any size holds no more than the
limits at once and is still one checkpoint, marked `FLAG_COMPACTED` when
nothing else changed; `content_cno` is unaffected.
- **`relocate` still never publishes.** Unlike a write it does not fall back to
a flush where no partial can be written (`MicroGroup`, or parts turned off):
the blocks stay dirty until the caller's flush, as before, so a pass is never
split into two checkpoints. A partial that fails is returned as the error,
with the blocks still marked for the flush.
- Blocks a partial writes go to the read cache, within its limit, as a write's
do.
### Tests
- `functional_compaction`: a pass past 6-block limits in nine calls writes
partials, stays within the limits, publishes once and keeps the flag;
`MicroGroup` and parts turned off write and publish nothing (now 11).
- `integration_s3_compaction`: the same on S3 in chunks past 2 MiB limits, with
`fs_content_cno` and `fs_content_cno_fast` agreeing.
## [0.10.3] - 2026-10-05
Input hyperfile does not expect is an error, never a panic or a hang.
### Fixed
- **A staging URI that does not name a bucket and a path panicked.** Open,
create and unlink unwrapped its parse: `s3://`, `http://…`, an empty string,
and for create `s3://bucket` with no path. Both the root and inode URIs are
now parsed before any request is sent, and a bad one is `InvalidInput`,
quoting the URI and the reason. A URI with an empty bucket (`s3:///x`) is
refused by the parse itself instead of by the SDK, and
`StagingConfig::new_s3_uri` no longer turns `s3://` into `s3:/`.
- **A HeadObject without a usable `Date` or `Last-Modified` panicked, or made
open wait for good.** A writable open uses the server's time to decide
whether a checkpoint no inode points at yet is still being published. A
`Date` that was not RFC 2822 panicked, a missing `Last-Modified` panicked, and
a missing `Date` left the time at 0, so the checkpoint looked forever fresh
and the open never returned. Each is now `InvalidData`: guessing the time
could take up a checkpoint whose writer is still publishing it. Seen only
behind a proxy or with an S3-compatible store; S3 always sends both.
### Tests
- `integration_s3_bad_input`: each malformed URI through open, create and
unlink; a garbled or missing `Date` and a missing `Last-Modified` on the
HeadObject of a writable open over a pending checkpoint.
## [0.10.2] - 2026-10-04
Two WAL transaction fixes, a batch delete that no longer hides keys it could
not delete, and the error classification made public.
### Fixed
- **A file with a transaction left open right after a publish opened with a
panic, every time.** The transaction's marker limits recovery to the records
before it; with none, the replay published nothing and recovery's check that
the checkpoint moved failed. A crash or process exit inside such a
transaction was enough. The group is now set aside like the rest of an open
transaction, its records counted in `records_dropped`.
- **An abort that could not delete its log records could bring them back.**
The transaction marker was deleted in the same batch as the records, so S3
could delete the marker and refuse records, and the next open replayed, in
`Latest` mode, what the abort had discarded. The marker is now deleted last,
once everything else in the group is gone.
- **An abort that failed could not be asked again**: it closed the transaction
first, so a second abort was refused as "no transaction is open". A failed
abort now returns the error and leaves the transaction open.
- **A DeleteObjects that refused some keys was taken for success.** The 200
response lists each key not deleted, and only the status was read, so
`unlink` reported success with objects left behind and logged nothing. Each
refused key is now logged with its code and fails the call, with the kind of
the first failure. Running `unlink` again still does not finish the job —
the inode is already gone; see "Known gap: orphaned objects" in
`docs/posix.md`.
### Added
- `hyperfile::s3_error_kind`, `s3_code_kind`, `is_condition_failure` and
`is_retryable`, exported at the crate root, so a caller that also talks to
S3 classifies its own errors by the same rule. `s3_code_kind` maps an S3
error code — all a DeleteObjects reports per key — to the kind its status
has; `is_retryable` is the retry column of the table on `s3_error_kind`:
`ConnectionReset`, `TimedOut`, `ResourceBusy`.
### Tests
- `integration_reactor_s3_wal`: a transaction left open right after a publish,
in both recovery modes (now 37).
- `integration_s3_partial_delete`: DeleteObjects refusing keys with
`AccessDenied` and `SlowDown`, and refusing none.
- `wal::abort_tests` (crate-internal, `--features wal`, ignored, S3): an abort
whose delete is answered 503, and one whose records are refused part-way.
- `public_error_kind`: the exported functions from outside the crate; needs no
bucket.
## [0.10.1] - 2026-10-04
Error kinds a caller can decide retries by: the network is `ConnectionReset` or
`TimedOut`, a busy store `ResourceBusy`, a handle that has to be reopened
`StaleNetworkFileHandle`, and nothing worth retrying comes back as `Other`.
### Changed
- **A GetObject body that breaks off part-way is `ConnectionReset`**, not
`Other`: a dropped connection, the SDK's stalled-stream protection, or fewer
bytes than the response promised. The message names the key and range. A
response that arrived but could not be read (`SdkError::ResponseError`) is
`ConnectionReset` too.
- **A handle that has to be reopened is `StaleNetworkFileHandle`**, not
`Other` (see 0.10.0: a failed write whose block map could not be put back).
- **A busy store reaches a reactor caller as `ResourceBusy`**, not `Other`. The
handler read `ResourceBusy` as "requeued, answered later", which 0.10.0 worked
around by changing the kind; a requeue now carries a private marker, and the
handler tells it apart by that alone. The setattr, release and transaction
arms, which requeued on any `ResourceBusy` and so never answered a real one,
answer it now.
- A WAL prefix that is not empty at create is `AlreadyExists`, and a flush while
a transaction is open is `InvalidInput`; both were `ResourceBusy`, which said
"retry" about something retrying cannot change.
- A failed batch delete has the kind of its first failure, not `Interrupted`; a
reactor short read is `InvalidData`.
The rule, written on `s3_error_kind`:
| `ConnectionReset`, `TimedOut`, `ResourceBusy` | yes |
| `AlreadyExists` | not as is: a conditional write refused, or a name taken |
| `NotFound`, `PermissionDenied`, `InvalidInput`, `InvalidData`, `Unsupported` | no |
| `StaleNetworkFileHandle` | no — reopen |
| `Other` | no |
### Tests
- `integration_s3_error_kinds`: a GetObject body cut off part-way, a reactor
write's fetch answered 503, and writes racing flushes so the handler requeues
them. On 0.10.0 the first two come back `Other`.
## [0.10.0] - 2026-10-04
A write that fails part-way — the network down, say — returns an error and
leaves the file as it was, instead of panicking the process. A publish that only
moves blocks is marked, so consumers can tell compaction from a change of
content.
### Breaking changes
- **`HyperTrait` has four more required methods**: `compaction_flag`,
`fold_partial_into_compaction`, `checkpoint_published` and `attr_only_raw`. An
implementation outside this crate has to add them. (0.9.0 likewise added
`refs_snapshot`, `block_source_get`, `refs_clear` and `refs_remove` without
saying so here.)
- **A write that fails no longer panics; it returns the error.** Callers that
relied on a failure ending the process should handle it.
### Fixed
- **A write whose index lookup failed part-way panicked** —
`bmap.insert(..).expect(..)` in sixteen places across `write`, `write_zero`,
`write_ref`, `write_batch`, `write_aligned_batch`, the reactor's write and
zero-write absorb paths and `refresh_bmap`. An insert can read an index node
back from storage, so a network drop of a few seconds killed the process.
Returning the error alone would not have been enough: the blocks before the
failure were in the map and the cache, the ones after only in the cache, and
the size not yet updated. A write is now done in three steps — what it needs
read, changing nothing; the map, all or nothing, putting back what it set if a
block fails; then the cache, references and inode, which cannot fail. So a
failed write leaves the file exactly as it was, the handle stays usable, and
the same write repeated is the same as one that succeeded the first time,
`i_blocks` included. A block that was a hole may be left an explicit zero
block, which reads the same. Should putting the map back fail, the handle
refuses reads, writes and flushes from then on, asking to be reopened.
`block_mut` and `truncate_last_data_block`, which put a block in the cache
before its map entry, now set the map first.
Measured on memory staging, a 4 KiB write costs 4–6 % more CPU, a 1 MiB write
2–4 %; requests to storage are the same.
- **Reactor write paths under a failing store.** `write_batch` returned on a
failed lookup while its block loads still ran, dropping the blocks they were
writing into — a use after free, seen as a segmentation fault; the retrieve
paths of `write` and `write_zero` had the same early return. `write_batch` also
ignored a load's error, leaving the block zeroed for a partial write to
persist. And a store answering 5xx or 429 maps to `ResourceBusy`, which the
handler reads as "requeued, answered later": such an error from these paths was
answered by no one. Loads are now stopped and waited for before any early
return, a load's error fails the write, and store errors leave with a kind the
handler answers.
### Added
- **`InodeRaw::FLAG_COMPACTED`** (`0x4` in `i_flags`; layout and format byte
unchanged, and earlier versions carry the bit through). Set by a publish whose
every dirty block came from relocation, with nothing else changed since the last
publish and its number directly following it; cleared by any publish that
changes content. An attribute-only inode write keeps it, unless it publishes a
content change, such as a truncate that only extends.
- **`content_cno()`** on `HyperFile` and `InodeRaw`, **`Hyper::fs_content_cno`**,
and **`Hyper::fs_content_cno_fast(client, uri)`**, which reads the stored inode
without opening the file: the checkpoint number of the newest published content,
`last_cno − 1` while the flag is set and `last_cno` otherwise. The same across
compactions, greater after any content change. For comparing only — it may name
a pruned checkpoint. Refused for device nodes and inline files.
- **`HyperFile::relocate` / `Hyper::fs_relocate(ranges)`**: mark blocks to be
rewritten at the next flush with their current bytes, for compaction. Read with
the read path's coalescing; all or nothing like every write; no change to mtime,
ctime or the read cache; holes, zero blocks, references and blocks already dirty
skipped, and the count returned is what was marked. Not available with the WAL.
### Changed
- **`write_batch` reads only the blocks it writes in part.** It used to fetch
every block it did not hold, whole ones too. Measured on S3, a workload of
whole-block batches went from 1426 data requests to none, a mixed one from
2046 to 718; on memory staging a 256-block batch takes 0.41 ms instead of
1.09 ms.
### Tests
- `integration_s3_write_failure`: an index read refused in the middle of a write,
for each write path — the write fails, the file reads and stats as before, the
repeat stores what a twin that never failed stores. On 0.9.1 the direct paths
panic, `write_batch` crashes, and the reactor write never returns.
- `functional_compaction` and `integration_s3_compaction`: when a publish is
marked and when not, and relocation in few requests on S3.
## [0.9.1] - 2026-09-29
A flush no longer fails with "key not found through btree node lookup" after index
nodes have been read back from S3.
### Fixed
- **A flush failed assigning index pointers, and every retry with it.** S3 staging's
default loader (`meta_loader_batch`) returns, with each index node asked for, the
other nodes of the same stored segment. btree-ondisk 0.20.0 installed those over
nodes already in memory, including ones changed since they were read: the change
was lost, parent and child disagreed on their keys, and the next flush failed with
`NotFound: key not found through btree node lookup`. The published checkpoint was
untouched, but no further checkpoint could be written from that handle.
It takes index nodes evicted and read back while others from their segment are
changed — a node cache smaller than the index a flush touches; with
`NullNodeCache` an evicted node can only be read back from S3. It has been possible
since 0.8.0; checkpoints past 1 GiB, which touch over a thousand index nodes each
against a default node cache of 1024, made it common. Fixed in btree-ondisk
0.20.1, which this version requires.
On an older version, a larger `node_cache_blocks` makes it less likely; it does
not rule it out.
### Tests
- `integration_s3_node_reload`: a file built with `S3Staging`, its loader and
`NullNodeCache`, a node cache of 16, and flushes of overwrites, small writes and
zero writes across 16 GiB, read back cold. Fails on btree-ondisk 0.20.0.
## [0.9.0] - 2026-09-29
A flush larger than 1 GiB no longer stores data that reads back wrong: a checkpoint
larger than one object is written as several and published as one. Blocks can be
written by reference, so a caller that keeps its own copy of what it writes need not
hold a second one in memory until the flush.
### Breaking changes
- **A checkpoint may be several objects even when no partial segment was written.**
Under the parted format, a flush larger than `segment_max_object_bytes` writes
data-only parts `<segid>.1…` before part 0. Readers of this version and 0.7+ read
them; a tool that assumes one object per checkpoint unless partials were written
does not.
- **A flush that cannot be split is refused.** Under `Flat` or `MicroGroup`, with
the `wal` feature, or with an index larger than one object, a checkpoint that
needs more than one object now fails with `InvalidInput` before anything is
written. It used to succeed and store wrong data.
- **Block pointer encoding checks its ranges in every build** and panics past them,
instead of masking the offset into another block's place.
- **`HyperFileRuntimeConfig` has a field, `segment_max_object_bytes`.** A config read
from JSON or built with `..Default::default()` is unaffected; a struct literal
naming every field has to add it.
### Fixed
- **A flush larger than 1 GiB stored blocks at the wrong offsets.** A block pointer
names an offset within one object in a field that holds 1 GiB, and only a
`debug_assert` guarded it: in a release build the flush reported success and the
data past the first gigabyte read back wrong. A checkpoint is now laid out before
a number is taken or a pointer assigned. Part 0 takes the summary, every dirty
index node and the lowest blocks that fit; the rest go into data-only parts,
marked partial. Data parts are uploaded first, part 0 last, then the inode, so a
flush failing part-way leaves nothing open would take up; when the checkpoint
already has partials, the part numbers a flush uses are reserved first so a retry
never reuses one. A partial segment larger than one object is split the same way.
Measured on S3: 1100 MiB in one flush is two objects and every MiB reads back
from a cold handle.
- **Memory staging published index nodes under keys the bmap never asked for.** It
keyed them by a pointer format of its own, `MicroGroup`, while containers are
created `PartedSegment`; the two agree only in part 0 and among a segment's first
16 nodes. A cold read of a larger flush, or of a checkpoint written as partials,
found holes. Test-only staging; S3 was never affected.
### Added
- **Writing by reference.** `HyperFile::write_ref(off, len)` records that whole blocks
are to be read from a `BlockSource` (`set_block_source`) when the checkpoint
holding them is uploaded; `refs_len` counts them. Each object reads its
referenced bytes on the blocking pool just before it is sent, so what is resident
is bounded by the parts in flight, and what is stored is byte for byte what
writing the data would have stored. The source must keep those bytes readable
until the flush publishing them returns. Every other path that changes a block —
`write`, `write_zero`, `truncate`, `write_batch`, `block_mut` — supersedes a
reference, reading it first when it changes only part of the block; reads and
`block` read through it; a flush that rolls back discards references with the
rest; partial segments hold them too. Not available with the `wal` feature.
Contributed by hyperbox-zfs, with fixes to four of its paths before release.
- **`segment_max_object_bytes`**, the most bytes one stored object may hold. Default
and upper limit 1 GiB, what a pointer can address; below 16 KiB it is taken as
16 KiB.
### Tests
- `functional_write_ref`, `functional_split_checkpoint` on memory staging, and
`integration_s3_large_checkpoint`, which writes and reads back 1100 MiB.
- `concurrent_two_writers_fail_fast_policy` failed now and then because its two
writers overlapped only by chance; they now both open before either writes.
## [0.8.0] - 2026-09-27
Opening a container no longer trusts whatever sits past its inode: a checkpoint that
reached storage without its inode is taken up only once it checks out, anything else
there is stepped past, and a read-only open writes nothing. Multipart segment uploads
are held to a single put's terms, and unflushed writes cost their own size in memory
instead of twice it.
### Breaking changes
- **`Staging` has a required method, `segment_exists(segid)`**, answering whether that
exact object is on storage. Only an implementation of `Staging` outside this crate
is affected.
- **The segment header's `s_next` is `s_base`.** It was never read and always written
as 0; it now holds the fingerprint of the inode the segment was built on. The format
on storage is unchanged in layout, and segments written before this carry 0, which
is accepted.
- **`FileReqWalFlushDone` has a `base` field**, the fingerprint of the inode the
published checkpoint was built on.
- **A data block's buffer is 16-byte aligned, not page-aligned.** A caller of the block
API (`fs_block*`, `fh_with_block*`) that hands a borrowed block to `O_DIRECT` I/O has
to copy it into an aligned buffer first. See Fixed below for why.
- **`HyperFileRuntimeConfig` has a field, `segment_mpu_max_concurrency`.** A config
read from JSON or built with `..Default::default()` is unaffected; a struct literal
naming every field has to add it.
### Fixed
- **Open takes up a checkpoint that reached storage without its inode only once it
checks out.** It used to adopt whatever object sat at the next number, after a 5 s
wait and without looking at it, and write it to the inode. An object there that was
not a segment was written over the inode for good; a debug build panicked on the
counter it carried and a release build truncated it. A read-only open, and
`open_cno`, wrote the inode too, and a principal allowed only to read could not open
at all. A flush already reported failed came back on the next open. And the session
kept its view from before the adoption, so its next attribute change rolled the
container back.
Open now walks the numbers past the persisted inode. A commit — part 0 of a parted
checkpoint, the bare name otherwise — is taken up if its header, checkpoint number,
base fingerprint, format, embedded counters and index root all check out, and its
inode becomes the base for the next number; anything else is stepped past without
being read; a free number ends the walk. A writable open then points the inode at
what it took up, once, with a conditional write, after waiting out 5 s on a commit
that fresh. A read-only open takes the same commits up in memory and writes nothing;
`open_cno` does neither. A stored counter out of range is `InvalidData`.
A flush that fails and rolls back now abandons its numbers: the persisted counter is
moved past every number the session used, so a checkpoint it may have left is never
taken up, and a flush reported failed stays failed. The reactor's `fh_flush` and
`release` do not roll back, and what they leave can still be taken up.
- **A stale writer's checkpoint is not taken up over another writer's publish.** Each
segment records the fingerprint of the inode it was built on, and open requires it
to match. Without that, a writer retrying after another had published could land a
segment built on the old state at the next number, and the next open would take it
up. The fingerprint catches staleness and junk, not a writer forging it.
- **WAL replay no longer deletes a log prefix still waiting to be replayed.** An
ordinary publish deletes the log for the segment before it, which during replay is
a prefix not yet replayed. Replay now publishes with the log detached.
- **A multipart segment upload is conditional, and aborted when it fails.** On the
default build a segment longer than `segment_mpu_chunk_size` was completed with no
condition, so it could replace an object already under its name, and a bucket policy
enforcing conditional writes refused it. Both upload paths now complete with
`If-None-Match: *`, or `If-Match` given a prior state. A refused complete is
recognised by its error code as well as its status: a directory bucket answers one
with 200 and the error in the body. An upload that fails once created is aborted,
after the part uploads still running are stopped, and the error says what became of
it. The wal path also no longer returns while a part task may still read its buffer.
- **Unflushed writes hold their own size in memory, not twice it.** A data block on the
heap was allocated page-aligned, and glibc keeps a header in front of every
allocation, so a page-aligned block's header took the page before it: every 4 KiB
block held 8 KiB. 1 GiB of writes grew RSS by 2052 MiB; it now grows it by 1068 MiB.
Blocks are 16-byte aligned, which plain malloc gives; nothing relied on more, and
alignments above 16 measured no faster in copying or scanning a block.
- **A slice of a segment body reserves room for its own pieces**, not for every piece
of the body. Each part of a multipart upload is such a slice; for 1 GiB in 16 MiB
parts that was 64 lists of 8 MiB each.
### Added
- **`segment_mpu_max_concurrency`**, the most parts of one segment's multipart upload
in flight at once. Default 8; zero is taken as one. Every part used to start at once.
On 1 GiB in 16 MiB parts, flush-time RSS rose by 43 MiB under 8 and by 80 MiB with no
bound, in 1.5 s and 1.0 s.
### Documentation
- **`docs/s3-permissions.md`**, the S3 permissions each use of a container needs, on
general purpose and directory buckets, checked with restricted credentials; the
README summarises it. `s3:ListBucket` on the prefix is needed even to read, since a
missing object is answered 403 without it. Aborting a failed multipart upload needs
no grant of its own unless it is denied. A bucket policy enforcing conditional
writes is compatible.
## [0.7.3] - 2026-09-11
### Added
- **The crate compiles for macOS**, on Apple Silicon and Intel alike. Until now it was
Linux-only: 35 errors in the library and 80 with the test targets, across four files.
What that check covers and does not is written down in `tests/README.md`, which gains
a target axis alongside the eight feature rows. Compiling is verified; **linking and
running are not** — the link step wants the Security and CoreFoundation frameworks
from a macOS SDK, so a cross build from Linux cannot reach it, and running the suite
stays a macOS job.
One target rather than one per feature row, and that is measured: before any of the
work, all eight feature combinations produced the same 35 errors in the same four
files. The portability surface is entirely in code that compiles unconditionally.
- **`FileFlags::direct()` and `FileFlags::noatime()`**, for asking without naming a host
bit. `FileFlags::from` takes the host's own `open(2)` flag word and darwin has neither
flag, so on darwin there was no bit to pass and no way to request either behaviour.
Every other flag hyperfile acts on exists on both platforms, so `from` still carries
it.
### Fixed
- **A width in the format string reaches a segment id.** `Display` wrote to the
formatter directly, which silently discards the width, fill and alignment it carries
— those reach a custom `Display` only through `Formatter::pad`. So `{:<12}` had no
effect on a `SegmentId`, and a caller putting one in a column got a table out of
alignment rather than an error. A segment id names a stored object, so it turns up in
listings and log lines, which is exactly where something asks for a column.
Object naming is untouched: it builds the name from `seq_id()` rather than through
`Display`.
- **`libc::stat`'s field widths are named per target.** They are not one shape even
within Linux: `nlink_t` is 64-bit on x86_64 and 32-bit on aarch64 and `blksize_t`
follows, while darwin narrows `mode_t` and `nlink_t` to 16-bit and makes `dev_t` a
*signed* 32-bit. `Inode::to_stat` now spells out the five fields whose width varies,
per target, with one target's widths kept together.
`update_stat` needs no platform arms at all, and the asymmetry is the point: that
direction converts *into* the inode's own fixed-width fields, so every host's 64-,
32- or 16-bit value widens with the same cast. Only handing out the host's own struct
has a moving target.
Persistence is unaffected, which is the question this raises: the on-disk inode is
fixed-width, so the host's `stat` widths never reach it, and the mode *values* —
`S_IFMT`, the `S_IF*` types, the permission bits — are identical on both platforms.
A mode written on one means the same thing read back on the other.
- **A target whose `libc::stat` widths are not written down no longer builds.** The
previous form named x86_64 and aarch64 and stopped, so on any other architecture
neither arm compiled and `st_nlink` and `st_blksize` kept the zero from the zeroed
initialiser — a file reported with no links and no block size, silently. This is a
behaviour change for such a target: it now fails to build where it used to compile
and answer wrongly.
- **The Linux-only `open(2)` flags are defined for hosts without them.** `O_DIRECT`,
`O_LARGEFILE`, `O_NOATIME` and `O_PATH` were reached through `libc`, which does not
define them outside Linux. They now come from one module, with Linux's values, so a
flag word originating on Linux still reads as intended; a host without the flag cannot
set the bit itself, so the predicate answers no there.
- **The punch-hole reclaim is Linux's, so only Linux does it.** `fallocate` with
`FALLOC_FL_PUNCH_HOLE` has no counterpart in darwin's libc. Nothing is lost from the
read path by skipping it — a released slot is written in full before it is read again
— but the cache file ends up fully allocated rather than sparse once every slot has
been used once.
## [0.7.2] - 2026-09-06
### Fixed
- **The `bench` feature compiles.** Four expressions in `src/staging/bench.rs` were
left over from `SegmentId` becoming a type. The same shape as the `blocking + wal`
breakage 0.7.1 repaired, and worse in one respect: no row of the build matrix turned
`bench` on, not even alone, so it compiled nowhere and therefore failed nowhere.
- **The staging benchmark reads the object it wrote.** Fixing the types was not
enough. Under this release's default format the writer goes through `new_segwr`,
which names part 0, while the reader spelled the bare checkpoint name itself — so
every read would have missed. The rule lives in one place now,
`Segment::object_of`, called by both.
Its iteration also advances by checkpoint rather than by part, which is the only
choice that works: a segment object is written create-only, so each iteration needs
an id nobody has used, and advancing by part would name the same object every time
since a checkpoint's contents go in part 0 whatever part the id carries.
### Changed
- **The build matrix has a second rule, because the first was not enough.** "Every row
reaches a `#[cfg]` no other row does" keeps the rows from being redundant; it does
not say every gate is reached, and `bench` fell through that gap with one `#[cfg]` in
the whole crate. So: every feature declared in `Cargo.toml` is turned on by some row.
`every_declared_feature_is_built_somewhere` checks it by reading the manifest, so
adding a feature fails that test until a row covers it. The matrix itself is in
`tests/README.md` and now has eight rows.
## [0.7.1] - 2026-09-06
### Fixed
- **`--no-default-features --features blocking,wal` compiles.** On 0.7.0 it does not,
so a consumer using that combination cannot build the crate at all.
Two functions are gated on both features. Each feature was built alone and no
combination built both, so when the flush family's return type changed from a
checkpoint number to a `SegmentId`, three paths were updated and the fourth was not.
Nothing failed, because nothing compiled it.
The same combination turned up a `#[cfg]` wider than the code it guards:
`add_inflight_read` was gated on `wal` while its only caller is behind `reactor`.
The build matrix is now written down in `tests/README.md`, derived from the gates in
the source: every row covers at least one `#[cfg]` no other row reaches, so a new
gate either falls under an existing row or needs a new one. Seven rows, and the one
that was missing is marked as such. Zero warnings rather than zero errors is part of
the criterion, because a warning in one combination is often a gate that does not
match its code — which is how the missing row paid for itself immediately.
## [0.7.0] - 2026-09-05
A checkpoint no longer has to be one object, which lets a container write data out
under memory pressure without publishing a checkpoint nobody asked for.
### Breaking changes
- **A container created by this version cannot be read by a `0.6` one.** The default
block-pointer format is now `BlockPtrFormat::PartedSegment`, under which a
checkpoint is written as `<segid>.<part>` objects. A container carries its format
for life, and an earlier build refuses a format it does not know rather than
misreading it — a clean failure, but a one-way door.
The default moved because the behaviour the format enables is the reason it exists,
and leaving it opt-in would have left that behind a config change nobody finds. Set
`parted_segment_enabled` to `false` to keep the behaviour off; nothing turns the
format off after the container is created.
Containers created by `0.6` are read exactly as before, under the object names they
always had.
- **`SegmentId` is a type, not an alias for `u64`.** It names a stored object: a
checkpoint number and, for a container that streams its checkpoints, which of
that checkpoint's objects.
```rust
pub struct SegmentId { seq_id: u32, part_id: Option<u16> }
```
Reaching the parts: `seq_id()`, `part_id()`, `as_cno()` for the checkpoint number
as the inode stores it. Building one: `SegmentId::new(seq)`,
`SegmentId::with_part(seq, part)`, `SegmentId::new_from_cno(cno)` from a persisted
number.
`seq_id` is a `u32` because that is what a block pointer can address — 30 bits of
it. The `u64` it replaces promised range the format cannot name.
Surfaces that change shape: `block_placement` and its batch form return
`Option<(SegmentId, u64)>`; `PlannedRead::Get` carries a `SegmentId` that names an
object rather than a checkpoint; every method of `Staging`, `SegmentReadWrite` and
`StagingIntercept` that took a segment id now takes one of these.
What does *not* change: `fs_flush`, `fh_flush`, `fdatasync`, `release`,
`commit_txn`, `last_cno` and `open_cno` still speak plain checkpoint numbers,
now under the name `Cno`. That is what a caller pins, logs and hands back.
- **`DataCache` gained `demote_dirty`**, which moves named blocks out of the dirty
tier and leaves the rest. An external implementation has to provide it.
- **A container created with `BlockPtrFormat::PartedSegment` cannot be read by an
earlier build.** The format byte is checked when the container is opened and an
unknown one is refused, so this is a clean failure rather than a wrong read.
Containers in every other format are unaffected, and are still written under the
object names they always had.
### Added
- **`BlockPtrFormat::PartedSegment`**, under which a checkpoint may be written as
several objects named `<segid>.<part>`. Now the default; see **Breaking changes**.
The part index goes where the micro-group index used to sit, a field that took no
part in addressing — `decode` ignored it and its only reader was a debug string.
So a part is addressable from the pointer alone with no bit growth and no loss of
range: 14 bits of part over an 18-bit in-part block index raises what one
checkpoint can hold from 1 GiB to 16 TiB rather than dividing the old ceiling.
A separate field rather than the high bits of the offset, because the offset
restarts at zero in each part — so a part cut short by memory pressure wastes no
address space, and the cut can fall wherever it needs to.
- **Partial segments.** A dirty-data threshold has always published, which ties how
much a writer is holding to where the checkpoint history has a point in it. The
caller gets a checkpoint it never asked for, at a moment of its own work that it
did not choose.
Where nothing may publish the coupling is worse. Inside a transaction, and
throughout `WalRecoveryMode::Barrier`, a crossing is refused with `OutOfMemory` —
a checkpoint there would put a state nobody declared beneath the floor recovery
lands on. So a unit of work larger than the threshold could not be written at all.
A partial segment is somewhere to put that data which is not a checkpoint. On
either dirty-data threshold, the flush interval, or the point that used to be the
refusal, the dirty set goes out as one object and accumulation continues. The
checkpoint stays open until an explicit `flush`, `fdatasync`, `release` or
`commit_txn` asks for one — and with no partial outstanding that writes exactly
the single segment it always did. The ordinary case is unchanged, which is the
point of doing it this way round.
A partial has the shape of any other segment: its own summary, its own metadata
blocks, its own data blocks, and the inode inline. It writes *every* dirty node,
so the bmap root in that inode reaches every block the checkpoint holds so far — a
partial is a whole state, not a fragment, and its header says so through
`SEGMENT_FLAG_PARTIAL`.
**It is not a durability event.** The inode object is untouched, so a reader that
opens the container normally still sees the previous checkpoint, and a crash
leaves the partials referenced by nothing. Without a WAL the data in them is gone.
No flush returned, so no promise is broken, but "written out" must not be read as
"durable". With a WAL nothing changes, because the writes were durable before any
of this.
Two configuration values, both in `HyperFileRuntimeConfig`:
`parted_segment_enabled`, on by default and meaningless unless the format allows
parts, which exists to take the behaviour back; and
`data_cache_dirty_min_bytes_to_part`, defaulting to what the byte threshold
defaults to, because a partial smaller than the amount that triggers a write buys
little memory for a lasting cost in fragmented reads.
See [docs/flush.md](docs/flush.md#partial-segments-writing-out-without-a-consistency-point)
for the ordering that carries the safety argument, what partials cost, and the
interactions that each needed handling.
- **`Cno`**, an alias for the checkpoint number a caller sees. An alias rather than
a newtype: a caller compares these, prints them, and hands them back to
`open_cno`, and the value that needs type safety is the one that names an object.
### Fixed
- **A read concurrent with a flush no longer goes to an object that is not there
yet.** A WAL flush returns before its upload lands and serves reads from a pinned
buffer meanwhile. The test for whether a pointer was covered by that buffer asked
whether the checkpoint was parted, on the grounds that a streamed checkpoint has
its data on storage before any pointer to it resolves. That is true of a partial
segment, which is uploaded before its pointers are assigned, and false of the
consistency point, which assigns first and uploads asynchronously.
Unreachable until the default format changed, because no pointer was parted. The
test is now `SegmentId::is_partial`, which names the distinction rather than leaving
it to be inferred — and carries a unit test, so the same mistake fails without
needing a concurrent read against a flush to find it.
- **A batched read of a partial segment is no longer answered out of another object's
buffer.** A flush pins the buffer of the object it is uploading so reads can carry
on. The `read_many` and `read_ahead` workers consulted that buffer for every request,
reducing the pointer to its checkpoint to look it up — so a pointer into a partial
matched, and the offset was read out of the buffer holding the checkpoint's summary
and metadata. Real bytes, belonging somewhere else, with no error and nothing
downstream to notice.
The buffer holds one object, so only a pointer that is not a partial may be served
from it — the test the planner already applies when it decides a request is served
from memory at all.
`flushing_segments` is keyed by `Cno` rather than `SegmentId` now. The defect was an
object id reduced to a checkpoint at a lookup, and a key type that is a checkpoint
number cannot be handed an object id.
- **The in-memory fallback names an object that exists.** When the pinned buffer has
gone, the read falls back to storage — and asked for the bare checkpoint name, which
in a container that streams its checkpoints is nobody's object. The request carries
what the pointer named now, and the lookups reduce it, which is the only arrangement
where both the buffer and storage are asked for what they hold.
- **Listing a container's checkpoints recognises a part suffix.** `list` parsed a
whole object name as a number, and a name carrying a part suffix does not parse — so
every object of a container in this release's default format was skipped, which is
indistinguishable from a container with no history: no error, an empty list. A tool
walking the history of one would report zero checkpoints.
Each checkpoint is also counted once now. Its objects share its number, so
recognising them all listed it once per object. The memory staging already
deduplicated and the S3 one did not, so the same change had been made correctly in
one place and half-made in the other.
The same mistake as two tests recognising a segment object by "ten digits", which
failed and were noticed. This is product code with no caller in the test suite, and
`list` had no coverage at all — it does now.
- **`SegmentReadWrite::open` resolves a checkpoint number.** It named an object
literally, and a caller holding a checkpoint number cannot know whether the
container streams its checkpoints — that is recorded in the inode it is trying to
reach. Given a bare number and no object of that name, the summary part is tried.
`S3Staging::open_exact` asks about exactly one object, which walking a checkpoint's
parts needs.
- **A threshold crossing no longer restarts a flush already underway.** Recovery
replays acknowledged writes through the ordinary write path while holding the
flush lock, and a replay larger than the dirty threshold crossed it — asking for
that lock against the hold the same task already had. The process stopped there,
with no error.
Reachable before this release and not reached: every test that replays keeps its
thresholds above what it writes, so no replay ever crossed one.
`State::is_flushing` was gated on the reactor feature while the flag it reads was
set and cleared unconditionally. The deadlock is on the replay path, which has
nothing to do with a reactor, so the gate is gone.
## [0.6.16] - 2026-08-31
### Fixed
- **A group nobody will apply no longer keeps the work above it out of reach.**
Declining an unsealed group is right, and leaving it in the log is right. What
was wrong is what happened next: the following session wrote on top of the same
checkpoint, sealed its own group, and recovery walked into the declined one
again and stopped there — never reaching the sealed group behind it.
Nothing lifts that. The group does not go away, and `last_ondisk_cno` does not
move until something publishes, so every later open stops in the same place. A
consumer saw three reopens in a row report the same checkpoint on disk while
their flushes returned higher ones each time, with acknowledged writes absent
afterwards. Arriving here takes nothing more than a crash in mid-flush, which
makes it the ordinary outcome of `Barrier` rather than a corner of it.
The comment justifying the stop said that skipping a group and applying a later
one would produce a state that never existed. True in general, false here: the
session that wrote the later group opened at the same checkpoint recovery lands
on, was told nothing had been applied, and wrote from there. Its group is a
delta on that checkpoint. The state that never existed would be one with the
declined group in it, and nothing proposes applying that.
So a group that is not whole is asked one more question before it may stop
anything: is there a barrier above it? A barrier above means a later session
sealed on top of a base that did not include this group. Under `Barrier` such a
group is set aside whole, including records before a transaction left open,
which the later session did not see either; `Latest` is unchanged except that
such a group no longer stops the replay. Records set aside are counted in the
report and not deleted.
- **A request no longer waits for the flush lock on the handler task.** The
reactor's WAL flush hands the flush lock's guard to a spawned publish, and the
guard comes back as a callback — to the handler task, the same one that runs
every request. A request that waits for that lock stops the task from ever
taking the callback, so the guard never returns and the handle is wedged: no
error, no timeout, until the process goes away.
`setattr` did that, because it persists through a flush. `fsync` followed by
`chmod` wedged the handle whenever the second landed before the first one's
publish had finished — a window of a couple of object-store round trips, and a
pairing a filesystem above performs constantly.
`release` was already right about this, and completely: it reports the conflict
and its arm puts the request back on the queue rather than waiting. The
mechanism existed in one place and was never generalised.
- **Aborting a transaction takes its records out of the log.** `abort_txn` rolled
back memory and removed the transaction marker, and left the writes themselves
in the log. Nothing about an abort seals them, so the next open found unsealed
records and `Latest` applied them — putting back exactly what was discarded.
Its own documentation promised the writes were gone from later opens too. The
test covering the other half said as much in its name and never reopened to
check this one, so the gap sat behind a test that looked like it covered it.
The whole group goes, not the interval's share of it: the rollback returns the
file to its last published state, which discards writes made before the
interval opened as well, and those are in the same group. Awaited rather than
spawned, unlike every other log delete here — a lost delete elsewhere costs a
little storage, a lost delete here undoes the abort.
### Added
- **Transactions on the handler**: `fh_begin_txn`, `fh_commit_txn`,
`fh_abort_txn`, `fh_in_txn`. They existed only on `Hyper`, and a caller that
hands its `Hyper` to a handler task can afterwards send only the request types
the context defines — so the feature was unreachable from the surface that needs
it most. Dropping to the direct API is not a trade a reader can make: those
methods take `&mut self`, so a read path that spawns its prefetch alongside the
read cannot use them.
The interval lives in the handler task, because that is where the file lives, so
every write sent between begin and commit falls inside it whoever sent it —
the only shape that works where each call is one round trip. `fh_in_txn` because
a failed commit leaves the interval as it was and the caller cannot tell from
its own side.
Commit and abort re-queue rather than wait when a flush is in flight, for the
reason above.
### Corrected
- **What `publish_every` buys.** Its documentation in 0.6.15 said publishing was
93 to 99 per cent of a consumer's fsync latency, and that raising the value
traded that cost away. That attribution was wrong, and the same consumer
falsified it by measuring: flush latency is flat across 1, 2, 4 and 8 (0.213,
0.210, 0.228, 0.197 seconds) while container objects fall from 11 to 4.
Publishing was never on the flush's critical path. A flush answers after one log
append — the barrier sealing its group — and hands the segment upload and the
inode write to a spawned task. Deferring an upload nobody waits for saves
nothing. The figure that was attributed to publishing is the synchronous part of
the path, which is the log writes.
So `publish_every` is a knob for object count, not for latency: fewer objects to
store, to list, and to prune. The recovery cost stated alongside it stands by
construction but is still unmeasured — a crash that lands on a publish boundary
has nothing outstanding, which is what the measurements above happened to
arrange, since their run length was divisible by every value tried.
## [0.6.15] - 2026-08-31
### Fixed
- **A session no longer writes into a checkpoint whose records recovery
declined.** `Barrier` recovery leaves what it will not apply in the log, and
the next session wrote its own records into the same checkpoint because nothing
moved the sequence past what was there.
Record keys are `<seq>_<offset>_<len>`, and a fresh session's counter starts at
zero while `last_seq` has not moved — so the first write lands on the key an
unapplied record already occupies, and the create-only PUT fails with 412.
Whether it collides needs the offset and length to match too, which replaying
a similar workload makes likely: a consumer saw a run get partway through and
then fail, and reported the data as appearing "partially in".
Worse, the two sets would have ended up in one group, and a later barrier would
have listed both and called the result complete — a seal vouching for work that
was deliberately set aside.
- **Applying nothing is no longer read as a recovery failure.** Recovery returns
the checkpoint it reached, and zero when it reached none. The retry loop took
that zero for failure, spent its attempts, then set the fail-stop flag — after
which the container refused every write, having opened successfully, with the
error that explained it discarded by `do_open`.
Under `Barrier` that zero is the ordinary result, so the mode's most common
path ended in a read-only container. The outcome belongs in
`WalRecoveryReport`, which already carries it, so the return value no longer
has to. The test covering it went from 3.4 s to 0.3 s, which is the retries and
their backoff going away.
- **In `Barrier` mode, only the caller publishes.** A dirty-data threshold could
still publish, and recovery cannot land earlier than the newest published
checkpoint — so a threshold crossing partway through the caller's own unit of
work put a checkpoint nobody declared beneath the floor, and what the mode
sells was gone. Whether that publish also sealed a barrier is beside the point;
publishing at all does it.
A write that would need a publish to make room is refused with `OutOfMemory`
instead. The memory bound stays — what changes is the action on crossing it,
from silently publishing a torn point to saying so. `Latest` is unchanged,
since there the bound is why the threshold exists.
- **Recovery decides whether to run on the same basis the replay filters by.**
The trigger read `last_seq`, the replay filters by `last_ondisk_cno`. Two bases
for one question, which is how a group becomes unreachable — the trigger says
there is nothing to do while the filter would have found something.
They agreed until a session began advancing `last_seq` on its own (the first
fix above). After that an inode-only publish — an attribute change with no
dirty data, which a filesystem above does constantly — wrote the advanced
`last_seq` out, and the trigger then concluded there was nothing to recover
while the records sat there. So the promise that what `Barrier` sets aside a
later `Latest` open can take did not survive a `chmod`.
### Added
- **`publish_every`** on `HyperFileWalConfig`: how many flushes may be satisfied
by the log before one of them publishes a segment. Default 1, which is the
previous behaviour.
Publishing costs about the same however much was written — a consumer measured
0.21 to 0.26 seconds for 1 MiB and for 32 MiB alike, 93 to 99 per cent of their
fsync latency — so it is a fixed cost per flush, and this is how to stop paying
it on every one.
Durability is untouched: every write is in the log before its call returns
either way. What is deferred is the checkpoint, not the data, so a flush can
return with no checkpoint existing for what it flushed and a caller needing one
has to publish. Recovery replays each deferred group in turn, so a mount after
a crash pays what those flushes did not — the knob is a dial between flush
latency and mount latency.
Nothing is deferred inside a transaction, whose commit is a publish by
definition, or on release, where deferring is a bet that another flush is
coming.
Deleting superseded log prefixes was widened to match. A publish used to drop
one, which was right when it stood for one group; standing for N it has to drop
N, or the log grows in proportion to how often publishing was deferred and the
saving is only moved.
## [0.6.14] - 2026-08-31
### Fixed
- **Batch writes did not reach the log, so the durability guarantee did not
cover them.** `write_inner` wrote a log record; neither batch path did.
Worse than a missing feature. A WAL flush answers before its upload lands, and
the reason it may is that the log already holds the data. For batch writes it
did not, so the flush cleared the dirty set and returned `Ok` over bytes that
were on no storage at all — not in a segment, since the upload had not
finished, and not in the log, since nothing wrote them there. A crash in that
window lost acknowledged writes. It also made `writes_durable_on_ack` report a
guarantee that held for one write path and not another, which is why the
partial-block batch path is fixed here too and not only the aligned one.
Records are coalesced across adjacent blocks of the same kind: one record per
block would be one object per block, and the batch API exists because that cost
was measured and rejected — a consumer moved an 8 MiB fsync from 69 s to 0.46 s
by switching to it. A 24-block batch with one hole writes three records.
- **A refused flush rolled back.** `fs_flush` goes through
`flush_with_rollback`, which treats every error as "the flush failed partway,
put the file back" and discards the dirty set. Right for a storage failure,
wrong for a refusal: a flush declined before it started never touched
anything, so rolling back threw away acknowledged writes over a request that
was merely not allowed.
- **Time alone no longer publishes when a log is configured.** A timer's purpose
without a log is to bound how long an acknowledged write sits only in memory,
and a log already bounds that. All it added was a checkpoint at a moment
nobody declared — and since recovery has to reach the newest published
checkpoint, one landing mid-transaction leaves `WalRecoveryMode::Barrier`
nothing sound to stop at. The dirty-data thresholds still publish, because
dropping them would trade a bounded cache for an unbounded one.
Note that `data_cache_dirty_max_flush_interval` set to `0` does not disable
the timer: `elapsed() >= Duration::from_millis(0)` is always true, so `0`
publishes on every check.
- **`update_cache` warned about correct behaviour.** It logged "this is not by
design" whenever a block was clean in the read cache rather than dirty, which
happens routinely, and a consumer chasing a corruption report spent time on it
as a suspect. A whole-block write replaces every byte, so what was underneath
does not matter; only a *partial* write into an unprepared block indicates
anything, and only that warns now.
### Added
- **Barriers: which log records belong together.** A flush seals the records it
is about to publish, writing a barrier that carries the `(seq, offset, len)` of
every record it covers. Without one, a crash mid-flush leaves a partial record
set indistinguishable from a complete one, and replaying it hands the layer
above half of its own transaction. The manifest is a list rather than a count
so recovery can check it against what is stored without depending on a listing
being complete or on the order records went out in.
Two orderings make it mean something: after every record it covers is
confirmed written, and before the segment build moves past the checkpoint those
records are filed under. Together they give *a barrier exists* ⟹ *the group is
complete and was never published*.
- **Recovery modes**, via `HyperFileWalConfig::with_recovery_mode`. `Latest`
(default) applies what it can and keeps every acknowledged write. `Barrier`
stops at the last sealed, complete group, so it needs no repair — at the cost
of everything acknowledged after that seal.
Those two cannot both keep the durability promise, and this is the case
`writes_durable_on_ack` was named for: under `Barrier` it answers `false`, so a
caller skipping flushes on the strength of it degrades instead of losing data
quietly. Making `Barrier` the default would have withdrawn a shipped guarantee
silently on upgrade.
A group that is unsealed or incomplete **stops** the replay rather than being
skipped. Skipping one and applying a later one produces a state that never
existed, which is worse than a torn one: that at least was some moment's truth.
- **`wal_recovery_report`** — whether anything was replayed, whether the landing
point is a state whoever wrote it declared consistent, and how many records
were present but not applied. The middle one lets a caller that can repair its
own contents skip doing so, which is work proportional to the whole container.
- **Transactions**: `begin_txn` / `commit_txn` / `abort_txn` / `in_txn`. For work
whose midpoints nobody should come up in — a repair pass can write far more
than the dirty-data thresholds allow, and a threshold crossing partway through
would make a half-fixed container the newest checkpoint.
Gives atomic publication and atomic recovery: a marker in the log means an
unfinished transaction is not applied, in *either* recovery mode, since it is
something the caller declared rather than an inference about where a crash
fell. Writes made before it began are ordinary and still applied, so an
interrupted transaction costs redoing it rather than repairing half of it.
Gives neither isolation — writes inside one are visible to readers
immediately — nor implicit undo: never committing leaves the writes in memory
until close, and `abort_txn` is what undoes one here and now. Both are stated
in the docs, because the name would otherwise promise them.
## [0.6.13] - 2026-08-28
### Added
- **`writes_durable_on_ack`**, on `Hyper` and `HyperFile`. True when a write
that has returned `Ok` is already recoverable with no further flush: its
bytes are in the write-ahead log, and opening the container again replays
them.
A caller that skips flushing on that basis was relying on a property this
crate had never stated as an interface, and had to work out whether it held
by checking that the WAL URI starts with `s3://` — a copy of a judgement this
crate owns. The copy agrees today. What makes it worth removing is the
direction it can diverge: add any path where a log is configured but the
guarantee does not hold — a feature switch, a degraded mode, a deferred init
that failed — and the caller skips a flush it needed, with nothing reporting
it and the data gone after the next crash.
Named for the guarantee and not for the mechanism on purpose. If this crate
ever keeps a log while not offering the property — batching log writes so a
write can return before its bytes are down — it has to start answering
`false`, which a name like `wal_enabled` could not do honestly.
### Documentation
- **The durability contract is stated in the terms that were ambiguous**, in
[docs/wal.md](docs/wal.md): which bytes it covers (those each `Ok` write
reports, with no ordering promised between separate writes), from when (the
moment the call returns), that it is independent of `commit_bytes`,
`commit_interval_ms` and of any publish having happened, and that reopening
with the same log configuration replays it with no manual step.
It also says what the guarantee is not, to keep it apart from checkpoint
views: it does not make a cno openable, since reading those bytes back needs
the container reopened rather than a checkpoint published.
Pinned by `reactor_wal_writes_are_durable_on_ack_without_any_flush`, which
sets the publish thresholds out of reach so nothing incidental can produce a
segment, writes, drops the handle without flushing, and compares bytes after
reopening — alongside a no-log control, without which the first half would
pass just as well if writes were simply never lost.
- **Corrects the TODO about satisfying a flush from the log.** It claimed a
background replay was a prerequisite, because nothing would otherwise bound
the log. The commit thresholds bound it. Measured on one workload, 256 MiB /
1 h gives 3 container objects and a log peak of 869 objects, while 16 MiB /
2 s gives 10 objects and a peak of 86 — both of the things publishing on
every flush was wanted for, which costs 200 objects. So the background
replay is an optimization after all; what it would add is bounding the log
by time since the last publish without traffic having to trigger one.
## [0.6.12] - 2026-08-27
### Fixed
- **A container format this build cannot represent is refused instead of
read.** Opening decoded the inode's `i_meta_config` and believed the result.
A container written before that field was populated carries zero, which
decoded to a root of 0 bytes and blocks of 1 byte — and since `do_open` ends
by overwriting the caller's config with the container's, that 1-byte block
size then decided how every read and write was cut up. A consumer opened
such a container and got a device reporting 33280 bytes where 2 GiB had been
written. Nothing about it was detectable from outside: a wrong size looks
like a small file.
No new format version was needed. The minimums to check against were
already named in `config.rs` — `MIN_ROOT_SIZE`, `MIN_META_BLOCK_SIZE`,
`MIN_DATA_BLOCK_SIZE` — and a zeroed field decodes below all three. The open
now reports the raw value it could not make sense of.
Two further ways the same decode could go wrong, neither of them the
reported symptom: `BlockPtrFormat::from_u8` panicked on a format byte it did
not know, and the block-size shifts come from a whole byte, so a stray value
shifted by up to 255. There is now a `try_from_u8`, and the shifts are
range-checked before the shift rather than after.
`Display` and `to_stat` decode the same field and must not panic over a
field they only report — an inode built in memory has not populated it yet —
so those fall back to the default. Rejection stays in the one place that
decides whether the container is usable at all.
- **A publish that cannot be completed no longer panics.** A WAL flush answers
as soon as the log holds the data and uploads detached, so when that upload
fails the caller has already been told, truly, that the data is durable.
There is nobody to return an error to. Replaying the log is the remedy, and
it can fail for the same reason the publish did.
That case ended in `panic!("please fix wal with offline tools")`, which for a
server built on this crate is the whole process, including the reads it was
still serving correctly. There were two such panics; the one in the dispatch
arm is the one actually reached.
Recovery now retries a bounded number of times with backoff. When those are
spent the file stops accepting modification and goes on serving reads —
every acknowledged write is in the log, so nothing is lost and offline
repair has what it needs, whereas continuing to accept writes would pile
more data behind a publish that is not happening. Reads through that handle
may not show the newest data, since the flush had already repointed the map
at the segment it could not publish; reopening replays the log and produces
it.
Recovery also no longer mistakes an already-published checkpoint for a
failure. A segment object is written create-only, so replaying one that
reached storage fails with 412 — which is what the common case looks like,
the segment landed and only the inode did not. Counting that as a failure
would take a file read-only over data that is present and correct. Before
this change that path panicked, since `do_open` calls recovery through
`let _ =`, which a panic goes straight past. See the TODO in
[docs/wal.md](docs/wal.md) for verifying rather than inferring this.
### Added
- **`Hyper::open_cno`** opens a published checkpoint read-only, so a
checkpoint can be read as a second view while the container carries on.
`HyperFile::open_cno` was already public but unreachable in practice: it
wants the three generic parameters constructed, and `Hyper` keeps its inner
file `pub(crate)`, so a caller who built one by hand still could not wrap it
and had none of the `fs_*` surface.
This is not what `hypercli file rollback` does — that publishes an old inode
as the current one, so it moves the container and every reader with it, and
the device has to come off first.
Writing the tests found that `release` flushed unconditionally, which is the
more serious half of this: a checkpoint handle holds a historical inode, so
closing one attempted to publish the past as the present. The on-disk state
check caught it, so nothing was corrupted, but it cost three flush retries
and ended in `ResourceBusy` — and it rested on a conflict being detected
rather than on not writing. A read-only handle now skips the flush, which
also drops a pointless flush attempt from every ordinary read-only close.
A checkpoint is opened through the same `do_open` as anything else, so the
container-format check above applies to historical segments too.
### Documentation
- **Which flush paths publish asynchronously.** `flush_process` has no WAL
branch, but a reactor-mode explicit flush does not go through it: `fh_flush`
reaches a WAL-specific dispatch arm that hands the upload to a spawned task
and returns. Only the direct API's `fs_flush` publishes synchronously. The
third request was to make explicit flush take the WAL path — it already
does.
What the asynchronous publish does not change is the object count. Measured
over 50 writes each followed by an explicit flush, 64 KiB apiece: 51 objects
and about 760 ms whether or not a WAL is configured. Every flush still builds
a segment, and the spawned upload holds the flush lock, so the next flush
waits for it regardless of having returned early.
So a consumer's measured gain — 200 objects against 1 — comes from not
producing a segment per flush rather than from not waiting for one. That is
recorded as a TODO along with why it is not a small change: nothing bounds
the log once flushes stop producing segments, so replaying it to storage in
the background is a prerequisite rather than an optimization on top.
## [0.6.11] - 2026-08-27
### Fixed
- **`BMap::read` was handed a bmap root with no alignment guarantee.** All
three places that reopen a map from a persisted inode copied the root out
into a local first:
```rust
let b = raw_inode.i_bmap;
BMap::read(&b, ..)
```
`BMap::read` wants an 8-byte-aligned buffer. `BMapRawType` is `[u8; N]`,
whose alignment is 1, so the copy sat wherever the compiler put it —
`InodeRaw` being `repr(C, align(8))` does not carry over to a value copied
out of it, since the alignment belongs to the struct and not to the field's
type.
It worked, and would have kept working on x86-64, where such a local gets an
aligned slot in practice. That is what made it worth fixing rather than
leaving: nothing enforced the property and nothing would have reported its
loss.
Borrowing instead of copying makes it aligned by construction, since
`i_bmap` sits at offset 104 of an `align(8)` struct. A unit test pins both
halves of that, because neither is visible at the call site.
Raised by btree-ondisk against callers that reopen a map from a `Vec<u8>`.
We were not doing that; we were doing something weaker, since a `Vec`'s
buffer at least comes from the allocator.
### Changed
- **btree-ondisk 0.18 → 0.20.0.** Nothing in 0.19.0 or 0.20.0 changes an
existing signature or behaviour.
0.20.0 adds a reachability API — `collect_ptrs` for maps whose leaf values
are storage locations, `collect_meta_ptrs` for maps whose leaves hold inline
payload. Neither is called here; hyperfile has no reachability question to
ask, and never used `nonleafnode_iter`, the method they replace. What the
bump does for this crate is enforce something previously assumed:
`collect_ptrs` requires `V == P`, checked before any node is read, and this
crate's `V == P == u64` is what makes the call legal for the callers that do
ask — the library now draws that line itself.
0.19.0 comes along with it, adding `BtreeNodeDirty::id()`. Not adopted.
Recorded in case it ever is: a node that has never been placed carries an
internal sequence number, and `is_invalid()` returns false for those, so
discriminating with it would hand a sequence number back as a storage
offset. `is_valid_extern_assign()` is the correct test.
## [0.6.10] - 2026-08-26
### Added
- **`fh_seek_hole` and `fh_seek_data`**, the handler equivalents of
`Hyper::fs_seek_hole` and `fs_seek_data`. A caller in reactor mode
previously had no way to ask where a file's holes are.
Both are a map walk, and the core methods take `&self`, so the dispatch arm
answers them directly — nothing spawned, no permit taken, and the map never
leaves its task. One request op carries a whence discriminant rather than
there being two, so the pair cannot drift apart in how it is dispatched.
Unflushed writes count as data, so the answer accounts for what has been
written but not yet persisted. This is asserted separately on the handler
even though it reaches the same code as the direct API, because a handler
write is multi-hop: a query can arrive between hops and see state the direct
API never presents.
Tested against spelled-out expected offsets *and* against the direct API on
the same file. Spelling them out is what makes the test mean something — a
cross-surface comparison alone passes when both surfaces are wrong together.
Swapping the two whences in the arm fails both new tests.
## [0.6.9] - 2026-08-26
### Added
- **`read_ahead_gets` and `read_ahead_bytes`**, so a read's own cost can be
told from what read-ahead spent on its behalf.
`data_gets` counted every data request alike. Nothing could separate the
two, because the counter is incremented where the request is made and
staging sees a ranged load with nothing about its purpose. That leaves a
measurement with read-ahead enabled unable to attribute what it cost, and
the only alternative — turning read-ahead off — measures a different system.
Subsets rather than separate totals: a read-ahead's request is still
counted in `data_gets`, so existing assertions keep their meaning, and
```
a read's own requests = data_gets - read_ahead_gets
a read's own bytes = data_bytes - read_ahead_bytes
```
Attributed by the read-ahead path itself on both surfaces, once per
successful load. Three cases resolved towards the honest answer rather than
the tidy one: a fetch that fails after staging counted it is not
attributed, so the counters disagree by that request; the `wal` path that
copies from a segment still pinned in memory issues no request and counts
nothing, matching `data_gets`; and `fh_read_many`'s fetches are not
attributed here, since they are asked for rather than speculative.
Both attribution sites were confirmed load-bearing by removing them in
turn — each makes the assertion that every request during a warm was
read-ahead's fail with 0 of 1.
### Notes on what prompted this
A consumer brought two files of the same size and contents, one costing 259
object requests to read and the other 128, and took the difference to be a
matter of how the files were laid out. The placement queries added in 0.6.8
answered that on their first use, and answered it against the hypothesis:
both files sat in a single segment with no discontinuity anywhere, so by the
criteria those queries expose, both layouts were as good as a layout gets.
What differed was how many times their own caller asked — 384 against 256 —
and whether that was prefetch volume could not be established, because the
counters did not say. Hence these two.
Worth recording alongside the queries themselves: a prediction made here from
the 0.6.8 invariants was wrong, and wrong for a specific reason. Reasoning
that each 128 KiB window costing two requests implied a segment change per
window assumed one request *arriving* per window, which was never established
and turned out to be false. The invariants held; the inference from them did
not. A cost figure cannot be read without knowing how many times the caller
asked, which is the same class of mistake as reading a coalescing figure
without knowing how much was asked for at once — see the note under 0.6.8.
## [0.6.8] - 2026-08-24
### Added
- **Two read-only queries for where a file's blocks are**, and what reading
them would cost. See [docs/placement.md](docs/placement.md).
A read is served by one object request per stretch of blocks that adjoins
within a single segment, so where the blocks landed decides what reading
the file costs. Nothing exposed that. A consumer arrived with two files of
the same size and contents, both left in one or two extents of their own
address space by different routes, where reading one cost 259 object
requests and the other 128 — with no way to tell whether the expensive one
was spread over many segments, over few in a poor order, or something else
again. Everything they could observe was the same.
```rust
let plan = fh.fh_read_plan(off, len).await?; let places = fh.fh_block_placement(start, n).await?; ```
`read_plan` returns one entry per object request, in the order they would
be made, plus an entry for every stretch that needs none, so the entries
account for the whole range. `block_placement` returns which segment each
block is in and where, `None` for a hole or a block not yet flushed. Batch
forms take several ranges at once. All four on `Hyper` as `fs_*`, on
`HyperFileHandler` as `fh_*`, and on `HyperFile` under the bare names.
The plan is the planner the read path uses, not a description of it. That
is the reason to expose it rather than leaving callers to work the merging
rule out from raw placement: a second implementation of the rule can
disagree with the first silently, and in the direction of reporting a
layout as fine when it is not. So the predicted request count is asserted
against what a real read issues, on layouts built to make the two easy to
disagree about.
**The plan follows the data cache.** A resident block needs no request, so
it is reported as local. That makes the answer what a read *now* would
cost, which is the question, but it means a warm file looks free. Judging a
layout wants a cold handle, or `block_placement`, which does not consult
the cache.
Metadata only: no data request is issued, which is asserted rather than
assumed, and index reads the walk needs land in the existing `meta_gets` so
a measurement using these can subtract what asking cost from what it is
measuring. Neither query influences placement, exposes segment contents, or
promises the answers will be anything in particular.
Batching earns its place on the handler, where it collapses a crossing per
range into one — worth more here than for `fh_read_many`, since with no
object request to wait on the crossing is most of the cost. It also cuts
the other way: the map does not leave its task, so a long batch occupies
the file for its whole walk. On the direct API there is no crossing, so
those forms save only the repeated call.
### Notes on what the queries revealed
Building layouts these queries could be tested against turned up something
about flush that was not written down anywhere:
- A flush emits its blocks **sorted by index and packed**. So two
file-consecutive blocks that are both in one segment are always adjacent
within it, and writing a file's blocks in reverse order lays them down
ascending regardless.
- Requests therefore break where the **segment** changes, which is a property
of when each block was last flushed rather than of the file's own order.
Overwriting one block in the middle of a flushed run splits a read of that
run into three requests: the blocks before it, the new block in its new
segment, the blocks after it.
- Two *consecutive* requests can share a segment only when the request-size
cap splits a run. A 20 MiB contiguous run reads as two requests under the
16 MiB `read_get_max_bytes` default.
That last point decided a test. Three ways a hand-rolled cost model would go
wrong were each injected and confirmed to fail the suite — merging across
segments, merging across the request-size cap, and reporting a dirty block as
placed. The cap case passed the whole suite until a test for a run longer than
the budget existed, because no smaller layout can produce the situation.
### Documentation
- **What coalescing does, and how much it depends on the ask.** A request is
extended only while the next block's location is strictly adjacent in the
same segment, so a read fetches nothing the caller did not ask for; the
`read_get_max_bytes` budget caps a request's size rather than a gap it may
span. Worth stating because it is easy to assume the opposite.
The consequence is that **how fragmented a file looks depends on how much
is asked for at a time.** A consumer measured the same file as effectively
unfragmented at the library layer and ten times slower through a
filesystem — 25.87 against 2.650 MiB/s, 135 requests against 854. Nothing
about the file changed: the library read asked for 4 MiB at a time, giving
the planner 1024 blocks to walk and one budget to spend, while through the
mount the kernel asks 128 KiB at a time and each call pays for whatever
runs it straddles. Every measurement in `docs/concurrency.md` is a
library-layer one, so they are an upper bound on what coalescing can do
rather than what a caller asking in small pieces will see.
## [0.6.7] - 2026-08-22
### Added
- **`fh_read_many`** reads a list of block indices, fetching the ones that
are not cached, and calls a closure once per index. One crossing for the
whole list.
`fh_with_blocks` from 0.6.5 answers from the cache and reports the rest
absent, which leaves a caller that wants the blocks doing it in two steps
— warm a range, then visit it — where the second waits on the first, and
where a range has to be named although what the caller holds is a list. A
directory walk reads records scattered across an inode table.
Where the closure runs depends on whether anything has to be fetched:
- **Nothing missing** — it runs on the handler task, borrowing out of the
cache. No copies.
- **Something missing** — the fetch leaves the handler task and the
closure goes with it, running where the fetched bytes are. Blocks that
were resident are copied first, since a spawned task cannot borrow the
cache. About 0.4 µs a block, reached only when an object request is
already being waited on, so it never decides anything.
The second case also stops an expensive closure from stalling everything
else. The first does not, so a batch closure should stay cheap and carry
its work out rather than doing it inside.
What was fetched is kept, and queued to be cached before the answer, so a
caller that reads the same list again finds it there rather than
eventually.
A list with gaps raises a question the range-based paths never had to
answer: fetch the gaps, or split into one request per run? On an object
store a request costs far more than the bytes a gap spans, so runs are
merged while the gap stays under `read_get_max_bytes` and `plan_read`
decides how the merged range breaks up. Measured over 38 blocks scattered
every eighth across a cold file: one request per run took 32 ms, a single
merged request 28 ms, one block at a time 197 ms. Worth stating because
the request that prompted this assumed the opposite — that spanning gaps
would be the expensive part.
No direct-API equivalent: `fs_block` costs 0.02 µs, so a loop over it
already is what a batch would be.
### Notes on when a batch helps
A batch is not always cheaper, and the way it fails is not obvious. What
these measurements established, over 300 cold blocks asked for as 100 lists
of three — a listing's shape, where the kernel asks for a window at a time:
| one block at a time, dispatched concurrently | 488 ms | 300 |
| `fh_read_many` per list | **445 ms** | **100** |
Requests fall threefold; wall clock barely moves. Both follow from the same
thing: a caller that dispatches concurrently has already overlapped the
latency, so what remains for a batch to save is requests, not time. A
consumer measured a 1.65× gain on cold blocks purely by dispatching
concurrently instead of awaiting one at a time — see
[docs/concurrency.md](docs/concurrency.md), where the cold row of the cost
table is the larger of the two effects.
And a batch that costs more crossings than the list has blocks loses. An
earlier two-crossing version of `fh_read_many` did exactly that on lists of
two or three, which is how the one-crossing shape above came about.
## [0.6.6] - 2026-08-22
**Upgrade from 0.6.5 if you use read-ahead.** `fs_read_ahead` /
`fh_read_ahead` as shipped in 0.6.5 could install a stale block and serve
it to a later read. It is silent — no error, no panic, wrong bytes. Do not
use 0.6.5's read-ahead.
### Fixed
- **A read-ahead whose fetch the file outran installed what it fetched.**
Read-ahead plans against the bmap as it is, fetches, and installs the
blocks afterwards. The only guard was `absorb_block`'s "drop it if the
block is already resident", which rests on an assumption that does not
hold: that a block which changed is still in the cache. `truncate` drops
every cached entry above the new size, so a block can be both changed
and absent — and then a copy fetched before the truncate is installed
after it.
Reported as silent bad data: fsx failed within a thousand operations,
deterministically at one offset, and an A/B on a single switch in one
binary put it on read-ahead. It was hard to see from this side because
fsx runs through a filesystem — the read it flagged was answered by the
kernel page cache and never reached hyperfile, so the read that actually
returned stale bytes was three operations earlier. Seven tests written
against the flagged read all passed.
Replaying the reporter's operation log directly, with no page cache in
the way and the read-ahead fired without waiting as a real one is, showed
the window opening 29 times and installing a stale block 3 times.
`State` now carries a mutation counter, bumped by everything that changes
what a block should contain. A read-ahead records it after planning and
the spawned fetch compares before handing anything back, discarding the
whole fetch if it moved. Replaying the same log then discards 9 fetches
and installs nothing stale. Coarse on purpose: any change invalidates
every read-ahead in flight, which costs a missed optimization rather than
correctness, and a counter cannot be wrong about blocks it does not know
about. The reporter has since verified this against their own fsx runs.
- **Read-ahead no longer reports cache hits.** `plan_read` records every
block it finds resident in `cache_hits`, and read-ahead plans the same
way a read does, so warming an already-warm range reported hits by the
thousand. Over a 16 MiB pass the number came out at 37187 where 4096
block accesses were the most possible — inflated about ninefold, which
made the counter unable to answer the question it exists for.
A hit is a read this layer served; a read-ahead is not serving anybody.
The numbers are now usable rather than merely smaller, and they show
something the inflated ones hid: a 1 MiB read-ahead window has fewer hits
than a 4 MiB one and still reads faster, because the gain here comes from
overlapping the fetch rather than from the hit rate.
### Tests
- `replay_fsx_log` replays an `fsx -d` operation log against hyperfile,
checking every read against an in-memory model. Optional — with no log it
says where it looked and passes; `HYPERFILE_FSX_LOG` points it at one.
Worth having because running fsx through a filesystem puts the kernel
page cache between it and hyperfile, which is how the bug above hid.
Replaying the operations directly catches a bad read where it happens.
Reads are preceded by a read-ahead fired without waiting, which is the
shape that found it.
### Notes on tuning read-ahead
Not a change, but what the reporter and these measurements established,
since the window size decides whether read-ahead helps at all:
- The window is a **pipeline depth**, not a cache-capacity figure. A wide
read-ahead is one coalesced request, and none of it is available until
all of it arrives, so too large delays the first usable byte. Too small
coalesces badly and costs more requests than no read-ahead at all.
- There is an optimum in between, and it moves with how fast the caller
consumes. Measured here at about 1 MiB reading in a tight loop; the
reporter measured 2 MiB through a filesystem, where each read costs a
round trip and the pipeline can be deeper.
- Do not derive it from `data_cache_blocks`, and do not reuse the value
used to widen the kernel's own read-ahead. Setting the window equal to
the cache is a bad point specifically: warming the next window evicts the
one being read.
## [0.6.5] - 2026-08-22
> **The read-ahead added here is defective — use 0.6.6.** It could install
> a stale block and serve it to a later read, silently. See 0.6.6.
Both additions came from one report about reads: bytes fetched ahead had
nowhere to live, and block access cost a channel crossing each.
### Added
- **Read-ahead** — `fs_read_ahead` / `fh_read_ahead` fetch a range into
the data block cache and return nothing but a count. A later read of
those bytes asks the ordinary way and finds them.
A byte read consults the cache and does not fill it, which is a
reasonable default — repeated reads mostly do not reach this layer, and
caching every read would evict what the write path is holding — but it
left read-ahead with nowhere to put anything. Bytes fetched
speculatively through a read were dropped and fetched again, so the
speculation was pure cost: a consumer measured object requests going
from 519 to 999 with cache hits unchanged at 692, for no throughput.
Only blocks brought in this way are cached, so a plain read still
cannot evict the write path's blocks. Planning is shared with `read`,
so requests are coalesced the same way: 64 contiguous blocks warm in
one request, and the 64 reads after it cost none. On the reactor
surface the fetches run off the handler task and each block returns
through the existing absorb hop, so a wide read-ahead does not hold up
other operations for the length of its requests. The range widens to
whole blocks and clamps to `i_size`; blocks already cached are left
alone and holes are skipped, since they read as zeroes without a
request.
- **Batch block access** — `fh_with_blocks(&[BlockIndex], f)` visits many
blocks in one crossing, calling `f(blk_idx, Some(bytes))` for a
resident block and `f(blk_idx, None)` for one that is not.
The cost of block access on the reactor surface is the crossing, not
the copy: a crossing is 16.2 µs measured here against a fraction of a
microsecond to copy a 4 KiB block. A caller touching hundreds of
blocks therefore spends nearly all its time in the channel. Over 300
warm blocks, a loop over `fh_with_block` takes 4.87 ms where one
`fh_with_blocks` takes 23.9 µs — 204x.
Nothing in a batch reaches staging, which is what keeps it cheap: an
uncached index is reported absent rather than fetched, so the handler
answers the whole batch without awaiting. Pair it with `fh_read_ahead`
and a region costs two crossings however many blocks it holds. The
closure may borrow from its environment, guarded as `fh_with_block`'s
is.
No direct-API equivalent: `fs_block` costs 0.02 µs, so a loop over it
already is what a batch would be.
### Changed
- `absorb_block` is no longer gated on `reactor`, so the direct API's
read-ahead can install blocks the same way the block path does.
### Documentation
- The data-cache tables in [docs/posix.md](docs/posix.md) and
[docs/block-api.md](docs/block-api.md) list the new entry points, with
why read-ahead has to be its own rather than something a plain read
does.
- [docs/block-api.md](docs/block-api.md) gains "Visiting many blocks at
once", including the measurement above and why a refcounted zero-copy
handle was not the answer to it.
### Tests
- Read-ahead is covered on the reactor surface, on the direct API, and on
memory staging where it needs no bucket. The assertions are on the
counters, since the property is invisible to timing: warming costs one
request and the reads after it cost none.
- `fh_with_blocks` is covered for a warmed batch and for uncached indices,
including that a batch issues no object requests.
## [0.6.4] - 2026-08-22
### Fixed
- **`HyperFileMetaConfig::block_ptr_format` now decides the format a new
file uses.** It is presented as that setting, carried through the config
builder, serialized and printed — and `HyperFile::new` ignored it,
writing `BlockPtrFormat::MicroGroup` regardless. `Flat` was unreachable
however it was configured, and nothing noticed because `open` reads the
format back from the bmap's user data.
The default moves from `Flat` to `MicroGroup` along with it. That is the
necessary half: `MicroGroup` is what every file created so far actually
got, so making the setting authoritative while leaving the default at
`Flat` would silently switch new files to a different on-disk encoding.
Aligning the default keeps what callers get exactly as it was.
Note for anyone reading `HyperFileMetaConfig::default()`: the value of
`block_ptr_format` has changed, even though the format files receive has
not. The `Flat` path had never run end to end before, since nothing
could select it.
### Added
- **In-memory staging** — `staging::memory::MemoryStaging` keeps segments
and the inode in a map instead of an object store, so the core can be
tested with no bucket, no credentials and no network. Only where the
bytes land changes: everything below `Hyper` is generic over the staging
backend, so the segment format, the bmap, block pointers, the caches,
flush and reopen all behave as they do against a bucket. `open` parses
through the shared `SegmentSum::from_slice` and `build_block_map` derives
its geometry the same way the S3 path does, so neither can drift.
Meta nodes are served by `btree_ondisk`'s `MemoryBlockLoader`. A
`BlockPtr` is already a single encoded value, so it is used as the flat
map's key directly and nothing decodes it on the way back; the map is
filled in `done`, where the key each node needs is recoverable from the
segment header.
Not available under `blocking`, which turns on `btree-ondisk/rc` where
that loader holds an `Rc` and is not `Sync`.
What it does not cover, because it belongs to S3 rather than to
hyperfile: conditional writes and the OCC built on them, multipart
upload, and the error kinds a real service returns. It also does not
reach the reactor or the `fs_*` wrappers, which are bound to S3 staging
through `Hyper`.
- `StagingType::Memory` and `StagingConfig::new_memory`.
### Tests
- New `functional_memory_staging`, eleven cases and the only suite that
needs nothing — no environment, no `--ignored`, about 0.1 s. Covers
round trips, several flushes across segments, an unaligned write
preserving the bytes around it, holes reading as zeroes, truncate both
ways, `write_zero`, a 600-block file whose bmap outgrows one meta node,
the read counters, `unlink`, storage isolation between handles, and a
full round trip on each block pointer format.
## [0.6.3] - 2026-08-21
Read the first two entries before upgrading: both are silent data loss on
the reactor surface, and one needs no optional feature.
### Fixed
- **A write concurrent with a flush zeroed the rest of its block, under
`wal`.** A 64-byte write into a 4 KiB block, issued while a flush ran in
another task, returned success and left those 64 bytes correct with the
other 4032 zero. The flush also returned success. Reproduces on 0.6.2,
so it has been there at least that long; without `wal` the same cases
are all correct.
A WAL write is a multi-hop pipeline and the handler task is free between
hops, so one write's two halves can straddle a flush. The first half
decides which blocks to fetch and the second applies the data, and
`update_cache` fabricates a zero-filled block when nothing is resident —
right for a write covering a whole block, silent loss for a partial one.
A block the write had fetched was sitting in the dirty list unmodified
and the flush swept it into its segment; a block that needed no fetch,
because it was already dirty, was swept the same way with nothing in
hand to replace it.
Fetched blocks now travel on in the request and are installed
immediately before the data is applied, in the same arm, where no flush
can interleave. A block taken anyway is recovered before the write is
applied — from the pinned segment with no object request where possible,
otherwise by going back for a staging read off the handler task.
- **A write whose read-modify-write could not read the block reported
success and persisted zeroes.** Affects the default build; no feature
needed. A partial write needs the block's current contents, and that
spawned staging read had its result discarded, with the task reporting
the full byte count either way. A failed read left the block as the
zeroed buffer it was allocated as, the write applied its bytes to that
and marked it dirty, and the next flush persisted the result.
Demonstrated by deleting the segment object and writing 64 bytes into a
4 KiB block: `Ok(64)`, `Ok(())` from the flush, and the block reading
back with 1000 checked bytes zero. The spawned loads now carry a
`Result` and such a write fails instead. The direct API was never
affected — it propagates the error.
- **One reactor handle used from several threads could stop answering,
permanently.** Reported from a FUSE filesystem reading through
`fh_read_owned` from spawned tasks while another thread wrote and
flushed; every thread parked, nothing outstanding on the caller's side,
two occurrences left running 13.5 and 19 minutes.
A writable handle without `range-lock` has a single per-file permit, and
a write carries it across a callback hop that only the handler task can
run. Anything that *waited* for that permit on the handler task
therefore waited on work only it could perform. `spawn_read`,
`block_mut`, `truncate` and the batch writes all did; each now takes the
permit without waiting and puts the request back on a miss. Verified by
reverting each in turn.
- **Under `range-lock`, continuous readers could starve a flush
indefinitely.** The owning task sat in `fh_flush` and never returned;
six readers looping on one handle were enough. A flush waits for
in-flight ranges to drain, and reads hold ranges too, but nothing held
new ones back during the wait — so the range map was never observed
empty. A livelock rather than a deadlock, so it also burned CPU.
A deferring flush now sets a flush-pending flag and new operations defer
instead of taking a fresh range lock. Without it the case fails 3 runs
out of 3; with it, 5 out of 5, and the suite drops from ~50 s to ~19 s.
- **A failed staging read was reported as a successful read.** The
reactor's spawned ranged GET discarded its result and reported the full
byte count, so an unreadable segment produced a successful read over a
buffer nothing had written into.
- **The WAL read fast path never ran.** With `wal`, a flush pins its
segment in memory precisely so reads can be served from it, but the read
paths deferred on `state.is_flushing()` with no WAL exception — and for
a WAL flush that flag is set from the kick until the write-out
completes, which is also when the buffer is released. So no read was
ever planned against a pinned segment: a probe on the planner's
in-flight decision fired zero times across every WAL-enabled suite.
Worse than inert. A deferred read is requeued and retried, so reads spun
for the duration of every flush — 118,535 requeues to complete 168 reads
in one test. Reads now proceed, and the same test completes about 10,000
reads, all served without an object request.
### Changed
- **Writes may overlap a WAL flush too**, for the same reason reads may.
What this costs a concurrent reader depends on what a write holds while
it overlaps: without `range-lock` the per-file permit is a single permit
for the whole file and a write holds it across its WAL write, so a
reader on unrelated blocks still pays (720 reads to 152). With
`range-lock` the permit is unbounded and the range is what excludes, so
a reader elsewhere is unaffected (848 against 712, inside the spread)
while a reader on the blocks being written pays, as it should.
A workload that reads and writes heavily through one handle wants
`range-lock`, and wants its readers and writers not to chase the same
blocks.
- A block a flush took out from under a write is recovered in place where
it can be — from the pinned segment, no object request and no extra
queue hop. 14 of 15 recoveries take that path in one test run; the
remaining case needs staging and keeps its object request off the
handler task.
### Added
- `ReadTimingSnapshot::inflight_reads` counts block reads answered from a
segment that is still being written out, which only happens with `wal`.
Throughput cannot distinguish that from a read that waited for the flush
and then read staging, so this is what a test can assert on.
Note this adds a field to a public struct that is not
`#[non_exhaustive]`. Code that constructs `ReadTimingSnapshot` with a
struct literal, or matches it exhaustively, needs updating; code that
reads it from `fh_read_timing` does not.
### Documentation
- New [docs/flush.md](docs/flush.md): what a flush does, why a non-WAL
flush stops the world and a WAL-protected one does not, what a write has
to do differently when its halves straddle one, the drain and its
admission control, and the measured cost to a concurrent reader. This
material was previously split between `concurrency.md` and `wal.md`;
both now keep a paragraph and a link rather than a copy.
- `HyperFile::flushing_segments` documents the invariant the design rests
on: `wal_set_mem_segment` and `wal_flush_done` pair up, the latter
dropping the pinned buffer in the same arm as it advances
`last_ondisk_cno`.
### Tests
- New `integration_reactor_s3_contention`, five cases: the reported
deadlock shape, the same with `truncate`, several writers plus readers
with the resulting data checked block by block, every permit-taking
operation driven at once, and colliding unaligned writes checking the
bytes neither writer touched. Each carries a watchdog, because the
failures it covers are hangs rather than assertions. Run it under
`range-lock` as well as default features — the mechanisms differ — and
in debug, where one of the races only showed up.
- `reactor_wal_read_block_in_segment_still_uploading` and
`reactor_wal_write_during_flush_keeps_the_untouched_bytes` cover reads
and writes against a segment still being written out.
- `reactor_read_reports_a_failed_staging_load` and
`reactor_write_reports_a_failed_read_modify_write` cover the failure
cases, the latter also checking the handle still works afterwards, since
a failing request holds the permit and its range.
### Internal
- One write-retrieve function instead of two near-identical ones, told
where to send the request when its loads finish.
## [0.6.2] - 2026-08-21
### Fixed
- **`fh_with_block` serialized every concurrent reader.** The read-only
block action was fetched inside the handler arm, and the handler takes
`&mut self` and runs one context at a time, so concurrent readers
queued behind each other — concurrency was 1 whatever the caller
spawned. Reported from a cold-metadata walk: eight concurrent walkers
over 800 records scaled 1.05x. Measured here at 0.96x, 48 cold blocks
taking 226 ms concurrently against 218 ms serially.
This was the last operation still doing that; the byte reads and all
the writes already fetched off the handler task.
Split by whether the arm has to await. A cache hit awaits nothing and
stays on the handler task, where overlapping it would buy nothing. A
miss is an object-store round trip and moves off: the block it fills
comes from `Cache::new_block`, so it is owned rather than borrowed
from the file, and the load, the closure and the cancellation gate
travel into a spawned task together. `HyperFile::block` cannot be used
for this — it yields a guard borrowed from `&mut self` — which is why
the arm ran to completion before.
The filled block is requeued afterwards to be cached, the way the
write path's retrieve requeues its fetched blocks. Skipping that would
have given away the reason to use the block path: caching keeps
repeated access at one request instead of one per access, and the
reporter measured the byte path costing 2.3x the requests for exactly
that reason.
Measured after the change: 48 cold blocks, 223 ms serially against
26 ms concurrently, 8.7x, with the request count unchanged.
- The read-only block path now defers while a flush is in progress, as
the byte paths do. It did not matter while the fetch ran inline, since
a non-WAL flush occupies the handler arm and a fetch could not overlap
it; now that the fetch spawns, it could.
### Documentation
- **What the reactor costs**, in `docs/concurrency.md`. Every `fh_*` call
is a request and a response across a channel, which is two thread
wakeups when the reactor is idle. Measured per block access:
| | direct | reactor, serial | reactor, concurrent |
|---|---|---|---|
| cache hit (work itself 0.02 µs) | 0.02 µs | 16.2 µs | 1.25 µs |
| cold, one object request | 4623 µs | 4687 µs | 475 µs |
The cost is latency, not throughput: the wakeups are only paid when
the queue is empty, so keeping requests outstanding removes most of
them. On anything that reaches the object store the two modes are
within 1.4%. The reactor's return is concurrency and only concurrency,
so it punishes serial use and rewards concurrent use — which is worth
knowing before benchmarking one mode against the other sequentially
and drawing a conclusion.
Referenced from the `reactor` and `blocking` feature documentation,
and from `docs/block-api.md`.
- `docs/block-api.md` gains a table of which reactor entry point fetches
off the handler task, and why `fh_with_block_mut` deliberately does
not.
## [0.6.1] - 2026-08-20
> **Affects 0.6.0 only.** If you took `fh_read_owned` from 0.6.0
> expecting reactor-mode concurrency, you got a concurrency of 1. Fixed
> below; no other release is affected, since that entry point was added
> in 0.6.0.
### Fixed
- **`fh_read_owned` serialized every concurrent reader.** It ran the
whole read — object-store round trip included — inside the handler
task, which takes `&mut self` and handles one context at a time. Every
concurrent reader queued behind the one before it, whatever the caller
spawned.
Reported with measurements from a mount: eight concurrent readers got
3.892 MiB/s against 18.24 for `fh_read`, and concurrent was *slower*
than a single reader (3.892 against 4.226), which is what queuing looks
like.
The owned read had its own op and its own arm, and that arm used the
serial read path deliberately: the spawning path sends its response
from a detached task after the arm returns, which would put the
buffer's lifetime back in question. That reasoning is right for
`fh_read`, whose buffer belongs to the caller, and wrong for a buffer
allocated for the request, which belongs to nobody else and can move
along with the work.
It now does what the write side has done all along: `fh_write_owned`
shares `FileReqOp::Write` with the borrowed form and distinguishes
itself with an `owned` field, which is why writes have had both
concurrency and cancellation safety. Reads match — one op, one spawning
arm, `FileReqRead::owned` deciding whether the response carries a byte
count or the buffer itself, and travelling with the request through the
flush-wait and range-lock requeues.
Measured after the change, eight concurrent 256 KiB reads against the
same reads done one after another: 34–38 ms versus 67 ms in release,
52–54 ms versus 96–107 ms in debug, stable across five runs.
Serialized they would match.
### Added
- `fh_dirty_block_count`, the reactor counterpart of
`Hyper::dirty_block_count`. Without it a caller deciding whether an
fsync has anything to write could not ask, and had to assume there was
always something pending and run the commit path every time.
### Changed
- `spawn_read` now answers its own errors instead of returning them for
the dispatch arm to report. The arm used to clone the response sender
to keep a second handle for the error path; an owned read answers on a
oneshot, which cannot be cloned. One owner of the reply is clearer
regardless. The two requeue paths still deliberately do not answer,
leaving that to the retried request.
## [0.6.0] - 2026-08-20
> **`fh_read` and `fh_write` must not be cancelled.** This has always
> been true and was never written down; see *Documentation* below. If
> you wrap either in a `timeout` or a `select!` branch, switch to the
> new owned variants.
### Added
- **A reactor block closure may now borrow the caller's locals.**
`fh_with_block` and `fh_with_block_mut` required `F: ... + 'static`,
so the closure could capture only owned values:
```rust
let v = fh.fh_with_block(idx, |blk| blk.to_vec()).await?;
out.copy_from_slice(&v);
fh.fh_with_block(idx, |blk| out.copy_from_slice(blk)).await?;
```
`Send` is still required — the closure runs on the reactor's thread —
and `R` is no longer `'static` either. One copy per block per
direction goes away, on the path whose purpose is to avoid copies.
The closure must not run once the caller is gone, so the request
carries a gate: the dispatcher checks it before calling the closure
and skips it if the caller has left, and the caller's guard sets it on
drop, however it leaves. The gate is taken after the block is in hand,
so the critical section is one closure body with nothing awaiting
inside it.
- **Cancel-safe byte read and write on the reactor surface**:
```rust
fh_read_owned(off, len) -> Bytes
fh_write_owned(off, buf: Bytes)
```
Nothing of the caller's is borrowed: the read allocates in the reactor
and hands the buffer back, and the write moves a reference-counted
`Bytes` into the request. Dropping either future is harmless.
These do not replace `fh_read` / `fh_write`, which keep their
signatures and remain the zero-copy path. Use the owned variants where
the operation may be cancelled.
### Documentation
- **`fh_read` and `fh_write` cannot be cancelled**, and now say so.
Both hand the reactor a bare pointer to the caller's buffer, which is
valid only while the caller stays parked on the response — dropping
the future can leave the reactor reading from or writing into freed
memory. With `wal`, a write's buffer is read straight into an
object-store PUT, so the window is a full round trip.
Long-standing behavior, not a change. It is called out here because
wrapping a read in a `timeout` is a natural thing to write, and
because both are safe functions, so nothing stops it.
Neither could be fixed the way the block closure was. The reactor
touches the buffer from inside the object-store request rather than in
one step afterwards, so there is no moment at which the work can
simply be skipped, and waiting for it would mean waiting across I/O —
which deadlocks, because the reactor's I/O can be driven by the
caller's runtime. Hence the owned variants above rather than a fix in
place.
- `docs/block-api.md` gains a table of which reactor entry point is
cancel-safe and who owns the buffer in each.
### Changed
- `FileReqWrite` gains an `owned: Option<Bytes>` field, which
`fh_write_owned` uses to keep the write's bytes alive for as long as
the request travels — through the absorb and requeue stages, and past
a caller that has gone away. `handler.rs` is public, so this is a
visible struct change; the write pipeline itself is unchanged.
### Tests
- Six cases added to `integration_reactor_s3_block_api`, run under
default, `wal` and `range-lock`: a closure mutably borrowing a
caller-owned buffer and filling it from the block, plus the write
direction; a hole leaving the borrowed local untouched; 32 rounds of
cancelling a borrowing closure mid-flight; the owned read agreeing
with `fh_read` byte for byte across aligned and unaligned windows and
being empty at EOF; the owned write round-tripping a full block and an
unaligned partial write over it; and 24 owned reads plus 24 owned
writes all actually cancelled mid-flight, after which every block is
uniformly one value or the other, so no cancelled write was partially
applied.
## [0.5.2] - 2026-08-20
### Added
- **Read-side counters**, the counterpart of `FlushTiming`. Read cost is
dominated by object-store round trips, and wall-clock timing cannot
tell one request for a coalesced range from many, nor a cache hit from
a fetch.
```rust
Hyper::read_timing() -> &ReadTiming Hyper::read_timing_reset()
```
Requests are split by what they fetch — data blocks, index nodes, the
inode — because the three answer different questions: how well block
reads coalesce, how many nodes a lookup descended, and a per-open
constant. `cache_hits` counts block reads that never reached staging,
and `staging_ns` lets a share of wall time be attributed rather than
inferred.
For scale, the same 1 MiB of data with only the request granularity
varying:
| | `data_gets` | `meta_gets` |
|---|---|---|
| one `fs_read` of 1 MiB | 1 | 4 |
| 256 `fs_read` of 4 KiB | 256 | 4 |
| 256 `fs_block` | 256 | 4 |
- The same counters on the reactor surface: `fh_read_timing` and
`fh_read_timing_reset`, plus `fh_flush_timing` and
`fh_flush_timing_reset`, which the reactor handle never had. These
return owned snapshots rather than references to the live counters,
which cannot leave the reactor task.
### Documentation
- **Which entry points populate the data cache is now stated.** The
block API populates it, the byte API does not, and both read from it:
| entry point | populates the data cache |
|---|---|
| `fs_read` / `fh_read` | no |
| `fs_block` / `fh_with_block` | yes |
| `fs_block_mut` / `fh_with_block_mut` | yes |
So reading the same bytes twice through `fs_read` fetches them from
staging twice, while borrowing the same block twice fetches it once. A
byte read after a block borrow of the same data is served from memory;
a block borrow after a byte read is not.
This is long-standing behavior, not a change, but nothing said so —
`docs/block-api.md`'s cache section described only `fs_block`, and
`fs_read` and `fh_read` had no doc comment at all. Worth checking if
you warm a range with `fs_read` and then borrow its blocks
individually: every borrow fetches again. `read_timing` now shows
this in one measurement.
Covered in `docs/block-api.md`, `docs/posix.md`, and the rustdoc for
each entry point.
### Tests
- `integration_s3_read_timing`, 7 cases, checking the counters count
what they claim: 64 contiguous blocks in one segment costing a single
request, the cache-population asymmetry in both directions, index
requests counted separately from data requests (including that the
batched index fetch reports both of the requests it makes), holes and
unflushed writes costing nothing, and `reset` zeroing everything.
- Three cases added to `integration_reactor_s3_block_api` covering the
reactor accessors, and asserting the cache rule holds on that surface
too — untestable before the accessors existed.
## [0.5.1] - 2026-08-19
> **If you used `fs_block_mut` or `fh_with_block_mut` on 0.5.0, some of
> those edits may never have been persisted.** See the first entry
> below. Data written through the byte API is unaffected. There is no
> way to recover a lost edit after the fact — the segment holding it was
> written, but nothing points at it — so re-apply anything you cannot
> verify.
### Fixed
- **`fs_block_mut` silently lost an in-place edit of an already-cached
block, for any reader that had not made the edit.** The flush returned
`Ok` and wrote a segment, a read on the same handle returned the new
bytes, and a fresh open returned the *previous* version. No error was
reported anywhere.
`block_mut` had a fast path for a block already in the cache: promote
it into the dirty tier, mark it dirty, return — without touching the
block map. `flush_process_build_segment` collects the set of dirty
meta nodes *before* it assigns pointers to data blocks, so a map node
that is not already dirty when the flush starts is never written: the
pointer the flush stored existed only in memory, and the persisted map
still named the old segment. The byte write paths avoid this by
inserting a placeholder for every block they dirty, which is what
marks the containing node dirty; the fast path skipped exactly that.
Two things hid it. While the whole map still fitted in the inode's
inline root the update rode along with the inode, which every flush
writes, so it only appeared past the spill threshold — 7 blocks from
index 0, 4 from a large index. And any byte write in the same flush
window masked it, because that write marked the node for every
block-API edit sitting beside it. A caller that happens to write one
structure of its own through the byte API on every commit would never
have seen it.
Reported against 0.5.0 with a reproduction; the block API was
introduced in 0.5.0, so no earlier release is affected.
- **A shrinking `truncate` within the current last block lost its tail
zeroing** the same way, and for the same reason:
`truncate_last_data_block` returned early when the block was already
cached, leaving it dirty but the map node clean. The discarded region
read back as the pre-truncate bytes after a reopen.
This one predates the block API and is reachable through the byte API
alone. It produces the same symptom as the truncate bug fixed in
0.4.3, by a different route: 0.4.3 was a cross-block shrink going
through the bmap sweep, this is a shrink that stays inside one block.
Existing truncate coverage missed it because every case wrote few
enough blocks to keep the map inline.
- `fs_block_mut` did not update `mtime` when the block was already
cached.
### Changed
- The ordering constraint behind both defects is now recorded at both
`bmap_lookup_dirty` call sites in the flush path: any path that puts a
block into the dirty tier must insert into the bmap for that index in
the same flush window. Neither of the two places that violated it
looked wrong on its own, and one of them carried a comment asserting
the opposite.
### Tests
- `block_mut_edit_survives_after_the_bmap_spills`: 8 and 30 blocks from
index 0, and 8 from index 33554432; every block edited, every block
checked after a reopen.
- `block_mut_and_byte_writes_interoperate_after_a_spill`: a block first
written by `fs_write` then edited in place, and the reverse.
- `block_mut_updates_mtime`.
- `smoke_truncate_same_block_tail_persists_after_bmap_spill`: byte API
only.
- Each was verified to fail with its fix reverted.
## [0.5.0] - 2026-08-19
### Added
- **Block API.** A caller that uses hyperfile as block storage rather
than as a file can now borrow the cached block instead of copying
through it. On `Hyper`:
```rust
fs_block(idx) -> Option<BlockRef> fs_block_mut(idx, create) -> Option<BlockMut> fs_block_state(idx) -> BlockState ```
`BlockMut` needs no write-back call: the block is dirty from
acquisition, so an early return between acquire and drop cannot lose
the modification, and repeated borrows inside one flush window
collapse to a single version, matching `write`. The point is to remove
the reason such callers keep a shadow copy of every block they intend
to modify, which is a recurring source of
copy-diverges-from-original bugs.
`BlockState` separates `Unmapped` / `Zero` / `Mapped`. Through the
byte API all three read back as zeros, so a caller holding its own
index could not tell "never written" from "real data that happens to
be zeros" — the difference between a consistent index and a lost
block.
Neither borrow moves `i_size`; these callers manage their own address
space. A block dirtied above EOF is still durable, but `read` stops at
`i_size`, so it is reachable only through this API. `i_blocks` is
updated either way.
- **Closure-scoped block access on the reactor surface**:
`fh_with_block(idx, f)` and `fh_with_block_mut(idx, create, f)`.
The reactor cannot hand out borrow guards. The `Hyper` lives inside
the reactor task, so a guard would reach the caller only once the
response arrives, after which the reactor is free to serve the next
request and invalidate it. So the action travels to the block: `f`
runs inside the reactor task while it holds the real borrow, and only
owned values cross the channel.
- `docs/block-api.md`, covering both surfaces, the three hole states,
why `i_size` is left alone, and why the reactor surface takes a
closure.
### Fixed
- **The local-disk *data* cache aborted the process on release.**
`close` called `libc::close` on a descriptor owned by a
`std::fs::File`, and both `Cache::shutdown` — the normal path from
`release` — and `Drop` called it:
fatal runtime error: IO Safety violation: owned file descriptor
already closed, aborting
- **The same tier corrupted memory on an extending write.** It mapped
the file's own address space and located a block at
`addr + blk_idx * data_block_size`, but the cache is told the new size
only *after* the blocks are created, so a write past EOF minted a view
outside the mapping and wrote through it. Observed as `SIGSEGV` and as
dynamic-loader assertion failures. Three related consequences of the
same addressing scheme: a newly created file has size zero and `mmap`
rejects a zero length, so the tier could not create a file at all; the
mapping could not grow, since `mremap` was called without
`MREMAP_MAYMOVE` and its result asserted unmoved, which it must be
because views already handed out point into it; and a sparse file
could not be mapped, because the mapping had to span the address space
rather than the resident blocks.
The mapping is now a fixed pool of block-sized slots, sized from the
cache capacity and the flush threshold, and a block takes whichever
slot is free. A block index is never used as an offset. A block's slot
is derived from its address rather than tracked separately, so the two
cannot disagree. When the pool is exhausted — one large write dirties
every block it touches before the flush check runs — blocks fall back
to heap allocations, costing memory rather than correctness.
- **`write`, `flush`, `read` within one open panicked on the same tier**
with `assertion failed: !block.is_locked()`. `plan_read` probed the
cache with `Cache::get`, which mlocks the block it hands out of the
clean tier and asserts on the next `get` that it was not already
locked; the read executor then called `get` for real. Reopening the
file masked it, since the clean tier starts empty and reads miss.
None of these three were reachable through the default (in-memory)
data cache, and no test selected the local-disk tier, which is why
they went unnoticed.
### Changed
- Blocks newly created in the local-disk cache are zeroed explicitly.
Releasing a slot punches its hole, so a recycled slot did read as
zeros, but depending on that made every release path load-bearing for
correctness rather than only for reclaiming space. Partially-filled
new blocks have caused two stale-data bugs already (0.4.3, 0.4.5).
- The local-disk cache file's size now follows the cache's capacity
instead of the file's size. It is scratch space with no persistent
contents — nothing repopulates the cache at open — so this is not an
on-disk format change, but the file on disk will be a different size
than before.
### Tests
- `integration_s3_block_api`: 9 cases, run under all three data cache
configurations (in-memory, local-disk, disabled), covering the borrow
contents, hole/zero/data discrimination, in-place persistence,
`create`, one-version-per-flush-window, `i_size` being untouched, and
access-mode enforcement.
- `integration_s3_local_disk_cache`: 7 cases, the first suite to select
that tier. Creating an empty file, releasing repeatedly without
aborting, writing far past EOF, a 4 TiB sparse address space, slot
recycling under constant eviction, truncate releasing slots without
leaving stale bytes, and pool exhaustion.
- `integration_reactor_s3_block_api`: 6 cases, including a check that
the same edits applied through `fs_block_mut` and through
`fh_with_block_mut` produce byte-identical files.
- All three should be run under debug assertions as well as release:
much of what they cover only fails with `debug_assertions` on. See
`tests/README.md`.
## [0.4.6] - 2026-08-17
### Fixed
- **A segment smaller than 512 KiB could not be opened.**
`SegmentReadWrite::open` speculatively read `SEGMENT_HEADER_FETCH_SIZE`
(512 KiB) into a pre-sized buffer, and the underlying getter rejects a
response shorter than the buffer ("feched size 28672 less than input
buffer size 524288"). A short read is expected here, since the range
deliberately asks for more than the object may hold.
New `S3Ops::do_get_object_speculative` returns the body as `Bytes`
rather than filling a caller-supplied buffer, which removes the
length mismatch by construction; the exact-length getter is unchanged
for its other callers, which all request an exact byte count. It also
maps `NoSuchKey` to `ErrorKind::NotFound`.
`open` has no caller inside hyperfile — reads go through the block map
and the block loaders — so this only affected segment tooling:
`hyperfile-cleaner` uses it, which broke `prune` (and hypercli
`file prune` / `file du`) on files written with small flushes, i.e. one
flush per filesystem commit.
- **Unbounded allocation and an integer underflow in the same path.** The
top-up read for a summary larger than the speculative fetch sized its
buffer from `s_bytes`, a `u32` off the segment header, so a corrupted
value could dictate an allocation of up to ~4 GiB. The size was also
computed before the guard that is its only consumer, so it underflowed
for any summary below 512 KiB — the normal case, since `s_bytes` covers
the summary rather than the segment (240 bytes for an 8 KiB segment,
4304 for a 1 MiB one). That panicked under debug assertions and was
silently discarded in release.
The top-up now uses the speculative read as well, so it is bounded by
data that actually exists and no length is derived from the header. A
summary still short afterwards is reported as `InvalidData` naming both
sizes, instead of over-allocating or parsing a truncated summary.
- The speculative range end was off by one: `bytes=0-524288` is
inclusive, so it requested 524289 bytes.
### Tests
- New `integration_s3_segment_open` suite covering single-flush segments
of 4 KiB, 8 KiB, 512 KiB, 1 MiB and 16 MiB — below, at and above the
speculative fetch size. `open` previously had no coverage because it
has no in-crate caller. The suite must also be run under debug
assertions, since the underflow above was invisible in release; see
`tests/README.md`.
## [0.4.5] - 2026-08-15
### Fixed
- **Panic on every data-cache lookup when the clean-block cache is
disabled.** The guard
`(data_cache_blocks > 0).then(|| cache.op()).unwrap()` is inverted:
`bool::then` yields `None` for a false condition, so the `.unwrap()`
panicked exactly when the cache was *disabled*.
This was reachable from hyperfile itself, not just from a bad config
value: both the create and open paths force `data_cache_blocks` to 0
when `O_DIRECT` is set and the `wal` feature is off, so **`O_DIRECT`
was unusable without `wal`** — the first read after a flush aborted
the process. For a FUSE daemon that means the mount dies and
applications see `ENOTCONN`.
Fixed at all 12 sites across `mem_cache` and `local_disk_cache`. The
ten `get` / `pop` sites now use `.flatten()`. The two `clear_dirty`
sites are restructured, because there the guard also decides the fate
of a block that cannot be cached: the in-memory cache drops it (the
data is already persisted and the buffer is a heap allocation), and
the local-disk cache drops it **and punches the backing hole** — its
blocks are unzeroed mmap views, so leaving the bytes behind would let
a later write to the same index observe pre-drop content, the same
failure class as the stale-clean-block bug fixed in 0.4.3.
## [0.4.4] - 2026-08-14
### Added
- **POSIX access mode enforcement.** An operation the handle's access
mode does not permit now fails with `EBADF`:
| Handle | Rejected |
| --- | --- |
| `O_RDONLY` | `write`, `write_zero`, `write_batch`, `write_aligned_batch`, `truncate` |
| `O_WRONLY` | `read` |
Previously a write on a read-only handle landed in the data cache
and a subsequent flush could persist it, and a read on a
write-only handle succeeded. POSIX lists `[EBADF] ... not a valid
file descriptor open for writing` (`write()`) and `... open for
reading` (`read()`) as mandatory errors. `ftruncate()` permits
`[EBADF]` or `[EINVAL]`; `EBADF` is used so every access-mode
violation reports one errno. Enforced on the direct `fs_*` API, the
reactor `fh_*` API, and the tokio wrapper.
`lseek` is not gated — POSIX requires no particular access mode for
it, and neither do the `SEEK_DATA` / `SEEK_HOLE` extensions behind
`fs_seek_data` / `fs_seek_hole`.
Two categories of internal I/O deliberately bypass the checks and
keep working: write-side operations that read (a sub-block write's
read-modify-write, and a shrink to a non-block-aligned size reading
the tail block), and WAL crash recovery, which replays previously
acknowledged writes during `open` — including a read-only open, so
a file that crashed mid-flush still presents correct contents.
- `HyperFileFlags::is_readable()` and `is_writable()`, keyed off the
access mode alone. `O_APPEND` does not grant write access on its
own, matching Linux, where `open(O_RDONLY | O_APPEND)` followed by
a `write` fails with `EBADF`.
### Changed
- **Compatibility note.** Callers that relied on the previous
permissive behaviour — writing through an `O_RDONLY` handle, or
reading through an `O_WRONLY` handle — now receive an error where
the call used to succeed. Open with `O_RDWR` if a handle needs both
directions.
- Because `std::io::ErrorKind` has no `EBADF` variant, the error is
built with `std::io::Error::from_raw_os_error`. Test it with
`err.raw_os_error() == Some(libc::EBADF)`; `err.kind()` is the
unmatchable `Uncategorized`. See `docs/posix.md` for why
`PermissionDenied` (EACCES) and `InvalidInput` (EINVAL) were
rejected as inaccurate.
### Documentation
- `docs/posix.md`: the open-mode table previously claimed `fs_write` /
`fs_truncate` "return `EBADF`-shaped errors from the underlying
staging layer", which was never true — S3 has no notion of the
handle's access mode. Replaced with an "Access mode enforcement"
section carrying a rejected/allowed matrix and the errno rationale.
## [0.4.3] - 2026-08-14
### Fixed
- **Data integrity: a shrinking truncate left stale clean-cached
blocks above the new EOF.** On an open handle, `write` at offset
X → `flush` → truncate below X → truncate back above X → read at
X returned the pre-truncate bytes instead of zeros. POSIX requires
the grown-back region to read as a hole.
The truncate path discarded only *dirty* cache entries above the
boundary. The `flush` is what makes a block clean, and a clean
block lives solely in the clean cache tier, which was never swept
— while the read path consults that tier first. Only the handle
that performed the truncate was affected; the persisted state was
always correct, so a close/reopen hid the problem.
Both cache tiers are now swept. The local-disk cache additionally
punches the backing hole for each dropped block, because its
blocks are unzeroed mmap views and a later partial write to the
same index could otherwise resurface pre-truncate bytes from the
cache file.
Found by `fsx` (xfstests) against a FUSE filesystem layered on
hyperfile. `write_zero` and the (absent) hole-punch paths were
audited and have no equivalent gap.
### Changed
- The internal `Cache` trait method `truncate_dirty_blocks_above` is
renamed `truncate_blocks_above`, reflecting that it now drops clean
entries too. The trait is `pub(crate)`, so this is not a public API
change. Its return value still counts dirty removals only, for
`i_blocks` accounting.
## [0.4.2] - 2026-06-28
### Added
- **Keep-open inode flag** (foundation for opt-in cross-mount open-but-unlinked
in a filesystem layer):
- `InodeRaw::FLAG_KEEP_OPEN` (`i_flags` bit) + `is_keep_open()` /
`set_keep_open(bool)`.
- `Inode::is_keep_open()` / `set_keep_open(bool)` (marks the inode attr-dirty).
- `HyperFile::set_keep_open(bool)` / `is_keep_open()` and
`Hyper::fs_set_keep_open(bool)` / `fs_is_keep_open()` — set (and persist on
flush) or read the flag on an open file.
### Compatibility
- Additive and on-disk compatible with `0.4.0`/`0.4.1`: no layout change
(`size_of::<InodeRaw>()` is still 160) and the flag is a previously-unused
`i_flags` bit.
## [0.4.1] - 2026-06-28
### Added
- **Device-node `rdev` accessors on `InodeRaw`** (so a filesystem can persist
character/block device nodes):
- `InodeRaw::set_rdev(rdev)` / `InodeRaw::rdev()` — store/read a device node's
`rdev`. A device node has no data segments, so the value is kept in the
otherwise-unused `i_last_cno` slot; call `set_rdev` after `set_inline(&[])`
(which zeroes the tail).
- `Inode::to_stat` now reports `st_rdev` from the inode for char/block
(`S_IFCHR`/`S_IFBLK`) modes, ignoring the passed `rdev`; non-device inodes
are unchanged (they use the passed value).
### Compatibility
- Additive and on-disk compatible with `0.4.0`: no layout change
(`size_of::<InodeRaw>()` is still 160) and the new behavior only affects
device-mode inodes, which `0.4.0` never produced.
## [0.4.0] - 2026-06-27
### Breaking changes
- **`InodeRaw` on-disk layout reordered.** The tail fields are now laid out
contiguously as `i_size, i_blocks, i_last_seq, i_last_cno, i_bmap` (previously
`i_blocks`/`i_size` sat near the top). `size_of::<InodeRaw>()` is unchanged
(160 bytes), but the **serialized byte layout differs**, so inodes written by
`0.3.x` are not readable by `0.4.0` and vice versa. There is no in-place
migration; this is intended for fresh trees. Consumers that embed `InodeRaw`
(e.g. `hyperdir` dirents/scatter) must be rebuilt against the same version so
both sides agree on the layout.
### Added
- **Inline-small-file format primitives on `InodeRaw`** (foundation for storing
a tiny file entirely within its inode, with no separate `FILE/<uuid>` object):
- `InodeRaw::FLAG_INLINE` — `i_flags` bit marking a fully-inlined file.
- `InodeRaw::inline_offset()` / `inline_cap()` — the inline payload region (the
contiguous `i_blocks..i_bmap` tail), `80` bytes on the current layout,
computed via `offset_of!` so they track the layout automatically.
- `is_inline()`, `inline_data()`, `set_inline(&[u8])`, `clear_inline()` — read
and populate the inline bytes, with `i_size` as the length.
- No behavior change on its own: this commit ships the format + accessors only;
the create/read/write/spill paths that use it land in a later change.
## [0.3.3] - 2026-06-01
### Fixed
- **Precise error kinds for S3 failures**: S3 operations previously
collapsed most failures into `ErrorKind::Other`. They now map the
SDK error to the semantically closest `std::io::ErrorKind` (404 →
`NotFound`, 403 → `PermissionDenied`, 409/412 → `AlreadyExists`,
429/5xx → `ResourceBusy`, 400-class → `InvalidInput`, timeouts →
`TimedOut` / `ConnectionReset`), so consumers (e.g. a FUSE layer)
can react precisely instead of treating every failure as EIO.
Call sites with bespoke handling (OCC conflict detection on the
conditional inode write/delete, 404 on inode read) keep their
explicit checks, so that behavior is unchanged.
### Changed
- During flush, a `429`/`5xx` from the inode write now maps to
`ResourceBusy` and is retried by the flush retry loop with
backoff (bounded by the existing max-retries cap) instead of
failing immediately.
## [0.3.2] - 2026-05-31
### Fixed
- **Create-time timestamps**: a freshly created, never-written file
or directory left `atime` and `mtime` at epoch 0; only `ctime`
was set. POSIX requires a newly created object to report `atime`,
`mtime` and `ctime` all at creation time. `Inode::default_dir` /
`default_file` now stamp all three (from a single clock reading,
so they are equal) instead of only `ctime`. No change to
write / chmod / chown / utimens timestamp semantics.
## [0.3.1] - 2026-05-30
### Added
- **SEEK_DATA / SEEK_HOLE**: `Hyper::fs_seek_data(off)` and
`Hyper::fs_seek_hole(off)` for sparse-file extent discovery.
`fs_seek_data` returns the smallest offset `>= off` holding data
(or `None`, which the caller maps to `ENXIO`, if only holes remain
to EOF); `fs_seek_hole` returns the smallest offset `>= off` in a
hole, treating EOF as an implicit hole.
- A block counts as data if it is an unflushed write in the dirty
cache or a non-zero block in the bmap; a hole is an absent or
zero block that is also not in the dirty cache. The seek path is
therefore consistent with `fs_read`, including for
written-but-not-yet-flushed blocks.
- The bmap scan uses `BMap::seek_key` to skip runs of absent keys
in O(log n), so SEEK_DATA over a large fully-sparse file is
cheap. SEEK_HOLE over a large fully-dense region remains O(n)
because zero blocks are present bmap keys indistinguishable from
data without a per-key lookup.
- `fs_seek_hole(off == size)` returns `size`; `off > size` returns
`None`.
## [0.3.0] - 2026-05-27
### Highlights
- **Read coalescing**: contiguous in-segment block reads are merged into a
single ranged S3 GET, giving **100×+ speedup** on sequential reads
(4 MiB sequential read: 4290 ms → 41 ms; 0.9 MB/s → ~100 MB/s) and
~1000× fewer S3 GET requests on the same workload.
- **Zero-copy flush**: peak transient memory during flush dropped from
257 MiB to 0.16 MiB on a 1 MiB write (Phase 1 + Phase 2 combined),
and from 321 MiB to 7.67 MiB on a 64 MiB write.
- **POSIX coverage**: `O_TRUNC`, `O_APPEND`, `O_EXCL`, `O_NOATIME`
honored at the open / write paths, and `fdatasync` is now a separate
API from `fsync`. `ctime` semantics fixed to match Linux.
- **WAL hardening**: WAL objects are deleted after successful flush,
crash-recovery is covered by integration tests, and S3Wal pure-logic
paths have unit tests.
- **Reactor + concurrency hardening**: handler death surfaces as
`BrokenPipe` instead of hanging, flush conflicts are now resolved by
a configurable policy, and `concurrent-segment-build` no longer
loops forever on small flushes.
- **Test growth**: ~13k lines of new tests across 9 integration
binaries, 217 unit tests (was ~10).
### Performance
- Coalesce contiguous in-segment block reads into one ranged GET on
both the direct (`HyperFile::read`) and reactor (`spawn_read`)
paths. Two new safety caps on `HyperFileRuntimeConfig`:
`read_get_max_bytes` (default 16 MiB) and `read_max_concurrency`
(default 10).
- Zero-copy flush body via scatter-gather: the segment is built as a
list of `Bytes` slices and uploaded directly without copying through
an intermediate buffer.
- Zero-copy cache → flush handoff via `Bytes::from_owner`, so dirty
data block buffers move ownership into the flush path instead of
being copied.
### Added
- **POSIX**:
- `fs_fdatasync` / `fh_fdatasync` on `Hyper` / `HyperFileHandler`,
distinct from `fsync` (skips inode metadata flush when only data
blocks are dirty).
- `O_EXCL` honored on `do_open_or_create`: returns `AlreadyExists`
when the file already exists.
- `O_APPEND` honored on `fs_write` / `fs_write_zero`: writes are
redirected to the current end-of-file regardless of the requested
offset.
- `O_TRUNC` honored on open: file size is set to 0 after a
successful open if requested.
- `O_NOATIME` honored on open: `atime` is not bumped on read when
set.
- `fs_rename` / `fh_rename` reserved as APIs returning `Unsupported`
(not yet implemented; reserved so users can plan around it).
- **API**:
- `HyperFileRuntimeConfig::flush_conflict_policy` (`FailFast` |
`RetryWithBackoff`) controls how the flush retry loop reacts to
a `PreconditionFailed` from the conditional inode write.
- `HyperFileHandler::fh_from_hyper` to construct a handler from an
existing `Hyper`.
- `Staging::load_range` trait method (parallel to
`load_data_block`), implemented on `S3Staging` as a single ranged
`GetObject`.
- **Examples / benches**:
- `examples/bench_flush.rs`: flush latency baseline harness with
per-phase breakdown.
- `examples/bench_flush_mem.rs`: peak-memory measurement during
flush, used to validate the zero-copy work.
- `examples/bench_read.rs`: throughput / latency for sequential,
stride, and random read patterns.
- **Tests**:
- 9 focused integration binaries (`integration_s3_smoke`,
`_rollback`, `_contract`, `_concurrent`,
`integration_reactor_s3_smoke`, `_failure`, `_range_lock`,
`_wal`, `_all_features`).
- 217 unit tests covering handler, RangeLock, S3Wal pure-logic
paths, segment byte-layout, and core modules.
- **Docs**:
- `docs/posix.md`: POSIX coverage, permissions, rename, fsync /
fdatasync, timestamps, supported / unsupported open flags.
- `docs/wal.md`: WAL design and operational guidance.
- `docs/concurrency.md`: direct vs reactor access modes, locking,
range-lock semantics, reactor failure modes.
- `docs/tests.md`: testing guide and feature matrix.
- Cargo features documented; reactor/blocking exclusivity called
out.
### Fixed
- **Truncate**:
- Shrinking to a block boundary preserves the last block (was
incorrectly dropping it).
- Dirty cache entries above the new size are evicted (was leaking
stale dirty data into the next flush).
- **Stat**: `st_blocks` reflects sparse holes (was always reporting
size / 512).
- **ctime**: bumped on every metadata-affecting operation
(`set_size`, `set_mode`, `set_uid_gid`, `update_stat`); previously
`update_mtime` and `update_stat` skipped `ctime` and Linux clients
saw stale values.
- **Reactor**:
- Handler-task death surfaces as `BrokenPipe` to the user instead
of hanging on a closed mpsc channel.
- `spawn_write` / `spawn_write_zero` use a non-blocking permit
acquire so a busy reactor returns `ResourceBusy` instead of
parking the dispatch task.
- `concurrent-segment-build` no longer loops forever on flushes
smaller than the segment buffer.
- **Flush correctness**:
- Transient flush failure (e.g. PreconditionFailed) is rolled back
cleanly instead of silently committing a partial state.
- `concurrent-segment-build` cfg gating no longer leaves dead
`file_off` assignments in segment build.
- **Memory safety**:
- `DataBlock` flags use `AtomicU64` instead of `write_volatile`
(latter was UB under the C++20 memory model the compiler now
targets).
- `SegmentSum` serialization no longer uses `unsafe transmute`;
encoded form is now a regular `repr(C)` struct with pinned-byte
tests.
### Changed
- **Breaking** — see also the dedicated section below.
- `Hyper::last_cno` returns `Result<u64>` (was `u64`) so the reactor
can surface handler-death symmetrically with read / write APIs.
- `FileResp` is now an `enum` (was a union with manual `Drop`); the
enum is `Send + Sync`, drops cleanly, and removes one source of
unsafe code.
- WAL objects are deleted after a successful flush (both reactor and
direct API paths). Previously they accumulated until manual prune.
- Conditional inode write failure now surfaces as
`ErrorKind::AlreadyExists` (was a generic `Other`), which the
retry loop now uses to drive `FlushConflictPolicy`.
- Default integration test layout: one large `integration_s3.rs` was
split into four focused binaries; reactor counterpart added.
### Breaking changes (from 0.2.0)
If you depend on `hyperfile = "0.2"`:
- `Hyper::last_cno() -> u64` is now `-> Result<u64>`. Callers must
handle `Result`.
- `FileResp` is now an enum. If you were constructing or matching it
manually (you shouldn't have been; it's `pub` only by virtue of
being part of the reactor channel ABI), update accordingly.
- `HyperFileRuntimeConfig` gained three new fields:
`flush_conflict_policy`, `read_get_max_bytes`,
`read_max_concurrency`. All have `#[serde(default = ...)]` so
existing serialized configs continue to deserialize cleanly. Code
that builds `HyperFileRuntimeConfig` field-by-field via the public
struct literal needs to add the new fields.
- `Staging` trait gained a required `load_range` method. Custom
staging backends must implement it (one ranged-read primitive).
- POSIX flag handling is now active. Code that opened with `O_TRUNC`
or `O_APPEND` and relied on the flag being silently ignored will
see different behavior. This is intentional and matches Linux.
- `ctime` bumps on more operations than before. If your tests
hard-code `ctime` values, they may need updating.
### Dependencies
- `btree-ondisk` `0.17` → `0.18` (transitively `0.16`-compatible
resolution still works for downstream consumers).
- `hyperfile-reactor` `0.2` → `0.3.2`.
- Other AWS SDK / tokio bumps tracked in `Cargo.lock`.
### Removed
- Stale doc note about a `multi-handle-hang` limitation under WAL
(the underlying race was fixed; the test that exercised it is
back).
- Stale doc note about a `concurrent-segment-build` known gap (gap
closed; suite now in `integration_reactor_s3_all_features`).
### Migration notes
For `0.2.0` consumers:
```toml
# Cargo.toml
[dependencies]
- hyperfile = "0.2"
+ hyperfile = "0.3"
```
If you maintain a custom `Staging` implementation, add:
```rust
async fn load_range(&self, segid: SegmentId, s3_off: usize,
buf: &mut [u8]) -> Result<()> {
// single ranged GET into buf
}
```
If you build `HyperFileRuntimeConfig` literally:
```rust
HyperFileRuntimeConfig {
// ... existing fields ...
+ flush_conflict_policy: FlushConflictPolicy::default(),
+ read_get_max_bytes: 16 * 1024 * 1024,
+ read_max_concurrency: 10,
}
```
Or use one of the constructor methods (`default()`,
`default_middle()`, `default_large()`) which set the new fields.
If you call `Hyper::last_cno`:
```rust
- let cno = hyper.last_cno();
+ let cno = hyper.last_cno()?;
```
---
## [0.2.0] - 2025-09-21
The 0.2 series is where Hyperfile became a usable random-read-write
file abstraction over S3, with WAL durability, optional caches, and
a reactor concurrency model. ~180 commits since 0.1.0.
### Highlights
- **WAL**: optional Write-Ahead Log support to preserve durability of
unflushed data, including detection of unclean WAL on open and a
full WAL flush-recovery path on reopen.
- **Reactor**: a dedicated tokio-runtime-based handler model
(`HyperFileHandler`) that decouples user calls from the underlying
flush worker via mpsc channels and a high-priority queue.
- **Range lock**: optional `range-lock` feature for fine-grained
concurrency on disjoint write ranges.
- **Concurrent segment build**: optional `concurrent-segment-build`
feature that appends dirty data blocks to the segment buffer in
parallel during flush.
- **Local-disk caches**: optional `local-disk-cache` feature for both
the data block cache and the bmap node cache, sized via runtime
config.
- **POSIX surface**: file mode, `fs_chmod` / `fs_chown`,
`fs_getattr` / `fs_getattr_fast`, default modes for files and
directories, `fs_unlink_with_interceptor` for testing.
### Added
- **WAL** (`feature = "wal"`):
- `WalReadWrite` trait with `S3Wal` implementation.
- `HyperFileWalConfig` plumbed through `HyperFileConfigBuilder`.
- `wal_flush_recovery` on open; replays WAL chunks if previous
process crashed mid-flush.
- WAL chunks listed via `list_segments` / `list_chunks` for
introspection.
- Read-through in-memory segments while a flush is in flight, so
readers can see in-flight writes without waiting for S3.
- **Reactor** (default; switched off by `--no-default-features
--features blocking`):
- `HyperFileHandler` constructed from `Hyper`; clonable and
shareable across tasks.
- Dedicated tokio multi-thread runtime for spawned read / write /
flush worker tasks.
- High-priority queue for absorb requests; permit-controlled
concurrent open in read-only mode.
- Spawn-style read / write / write_zero paths with handles
collected into an `ImmOrJoinSize` aggregate.
- **Range lock** (`feature = "range-lock"`):
- Per-file rangemap of in-flight write ranges, aligned to data
block size.
- `try_lock` / `unlock` for non-blocking acquire on disjoint
ranges.
- **Concurrent segment build** (`feature =
"concurrent-segment-build"`):
- `spawn_append` path for appending dirty data blocks to the
segment buffer concurrently with flush prep.
- **Local disk cache** (`feature = "local-disk-cache"`):
- `Cache` trait with `MemCache` (in-memory LRU, default) and
`LocalDiskCache` (mmap-backed, optional) implementations.
- `NodeCache` generic on `HyperFile` for the bmap; pluggable.
- mlock for hot blocks; graceful shutdown to flush pending writes.
- **POSIX**:
- `FileMode` with default file/dir modes; `fs_chmod`, `fs_chown`,
`update_stat` APIs.
- `fs_getattr_fast`: stat without opening the full file.
- File type bits preserved across chmod (input mode merged
instead of overwriting).
- `fs_unlink_with_interceptor` for testing failure paths.
- **Batch I/O**:
- `write_aligned_batch` / `write_batch` for bulk writes.
- Sorted-and-deduplicated input blocks.
- Batch meta-block loader (`feature = "batch"`) for fewer
round-trips when warming the bmap.
- **Configuration**:
- `HyperFileConfigBuilder` for building file configs from
meta / staging / runtime / wal pieces.
- JSON serialization of `HyperFileConfig` (template generation
via `hypercli config gen-template`).
- Meta config now stored in the inode, so reopens use the file's
own configuration instead of caller-provided defaults.
- Cache config: `data_cache_blocks` / `bmap_cache_limit` runtime
fields.
### Fixed
- bmap insert failure now panics (the dirty cache was already
polluted at this point and there is no path back to a consistent
state); previously the error surfaced as a regular `io::Error`
and let the caller continue with corrupt state.
- `update_cache` corner cases: throw away the cached entry on a full
block overwrite; ensure block is locked when moved from cache list
to dirty list; do not unlock if the block is still dirty.
- Truncate edge cases: handle `NotFound` on the highest key when
extending; correct handling when shrinking from 0 with -1 diff;
recompute target block index after split.
- Segid encoding bug in flat block-pointer format.
- Mul-overflow guard when computing block addresses on 32-bit
targets.
- Flush lock flow under WAL: defer `set_last_flush` until
`wal_flush_done`; correct flushing-state clear on metadata-only
flushes.
- Reactor write/return convention: pass write task results through
consistently so 0-byte writes don't panic.
- Strong-count trick to extend in-memory segment lifetime until
`wal_clear_mem_segment` (was racing with reader teardown).
### Changed
- **Breaking**: `HyperTrait` reshaped multiple times; reactor /
blocking exclusivity is now enforced via cargo features.
- Cache layer factored behind a `Cache` trait; concrete
implementations select via cargo features.
- Default runtime config split into `default()`, `default_middle()`,
`default_large()` profiles.
- Bunch of internal renames: `Block` → `DataBlock`, `AlignedBlock`
→ `AlignedDataBlock` → `AllocDataBlock`; `Writer` lifetime
annotation removed.
### Tooling
- `examples/submit_batch.rs`: programmatic batch submission demo.
- `staging benchmark` code base for read/write performance harness
(driven from `hypercli staging benchmark`).
---
## [0.1.0] - 2025-03-15
The 0.1 series imported the original prototype and turned it into a
working library with a tokio wrapper, basic POSIX-shaped APIs, and
the first wave of bug fixes. ~85 commits.
### Highlights
- File-like API over S3 with random read / write / truncate, all
block-aligned and backed by a B+tree-indexed log-structured
segment format.
- Tokio-friendly wrapper and runnable examples.
### Added
- **Core architecture**:
- `HyperFile`, `Hyper`, `HyperFileHandler` (`fh`), and the
handler-based async dispatch shape.
- `HyperFileConfig` / `HyperFileConfigBuilder` to assemble meta,
staging, and runtime configs.
- `StagingConfig::new_s3_uri` / `new_s3_staging` constructors.
- **File ops**:
- `read`, `write`, `write_zero`, `truncate`, `flush`,
`flush_ext` (returns last cno), `last_cno`.
- Random read/write demo example, write-zero example, batch-write
example.
- **Cache basics**:
- Dirty data blocks reflected into bmap on flush.
- Read path: `NotFound` from bmap → zero block (sparse files
work).
- **Tokio integration**:
- `HyperFileTokio` wrapper to use `Hyper` from
`tokio::io::AsyncRead` / `AsyncWrite` consumers.
- Tokio-friendly examples.
- **Default profiles**:
- `HyperFileRuntimeConfig::default_large()` for large-file
workloads.
### Fixed
- Several truncate bugs: hardcoded data block size; last-block trim
edge cases; handling of "not found" during the truncate-extend
path; reverse split for dir/filename in unlink.
- MD5 mismatch now panics (was silently accepted).
- Incorrect nanosecond multiplier in time math.
- Various conditional-compile mistakes around staging features.
- Removed unsafe transmute on inode struct.
- Self-reference address bug in lifetime cheat.
### Changed
- `Block` renamed to `DataBlock`; `AlignedBlock` to
`AlignedDataBlock`.
- HyperTrait simplified; inode ops merged into `Hyper`.
- Removed the cached `s3_client` field from `HyperFile` (moved to
staging layer).