# Completed checklist: windows-ioring-sys
Append-only. See [CHECKLIST.md](CHECKLIST.md) for pending and in-progress work.
## Moved 2026-08-22 -- M1 through M6: ring lifecycle through consumer documentation
### M1 -- Ring lifecycle and capability negotiation
- [x] **M1.1** -- Create the crate skeleton: `Cargo.toml`, `src/lib.rs`, README, and add it to the
workspace `members`. `windows-sys` features are `Win32_Foundation`, `Win32_System_IO`, and
`Win32_Storage_FileSystem` (which is where the IoRing bindings live). Everything is behind
`cfg(windows)`, as elsewhere in this repository.
- [x] **M1.2** -- A typed owner for `HIORING` whose destructor is `CloseIoRing`, per the repository rule
that a resource with a specialized destructor gets its own owner rather than a generic handle wrapper.
Not `Clone`.
- [x] **M1.3** -- Capability query and version negotiation (D-6). `QueryIoRingCapabilities` needs no ring,
so expose it as a free function for consumers deciding whether to use the crate at all. Ring creation
negotiates `min(highest we understand, caps.MaxVersion)`, stores it, and exposes it. Surface
`UM_EMULATION` rather than hiding it: a consumer reaching for this crate to maximize throughput needs to
know the ring is emulated.
- [x] **M1.4** -- Probe every op once at construction into a cached capability set, plus
`supports_raw(op_code)` for ops the OS has and this crate has not wrapped (D-7). The public op enum is
`#[non_exhaustive]`.
- [x] **M1.5** -- Integration test: create and close rings at each version the machine supports, assert
capability reporting is self-consistent (a ring never claims an op its capability set denies), and assert
a ring created at a negotiated version reports that version back through `GetIoRingInfo`.
### M2 -- Operation identity, buffer ownership, and rundown
- [x] **M2.1** -- Duplicate `IoBuf`/`IoBufMut` from `windows-overlapped-io-sys` (D-1), with the contract
extended to cover registration outliving a single operation. Record the extension in DESIGN-NOTES; do
**not** extract a shared crate yet -- that decision is M6+.
- [x] **M2.2** -- `Token<B>` owning its buffer, with a generation-stamped `usize` as the `UserData` carried
through the SQE and returned in the CQE (D-4). Validation on completion rejects a token that does not
match the generation, so a stale token cannot claim a later operation's completion -- the same guarantee
`OperationId` makes in `windows-overlapped-io-sys`, but cheaper here because `UserData` is ours to choose
rather than being an address we have to stamp separately.
- [x] **M2.3** -- `Drop` on an uncompleted token forgets its buffer rather than freeing it (D-4). Test that
the leak happens and that nothing is freed, because a use-after-free here would be silent and remote.
- [x] **M2.4** -- In-flight accounting and rundown: `CloseIoRing` must not run with operations outstanding.
Mirror the rundown discipline `windows-overlapped-io-sys` already uses, including its bounded, rechecked
wait rather than an unbounded one.
### M3 -- The submission builder
- [x] **M3.1** -- `Batch` holding `&mut IoRing`, submitting on drop, with `submit()` consuming it and
returning the submitted count (D-5). Document loudly that the submission queue is ring state, so a
dropped batch still submits -- the alternative would strand SQEs whose buffers a later unrelated submit
would hand to the kernel.
- [x] **M3.2** -- Per-op builders reached from the batch, each returning a token from `push()`:
`read`, `write`, `flush`, `cancel`. Options chain (`offset`, `drain_preceding` for
`IOSQE_FLAGS_DRAIN_PRECEDING_OPS`) so the common case stays short and the barrier stays discoverable.
Every method checks its op's capability bit from M1.4.
- [x] **M3.3** -- Submission-queue backpressure: `push()` surfaces `IORING_E_SUBMISSION_QUEUE_FULL`
(`0x80460002`, observed at exactly entry 64 on a 64-entry queue) as a distinguishable error rather than
auto-flushing. Auto-flush would silently change submission ordering and timing.
- [x] **M3.4** -- `submit_and_wait(n, timeout)` exposing the fused form, which is the primitive Model B is
built on (D-3) and not merely a convenience.
- [x] **M3.5** -- The narrow unsafe raw-SQE seam (D-7), documented with the same framing as the `device`
family's unsafe `ioctl`: it exists so a consumer is not blocked on us wrapping a new op.
- [x] **M3.6** -- Integration tests: a batch of many reads round-trips with every `UserData` preserved and
every buffer returned; a deliberately overfilled queue reports backpressure at the right entry and stays
usable afterwards; a dropped batch still submits; a cancel of a target that is not outstanding reports
`ERROR_NOT_FOUND` through the completion rather than at build time.
- [x] **M3.7** -- *(added during execution)* Completion retrieval: `IoRing::try_pop` popping one
`Completion` (identity plus result) without blocking, so a caller matches it against a held `Token` via
`claim_if`. **Re-plan:** M3's original items assumed `Token::claim_if` alone was enough to exercise the
M3.6 round-trip, but nothing in M1/M2 exposed a way to actually pop a *typed* completion outside of
`IoRing`'s internal, untyped rundown drain -- M3.6 cannot be tested without this. Added and implemented
in the same pass as M3.1-M3.6 rather than deferred, per the re-planning discipline.
### M4 -- Model A delivery: completion event and the thread pool
- [x] **M4.1** -- Refuse to construct the event-driven path when `SET_COMPLETION_EVENT` is absent, with
`io::ErrorKind::Unsupported`, rather than silently degrading into a thread-based loop. Behavior is owned,
not inherited: a consumer who asked for threadless delivery and got a thread has been told something
false. The threaded path is a separate, explicit choice (M6+).
- [x] **M4.2** -- Wire `SetIoRingCompletionEvent` to a `ThreadpoolWait` from `windows-threadpool-sys`,
using an auto-reset `WaitableHandle::event`. The drain discipline is drain-to-`S_FALSE`, re-arm, drain
again: the spike showed the event is set when completions land and auto-resets on wait, so the worst case
is a harmless spurious callback, but the ordering is what makes a completion arriving between the last
pop and the re-arm impossible to lose.
- [x] **M4.3** -- Teardown: quiesce the wait, run down outstanding operations, then close the ring, in that
order. Interaction with `CleanupGroup` documented, since a consumer will reasonably expect to put this
in one.
- [x] **M4.4** -- Integration test: submit with `wait_operations = 0` and assert completions are delivered
on pool threads without the submitting thread ever waiting; assert teardown with operations in flight
neither hangs nor closes the ring early.
### M5 -- Registration
- [x] **M5.1** -- Registered file handles: a typestate carrying the registered index, so a `read` against
a registered file cannot be written against an unregistered one by mistake.
- [x] **M5.2** -- Registered buffers, with the ownership rule that registration outlives any single
operation (M2.1) and the pinning cost documented, because it is the axis that punishes over-sharding
(D-8).
- [x] **M5.3** -- Integration test: a read addressing both a registered file index and a registered buffer
index round-trips; a registration outliving many operations stays valid; dropping the registration while
operations are in flight is refused rather than permitted.
### M6 -- Documentation for consumers, and the guidance that motivated this crate
- [x] **M6.1** -- Render the "Two delivery architectures" material from [DESIGN-NOTES.md](DESIGN-NOTES.md)
as crate-level rustdoc and README content, not only as a maintainer-facing design note. Consumers
reaching for this crate are trying to maximize I/O throughput; the Model A / Model B trade-off, the
reason the NUMA node is the wrong partitioning key, and the observation that buffer placement likely
dominates thread placement are the things they need and cannot easily derive.
- [x] **M6.2** -- A worked example of Model A: threadless delivery through the pool, which is the shape most
consumers should start with.
- [x] **M6.3** -- Topology guidance as documentation rather than as API (D-8): how to enumerate L3 domains
with `GetLogicalProcessorInformationEx`, why processor groups are a hard floor, and how to allocate a
node-local pool with `VirtualAllocExNuma`. Pointers, not a partitioning policy.
## Moved 2026-08-23 -- M7: `ring-copy`, a topology-aligned sample
- [x] **M7.1** -- The `Topology -> Policy -> RingPlan` pipeline, with `Policy` as named code (`ByL3`,
`ByNode`, `ByPackage`, `ByCore`, `Single`) rather than data. A plan names, per domain, the ring to
create, the processors to affinitize to, and where its buffer pool should be allocated.
- [x] **M7.2** -- Reject a plan the platform cannot express, rather than emitting an impossible affinity
mask: a fed-in description may carry one group with more than 64 processors, which is legal in the
description and unrepresentable on Windows (topology D-10).
- [x] **M7.3** -- The copy engine itself: read and write through per-domain rings, buffers allocated with
`VirtualAllocExNuma` and registered once per ring.
- [x] **M7.4** -- Switches for topology source (discovered, or a JSON file), policy, and **buffer
placement** (node-local versus deliberately remote). Buffer placement is the variable most likely to
show a real effect, because the device DMAs into the registered buffer on every operation, whereas
callback placement is a one-time cache-warmth question.
- [x] **M7.5** -- Report per-domain throughput, and **say plainly when the host cannot show a
difference** -- a single-node or virtualized machine will produce noise, and a benchmark that reports
noise as a result is worse than no benchmark. The machine this was designed on reported zero NUMA nodes.
## Moved 2026-08-28 -- M11: the completion event as a ring primitive
### M11 -- The completion event as a ring primitive (external consumer proposal, 2026-08-28)
Prompted by a consumer proposal and the spike that answered it; the exchange is recorded in
[DESIGN-SESSION-2026-08-28-completion-event-multiplexing.md](design-sessions/DESIGN-SESSION-2026-08-28-completion-event-multiplexing.md),
and the decisions are [D-19](DESIGN-NOTES.md#d-19) through [D-22](DESIGN-NOTES.md#d-22).
M11.1 and M11.2 build the primitive; M11.3 then consolidates `EventDelivery` onto it *and* fixes the
stranded-backlog bug in one change. Those were separate items when the fix looked like it needed to ship
ahead of the API work; they are merged because in practice the two land minutes apart, and splitting them
would mean writing a `SetEvent`-after-arm patch that M11.3 immediately deletes.
- [x] **M11.1** -- Added `IoRing::completion_event(&mut self) -> io::Result<OwnedHandle>`
([D-20](DESIGN-NOTES.md#d-20)): capability-checks `IORING_FEATURE_SET_COMPLETION_EVENT`, creates and owns
an auto-reset event, attaches it with `SetIoRingCompletionEvent`, signals it once, and returns a
duplicate. Idempotent -- a repeat call returns another duplicate of the same event rather than attaching a
new one, which matters because `SetIoRingCompletionEvent` *replaces* rather than adds, so a second attach
would silently detach the first subsystem's event. The rustdoc states the
[D-19](DESIGN-NOTES.md#d-19) contract in full: signalled on empty -> non-empty, drain to empty before
waiting again, a wake with nothing to pop is normal. Ordering detail worth keeping: the event is stored on
the ring *before* it is signalled, so no later failure can drop it and leave the ring signalling a closed
(possibly recycled) handle; and because a manual `Drop` body runs before its fields drop, `CloseIoRing`
always runs before the event handle closes. Contract tests are M11.2; a throwaway smoke check confirmed
the setup signal, auto-reset, idempotence, and -- the one that matters for M11.3 -- that attaching to a
ring with a completion *already* queued does signal.
- [x] **M11.2** -- Contract tests for the M11.1 primitive in
[tests/completion_event.rs](tests/completion_event.rs), written against the *stated* rules rather than
against current behaviour: eleven cases covering the setup signal, the backlog case (submit, let the
completions land, *then* attach, assert the returned handle signals), idempotence, the edge itself
(empty -> non-empty signals; a batch of eight produces exactly one wakeup; the edge re-arms after each
drain; a full drain leaves no leftover signal), the duplicate's independence (closing one duplicate,
and outliving the ring), the capability gate's `Unsupported` branch, and the multiplexed wait.
Each rule is named in the test that pins it, so a failure is readable without opening
[DESIGN-NOTES.md](DESIGN-NOTES.md).
Every case was **verified by sabotage**, which is what makes these contract tests rather than
behaviour snapshots -- and doing it corrected the suite three times, each correction worth keeping:
- Suppressing `completion_event`'s setup `SetEvent` fails all eleven, the backlog case on its own
message. That is the property M11.3 depends on.
- Making the repeat call attach a *second* event initially failed only **one** test. The other two
idempotence cases passed **vacuously**: asserting that the handle returned by the *second* call
signals is satisfied perfectly well by a freshly attached event, so a silently detached first handle
went unnoticed -- the exact bug those tests exist to exclude. Both were rewritten to assert through
the *first* handle, and to close the *second* duplicate rather than the first; the sabotage now
fails all three.
- The multiplexed test's first shape proved that draining mattered but not that draining on *every*
pass did. Sabotaging round 1's drain to a single `try_pop` fails at **round 2**, not round 3,
because round 2's unrelated-wake drain rescues the seven stranded completions -- which is precisely
why rule 2 says every pass and not merely every ring pass. A round 4 was added that breaks rule 2
deliberately and asserts the resulting lost wakeup as a timeout, so the deadlock is demonstrated
rather than described; a control run confirms the same wait returns immediately when the queue was
drained first, so the assertion measures the edge and not an impatient timeout.
These are integration tests because generating completions crosses the filesystem boundary, matching
[tests/event_delivery.rs](tests/event_delivery.rs). The suite runs in about half a second, nearly all
of it the one deliberately-asserted timeout.
- [x] **M11.3** -- `EventDelivery::new` re-expressed on top of `IoRing::completion_event`, which removed
the second `SetIoRingCompletionEvent` call site ([D-20](DESIGN-NOTES.md#d-20)) and fixed the
stranded-backlog bug ([D-19](DESIGN-NOTES.md#d-19)) in one change. `src/` now has exactly one
`SetIoRingCompletionEvent` call, in `IoRing::completion_event`.
The repro landed first and was **watched failing**: a ring handed over with eight completions already
in its CQ delivered nothing, and
`completions_queued_before_handover_are_still_delivered` in
[tests/event_delivery.rs](tests/event_delivery.rs) failed on a five-second delivery timeout. After the
consolidation it passes instantly. The two pre-existing tests in that file both hand over a *fresh*
ring, which is exactly why the bug survived CI. `completion_event`'s signal-once-on-attach is what
makes it pass; `ThreadpoolWait` takes the returned duplicate through
`WaitableHandle::assume_waitable`, and the drain/re-arm/drain callback body needed no change.
Also removed: `EventDelivery`'s own copy of the capability check and its `Unsupported` error message,
which duplicated `completion_event`'s word for word. One statement of that rule now, not two.
Documentation corrected in the same commit, per CONTRACT INTEGRITY's blast-radius rule. The rustdoc
had asserted the backlog guarantee while the code did not hold it, so it is now stated as a guarantee
the method *buys* -- naming the edge that would otherwise strand the backlog and the setup signal that
closes it -- with a note that a caller on an earlier version cannot rely on it. Swept
`SetIoRingCompletionEvent` / "already queued" / "handed over" across `src/`, `tests/`, `examples/` and
`*.md`: 6 files updated ([src/event_delivery.rs](src/event_delivery.rs),
[src/capability.rs](src/capability.rs), [examples/model_a_delivery.rs](examples/model_a_delivery.rs),
[DESIGN-NOTES.md](DESIGN-NOTES.md)'s D-20 status marker, its Model A description, and its "this bit us"
paragraph). Design-session transcripts and [COMPLETED-CHECKLIST.md](COMPLETED-CHECKLIST.md) were left
alone as historical record.
One newly reachable interaction documented on `EventDelivery::ring`: calling
`IoRing::completion_event` on a ring the pool is already waiting on now *succeeds*, returning a
duplicate of the pool's own event, which is two waiters on one ring and a
[D-21](DESIGN-NOTES.md#d-21) violation. Worth stating because the same call was previously worse and
equally silent -- it replaced the pool's event and stopped delivery outright.
- [x] **M11.4** -- `windows-threadpool-sys` is now optional behind a default-on `threadpool` feature
([D-22](DESIGN-NOTES.md#d-22)). Gated with it: the `event_delivery` module and its unit tests, the
`EventDelivery` re-export, [tests/event_delivery.rs](tests/event_delivery.rs), and
[examples/model_a_delivery.rs](examples/model_a_delivery.rs) -- the example via an explicit
`[[example]]` with `required-features`, so cargo *skips* it rather than failing to compile it.
Verified non-vacuous by `cargo tree`: `windows-threadpool-sys` disappears from the graph entirely
under `--no-default-features`, and the other two examples still auto-discover and build.
CI gained an `ioring-no-threadpool` job, which is the cost D-22 accepted the gate with: nothing else
in the workflow builds that configuration, because every `--workspace` step uses the default set and
`--all-features` turns the gate back on. It runs build, clippy `-D warnings`, test, and doc for
`--no-default-features`, plus isolated default-feature clippy and test steps (selecting the crate
alone, so `--workspace` feature unification cannot mask a gap -- the same reason
`windows-file-watcher` has isolated steps).
The `cargo doc` step is there because of a defect this item found rather than by symmetry: the
crate's ungated "Choosing a delivery architecture" prose intra-doc-linked ``[`EventDelivery`]``,
which resolves under `--all-features` and *dangles* without the feature. The repo-wide `docs` job
only documents `--all-features`, so it could never have caught it. The link is now plain code naming
the feature, matching how `windows-overlapped-io-sys` avoids linking its own gated items from ungated
prose; the failure was confirmed by restoring the link and watching
`cargo doc --no-default-features` fail with `unresolved link to EventDelivery`.
Also documented for consumers: a Cargo-features table in [README.md](README.md) with the
`default-features = false` snippet and D-22's actual rationale -- layering, not runtime cost, since
linking the thread-pool crate creates no threads (the Win32 default pool is process-wide and lazily
instantiated).
- [x] **M11.5** -- Both facts now stated wherever the wakeup shapes are described, as a blast-radius
sweep rather than a single edit. Grepped `drain_preceding` / `DRAIN_PRECEDING` / "two delivery" /
"completion event" / "delivery architecture" across `src/`, `tests/`, `examples/` and `*.md`; 18 files
matched, 4 changed, and the rest are accounted for below.
**Fact 1 -- Model B's wakeup source is separable from its identity.** The
[DESIGN-NOTES.md](DESIGN-NOTES.md) "Two delivery architectures" section described Model B as a thread
parked in `SubmitIoRing` and offered no alternative, which is exactly how the framing came to be read
as fixing the wakeup mechanism ([D-3](DESIGN-NOTES.md#d-3)'s amendment note). It gained a subsection
naming the two wakeup sources -- fused submit-and-wait, and a multiplexed `WaitForMultipleObjects` over
`IoRing::completion_event` -- as a table, with the point that identity is *who owns, submits and
drains* and the wakeup is a separate axis. [README.md](README.md) gained the same in consumer form.
[src/lib.rs](src/lib.rs) already stated it (M11.1); its Model B paragraph gained a forward pointer so a
reader who stops there does not leave with the fixed-wakeup reading.
**Fact 2 -- `drain_preceding`'s barrier stops at the ring's edge.** Stated in
[DESIGN-NOTES.md](DESIGN-NOTES.md)'s Category 2 and in [src/lib.rs](src/lib.rs), but **not on
`PushOptions::drain_preceding` itself** -- the one place a consumer actually meets the flag. That was
the real gap this sweep found. Its rustdoc now states all three measured properties: ring-wide and
spanning submissions with no cross-epoch pipelining ([D-24](DESIGN-NOTES.md#d-24)), powerless in both
directions across the ring boundary with `IoRing::completion_event` as what this crate offers instead,
and that the flag orders but does not flush -- while being what makes a flush cover preceding writes
at all ([D-23](DESIGN-NOTES.md#d-23)). README states the limit too, as the reason the multiplexed
shape has to exist.
**Accounted for, unchanged:** `IoRing::completion_event` and the `IoRing` type docs in
[src/ring.rs](src/ring.rs) already state both facts correctly (M11.1);
[src/event_delivery.rs](src/event_delivery.rs) and
[examples/model_a_delivery.rs](examples/model_a_delivery.rs) are Model A only and make no wakeup claim
beyond it; [src/batch/tests.rs](src/batch/tests.rs) and the integration tests are uses of the flag and
the primitive rather than statements about them (the cross-path limit is not testable in-crate -- it
is a fact about I/O this crate does not issue); [PLANS.md](PLANS.md) is an index summary with no
wakeup claim; and [COMPLETED-CHECKLIST.md](COMPLETED-CHECKLIST.md), the design-session transcripts and
the spikes are historical record, deliberately left alone.
- [x] **M11.6** -- [examples/model_b_multiplexed.rs](examples/model_b_multiplexed.rs): a caller-owned ring
whose `completion_event` is waited on via `WaitForMultipleObjects` alongside a manual-reset shutdown
latch, draining to empty on every pass, over three waves of reads. It also covers the half the item did
not name but which a consumer hits immediately -- shutdown while I/O is outstanding -- by quiescing
through `Batch::submit_and_wait` before the ring closes, which doubles as a demonstration that both
wakeup sources are Model B (M11.5's fact 1) and that switching between them changes nothing about
ownership.
**The central claim was verified by sabotage.** Replacing the drain-to-empty with a single `try_pop`
reproduces the lost-wakeup deadlock exactly: wave 0 pops one of eight completions, the queue never
returns to empty, the edge never re-arms, and the wait times out with all remaining work stranded. That
is [D-19](DESIGN-NOTES.md#d-19) as a running program rather than a paragraph, which is the whole point
of the item.
Two comments were **removed after measurement contradicted them**, rather than left as plausible
prose. The example requests shutdown while the final wave is outstanding, and an earlier draft claimed
that made the drain-on-shutdown-pass and the quiesce load-bearing. Six runs showed otherwise: the loop
always exits with zero in flight, because `WaitForMultipleObjects` reports the *lowest* signalled index
and the ring is index 0, so a set shutdown latch can never starve completions. That is a genuinely
useful fact for a consumer choosing handle order, so the example now states it and reports what
actually happened on each run instead of asserting an outcome it does not control.
The example uses no thread pool, so it is also the first real consumer of the `--no-default-features`
configuration M11.4 added to CI; verified building under both feature sets.
## Moved 2026-08-28 -- M12: durability exposed, and the flush barrier made explicit
### M12 -- Durability: expose it, and stop defaulting it wrong
The kernel exposes a durability parameter on writes (`FILE_WRITE_FLAGS`) and on flushes
(`FILE_FLUSH_MODE`); this crate hardcodes both, so a consumer sees ordering but no way to express
durability at all. Worse, [D-23](DESIGN-NOTES.md#d-23) measured that an unflagged flush does *not*
cover preceding writes -- which made `Batch::flush(&file, PushOptions::default())`, the obvious
spelling, a silent data-loss bug rather than a missing feature. **M12.1 removed that spelling**; the
remaining items expose the parameters the crate still hardcodes.
Decisions: [D-23](DESIGN-NOTES.md#d-23) through [D-25](DESIGN-NOTES.md#d-25). Measurements are
reproduced by the drain spike recorded in
[DESIGN-SESSION-2026-08-28-external-consumer-correspondence.md](design-sessions/DESIGN-SESSION-2026-08-28-external-consumer-correspondence.md).
M12.1 is first because it is a correctness defect in shipped 0.1.2, not an enhancement.
- [x] **M12.1** -- The barrier decision is now explicit for flushes ([D-23](DESIGN-NOTES.md#d-23),
[D-25](DESIGN-NOTES.md#d-25)). `Batch::flush` and `Batch::flush_raw` take a required
`FlushCoverage` -- `CoversPrecedingOperations` or `Unordered` -- **in place of** `PushOptions`,
rather than alongside it. Replacing it rather than adding to it is the point: keeping both would let
a caller write `FlushCoverage::Unordered` with `PushOptions::new().drain_preceding(true)` and mean
two contradictory things at once. `PushOptions` carries exactly one decision, and for a flush
`FlushCoverage` *is* that decision.
An enum rather than a `bool` because `flush(&file, true)` does not say what the `true` decides, and
a two-variant type makes the wrong choice unwriteable by accident rather than merely discouraged.
The `Unordered` variant is documented with the two uses that are legitimate (host sequencing, and a
flush not being used for durability at all) so it reads as a deliberate choice rather than an
escape hatch.
Rustdoc on `flush_raw` now states the measured contract in full: that the ring has no FUA and the
flush is therefore its only durability primitive, that an unflagged flush was measured completing
while 17 and then 23 of 32 preceding writes were still outstanding, and that durability is a
property of an epoch rather than of a write. `PushOptions::drain_preceding` cross-references it so a
reader arriving from the flag side learns the flush no longer inherits the decision.
A unit test pins the `FlushCoverage` -> SQE-flag mapping, **verified by sabotage**: making
`CoversPrecedingOperations` map to `IOSQE_FLAGS_NONE` compiles and passes every other test in the
crate, and would lose data only on power failure -- exactly the class of defect that needs a
mechanical check rather than review.
Swept the statements this changed: [DESIGN-NOTES.md](DESIGN-NOTES.md)'s epoch construction and
barrier-cost table now name the API instead of the raw flag, [D-25](DESIGN-NOTES.md#d-25) carries an
implementation-status marker, and the M12 intro above no longer describes a spelling that exists.
The two `flush_raw` call sites in [tests/submission_lifecycle.rs](tests/submission_lifecycle.rs) use
`Unordered` with a note saying why (they test backpressure, not durability). The broader durability
sweep across `lib.rs` and `README.md` stays M12.5, which is where those documents first have to
discuss durability at all.
**Breaking change**: both flush entry points changed signature.
- [x] **M12.2** -- [tests/flush_barrier.rs](tests/flush_barrier.rs) proves the barrier *behaviour*,
not the flag. The unit test in [src/batch/tests.rs](src/batch/tests.rs) pins the enum-to-SQE-flag
mapping, which shows the flag is set but not that setting it changes what the kernel does; this
reproduces the D-23 shape against a real device and asserts the difference. Verified by sabotage --
mapping `CoversPrecedingOperations` to `IOSQE_FLAGS_NONE` fails it on behaviour, not on a flag
comparison.
All three of the spike's failed iterations are encoded as requirements rather than left to be
rediscovered: `FILE_FLAG_NO_BUFFERING` (buffered writes finish in issue order), a pre-written extent
(extending writes serialize), and a **size asymmetry** between the two phases (uniform sizes do not
reorder). The third was not in the item's text and cost a full rewrite to find: a first version used
32 uniform writes and one flush, and skipped on this machine because it could observe nothing.
**The measurement disagreed with D-23, and the disagreement is now recorded rather than smoothed
over.** On this machine *no* preceding write ever completed after an unflagged flush -- 0 of 32,
against the spike's 17 and 23 -- yet 11 of 32 writes queued *after* the flush completed before it.
Reordering was plainly happening; it just did not manifest as the flush overtaking the writes ahead
of it. So the control accepts *either* direction of reordering and skips only when it sees neither;
requiring D-23's specific observable would have made the test silently vacuous here.
[D-23](DESIGN-NOTES.md#d-23) now carries the amendment, and "The two measured facts" states the
consumer-facing consequence: **observing that your flush lands last is not evidence you can omit the
barrier** -- it is incidental behavior of one device stack, which is exactly what PLATFORM INTEGRITY
says never to bind to.
The covering case also asserts [D-24](DESIGN-NOTES.md#d-24)'s other half (writes queued after a
drained flush are held until it completes), which is the observable that actually discriminates on
this hardware. Short-write detection guards the whole thing: an unbuffered write with a misaligned
offset or length would otherwise make every count meaningless.
- [x] **M12.3** -- `FILE_WRITE_FLAGS` is exposed as `WriteCaching` (`Cached` / `WriteThrough`), a
required argument on all four write entry points -- `write`, `write_raw`, `write_registered`,
`write_registered_raw` ([D-25](DESIGN-NOTES.md#d-25)). A typed enum rather than a raw flag word, and
`Cached` is `#[default]` so the previous hardcoded behaviour has a name rather than being the
absence of one.
The rustdoc states what write-through **is** -- a first-level cache directive whose value is
latency shaping, since data already at the device makes a later flush shorter -- and what it is
**not**: not a durability guarantee and not FUA, because whether it becomes a Force Unit Access bit
depends on the driver, the volume, and whether the device's write cache is enabled, none of which
this API can see or promise. A write that completes with `WriteThrough` may still be in a volatile
device cache. That conflation cost the originating exchange a wrong recommendation, so it is stated
at the type rather than left in a design note.
A unit test pins the mapping, including that `Cached` stays the no-flag value: silently enabling
write-through would change latency behaviour for every existing caller without changing any call
site.
- [x] **M12.4** -- `FILE_FLUSH_MODE` is exposed as `FlushMode` (`Default`, `Data`, `MinMetadata`,
`NoSync`) on both flush entry points ([D-25](DESIGN-NOTES.md#d-25)), with `Default` as `#[default]`
-- the mode a durability barrier wants and what the crate hardcoded before.
`NoSync` carries the loudest documentation in the crate, because it is **the one mode that makes
nothing durable**: it pushes data out of the system cache and stops, so anything in a volatile
device cache is lost on power failure exactly as if no flush had been issued. Its completion is not
a commit point and must never be reported to a caller as one. Stated in passing, as the item asked:
the existence of a distinct "no sync" mode is itself the evidence that the other three *do* issue
the sync -- nothing in the Win32 documentation says so directly.
A unit test pins all four mappings. It matters most for `NoSync`: confusing it with any other value
would turn a commit point into a no-op that still reports success.
**Note on sequencing.** These two were implemented back to back and are committed together, citing
both IDs. They are genuinely independent -- separate kernel parameters on separate entry points --
and that independence was preserved in how the code was written, so re-slicing the commit would be
bookkeeping rather than history.
**Breaking change**: all four write entry points and both flush entry points changed signature.
- [x] **M12.5** -- Durability is now stated wherever it is discussed, as a blast-radius sweep.
Grepped `flush` / `durab` / `write_through` / `WriteThrough` / `drain_preceding` / `FUA` across
`src/`, `tests/`, `examples/` and `*.md`.
**The finding was an absence, not a contradiction.** [src/lib.rs](src/lib.rs) and
[README.md](README.md) between them mentioned durability *zero times* -- one incidental hit each, both
the word "flush" in the list of the kernel's seven ops. A consumer reading either document
front-to-back would have learned that the ring exists, how to choose a delivery architecture, and
how to size an execution domain, without ever being told that a flush is the only way to commit
anything or that the obvious spelling of one commits nothing. Both now carry a `Durability` section
stating the three facts the item names -- no FUA, the flush is the only durability primitive, a
flush without the barrier covers nothing -- plus the consequence that ties them together: durability
is a property of an epoch, never of an individual write. README's third point also carries M12.2's
measurement, that seeing your flush land last is device-dependent and not evidence the barrier can
be omitted.
**Accounted for, already correct:** the flush and write rustdoc state all three facts in full
(M12.1, M12.3, M12.4), which is where the summaries point; `PushOptions::drain_preceding` states the
barrier's reach and its relationship to durability (M11.5);
[DESIGN-NOTES.md](DESIGN-NOTES.md)'s "Durability on the ring" is the long form all of them defer to.
[tests/flush_barrier.rs](tests/flush_barrier.rs) and [src/batch/tests.rs](src/batch/tests.rs) are
checks of the contract rather than statements of it; the `ring_copy` example's writes are a copy
pipeline with no durability claim to make, and now name `WriteCaching::Cached` explicitly rather
than inheriting a hardcoded flag.
## Moved 2026-08-29 -- M13: the epoch-committed log worked example
### M13 -- Worked example: consumer-side durability (an epoch-committed log)
[D-26](DESIGN-NOTES.md#d-26) puts durability *policy* with the consumer and Windows *mechanism*
here, which leaves a gap: without a demonstration, every consumer rediscovers the same composition,
and the three measured contracts ([D-19](DESIGN-NOTES.md#d-19), [D-23](DESIGN-NOTES.md#d-23),
[D-24](DESIGN-NOTES.md#d-24)) are exactly the kind that are learned by deadlock or by data loss.
This milestone closes that gap with a worked example, not a library -- it demonstrates the pattern
without this crate owning the policy.
The example is a miniature write-ahead log: records appended through the ring, made durable by
group commit, with durability reported by epoch. It is deliberately the shape a real consumer
needs, and it exercises `windows-ioring-sys` and `windows-threadpool-sys` together.
**Depends on M11.1** (`IoRing::completion_event`) and **M12.1** (explicit flush barrier). Both have
landed, so this milestone is unblocked.
- [x] **M13.1** -- Scaffolding under [examples/epoch_log/](examples/epoch_log/) --
[main.rs](examples/epoch_log/main.rs) and [contract.rs](examples/epoch_log/contract.rs) -- with the
sample's own durability contract written down first, before any code that implements it.
The contract is phrased as *this program's specification*, not as a description of what the ring
happens to do, which is the Design Autonomy rule applied rather than cited: the mechanisms it picks
(a covering flush, the ring's completion event) are recorded as chosen *because* they satisfy the
specification, and the file states plainly that a dependency which stops satisfying it is the thing
that is wrong. The guarantee is the item's sentence -- **a record is durable when the commit of the
epoch containing it has completed** -- with each of its four load-bearing phrases unpacked, since
"has completed" doing the work of "was submitted" is exactly how this contract gets misread.
All three non-guarantees the item names are stated as plainly as the guarantee (no per-record
durability, no ordering within an epoch, no atomicity past the device's power-fail atomic write
unit), plus two the writing surfaced: nothing is promised about records after the last committed
epoch -- present, absent, and torn are all legal outcomes of one crash -- and the whole contract
rests on the device honoring the flush, which nothing here can verify. Those are recorded as an
explicit `Assumes` clause rather than left silent.
The contract is also machine-readable (`CONTRACT: &[Statement]`, grouped by `Clause`) and the
sample prints it, so a reader who only runs the program still learns what it does and does not
promise. M13.5's verification pass has something concrete to refer back to.
- [x] **M13.2** -- The append path, in [record.rs](examples/epoch_log/record.rs) (the format) and
[append.rs](examples/epoch_log/append.rs) (the arena and the push). A record is a 20-byte header --
magic, sequence, payload length, checksum -- followed by its payload, with each field justified by
a clause of the contract rather than by convention: the length because replay has no framing of its
own, the sequence because the contract guarantees no ordering *within* an epoch so the on-disk
order is not the logical one, and the checksum because the contract says the tail may be **torn**
and a reader that cannot tell torn from whole cannot honour that clause. The checksum covers the
header too, so a torn header carrying a plausible length is caught rather than trusted.
Appends compose into a registered arena (`SLOTS` slots) and push with `write_registered_raw` over
exactly the record's bytes, deliberately not the owned-`Vec` form: an externally-managed arena is
what a real consumer has, and it is what makes this sample worth reading. `PushOptions::new()` and
`WriteCaching::Cached` are both the *unordered, uncached* choice on purpose -- records stream
unordered within an epoch exactly as the contract says, and the ordering is bought once by M13.3's
covering flush. The example runs the arena dry on purpose (24 records through 8 slots), so the
`WouldBlock`-then-drain path is exercised rather than hypothetical.
**This item found a real gap in the crate and fixed it at the layer, per the mono-repo policy.**
`RegisteredBuffers` exposed `get` but no mutable accessor, so a registered arena could only ever
carry bytes the *kernel* had produced -- there was no way to put a record a caller composed into
one. `ring_copy` never hit it because it reads into a registered buffer and writes back out of the
same one. Raised rather than worked around, and the engineer chose per-buffer accounting over a
narrow `unsafe` seam.
The subtlety that made it more than an accessor: **`&mut self` is not sufficient for safety here.**
An in-flight operation holds no borrow -- `write_registered` takes `&RegisteredBuffers` for the
length of the call and the `Token` keeps only a `RegisteredUse` -- so the borrow checker would
happily allow mutating a buffer the kernel is reading through. `get_mut` therefore pairs `&mut
self` with a runtime check, and the outstanding count moved from one per *registration* to one per
*buffer* so that a busy slot does not block its neighbours, which an arena being refilled needs.
`WouldBlock` and `InvalidInput` are distinct so "busy" and "no such buffer" cannot be confused.
Sabotage-verified: reverting `get_mut` to the old per-registration semantics fails the new
neighbour test with "buffer 1 still has 1 operation(s) outstanding".
Two tests in [tests/registration.rs](tests/registration.rs) cover it -- filling a registered buffer
and writing it back out, and the refusal/neighbour/out-of-range/release cycle. The example also
reads its own log back and decodes every record, so the claimed format is checked rather than
asserted; verifying the *contract* (durability, the torn tail) stays M13.5's job.
- [x] **M13.3** -- Epoch bookkeeping and group commit in [commit.rs](examples/epoch_log/commit.rs).
Records join whatever epoch is open, closing epoch *N* pushes **one covering flush** whose
`UserData` is *N*'s identity for the rest of its life, and observing that flush's completion is what
advances `durable_through`. The item's completion test -- a caller can await "epoch *N* is durable"
and get a truthful answer -- is `Committer::is_durable`, and the example awaits each epoch before
moving on.
**Truthfulness is the whole item, so it is asserted rather than assumed.** The example checks
`!is_durable(closed)` in the window between pushing a commit and observing it, checks monotonicity
across every epoch below the watermark, and checks that the still-open epoch never reports durable.
Sabotage-verified: advancing `durable_through` at push time instead of at completion fails on "a
pushed commit is not a completed one". A failed commit advances nothing, which is the truthful
answer for the epoch it was closing.
Two things the writing forced, both recorded in the module rather than left implicit. The barrier is
**ring-wide**, so a commit stalls the log -- next-epoch appends queue behind it and hold their arena
slots -- which is D-24's cost made visible rather than described. And because it is ring-wide, a
commit of *N* may in fact also cover records already pushed into *N+1*; the committer deliberately
**reports less than is true**, advancing only to *N*, because reporting less than reality is always
safe and reporting more never is.
**The record format gained an `epoch` field**, extending M13.2's header from 20 to 28 bytes. Keeping
the record-to-epoch mapping only in RAM would have made the contract verifiable in this
demonstration and unverifiable in the situation it exists for: a replay after a crash runs in a new
process with no memory of which record joined which epoch. The read-back check now asserts every
record carries the epoch that was open when it was appended.
The blocking wait here is the fused submit-and-wait; M13.4 replaces it with the multiplexed one,
which changes the shape but not the accounting.
- [x] **M13.4** -- The event loop in [event_loop.rs](examples/epoch_log/event_loop.rs): the ring's
`completion_event` waited on by `WaitForMultipleObjects` alongside a manual-reset shutdown latch,
with the drain outside the match on which handle woke us and draining *to empty*. Ownership does
not change -- this thread still owns, submits to, and drains its ring -- so it is Model B with a
different wakeup source, not Model A. The fused wait M13.3 used is kept beside it as the documented
contrast rather than deleted, since it is the right choice for a log whose only I/O is ring I/O.
**The item asked for the drain-to-empty rule to be visually obvious, and sabotage corrected which
half of it actually matters.** Replacing drain-to-empty with a single `try_pop` deadlocks the
example outright: the queue stops returning to empty, the edge never re-arms, and the next wait
blocks until its 30-second timeout with the log's work stranded. That half has teeth and is now
demonstrated.
Moving the drain *inside* the ring's arm, by contrast, **does not break a conformant loop** -- the
sabotage passed. On a loop that already obeys rule 1 the shutdown pass has nothing to pop, because
any completion that arrived signalled the event and was drained on the ring's own wake. An earlier
draft claimed the unconditional placement was "the bug this file exists to demonstrate the absence
of"; that claim was false and is removed. The placement stays because rule 1 says every pass and it
costs nothing, and the file now says plainly that this example cannot make it fail. This is the
the same overclaim M11.6 caught, found the same way.
- [x] **M13.5** -- The replay-and-verify pass in [replay.rs](examples/epoch_log/replay.rs), which is
what turns the sample from a demonstration into evidence. It is handed the watermark the log
reported and holds it to exactly the contract's asymmetry: every record in an epoch at or below the
watermark **must** be present, in sequence, with a matching payload and a validating checksum,
while records above it may be present, absent, or torn and are counted rather than judged. Refusing
to tolerate a torn tail would be its own bug -- the contract promises the tail is unreliable, so a
reader that treated it as corruption would reject a healthy log.
The example now appends `TAIL_RECORDS` into an epoch it **deliberately never commits**, because a
replay pass that never sees an uncommitted tail has not been asked the interesting question.
**The verifier is run three ways, and the third is the point.** Against the log as written (24
durable verified, 3 tail tolerated); against a copy cut mid-record to simulate a crash (24 durable
still verified, tail truncated, *no* violation reported); and -- the negative control -- against a
copy with one byte corrupted **inside** the durable region, which must be reported. A verifier that
cannot fail proves nothing, so the sample proves this one can: it reports
`MissingDurableRecord { reason: ChecksumMismatch }` and the run states out loud that the checker
which passed the first two cases was shown to be able to fail.
## Moved 2026-08-29 -- M14: crossing the ring boundary, and paying for the barrier
### M14 -- Worked example: crossing the ring boundary, and paying for the barrier
Second half of the example. M13 stays inside the ring; this milestone covers the two things that
forced the original consumer conversation -- operations the ring cannot express, and the cost of
[D-24](DESIGN-NOTES.md#d-24)'s full-barrier stall.
**Depends on M13.**
- [x] **M14.1** -- Order a non-ring operation against ring epochs: an `FSCTL`-class operation issued
through [`windows-overlapped-io-sys`](../windows-overlapped-io-sys), sequenced at an epoch
boundary, with its completion waited on in the *same* multiplexed wait as the ring's. This is the
case `drain_preceding` cannot express at all ([D-24](DESIGN-NOTES.md#d-24) orders SQEs against
SQEs), and the reason `completion_event` exists. Add the sibling crate as a dev-dependency.
**Done:** `examples/epoch_log/reclaim.rs` reclaims a retired segment with `FSCTL_SET_ZERO_DATA`
on a worker thread (that backend completes synchronously, which the event loop must not do), and
`EventLoop` now waits on three handles. Two measured facts, both sabotage-checked: the reclaim
running *alongside* appends does **not** make the third handle load-bearing (ring traffic wakes
the loop; removing the handle costs nothing), whereas the idle-path reclaim after the ring
quiesces does -- removing it turns a 78 ms run into a 30 s `WAIT_MS` block. The ordering itself
is enforced by the log, not the ring, and asserting it before each request makes that checkable.
- [x] **M14.2** -- A control-plane and background path on `windows-threadpool-sys`: checkpointing or
reclamation driven from the pool while the pinned log thread keeps the data path. Demonstrates the
hybrid the design notes recommend -- Model B on the hot path, Model A for everything else -- in one
program, which nothing in the crate currently shows. **Must also add an explicit `[[example]]`
entry for `epoch_log` with `required-features = ["threadpool"]`** (found while doing M13.1): the
sample uses no thread pool through M13, so it builds under `--no-default-features` today, and the
moment this item introduces `EventDelivery` it stops -- which the `ioring-no-threadpool` CI job
from M11.4 will catch as a build failure rather than a warning.
**Done:** `examples/epoch_log/checkpoint.rs` runs the checkpoint on a *second* ring handed to
`EventDelivery`; the pool thread that sees the covering flush complete is what authorises the
reclaim, so the chain crosses log thread -> pool thread -> reclaim worker -> log thread with the
log thread blocking for none of it. Two rings rather than one is forced by
[D-21](DESIGN-NOTES.md#d-21), not chosen. The `[[example]]` entry was added and its necessity
verified: removing it makes `cargo check --all-targets --no-default-features` fail with
`unresolved import windows_ioring_sys::EventDelivery`, which is exactly what the M11.4 job runs.
- [x] **M14.3** -- Implement all three epoch-commit strategies from "Durability on the ring" behind
one interface, selectable at run time: covering flush (ring stalls), host sequencing (a userspace
round trip per epoch), and alternating rings (neither, at the cost of doubled registration). The
point is that [D-24](DESIGN-NOTES.md#d-24) makes this a real fork with no free answer, and a
reader needs to see all three to choose.
**Done:** `examples/epoch_log/strategy.rs`. All three run the same workload and are checked two
ways: each log replays clean, and all three must be **byte-identical** to each other. Both checks
were sabotage-verified (a shifted offset trips replay; a dropped epoch trips byte-identity). The
limit is stated in the code rather than papered over -- neither check can observe whether the
ordering held on the *device*, which is only visible across a power cut, so the strategies are
argued from [D-23](DESIGN-NOTES.md#d-23)/[D-24](DESIGN-NOTES.md#d-24) rather than from a clean run.
- [x] **M14.4** -- Measure the three strategies on the running machine and print the comparison:
throughput, commit latency distribution, and ring idle time during the barrier. The example should
*demonstrate* the trade-off rather than assert it, and the numbers are machine-specific enough that
quoting ours would be misleading.
**Done:** throughput, commit-latency quantiles, and append stall are measured per strategy and
printed with an explicit "these describe THIS machine" note. The finding is that on this machine
the three are **indistinguishable** -- the cross-strategy spread (1.08x-1.72x) is the same size as
one strategy's run-to-run spread (1.42x) -- because every strategy pays one device flush per epoch
at hundreds of microseconds while their actual differences land in the tens. The program computes
and states that from its own data rather than hard-coding a ranking. Measurement also **found two
harness bugs** that no other check caught: a single deferred-commit slot shared by two lanes (so
half the commits were never awaited and `durable_through` was a claim), and pending commits keyed
by `UserData` in one map across two rings whose sequences collide. Both are recorded in
`strategy.rs` because both are easy to repeat.
- [x] **M14.5** -- Document the example: a module-level walkthrough, a pointer from `README.md` and
from the "Durability on the ring" section of [DESIGN-NOTES.md](DESIGN-NOTES.md), and an explicit
statement that it is a demonstration of a pattern rather than a supported API -- so that nobody
vendors it and then expects this crate to maintain its policy choices.
**Done:** `main.rs` opens with the not-API statement and its reasoning (D-8/D-26), a module table
giving each file's job *and the contract behind it*, what one run does, and two things the sample
cannot show. `README.md` points at the example from the Durability section and gains an "the
examples are demonstrations, not API" subsection under Cargo features.
[DESIGN-NOTES.md](DESIGN-NOTES.md) gains a subsection at the end of "Durability on the ring" that
also promotes M14.4's two findings out of the sample, since both are about the design rather than
about the demonstration.
## Moved 2026-08-30 -- M8 through M10: handle lifetime, cross-ring identity, and the contract audit
### M8 -- `FileRef::Raw(HANDLE)` lifetime safety (PR #20 review finding)
A caller can close or reuse the raw `HANDLE` passed to `Batch::read`/`write`/`flush`/`cancel`/
`register_files` before the kernel finishes with it: unlike a buffer (owned by a `Token` until claimed),
`FileRef::Raw` carries no lifetime and nothing in this crate borrow-checks a handle across the async gap
between push and completion.
- [x] **M8.1** -- Ownership model decided: `Batch::read`/`write`/`flush`/`cancel`/`register_files` become
`unsafe fn` when addressing a raw `HANDLE` (their existing `SAFETY` comments already state the
caller-keeps-it-alive obligation informally; this makes it a real, compiler-checked boundary), paired
with a safe `SharedFile` wrapper (`Arc<OwnedHandle>`) for the common case: each push clones the `Arc` into
the same `Token` that already tracks the operation's buffer (or, for `flush`/`cancel`, a standalone
`Token<Arc<OwnedHandle>>`), so the underlying handle survives until every operation referencing it is
claimed or leaked, regardless of what the caller does with its own `SharedFile` clone. Rejected:
borrowing `FileRef::Raw` for the `Batch`'s own lifetime (does not solve the completion-outlives-the-push-
call problem) and forcing every raw handle through an owning wrapper the way
`windows-overlapped-io-sys`'s endpoints do (defeats `FileRef::Raw`'s zero-setup reason to exist).
`FileRef::Registered` needs none of this and stays unaffected.
- [x] **M8.2** -- Added `SharedFile` (`pub struct SharedFile(Arc<OwnedHandle>)`) with a constructor from
`OwnedHandle`, `Clone`, and a raw-handle accessor for building an `IORING_HANDLE_REF`.
- [x] **M8.3** -- Marked `Batch::read`/`write`/`flush`/`cancel`/`register_files`'s raw-`HANDLE`-taking
forms `unsafe fn` with a `# Safety` section stating the real obligation (valid handle, correct access
rights, remains valid until the pushed operation's completion is observed or the ring runs down).
**Scope correction found during execution:** `read_registered`/`write_registered` take the identical
`impl Into<FileRef>` shape for their own `file` argument and carry the same hazard, so they were marked
`unsafe fn` too, with no separate checklist item -- the original enumeration omitted them by oversight,
not by decision. Committed as `feat(ioring)!`.
- [x] **M8.4** -- Added safe overloads `read_shared`/`write_shared`/`flush_shared`/`cancel_shared` (plus
`read_registered_shared`/`write_registered_shared`, for the same reason as M8.3's correction) taking
`&SharedFile`, cloning the `Arc` into the same `Token` (a tuple with the buffer for `read`/`write`-shaped
pushes, or `Token<SharedFile>` alone for `flush`/`cancel`, which have no buffer of their own) so a caller
never needs `unsafe` for the common case. `register_files` gets no `_shared` counterpart: a registration's
handles must stay valid for the ring's remaining life, a lifetime no single push's `Token` can express.
- [x] **M8.5** -- Integration test
(`dropping_the_callers_own_sharedfile_clone_does_not_close_a_still_outstanding_handle`,
`tests/submission_lifecycle.rs`): drop the caller's own `SharedFile` clone (its only external reference)
while a push against it is still outstanding; the read still completes correctly against a live handle,
proving the `Arc` clone inside the token -- not the caller's copy -- is what kept it open.
- [x] **M8.6** -- Renamed the API so the safe path gets the plain names: the raw/`unsafe` entry points
became `read_raw`/`write_raw`/`flush_raw`/`cancel_raw`/`read_registered_raw`/`write_registered_raw`, and
the safe `SharedFile`-taking overloads (previously `*_shared`) took over the plain names
`read`/`write`/`flush`/`cancel`/`read_registered`/`write_registered` -- since the safe path is the one
this crate wants to steer callers toward. `register_files` keeps its plain name unsafe, since it has no
safe counterpart to make way for.
### M9 -- Cross-ring identity and registration-drop safety (PR #20 review findings)
`UserData`, `RegisteredFile`, and `RegisteredBuffers` indices are each meaningful only against the
specific ring that minted them, but nothing enforced that when more than one `IoRing` exists in the same
process.
- [x] **M9.1** -- Added `RingId` (`ring.rs`): a monotonic, process-lifetime-unique `AtomicU64` counter,
never the ring's own `HANDLE` (which Windows can reuse for the next object after a ring closes). Every
`IoRing` gets one at construction; every popped `Completion` now carries the id of the ring that
produced it (D-17).
- [x] **M9.2** -- `Token::claim_if` now requires both the `UserData` identity and the `RingId` to match,
closing the gap where two different rings' own zero-based `UserData` counters could coincide. Added
`claim_if_rejects_a_matching_user_data_from_a_different_ring` (`src/token/tests.rs`).
- [x] **M9.3** -- `RegisteredFile`/`RegisteredFiles`/`PendingFileRegistration` and
`RegisteredBuffers`/`PendingBufferRegistration` now carry the minting ring's `RingId`; `handle_ref`
(fallible now) and a new `Batch::check_registration_ring` reject a `FileRef::Registered`/
`RegisteredBuffers` argument from a different ring with `io::ErrorKind::InvalidInput`, checked before any
`Build*` call runs. Added `a_registered_file_from_a_different_ring_is_rejected` and
`a_registered_buffers_from_a_different_ring_is_rejected` (`tests/registration.rs`).
- [x] **M9.4** -- `PendingBufferRegistration` now leaks its buffers (via `ManuallyDrop` plus a
deliberately empty `Drop`) if dropped without a matching `claim_if`, mirroring `Token`/
`RegisteredBuffers` (D-18) -- previously it dropped `Vec<B>` normally, freeing memory the kernel might
still reference from an already-queued `BuildIoRingRegisterBuffers` call. Added
`dropping_an_unclaimed_pending_buffer_registration_leaks_rather_than_frees` (`tests/registration.rs`).
- [x] **M9.5** -- `Batch::do_submit` now marks itself attempted (`self.submitted = true`) *before*
propagating `SubmitIoRing`'s `HRESULT`, not after: `submit_and_wait` takes `self` by value, so a failed
call still runs `Drop` once its caller's `self` goes out of scope, and the old ordering left `Drop` free
to silently retry an already-attempted submit -- which could succeed on the retry without the original
caller's `Err` ever saying so.
- [x] **M9.6** -- Output-abstraction cleanup in the two examples the review flagged
(`examples/model_a_delivery.rs`, `examples/l3_domains.rs`): routed through a single writer seam per the
repository's architectural pre-step, matching `src/bin/run_scenario.rs` (in `windows-file-watcher`) and
`examples/ring_copy/main.rs`'s existing pattern. `PLANS.md`'s bare `COMPLETED-PLANS.md`/
`COMPLETED-CHECKLIST.md` references were also made clickable relative links.
### M10 -- Finish the contract audit against the ten specification-gap categories
[DESIGN-NOTES.md](DESIGN-NOTES.md) -> "Specifying this contract" audits this crate against
[the ten categories](../../DESIGN-NOTES.md#specifying-a-delivery-contract). The first pass reached four of
them: D-17's `RingId` (categories 4/5, handled correctly), `Completion::synthetic`'s test-only gate
(category 10, handled correctly), D-14's registration-index continuity (category 4, recorded as an explicitly
unverified assumption), and the previously-unstated completion-ordering rule. Categories 1, 2, 3, 6, 8, and 9
were **not examined**, recorded as "not looked at" rather than "does not apply".
M10.1-M10.3 closed that out, so all ten are now examined. The audit also turned up two API gaps rather than
mere statement gaps; they are M10.4 and M10.5 below rather than notes in the design record, per "design notes
are not a work queue".
- [x] **M10.1** -- Audited category 3 (state-dependent legality unenumerated) against capability negotiation
([D-28](DESIGN-NOTES.md#d-28)): [DESIGN-NOTES.md](DESIGN-NOTES.md) -> "Category 3" now tabulates which
`Batch` methods each probed op gates, and states the four rules the prose left silent -- `supports` answers
for the kernel's op table rather than this crate's push surface (`Op::Nop` gates no `Batch` method at all,
which is exactly the op M6+.3's shutdown case reaches for); `supports_raw` accepts named codes too and
differs from `supports` in caching rather than truth, while never widening what `Batch` can push; every
legality check runs before `reserve_user_data`, so a rejected push strands no identity; and the registration
one-shot is enforced against the registered count, making the real rule "at most one registration that
assigned an index" and making a *failed* registration unretryable on that ring. Corrected the `supports`,
`supports_raw`, `Op`, `register_files`, and `register_buffers` rustdoc that stated these wrongly or not at
all, and added an integration test binding the zero-length-registration rule.
- [x] **M10.2** -- Audited the remaining categories (1, 2, 6, 8, 9) against the ring/token/registration
surface; [DESIGN-NOTES.md](DESIGN-NOTES.md) carries one section each. Category 1: file addressing and
buffer addressing are genuinely orthogonal (all four combinations reachable), but file addressing and
*safety* co-vary and in the wrong direction -- `FileRef::Registered` is reachable only through an
`unsafe fn` whose safety obligation is vacuous for that very input
([D-29](DESIGN-NOTES.md#d-29), fixed in M10.4). Category 2: **every successfully queued SQE produces
exactly one completion**, unconditionally -- the rule `run_down`'s termination silently depended on --
plus the two "may" readings that were too weak (`submit*` returns entries submitted, not completed; a
cancel is a request that yields a second completion rather than replacing its target's). Category 6: a
completion matching no live `Token` is normal, via four distinct routes. Category 8: this crate joins
nothing, deliberately, and now says so. Category 9: `io::Error::kind()` is lossy exactly where the
`HRESULT` is not, discriminating this crate's own rejections and never the kernel's
([D-30](DESIGN-NOTES.md#d-30), predicate queued as M10.5). Category 7 falls out of 2 and 6 jointly, so
all ten categories are now examined.
- [x] **M10.3** -- Resolved D-14's unverified registration-index continuity assumption, by **dissolution
rather than measurement** ([D-31](DESIGN-NOTES.md#d-31)). D-14 justified the eager advance on the grounds
that erring early "can only ever waste indices, never collide two registrations onto the same index" -- and
that collision needs a *second* registration, which the PR #20 review response forbade the day after D-14
was written. So `base_index` is always zero, no later base index is ever derived from the count, and the
kernel's claim timing has no observable consequence; measuring it would settle a fact with nothing
downstream of it. Took the checklist's second branch for the residue that *is* observable: the public
counts now state on themselves that they report a **reserved** count, not a confirmed one -- already
advanced before any completion is popped, and still advanced after a registration whose completion failed
(which is why that registration cannot be retried, [D-28](DESIGN-NOTES.md#d-28)). Added a test binding
reserved-at-queue-time, gave D-14 the required adjacent status marker, and corrected the audit section's
category-4 paragraph, which still described the assumption as live.
**Merge note (2026-08-29):** `main` had meanwhile recorded that the measurement *does* exist -- the
2026-08-27 session established that a second `BuildIoRingRegisterFileHandles` replaces the whole table and
re-bases indices at zero, that a table holds at least 65536 handles, and that an index is resolved at
submission -- and scheduled the follow-up workspace-side as [M19.1](../../CHECKLIST.md), with the
relaxation of the one-registration-per-ring rule as M19.2. See
[DESIGN-SESSION-2026-08-27-pseudo-async-namespace-operations.md](../../design-sessions/DESIGN-SESSION-2026-08-27-pseudo-async-namespace-operations.md).
That does not reopen this item, but it does bound the dissolution: **the dissolution's premise is that a
second registration is forbidden**, so if M19.2 relaxes that rule, D-14's collision concern becomes live
again and [D-31](DESIGN-NOTES.md#d-31) must be revisited rather than assumed to still hold. Recorded here
because nothing else would carry that dependency across the two checklists.
- [x] **M10.4** -- Gave `FileRef::Registered` safe entry points ([D-29](DESIGN-NOTES.md#d-29)), via a sealed
`FileTarget` trait with an associated `Guard` type ([D-33](DESIGN-NOTES.md#d-33)) rather than a parallel
family of `*_registered_file` methods. `read`, `write`, `flush`, `cancel`, `read_registered`, and
`write_registered` are now generic over it, so a `RegisteredFile` pushes without `unsafe` and the
file-registered x buffer-registered combination is expressible for the first time. `Guard` carries the
one real difference between the two targets -- what the `Token` must hold until the completion is
observed: an `Arc` clone for `SharedFile`, nothing needing to be kept alive for `RegisteredFile` (which
hands its `Copy` index back for symmetry). **Non-breaking**: `read(&SharedFile, ..)` still resolves to
`Token<(B, SharedFile)>` and every existing call site compiled untouched; generic params are ordered
`<B, F>` so an existing `read::<Vec<u8>>` turbofish still resolves. Sealing is load-bearing rather than
tidiness -- an outside impl could return `FileRef::Raw(arbitrary)` with a guard keeping nothing alive,
reintroducing what D-16 closed. Three tests: a safe registered-file read, the fully-registered
composition, and a cross-ring rejection confirming D-17's check still holds on the safe path.
- [x] **M10.5** -- Added named predicates for the ring conditions a consumer must branch on
([D-30](DESIGN-NOTES.md#d-30), resolved as [D-34](DESIGN-NOTES.md#d-34)), in three parts. A
`#[non_exhaustive]` `RingCondition` enum covering **every** `IORING_E_*` this crate names -- not only the
actionable ones, since narrowing it would be the "narrow the platform to serve the visible goal" failure
PLATFORM INTEGRITY forbids. Predicates (`is_submission_queue_full`, `is_completion_queue_too_full`,
`is_submit_in_progress`) for just the runtime-actionable conditions, because a predicate asserts a branch
exists; the rest stay reachable via `condition()`. And a sealed `IoRingErrorExt` putting those answers on
`io::Error` itself, which is what actually removes the hand-rolled
`get_ref().downcast_ref::<IoRingError>()` from call sites -- the crate's own integration tests had two
such helpers, and deleting them was the new API's first use. `IoRingError::name` is now **derived** from
`condition()` rather than matching the `HRESULT` a second time (CONTRACT INTEGRITY: prefer a derived fact
to a restated one) -- previously a new condition could be added to one `match` and silently missed by the
other. Did **not** map `IORING_E_*` onto `io::ErrorKind`; D-30's refusal stands. Bound to a *real*
kernel-reported queue-full in the existing backpressure integration test, not only to synthetic errors.
- [x] **M10.6** -- Fixed a live use-after-free in `Batch::register_buffers`
([D-32](DESIGN-NOTES.md#d-32)). `BuildIoRingRegisterBuffers` reads its `IORING_BUFFER_INFO` array when
the registration op *runs*, not when the `Build*` call returns; the crate built that array in a local
`Vec` and dropped it before `SubmitIoRing`, so the kernel read freed heap and the registration completed
with `ERROR_NOACCESS`. Since `register_buffers` is a **safe** `pub fn`, safe code could make the kernel
dereference a dangling pointer -- a soundness hole in shipped 0.1.2, not a test defect. A spike crossed
array-lifetime against buffer alignment and disproved alignment in both directions; it also established
that the array may be released once submit returns, and that `BuildIoRingRegisterFileHandles` genuinely
*does* read synchronously -- the asymmetry the rustdoc had wrongly generalized across, and the reason the
file-registration tests always passed. The array is now owned by the `IoRing` rather than the `Batch`,
because a failed submit leaves the SQE queued as ring state ([D-5](DESIGN-NOTES.md#d-5)) and a later
unrelated submit can be what runs it. Added a regression test that churns the heap between the push and
the submit, verified by sabotage; corrected both false "read synchronously" claims; moved the entry from
[UNRESOLVED-TEST-FAILURES.md](UNRESOLVED-TEST-FAILURES.md) to
[RESOLVED-TEST-FAILURES.md](RESOLVED-TEST-FAILURES.md).
## Moved 2026-08-30 -- M15 through M18: the testing-strategy response to eight defects
### M15 -- Deterministic memory instrumentation (release-gating)
Eight defects came out of the 0.1.x line and the M11-M14 branch:
[#47](https://github.com/MikeGrier/windows-threadpool-sys/issues/47),
[#48](https://github.com/MikeGrier/windows-threadpool-sys/issues/48), `get_mut` returning `&mut B`
([D-35](DESIGN-NOTES.md#d-35)), `Appender::claim` leaking an arena slot, the checkpoint path authorising a
reclaim after a failed write, a shared deferred-commit slot in the strategy harness, `get` racing the kernel
([D-36](DESIGN-NOTES.md#d-36)), and a `debug_assert` that could never fire. Sorting them by *what would have
caught them* is what M15-M18 are built from, because they need different tools and one population needs a tool
that does not exist:
- **A -- preconditions never varied.** Every `tests/event_delivery.rs` case handed over a *fresh* ring, so
"completion queue non-empty at handover" was never a test input. That is
[#47](https://github.com/MikeGrier/windows-threadpool-sys/issues/47). M17.
- **B -- failure paths never taken.** No test ever ran `completion.result()` returning `Err`, which is the
checkpoint defect, and is why [#48](https://github.com/MikeGrier/windows-threadpool-sys/issues/48) surfaced
as a *lucky* `ERROR_NOACCESS` rather than as corruption. M15 and M16.
- **C -- permissions, not behaviour.** `&mut Vec<u8>` *permits* `reserve`/`resize`/reassign; no execution ever
performs it. That is [D-35](DESIGN-NOTES.md#d-35) and [D-36](DESIGN-NOTES.md#d-36), the two most severe
findings, and **no runtime technique reaches this population at all.** M18.
**Deliberately rejected: a mock `IoRing`.** Both shipped defects were the kernel behaving differently from
this crate's assumptions -- the completion event is edge-triggered ([D-19](DESIGN-NOTES.md#d-19)), and
`BuildIoRingRegisterBuffers` reads its array at submit rather than at build ([D-32](DESIGN-NOTES.md#d-32)). A
mock encodes the assumption, so one written before those discoveries would have passed both bugs green -- it
would not merely have failed to find them, it would have manufactured evidence they were absent. A model
belongs here as an **oracle over observed sequences** (M16), never as a substitute for the kernel.
**Deliberately rejected: Application Verifier / PageHeap**, after measuring it rather than assuming either way.
See [D-37](DESIGN-NOTES.md#d-37); the short version is that it works but is keyed by *image file name*, and
cargo rehashes test binaries on every meaningful rebuild.
**M15 and M16 gate the 0.2.0 release; M17 and M18 do not.** 0.2.0 carries two security fixes and should not
wait on the state-space work.
- [x] **M15.1** -- Add a guard-page global allocator for test builds: `VirtualAlloc` each allocation with a
trailing `PAGE_NOACCESS` guard page, right-aligned so its end abuts the guard, and on free flip the block to
`PAGE_NOACCESS` and **never reuse the address**. That combination makes both an overrun and a
use-after-free a deterministic `0xC0000005` instead of a silent stale read.
Use `MEM_DECOMMIT` rather than plain `VirtualProtect` on free: never reusing an address is the point, but
never releasing the *commit* would grow a long suite without bound.
A working probe is at `.scratch/guardalloc/` -- measured clean/UAF/overrun as `0`/`0xC0000005`/`0xC0000005`,
against a system-allocator baseline that silently read a freed byte and exited `0`.
**Sabotage-verify against the real defect:** revert [D-32](DESIGN-NOTES.md#d-32)'s fix locally and confirm
the allocator turns it from a lucky `ERROR_NOACCESS` into a deterministic failure. An instrument never seen
to fire is indistinguishable from one that cannot.
**Done:** new `windows-guard-alloc` crate (`publish = false`), installed by `tests/registration.rs`. The
calibration was run and is the point of the item: reverting [D-32](DESIGN-NOTES.md#d-32) makes
`a_buffer_registration_survives_heap_churn_between_the_push_and_the_submit` -- the regression test that fix
added -- die with `STATUS_ACCESS_VIOLATION` instead of passing. Five subprocess tests in `tests/faults.rs`
prove the allocator fires at all (an access violation cannot be caught in-process), and a
`the_guard_allocator_is_installed_for_this_test_binary` assertion catches the silent failure mode where the
`#[global_allocator]` attribute is missing and everything passes uninstrumented.
- [x] **M15.2** -- Fill every allocation and every freed block with a **tracked** poison pattern, derived from
a per-run seed plus a per-allocation ordinal so the bytes identify *which* allocation they came from rather
than merely being "not real data". A fixed constant like `0xDD` is worth much less: it collides with real
payloads, and it cannot answer "where did this come from".
The tracking is also what keeps this compatible with this component's reproducibility rule: **log the seed at
test start and accept it from the environment**, so a failure is replayable exactly rather than being a
one-off. Record that in [DESIGN-NOTES.md](DESIGN-NOTES.md) as the terms under which non-fixed test data is
permitted here.
Keep the pattern cheap to write (a repeating 8-byte word carrying the ordinal, not a per-byte hash) since it
runs on every allocation.
**Done:** `windows-guard-alloc`'s `poison` module. The pattern is `splitmix64(seed ^ ordinal)` repeated
across the allocation, and the mixing function is a **bijection with a computable inverse**, so
`GuardAlloc::poison_check` recovers *which* allocation a region belongs to from its own leading bytes --
no snapshot needed, which is what M15.3 will build on. Seeding terms recorded as
[D-39](DESIGN-NOTES.md#d-39) and verified end to end: two unpinned runs differ, two pinned runs reproduce
byte-for-byte, and both decimal and `0x` hex parse.
**Two findings while implementing.** (1) *Poisoning freed blocks is dead code here* -- `dealloc` decommits,
so those bytes are unreadable and poison written there could never be observed by anything. It would cost a
memset per free for a guarantee strictly weaker than the one already in force, so it is deliberately not
done, and `dealloc` says why. The item text above is left as originally written rather than quietly edited,
since the planning error is the more useful record. (2) The first draft **hardcoded two multiplicative
inverses and both were wrong**; `the_multiplicative_inverses_are_actually_inverses` caught it. They are now
*derived* by a `const fn` Newton iteration, which makes that class of error unrepresentable rather than
merely tested for.
- [x] **M15.3** -- Verify the poison at the points where the *kernel* is the suspect, which is the gap guard
pages structurally cannot cover: a guard page only catches access to memory that should not be touched at
all, and says nothing about the kernel writing into a live, valid allocation. Both checks below are
invariants this crate asserts in prose today and verifies nowhere:
- **After a registered *write* completes** (`KernelAccess::ReadsBuffer`): the slot must be **byte-identical**
to what was submitted. The kernel is only permitted to read it.
- **After a registered *read* completes** (`KernelAccess::WritesBuffer`): every byte *outside*
`[span.offset, span.offset + information)` must still be poison. This binds the kernel to the
`RegisteredSpan` we declared, and would equally catch our own `checked_span` or offset arithmetic being
wrong.
Sabotage-verify each by deliberately submitting a span narrower than the write, and by mutating a slot
mid-flight.
**Done:** `tests/kernel_span.rs`, five tests. Both named checks, plus two the item did not anticipate: a
**short read** must leave the unfilled remainder of its span poison (the span is a permission, not a
promise, so `information` is the only thing that says where real data stops), and a read into one slot must
leave its **neighbours** untouched -- which is what per-slot ordinals buy over a single constant, since a
write landing in the wrong slot is visible as such rather than merely as "something changed".
All four assertions were sabotage-verified individually and each named the right failure: a byte before the
span, a byte past its end, a modified write source, and a disturbed neighbour. Every message carries the
seed, so a failure is replayable per [D-39](DESIGN-NOTES.md#d-39).
**Note the checks cut both ways**, which is worth more than the kernel-conformance reading: the offset the
kernel receives is computed by *this* crate in `checked_span`, so an off-by-one there fails these tests
exactly as a kernel bug would -- and is otherwise as invisible as one.
- [x] **M15.4** -- Verify poison at quiescence too: at teardown every registered buffer's never-written
regions still hold their expected pattern, and no buffer that was only ever a write source has changed.
This is the whole-arena form of M15.3 and is what catches a stray write attributed to no particular
operation.
**Done:** `windows-guard-alloc`'s `witness` module plus
`a_mixed_workload_leaves_every_unaccounted_byte_poisoned`. A `Witness` starts owning a region with nothing
permitted; each completion permits exactly the bytes it was entitled to change -- the **transferred** count,
never the requested one -- and `verify` walks the *gaps* between merged permissions at teardown. Slot 3 in
the test is a write source only, so nothing is ever permitted for it and it must be byte-identical.
Both directions were sabotage-verified: a byte in the gap between slot 0's two disjoint reads, and a byte in
the write-source slot, each reported with the slot, the offset, expected-versus-found, the seed, and how
many bytes were legitimately written (`0 byte(s)` for the write source). The 12 `witness` unit tests weight
the **false-positive** direction deliberately -- abutting ranges, overlapping ranges, out-of-order
permissions -- because a witness that accuses legitimate writes trains a reader to ignore it, which is the
same failure as missing one.
### M16 -- Make the contract executable, and take the failure paths (release-gating)
Population B, and the invariants this crate already *states* but nothing checks.
**Depends on M15** only for convenience -- the oracle is independent of the allocator, but the two are most
useful together.
- [x] **M16.1** -- Add `src/contract.rs` with a public `RingContract`: the executable form of
[the category-2 rule](DESIGN-NOTES.md#one-sqe-one-completion) -- "every SQE that successfully queues
produces exactly one completion" -- every token claimed-or-deliberately-leaked, and every per-buffer
outstanding count back to zero at quiescence.
**Correction (found while implementing):** this item originally cited D-29/D-30 for that rule. Both are
about something else -- D-29 is `FileRef::Registered`'s safety and D-30 is `io::Error::kind`'s asymmetry --
and the rule actually lives in the M10.2 audit's category-2 prose, which had no anchor to cite. It has one
now. The wrong citation had already reached [PLANS.md](PLANS.md) and
[DESIGN-NOTES.md](DESIGN-NOTES.md) itself; all three are corrected. Fed by the caller (`observe_push` / `observe_completion` / `check_quiescent`) rather than
wired into `Batch`, so a consumer can validate its own harness against the same definition.
It lives in **this** crate, not in the test harness: the layer that owns the invariant owns the oracle, and a
harness-side copy is a second implementation of the rule rather than a check of it. Follow
[`ContractChecker`](../windows-file-watcher/src/contract.rs)'s shape.
State plainly what it does **not** check -- it cannot observe device ordering or anything absent from the
completion stream -- since over-constraining is the same defect as under-specifying.
**Done:** `src/contract.rs`, 16 unit tests. Five violations: unexpected completion, duplicate completion,
outstanding at teardown, leaked token, buffer still in use. A completion provisionally marks its operation
**leaked**, corrected by `observe_claim` -- so forgetting to claim is the default failure rather than the
silent success, which is the shape of the `Appender::claim` defect. A leak is excused only when *stated*
via `observe_deliberate_leak`, because the difference between a stated and an unstated leak is the whole
point. Violations sort before they are reported, since `HashMap` order is unspecified and an oracle whose
output reorders between runs cannot be diffed. Half the tests are the **does-not-fire** direction, listed
in the module doc: completion order (the ring promises none between independent operations, and
[D-24](DESIGN-NOTES.md#d-24)'s barrier constrains execution rather than pop order), operation success,
anything about the device.
- [x] **M16.2** -- Bind the existing integration tests to `RingContract` and assert quiescence at teardown.
This is the item that pays for M16.1: the `Appender::claim` slot leak and the strategy harness's shared
deferred slot were both conservation failures, found by review and by measurement respectively, and both
would have fallen out of a quiescence assertion automatically.
Sabotage-verify by reintroducing one of them and confirming the oracle reports it.
**Done:** bound `tests/submission_lifecycle.rs` (two tests) and `tests/kernel_span.rs`'s mixed workload,
which also reports per-buffer counts. **The example's `Appender` now owns a `RingContract`**, because the
claim that the `Appender::claim` leak "would have fallen out automatically" had to be *proved* rather than
asserted -- and nothing outside the example could prove it. Calibrated against the real defect: taking the
pre-fix early-return path once makes the run die with `the token for user_data 0x1 was dropped unclaimed,
leaking whatever it held for the life of the process`.
**Binding real tests exposed a gap in M16.1**: `flush_raw` and `cancel_raw` return a bare `user_data` with
no token, so the oracle demanded a claim that cannot be made -- a false violation the caller has no way to
satisfy, which is the worst kind. Added `observe_tokenless_push` and a `PushedTokenless` state, with four
more tests. The backpressure test also exercises the rule's other half: the *rejected* push is deliberately
never observed, since a synchronously-failing `Build*` releases its reservation and produces no completion.
Noted for M16.3: reverting the fix's *ordering* alone does not reproduce the leak, because the early return
only triggers on a **failed** write -- which is precisely the population-B path that has no coverage yet.
- [x] **M16.3** -- Add a fault-injection seam at the completion boundary, so a test can make `result()` return
a chosen error for a chosen operation. `Completion::synthetic` already exists under `#[cfg(test)]` and is
half of this. Keep it test-gated for the same reason `synthetic` is: `Token::claim_if`'s safety argument
depends on every `Completion` tracing back to a real popped `IORING_CQE`.
**Done:** `Completion::with_injected_failure` plus an `InjectedFailure` enum (`Ring` / `Win32` / `Hresult`),
behind a new default-off `fault-injection` feature.
**The design turns on a distinction the item's framing missed.** `synthetic` *fabricates* a completion, and
the reason it must stay `#[cfg(test)] pub(crate)` is not merely convention: a fabricated completion can name
an operation still **in flight**, so claiming a token against one hands a buffer back while the kernel is
writing through it -- the exact use-after-free this crate exists to prevent. Building the seam that way to
test safety would be self-defeating. So this seam **transforms a completion the ring genuinely popped**
instead: same `UserData`, same ring identity, same finished operation, and only `result()` changes.
`claim_if`'s argument is therefore untouched, which is what makes the feature safe to expose at all.
**Found while writing the tests:** `Hresult(0)` would have injected *success*, silently falsifying the
"failure only, never success" guarantee and letting a test conceal the very defect it was written to find.
Now enforced by a panic rather than asserted in prose, with a `should_panic` test on it.
Also deleted a test that could not verify its own name -- `information` is private and unreachable once
`result()` is `Err`, so "reports no bytes transferred" had no assertion to write. The zeroing stays (it
would be wrong to model a state the kernel never produces) but the vacuous coverage does not.
`tests/fault_injection.rs` proves the seam is reachable *and* gated from outside the crate; CI's existing
`--workspace --all-features` job runs it, verified rather than assumed.
- [x] **M16.4** -- Use M16.3 to cover the failure paths that have no coverage at all: a failed read, write and
flush completion claimed normally, a failed registration, and `EventDelivery` delivering a failed completion.
Assert the *documented* degradation each time rather than merely that nothing panics -- the checkpoint defect
was precisely a failure that was noticed, recorded, and then not acted on.
**Done:** `tests/failure_paths.rs`, six tests, all five named cases plus the loop-closer M16.2 left open --
`claiming_before_checking_the_result_is_what_stops_a_failure_from_leaking` runs *both* orderings against
`RingContract` on the same injected failure, so the `Appender::claim` defect is a reported violation rather
than an argument. `EventDelivery` uses a **genuine** `ERROR_NOT_FOUND` from cancelling a non-outstanding
target rather than an injected failure, per M16.3's own guidance.
Two sabotage-verified: making a failed registration leak instead of drop, and making `EventDelivery` swallow
failed completions. Both fired with the right message.
**Corrected M16.3 while doing this.** That item's soundness argument covered `Token::claim_if` only, and
`PendingBufferRegistration::claim_if` is materially different: it treats a failed completion as proof the
kernel did *not* retain the addresses, so it **drops the buffers**. Injecting a failure there frees memory
the kernel genuinely holds registered. Inert only while nothing uses it -- the test takes the precaution
explicitly and `with_injected_failure` now documents the limitation, which it did not before.
**Found a race in the test, not the crate:** the `EventDelivery` callback published the error code *after*
the counter the waiting thread spins on, so the waiter read a stale zero. Fixed by publishing the data
before the flag -- the same ordering rule the epoch log's reclaim worker already follows.
### M17 -- Feed the detectors M15 and M16 built
Population A. [#47](https://github.com/MikeGrier/windows-threadpool-sys/issues/47) is one point in a space
nobody was sampling: `{fresh, dirty}` handover state crossed with operation kind, buffer kind, claim/drop and
drain timing. Enumerating that by hand is how it was missed the first time.
**What M15 and M16 changed about this milestone's purpose.** Neither found a single new defect in shipping
code: every finding was either historical ([D-32](DESIGN-NOTES.md#d-32), `Appender::claim`) or a defect in the
new instrumentation itself. That is not evidence the crate is clean. `RingContract`, the guard pages, the
poison, `Witness` and the fault seam are all **passive** -- they fire only on paths the existing hand-written
tests already walk, and #47 lived on a path no test walked. M17 is therefore not a fourth technique standing
beside M15 and M16. It is the **input generator for detectors that are already built and currently under-fed**,
which is also why its cost is lower than when it was first planned: the oracle, the memory checking and the
failure paths all exist now.
**Depends on M16.1** (the oracle is the property these tests assert).
**CONVENTION GATE -- CLEARED.** Randomized property testing is **permitted**, decided by the engineer and
recorded as [D-41](DESIGN-NOTES.md#d-41), so M17.3 is unblocked. Two conditions bind it: seed whatever can be
seeded (so a run replays from one number, and varying that number is how permutations are generated), and
where non-determinism is inherent and outside our control -- multiprocessor scheduling, kernel completion
order -- randomness is fair game. The corollary that keeps the second from becoming a loophole: inherent
non-determinism excuses *irreproducibility*, never *unverifiability*, so a test that cannot be replayed must
instead prove it reached the state it claims to exercise.
- [x] **M17.1** -- Close the two open cells of the handover precondition. A coverage inventory taken after M16
found this axis is mostly covered already, so this is no longer a matrix to build: *fresh* and
*queued-non-empty* are covered by [tests/completion_event.rs](tests/completion_event.rs),
*drained-then-resubmitted* by its `the_edge_re_arms_after_every_drain_to_empty`, and #47's own repro by
[tests/event_delivery.rs](tests/event_delivery.rs). What remains is (a) attaching while operations are
**in flight but no completion has landed**, for both `IoRing::completion_event` and `EventDelivery::new`,
and (b) drain-then-resubmit for `EventDelivery`. Gated on nothing; done first so #47's own axis is banked
before any convention decision.
**Done:** [tests/handover.rs](tests/handover.rs), four tests, each bound to `RingContract` so a lost,
duplicated or unclaimed completion is a contract violation rather than an inferred count.
**The planned approach for cell (a) was wrong, and its sabotage is what proved it.** The item called for
racing the attach against in-flight operations. That state does not exist for *buffered* reads: they
complete **synchronously inside** `submit_and_wait`, measured at 80 of 80 attempts across five shapes up to
512 reads of 64 KiB, always leaving a full queue at attach time and never a partial split. The first draft
swept a delay across the supposed window and passed; sabotaging [D-20](DESIGN-NOTES.md#d-20)'s setup signal
failed it on *attempt 0*, revealing the sweep sat entirely on the already-queued side and the test was a
slower restatement of `attaching_to_a_ring_whose_queue_is_already_non_empty_still_signals`. Recorded as
[D-40](DESIGN-NOTES.md#d-40). Cell (a) is covered two ways instead. **Deterministically:** one attach
serving a backlog queued *before* it **and** a wave submitted *after* it into the still-non-empty queue,
which is where a seam between the setup signal and the edge would strand work exactly as #47 did.
**And genuinely in flight:** unbuffered (`FILE_FLAG_NO_BUFFERING`) reads *are* asynchronous, so the state
is reachable after all -- reusing the aligned-buffer shape [tests/flush_barrier.rs](tests/flush_barrier.rs)
already had, extended to `IoBufMut`. That test **checks its own precondition**, counting what was already
queued at the instant of attach and failing if no attempt caught a read in flight, so it cannot decay into
the already-queued case on faster hardware; dropping only the `NO_BUFFERING` flag makes it fail with
exactly that message.
**Calibrated, and the two mechanisms are separable:** suppressing the setup signal fails only the two
mixed-queue tests; suppressing `EventDelivery`'s `activation.rearm` fails only the re-arm test. Disjoint
failure sets, so neither test is passing on the other's behalf.
**Cell (b) was genuinely open, shown by mutation rather than asserted:** with `activation.rearm` removed,
**all 107 pre-existing tests still pass** -- the entire suite was blind to `EventDelivery` silently
ceasing delivery after its first drain -- and only the new test catches it. The three targets that did not
get to run reference no `EventDelivery`, so they could not have caught it either.
Whether other test files should also bind to `RingContract` is left to **M18.3**'s `cargo-mutants` triage
to answer with evidence, rather than retrofitted blind now.
- [x] **M17.2** -- Decide and record whether randomized property testing is permitted here, as a
[DESIGN-NOTES.md](DESIGN-NOTES.md) decision. [D-39](DESIGN-NOTES.md#d-39) already fixed the terms under which
non-fixed test data is allowed in this component -- **seeded, announced, pinnable** -- and M15.2 proved them
end to end, so this is now the narrow question "does D-39 extend from poison to `proptest`?" rather than an
open-ended policy call. Inherit D-39's three terms rather than re-deriving them, and add the two it does not
cover: a committed regression corpus so any discovered failure becomes a permanent fixed case, and placement
as **integration** tests rather than unit tests (they cross the OS boundary and will exceed the one-second
unit budget).
**Decided: permitted**, recorded as [D-41](DESIGN-NOTES.md#d-41). D-39's three terms carry over to any
generator, and the two operational terms above are adopted. The engineer added a **second condition D-39 did
not anticipate**: where non-determinism is inherent and outside our control -- multiprocessor scheduling,
kernel completion order, thread-pool dispatch -- randomness is fair game, because a test depending on those
cannot be seeded and pretending otherwise would make it deterministic in its inputs while still
nondeterministic in what it observes. D-41 records the corollary that stops that becoming a loophole:
inherent non-determinism excuses *irreproducibility*, never *unverifiability*, so an unreplayable test must
prove it reached the state it claims to exercise. M17.1 is cited as the worked example of both halves -- its
racing draft decayed into testing the easier state, and its replacement guards against exactly that.
No dev-dependency is added here. D-41 permits randomized property testing without mandating a particular
crate, and M17.3 went on to satisfy its terms with a seeded `SplitMix64` rather than `proptest` -- see that
item for why.
- [x] **M17.3** -- Model the operation space as data: operation kind, buffer kind (owned / registered), file
target kind (raw / shared / registered), claim-or-drop, drain-now-or-later, and handover state. A generator
over *sequences* of these, each sequence checked against `RingContract`. The harness **must run under
`windows-guard-alloc` with poison enabled**: if it does not, generated sequences are checked for conservation
only, which discards every M15 detector at exactly the moment there is finally enough input to feed them.
Address space is not a constraint on doing so -- measured at 8 KiB per allocation, about 8.6e9 allocations
before 64 TiB, against a generative run needing a few million.
Bound by [D-41](DESIGN-NOTES.md#d-41): the generator is **seeded, announced and pinnable**, so one number
replays a whole run and varying it is how permutations are produced. Where a sequence's outcome also turns
on scheduling or kernel completion order, that residue is legitimately random -- but D-41's corollary still
applies, so any such sequence must **verify it reached the state it claims**, not merely fail to crash. Note
the guard allocator's seed and the generator's are two separate knobs and must not be conflated in the
announcement, or a replay will reproduce one and not the other.
**Done:** [tests/generated_sequences.rs](tests/generated_sequences.rs). 128 sequences per run, ~650
operations, all 15 reachable shapes, under the guard allocator and checked against `RingContract`.
**No `proptest`.** D-41 permits randomized property testing without naming a crate. Its terms are that one
number replays a whole run and is pinnable from the environment; `proptest`'s model is a persisted
regression file plus per-case seeds, a different shape, and its headline feature -- shrinking -- earns its
keep on long sequences. These cap at 10 steps and print every one, so a failure already arrives readable. A
seeded `SplitMix64` over one `u64` serves the stated terms exactly. If the step cap is ever raised
substantially, shrinking stops being redundant and the choice should be revisited; that condition is
recorded in the file's module doc rather than left implicit.
**Seeding verified end to end:** two runs pinned to the same pair of seeds produced byte-identical coverage;
a different seed produced different sequences; both decimal and `0x` hex parse. The two seeds are
independent knobs and both are announced with the replay command.
**The generator is self-verifying, per D-41's corollary.** A green run proves nothing unless it shows which
states it reached, so the test asserts every shape appeared, plus at least one deliberate drop, one
mid-sequence drain, and one attach against a ring with work outstanding (#47's own axis). Sequence count is
set *by* that assertion rather than by taste: at 48 the rarest shape (flush against a registered file, 3.3%
of operations) would be missed about 1 run in 4000 and flake CI; 128 puts that at ~3e-10.
**The first run found a defect in the generated program, not the ring.** It emitted a registered-buffer
operation whose token was deliberately dropped, and `RegisteredBuffers`'s drop guard fired.
`read_registered_raw`'s rustdoc already states that token "must be claimed", because claiming is the only
thing that releases the use -- `Token`'s drop is deliberately empty (D-4), so dropping instead pins that
buffer index for ever. Claim-or-drop is therefore a free axis only for owned buffers, which the generator
now encodes alongside the existing rule that two live operations never share one registered buffer index.
**Calibrated by sabotage, which also exposed a defect in the harness:** removing `RegisteredUse::drop`'s
`outstanding` decrement -- D-32's own class of bug -- is caught and reported as
`BufferStillInUse { index: 0, outstanding: 1 }` with the seed and the step that did it. The first attempt
reported it as a bare `debug_assert` from inside the crate instead, because the registration's drop guard
fires *while the error is being returned* and masked the diagnostic. The failure path now forfeits the
registration deliberately -- the same leak-not-free choice the crate makes in release -- so the report
survives. An instrument whose failure output names neither sequence, step, nor seed is barely an instrument.
Note this sabotage is a *smoke test* that the harness is not inert, not the calibration M17.4 requires:
it shows the oracle reports a fault injected into buffer accounting, not that the generator can emit the
shapes that trigger the defects which actually shipped.
- [x] **M17.4** -- Calibrate the generator before its green result is allowed to count. Show that it
rediscovers a defect known to be real -- #47's handover shape, or D-32's registration timing -- when the fix
is reverted. Every M15 and M16 item carried a sabotage step and this milestone had none: a generator that
cannot emit the shape that triggers #47 reports green for ever, and that green would feed straight into
release confidence. Until this item passes, M17.3's result is uninformative by construction. Note M18.3's
`cargo-mutants` run is a general form of the same calibration; this targeted version comes first because it
is cheaper and aims at two known-real defects rather than synthetic mutants.
**The calibration failed the generator, which is the entire reason this item exists.** With
[D-20](DESIGN-NOTES.md#d-20)'s setup signal removed -- #47 exactly as it shipped -- the M17.3 generator
reported **green**. It attached the completion event but drained with `try_pop`, and unconditional polling
recovers every completion whether or not the ring ever signalled, so a lost wakeup was invisible to it. A
generator can sample the right *states* and still be blind to the defect that lives in them.
**Fix:** once an event is attached, a sequence now **waits for the wakeup it is owed** before draining, and
a wait that times out with work still outstanding is reported. Waiting precedes draining rather than
following it, because a backlog queued *before* the attach is reachable only through the setup signal --
drain-first would consume it by polling and hide the signal's absence. Recorded as
[D-42](DESIGN-NOTES.md#d-42).
**Re-calibrated, and the detection is reliable rather than lucky:** with the sabotage restored the generator
caught it in **10 of 10 runs on fresh seeds**, always within the first six sequences, and the reported trace
is #47's own shape -- an operation queued, `attach completion event (queue had 1 outstanding)`, then a lost
wakeup. With the fix in place, 10 of 10 clean runs at ~80 ms.
**Second defect, D-32:** reintroducing the 0.1.2 use-after-free (build the SQE from an array dropped before
submit) is caught as a hard `STATUS_ACCESS_VIOLATION`. Worth recording precisely, because it is *not* what
0.1.2 did: there the same defect surfaced as a survivable `ERROR_NOACCESS` because the freed pages happened
to still be mapped. The only difference is the guard allocator decommitting them, which is M15.1's stated
purpose demonstrated against the real historical defect rather than a synthetic one. Note this one is caught
at the registration *setup* boundary, which every registering test crosses, so it calibrates the allocator
rather than the generated space -- #47 is the calibration that is specifically about the generator.
**Process note:** an intermediate "failure" run was traced to a stale test binary left by the previous
sabotage build, not to flakiness. Timestamps were checked against the rebuild before any conclusion was
drawn from a direct binary run; a calibration that mistakes a stale artefact for a result is worse than none.
- [x] **M17.5** -- Make discovered failures permanent: every sequence the generator finds becomes a named,
seed-free regression test committed alongside the corpus. A property test that finds a bug and then forgets
it has bought a single debugging session rather than a guarantee.
**The trap in this item is that the generator has found no defect in shipping code**, which makes "nothing
to make permanent" look like a reason to defer. It is not: the first real failure is exactly the moment
nobody wants to be inventing a corpus format, so the mechanism is built and proven now.
**Corpus:** named, seed-free entries replayed verbatim through the generator's own executor, each carrying
*why* it was recorded. Seeded with `regression_issue_47_backlog_at_handover` -- #47's shape exactly as M17.4
reported it, kept in the shape it was found rather than tidied into a minimal one, because a corpus that
rewrites what it was given cannot be trusted to have preserved the failing case. This is a real
strengthening rather than bookkeeping: the generator reaches that shape within a handful of sequences on
*most* seeds, and the corpus reaches it on *every* run.
**Verified load-bearing:** with [D-20](DESIGN-NOTES.md#d-20) reverted the entry fails deterministically,
reporting "reproduced a defect that was fixed", the recorded reason, and the step trace.
**The calibration itself is now automated**, which M17.4's could not be. That procedure edits `src/`, so
nothing in CI would notice if `wait_then_drain`'s wait were later "simplified" back into a poll -- the exact
change that made the generator blind to #47 ([D-42](DESIGN-NOTES.md#d-42)).
`the_lost_wakeup_detector_fires_when_no_wakeup_is_owed` constructs the lost-wakeup condition against a
*correct* ring instead -- attach, consume the setup signal, then wait again without draining, so no
empty-to-non-empty edge can occur -- and asserts the detector reports it. Sabotaging the wait makes it fail,
so the guard is real. A short 250 ms budget is used because that wait is *expected* to expire; `WAIT_MS`
would add five seconds per run to learn the same thing.
**Found while adding tests:** `temp_file` keyed its fixture on the process id alone, so the second test in
this binary would have raced the first for one path -- libtest runs them as concurrent threads. Now tagged
per test.
### M18 -- The population no runtime technique reaches
Population C, and the one that produced the two most severe findings of the M14 review round. `get_mut`
returning `&mut B` ([D-35](DESIGN-NOTES.md#d-35)) and `get` returning an unchecked `&B`
([D-36](DESIGN-NOTES.md#d-36)) are both defects of **what safe code is permitted to do**, not of what any code
path does. No fuzzer, oracle, allocator, poison scheme or chaos harness reaches them, because nothing has to
execute for the hole to exist. Both were found by review, which makes review a *primary* technique here rather
than a backstop -- and that deserves a written procedure instead of depending on who happens to read the diff.
Independent of M17; may run in parallel.
- [x] **M18.1** -- Audit every public type that hands out a borrow or an owned value, against one mechanical
question: **what can safe code do with this, and does the registration or the kernel still hold anything it
could invalidate?** `&mut Vec<u8>` permits `reserve`, `resize` and whole-value assignment; `&mut [u8]`
permits none of them. Record the finding per item even when the answer is "nothing", so the absence of a
hole is evidence rather than silence.
**Done:** [DESIGN-NOTES.md](DESIGN-NOTES.md) -> [Borrow-surface audit](DESIGN-NOTES.md#borrow-surface-audit-m181),
nineteen items, each recorded including the sixteen where the answer is "no hole".
**One finding, and it is the same shape as the two that prompted the audit.** `EventDelivery::ring` hands
out `&Mutex<IoRing>`, and *any* `&mut IoRing` permits whole-value assignment, so safe code can replace the
ring and silently stop delivery. Measured rather than argued: the replacement compiles, and a probe recorded
one completion delivered before the swap and **none** after, despite four further operations completing on
the replacement. The pool's wait holds a duplicate of the *original* ring's event, and nothing is ever
attached to the new one. Recorded as [D-43](DESIGN-NOTES.md#d-43); the fix is **M18.6** below.
**Worth carrying into M18.2:** the question that finds these is not "is this correct?" but "what else does
this type allow?" Note also that the two obvious fixes do not work -- a `Deref`/`DerefMut` newtype still
permits `*guard = ...`, and so does a `with_ring(|ring: &mut IoRing| ...)` closure. Anything that lets a
`&mut IoRing` escape keeps the hole.
- [x] **M18.2** -- Turn M18.1 into a recurring obligation rather than a one-time pass, as a
`DESIGN-INSTRUCTIONS.md` rule for this component: any change adding or widening a public borrow-returning
method answers M18.1's question in the PR. Both C-population defects were introduced by ordinary,
well-reviewed changes; what was missing was the specific question, not the diligence.
**Done:** [DESIGN-INSTRUCTIONS.md](DESIGN-INSTRUCTIONS.md) -- the repository's first, so it is kept tightly
component-scoped. It states the question, what counts as answering it (three points, with the M18.1 audit's
nineteen rows as worked examples), and that "no hole" is a complete answer that must still be written down.
**A document alone would not have made this an obligation**, which is the item's own point: the rule it
replaces was "remember to ask", and that failed three times. So the trigger is mechanical.
[BORROW-SURFACE.txt](BORROW-SURFACE.txt) is a generated inventory of every public function in `src/` whose
return type carries a borrow; `tools/check-borrow-surface.ps1` regenerates it and fails on any
disagreement, wired into CI beside the existing `encoding` and `baseline` sanity jobs -- whose own header
describes this same restatement-drift problem, so the pattern was already established here rather than
invented.
**Scope of the trigger, chosen deliberately:** a reference *or* a lifetime-carrying wrapper. A `&`-only
scan would have missed `Batch<'_>` and `RingScope<'_>` -- which is precisely where M18.6's fix lives, so the
narrower scan would have gone blind to the surface it was written for. Seven entries today.
**Verified load-bearing:** adding `pub fn sabotage_ring(&mut self) -> &mut IoRing` -- D-43's exact shape --
makes the check fail and print the question. It checks *shape*, not correctness, and the instructions say so
plainly: it cannot tell a safe accessor from a dangerous one, and tuning it until it stops firing would be
the failure mode to guard against.
**Found while writing it:** `.Count` on a `Where-Object` pipeline is a `StrictMode` error when the result is
empty or a single item -- the check crashed rather than passing on its first clean run. Fixed with `@(...)`.
- [x] **M18.3** -- Run `cargo-mutants` over the crate and triage the surviving mutants. There is a measured
rate to justify it: this branch produced two vacuously-passing tests (the M11.2 idempotence pair, which
asserted through a freshly attached event and so could not observe a detached one) and one `debug_assert`
that could never fire (`Option::take` had already emptied the slot it checked). Both are exactly what a
surviving mutant looks like.
**Result:** 306 mutants in 32 minutes -- 182 caught, 48 missed, 7 timeouts, 69 unviable. Counting timeouts
as detections (a mutant that hangs the suite fails CI as surely as one that trips an assertion), that is
**189 of 237 viable mutants caught, 79.7%**. Triaged into six categories in
[MUTATION-SURVIVORS.md](MUTATION-SURVIVORS.md), which is M18.4's input.
**It found a third vacuous test, which four review rounds had read past.** Both `Arc<[u8]>` tests in
`src/buf/tests.rs` compare `stable_ptr()` against *another call to the same function*, so returning a **null
pointer** -- the buffer address this crate hands the kernel -- passes both. They assert self-consistency,
never that the address is real. The neighbouring `&'static [u8]` test already shows the fix: compare against
an independently obtained address.
**And it caught the previous commit's author.** Every read-only accessor M18.6 added to `RingScope` survives,
because no test calls any of them. The surface was added on the stated principle that a platform layer is
not narrowed to its current caller, which stands -- but mutation testing showed within one commit that none
of it is exercised.
**A first reading of one survivor was wrong, and checking corrected it.** The `contract.rs` mutant deletes a
match arm in `check_quiescent`, which looked like the oracle's own busy-buffer detection going untested. It
is the **sort key**, not the detection: the test has one busy buffer, so ordering is unobservable. Recorded
as equivalent-under-current-tests rather than as a hole.
**Tooling, measured rather than assumed:** `cargo install cargo-mutants --locked` fails on this host --
the locked `winapi` does not compile for `aarch64-pc-windows-msvc` (285 errors). Installing unlocked, from
outside the repo so `rust-toolchain.toml`'s 1.98.0 pin does not apply to the tool, works. `--timeout 180` is
needed because several mutants hang, and a cap derived from the ~5 s baseline is too tight for a suite whose
own waits are 5 s. All of this is in the reproduction section of the survivors file.
Note `cargo-mutants` has no `cargo_*` MCP tool, so it is the rare case where the terminal is the only route.
- [x] **M18.4** -- Resolve the survivors: strengthen the test, or record why the mutant is equivalent and
harmless. Do not delete an assertion merely because a mutant survives it -- a dead assertion beside a live
one is worse than none, which is why M14.3's was removed rather than repaired.
**Result: 48 survivors down to 12, and the score from 79.7% to 94.9%** (219 caught, 12 missed, 6 timeouts,
69 unviable of 306). 18 new unit tests; no production code changed. Verified by re-running `cargo-mutants`,
not by assuming the tests would kill what they targeted -- which mattered, because four of them did not.
**The vacuous `Arc<[u8]>` tests are fixed** by comparing `stable_ptr()` against `Arc::as_ptr`, an
independently obtained address, exactly as the neighbouring `&'static [u8]` test already did.
**Four tests failed to kill what they aimed at, and re-running found each one.** Worth recording, because
every one of them looked correct: (1) the sort-order test used three busy buffers, and with the sort key
deleted `sort_by_key` is stable, so `HashMap` order came out sorted anyway -- now eight; (2) `checked_index`
is the *span* bounds check, not `get`'s, so an out-of-range `get` never reached it -- now submits a span;
(3) `user_data() -> 0` matched because a fresh ring's first operation *is* zero -- now burns ids first, and
then `-> 1` matched for the same reason, so it burns two; (4) `supports_raw` was asserted only in the
negative, which a constant `false` satisfies. **A test written against a mutant is not verified until the
mutant is re-run.**
**Two survivors are provably unkillable**, argued rather than assumed: `as_hresult`'s `|` versus `^` operate
on disjoint bit sets and are the same function; `with_injected_failure`'s zeroed `information` is
unreachable because `result()` returns `Err` first. Recorded with proofs.
**The rest are recorded with named reasons** in [MUTATION-SURVIVORS.md](MUTATION-SURVIVORS.md), separated
into unkillable, host-blocked (two `supports -> true` mutants need an `Op` this host does not support, and
widening a closed enum to satisfy a mutant would be inventing API for a test), queued (capability decoding,
spawned as **M18.7**), and two `Debug` impls whose exact wording an assertion would only change-detect.
"Hard to test" is written down as that, never as "equivalent".
- [x] **M18.7** -- Extract the capability flag decoding in `capabilities()` into a pure function over the raw
`IORING_CAPABILITIES`, so it can be exercised with synthetic flag values. Spawned by M18.4: four mutants on
`raw.FeatureFlags & FLAG != 0` survive because nothing can vary what `QueryIoRingCapabilities` returns, and
the decoding of `supports_completion_event` in particular gates every completion-event path in the crate --
[D-20](DESIGN-NOTES.md#d-20), and #47 lived downstream of it. A pure `fn decode(raw) -> Capabilities` plus a
table of flag combinations kills all four and makes the one capability that matters testable in both
directions, which no test can do today.
**Done:** `decode(&IORING_CAPABILITIES) -> Capabilities` split out of `capabilities()`, which now does the
syscall and nothing else. Pure refactor -- no behaviour change -- plus five tests, and a round-trip test
tying `decode` back to the query so the split cannot drift.
**All four mutants killed**, verified by a scoped re-run: `capability.rs` reports 12 mutants, 9 caught and
3 unviable, none missed. The killing cases fall straight out of the pure function: an empty mask kills
`& -> |` (because `flags | FLAG` is non-zero whatever `flags` holds), a mask carrying exactly one named flag
kills `& -> ^` (which clears the very bit it is testing), and any positive case kills `!= -> ==`. An
unknown-feature-bit case is included because this crate has already been surprised once by a Windows
reporting a version it did not name.
**Crate-wide: 307 mutants, 221 caught, 10 missed, 6 timeouts, 70 unviable -- 95.8%**, up from 79.7% before
M18.4.
**Recorded a caveat on the headline number rather than quoting it flat.** Two mutants -- `Batch::require`
and `IoRing`'s `Debug` -- were reported caught in one run and missed in another *with identical sources*.
The suite has timing-dependent tests and `cargo-mutants` runs four jobs in parallel, so a mutant can be
"caught" by a test that merely flaked. Both are now filed as unresolved rather than fixed, and
[MUTATION-SURVIVORS.md](MUTATION-SURVIVORS.md) says a single run's score is approximate.
**`Batch::require -> Ok(())` was reclassified on the evidence:** `require` returns `Ok` exactly when
`supports` is true, so on a host supporting every operation it is equivalent *in practice*. Filed under
host-blocked rather than provably-unkillable, because the equivalence is a property of the host, not the
code -- and its single "caught" result is consistent with the flake above.
- [x] **M18.5** -- Document the strategy as a whole in [DESIGN-NOTES.md](DESIGN-NOTES.md): the three defect
populations, which technique covers which, and -- most importantly -- **what none of them cover.** Two of the
eight defects came from spikes against the real kernel, and no oracle, generator or allocator would have
produced either, because both were cases of Windows behaving differently from the assumed contract. Record
spikes as a budgeted technique for each new Win32 surface rather than something that happens when a test
mysteriously fails.
**Done:** [DESIGN-NOTES.md](DESIGN-NOTES.md) -> [Testing strategy](DESIGN-NOTES.md#testing-strategy-m185),
plus [D-44](DESIGN-NOTES.md#d-44) making the spike rule citable from the decision index rather than reachable
only through prose.
**The item's own premise needed a correction, and checking found it.** "No allocator would have produced
D-32" is not quite true: the guard allocator *does* catch it, as a hard `STATUS_ACCESS_VIOLATION`, measured
in M17.4's calibration. The accurate statement is sharper and is what got written down -- every technique
here checks this crate's code against **this crate's stated contract**, and in both spike-found defects the
stated contract was the thing that was wrong. They detect a *consequence*, on a path some test already
walks; a spike produces the platform knowledge that decides what the code should be, and is the only
technique that runs **before there is code to test**.
**Two things the write-up records that no single milestone did.** First, M15 and M16 finding nothing is not
reassurance: they are passive, so zero findings meant the detectors had only seen the twenty-odd
hand-written scenarios that already passed -- which is why M17 exists to feed them. Second, **every one of
these instruments was wrong the first time and only sabotage found it**: M17.3's generator reported green
with #47 reintroduced, four of M18.4's mutation-killing tests did not kill their mutant, and M15.2's poison
inverse was fabricated twice. Budget the calibration, not just the instrument.
Also records the two rejected techniques -- a mock `IoRing` and PageHeap -- so neither is re-proposed as an
obvious win, and a per-technique table of what each actually found rather than what it was hoped to find.
- [x] **M18.6** -- Narrow `EventDelivery`'s ring surface so safe code cannot replace the ring out from under
the pool's wait. Spawned by M18.1, recorded as [D-43](DESIGN-NOTES.md#d-43), and **measured**: the
replacement compiled and silently stopped delivery -- one completion before the swap, none after.
**The two obvious fixes do not work.** A `Deref`/`DerefMut` newtype still permits `*guard = ...`, and so
does handing a `&mut IoRing` to a closure. Closing the hole means never letting a `&mut IoRing` escape at
all.
**Done:** `EventDelivery::ring -> &Mutex<IoRing>` is replaced by `EventDelivery::scope -> RingScope`.
The exposure rule is stated rather than ad hoc: **every read-only part of `IoRing`, plus batch
construction -- and nothing that can retarget the ring or steal the pool's completions.** So `batch`,
`outstanding`, `info`, `version`, `supports`, `supports_raw` and the registered counts; deliberately no
`try_pop` ([D-21](DESIGN-NOTES.md#d-21) makes the pool the single drainer), no `completion_event` (two
waiters on one ring), no `run_down`, and no `&mut IoRing`.
**Reviewed for sizing before starting, and one item was right.** Nine call sites across six files, all of
which stop compiling the moment `ring()` is removed -- so any split would have produced a non-compiling
intermediate commit, and the add-alongside-then-remove shape would have left the hole open across commits.
Larger than the item text implied, though: it also forced `handover.rs`'s `submit_wave` to split into a
`queue_wave` that fills a caller-owned `Batch` (four plain-ring call sites kept their old shape), because
`Batch::submit_and_wait` consumes the batch and the scope hands out a `Batch` rather than a ring.
**Proved by construction, not by review**, which is the whole point of M18: a `compile_fail` doctest
asserts the replacement no longer compiles. That doctest was itself verified -- adding a `DerefMut` impl
makes it **fail**, so it is passing for the right reason rather than on a typo in its own setup, and that
same experiment is the empirical proof of the "a `Deref` newtype would not close this" claim above. It is
paired with a normal doctest sharing the setup, since a `compile_fail` example passes on *any* error.
**Incidental fix:** call sites were inconsistent about mutex poisoning -- some `expect`, some
`unwrap_or_else(PoisonError::into_inner)`. `scope()` absorbs poisoning internally, matching what the wait
callback's own drain already did, so the question no longer reaches callers at all.
Verified: 148 tests pass, both affected examples (`model_a_delivery`, `epoch_log`) still run to exit 0.
## Moved 2026-09-07 -- M20.5: dissolved by D-47 rather than decided
### <a id="m205"></a>M20.5 -- Dissolved by [D-47](DESIGN-NOTES.md#d-47-detail) rather than decided: the `flush_barrier` assertion was measuring a claim the platform does not honour, so it was never a flaky test. *(completed 2026-09-07 21:40:23 -04:00)*
**Dissolved by [D-47](DESIGN-NOTES.md#d-47-detail), not decided.** This asked whether to make the
load-sensitive `flush_barrier` assertion load-independent or mark the test serial. Neither: the
assertion was measuring a claim the platform does not honour, so it was not a flaky test at all. Its
own contention hypothesis was disproved in the same measurement -- an idle ring failed too. Recorded
in [RESOLVED-TEST-FAILURES.md](RESOLVED-TEST-FAILURES.md).
## <a id="m19"></a>Moved 2026-09-09 -- M19: close the `get()` borrow hole
Found by the code review of the M15-M18 branch, and **measured**: safe code can hold the `&[u8]` that
`RegisteredBuffers::get` returns across a submit that makes the kernel write into that same buffer. A probe
observed the bytes change from `0x11` to `0xEE` through the live borrow while a *fresh* `get(0)` at that same
instant correctly refused with `WouldBlock`.
The [D-36](DESIGN-NOTES.md#d-36) fix checks `kernel_writes` at the instant of the call, but returns a slice
whose lifetime is tied to `&self` -- and `Batch::read_registered` takes the registration by **shared**
reference, so the borrow and the read coexist. `get_mut` is unaffected: `&mut self` conflicts with the shared
borrow, so the compiler already rejects the analogous sequence. Only `get` is exposed.
This matters more than its severity alone suggests: it is the same hazard class D-36 was filed to close, in
the API whose breaking change 0.2.0 is being cut for, and it is reachable with no `unsafe` anywhere.
- [x] **M19.1** -- Make the borrow's *existence* conflict with starting a read into that buffer, not merely
its creation. The options differ in what they cost, and the choice is the engineer's:
(a) `get(&mut self)`, which is one line and closes it completely, but forfeits the concession D-36
deliberately kept -- a caller could no longer read a buffer while its own *write* is in flight, which is
sound because a write means the kernel only reads;
(b) a `with_bytes(i, |bytes: &[u8]| ...)` callback, or a returned guard type holding a reader count that
`begin_use(KernelAccess::WritesBuffer)` then refuses against, which preserves (a)'s concession at the cost
of a wider API change.
**Chose (a), after the engineer's question collapsed the choice:** does a caller ever *need* access during
the hazard window? No -- while a read is in flight the bytes are indeterminate, partially written in
arbitrary order, and only become meaningful once the completion is observed; while a write is in flight the
caller wrote them and already knows. Earlier or later is always available, so the concession (b) preserves
has no legitimate use.
**The codebase agreed before the change was made:** all ~40 read sites across tests, examples and the
epoch-log sample already read at a quiescent point -- their own `expect` messages say "is quiet", "is quiet
again", "neighbour slot is quiet". Converting them needed nothing but `mut` on ten locals; not one held a
borrow across a submit.
The SAFETY comment now states an invariant the signature actually provides, and the rustdoc explains why
`&mut self` hands back a shared slice.
**The arena pattern is intact**, which was the thing worth checking: a `Token` holds a `RegisteredUse`, not
a borrow of the registration, so quiet neighbours stay readable while operations are outstanding.
- [x] **M19.2** -- Add the regression test the probe became: hold the borrow across a submit and assert the
bytes cannot change, or that the sequence no longer compiles. Verify it by sabotage like every other
instrument in M15-M18 -- reverting the fix must turn it red.
**Done as a `compile_fail` doctest on `get`**, since the hazard is now a type error rather than a runtime
one. **Sabotage-verified:** reverting the signature to `&self` makes it fail -- the sequence compiles again,
which is exactly the hole.
Paired with a `no_run` doctest asserting the *neighbour* case still compiles, because a `compile_fail`
passes on any error and a guard that over-constrained the arena would look identical to one that did not.
- [x] **M19.3** -- Sweep for the same shape elsewhere. The M18.1 audit asked what a returned value *permits*,
not how long its borrow *lasts*; those are different questions and only the first was asked. Re-check every
borrow-returning entry in [BORROW-SURFACE.txt](BORROW-SURFACE.txt) against the second, and record the
distinction in [DESIGN-INSTRUCTIONS.md](DESIGN-INSTRUCTIONS.md) so the recurring question covers both.
**Swept all seven entries; `get` was the only one.** `get_mut` never had it (`&mut self` already conflicts
with the shared borrow `read_registered` needs -- confirmed by compiling the analogous sequence and getting
`E0502`). `RingScope::batch` and `EventDelivery::scope` both borrow exclusively and confine what they hand
out. `RingContract::violations` holds no kernel resource and every `observe_*` takes `&mut self`.
`IoRingError::name` returns `&'static str` from a literal.
[DESIGN-INSTRUCTIONS.md](DESIGN-INSTRUCTIONS.md) now poses **both** questions, with a mechanical form for
the second -- take the borrow, then try to call everything that could start work against the same object --
and D-45 is added to its table of shipped defects of this shape.
**Swept the count restatements too:** that file said "three defects" in four places and is now four, which
is the restatement drift the repository's own conventions warn about.
## Moved 2026-09-19 22:40:58 -07:00 -- M20.4: the file-handle NUMA mechanism correction
### <a id="m204"></a>M20.4 -- Correct "What is not reachable" in [DESIGN-NOTES.md](DESIGN-NOTES.md): the file-handle-to-storage-node mapping is reachable on mechanism, and the conclusion it supported now rests on volume granularity, absence, and spanned volumes instead. *(completed 2026-09-19 22:40:58 -07:00)*
The authoritative text is the rewritten "What is not reachable" section in
[DESIGN-NOTES.md](DESIGN-NOTES.md); the research behind it is `F-1` in
[DESIGN-SESSION-2026-08-30-numa-sharded-io-execution-domains.md](../../design-sessions/DESIGN-SESSION-2026-08-30-numa-sharded-io-execution-domains.md).
Two things the item's own text had wrong, corrected while doing it rather than copied forward:
- It said to cite [file-handle-numa-spike.rs](design-sessions/spikes/file-handle-numa-spike.rs) as the
**unrun** instrument, and to state that no measurement of either call succeeding on an ordinary NTFS
data file could be found. `F-1a` of the same session had already smoke-run it: both calls succeed on an
ordinary NTFS data file and on a directory handle, and agree. The item was written from `F-1` without
`F-1a`. What remains unmeasured is narrower -- whether either call ever names a node that distinguishes
one device from another, which needs a multi-node host with storage whose PDO advertises a proximity
domain.
- The spikes [README.md](design-sessions/spikes/README.md) carried the same staleness, each instance
contradicted by its own body a few paragraphs later. Swept: 3 phrasings in 1 file, plus the sentence
promising that a multi-node run "would correct a claim DESIGN-NOTES.md currently makes", which this item
has now made false -- it settles an open question instead.
The heading stays "What is not reachable". What is not reachable is the *answer* a ring consumer wants,
which is still true; renaming it would dangle the pointers in the 2026-09-19 review session.
## Moved 2026-09-19 22:49:09 -07:00 -- M20.2: the ARM no-L3 measurement recorded as D-48
### <a id="m202"></a>M20.2 -- Record the 2026-08-30 ARM measurement as a decision, beside the zero-NUMA-node observation it is the sibling of. *(completed 2026-09-19 22:49:09 -07:00)*
Landed as [D-48](DESIGN-NOTES.md#d-48) in the decision index, plus a sibling paragraph in
"Why the NUMA node is the wrong key" where the existing zero-node observation lives, which is where the
item asked for it.
Two choices worth recording, because both were places this could have gone wrong:
- **The measurement is cited, not re-transcribed.** The capture is Measurement M-1 in
[DESIGN-SESSION-2026-08-30-numa-sharded-io-execution-domains.md](../../design-sessions/DESIGN-SESSION-2026-08-30-numa-sharded-io-execution-domains.md),
and D-48 links it rather than copying the probe output into a third place. Pasting the block would have
created exactly the restatement the repository conventions warn about.
- **The false clause it falsifies is marked, but not rewritten.** D-48 sits two paragraphs from the
sentence saying the last-level-cache domain "is meaningful on Intel and ARM too", which this
measurement shows is false on a shipping part -- so leaving it unmarked would have made the document
contradict itself. A one-line adjacent marker says so and points at `M20.1`. The restatement of the
rule and the sweep across the README, `lib.rs` and `policy.rs` remain `M20.1`, which is coupled to
`SH-4.12` and must follow it.
## Moved 2026-09-20 23:16:19 -04:00 -- M21.1: the last site asserting D-24's withdrawn half
### <a id="m211"></a>M21.1 -- Correct the last site that still asserts [D-24](DESIGN-NOTES.md#d-24)'s withdrawn half: the epoch-order assertion in the epoch-log committer, whose justification cited the hold-back claim [D-47](DESIGN-NOTES.md#d-47) removed. *(completed 2026-09-20 23:16:19 -04:00)*
The assertion is unchanged, because it was always sound -- just for the other reason. Every commit
carries `FlushCoverage::CoversPrecedingOperations`, so commit *N* is outstanding when commit *N+1* is
reached, and D-47's *surviving* half -- no operation queued before a drained flush was ever observed
completing after it -- is what orders them. The comment now says that, and says explicitly what it does
not rest on, so a later reader does not "restore" the withdrawn reasoning.
Sweep re-run before committing, as the item required. **21 matches across 14 files**, disposed as:
- **1 violation**, fixed: the justification in [commit.rs](examples/epoch_log/commit.rs).
- **1 historical site**, marked rather than rewritten:
[DESIGN-SESSION-2026-08-28-external-consumer-correspondence.md](design-sessions/DESIGN-SESSION-2026-08-28-external-consumer-correspondence.md)
records what the findings became on that date, and glossed D-24 as a stall that "holds operations
against unrelated files". A session is a faithful record of its moment, so the gloss stays and a
one-line note beside it says which half was later withdrawn.
- **2 false positives**, left alone: the spikes README ("checked in deliberately rather than held back",
about an instrument) and [fault_injection.rs](tests/fault_injection.rs) ("holds it until the completion
is claimed", about a token).
- **17 already correct** -- the decision index, both sides of the crate docs, the README, the stress
tests, strategy.rs, and the item's own text.
The item quoted the original sweep as "17 matches across 10 files". Re-running it found more of both,
and the file count needed care: `rg` groups the two design-session files under a single header, so the
first reading of its output undercounted by one. Counted with a command rather than by eye.
Verified by running the example in a debug build, where the `debug_assert` is live: it completed, the
negative control still caught the corrupted record, and all four epochs reported durable in order.
## Moved 2026-09-21 00:23:49 -04:00 -- M21.2: a bounded pop, generic over the wait
### <a id="m212"></a>M21.2 -- Publish a bounded pop and the wait it is generic over, then remove the two unbounded spins. *(completed 2026-09-21 00:23:49 -04:00)*
**Decided: the trait, plus a default impl and a convenience.** The crate cannot choose the wait for a
caller, and that is a contract rather than a shrug -- the completion event is auto-reset with exactly one
waiter per ring ([D-21](DESIGN-NOTES.md#d-21)), so a wait this crate picked could consume an edge the
caller's own loop was entitled to. Making it a parameter moves the obligation to the only party who can
discharge it.
The surface added to [ring.rs](src/ring.rs):
- `IoRing::pop_within(timeout)` -- the convenience, using `SubmitWait`.
- `IoRing::pop_within_with(wait, timeout)` -- the same loop over a caller-chosen wait; `?Sized`, so a
trait object works.
- `CompletionWait` -- one method, whose contract is deliberately weak: returning early or spuriously is
always permitted, because [D-19](DESIGN-NOTES.md#d-19) already makes a wake with nothing to pop normal.
An implementation cannot be subtly wrong about *when* to return, only wasteful.
- `RingWait` -- the ring narrowed to `block` and `outstanding`, so a waiter cannot pop the completion
its own caller is waiting for, nor submit work nobody asked for. Same narrowing, same reason, as
`RingScope` under [D-43](DESIGN-NOTES.md#d-43).
- `SubmitWait` -- blocks inside `SubmitIoRing` with nothing queued, touching no event, so it composes
with a caller who owns the completion event.
**The borrow question, answered even though the check says the surface is unchanged.** `RingWait` is only
ever passed *in*, never returned, so [BORROW-SURFACE.txt](BORROW-SURFACE.txt) is untouched (verified: 7
entries, unchanged). Asked anyway, since it is a lifetime-carrying wrapper: safe code reaching it can call
`block` and `outstanding` and nothing else; the ring it borrows is held exclusively for the call, so
nothing it could invalidate is live elsewhere; and the narrow type is the point rather than an accident,
because a bare mutable reference to the ring would have permitted exactly the two things a waiter must
not do.
**Measured while building it, and now documented on `RingWait::block`:** `SubmitIoRing` answers
`E_INVALIDARG` (`0x80070057`) -- not a timeout -- when asked to wait for a completion the kernel has no
pending operation for. Found by a test that drove the loop with a reservation having no real SQE behind
it. The precondition holds structurally: `pop_within_with` checks `outstanding()` before consulting the
wait, so `block` is unreachable with nothing pending.
**An early return that is not an optimisation.** With nothing outstanding and an empty queue no completion
is possible, so the loop answers immediately rather than sleeping out the bound. That turns "you forgot to
submit" from a timeout into an instant answer, and it is what the sabotage below pins.
**A panic path found and closed before it shipped.** The first draft computed an instant plus the caller
timeout directly, which panics on overflow -- so `Duration::MAX`, a reasonable spelling of "no deadline",
would have taken down the process. Now `checked_add`, with an unrepresentable deadline treated as one that
never arrives. Two tests cover it: one where the early return answers first, one where an operation is
outstanding so the overflow branch is actually reached.
**Sabotage-verified**, because a test that cannot go red proves nothing:
- Removing the clamp that keeps the wait timeout above zero turns `the_wait_is_never_handed_a_zero_timeout`
red.
- Removing the nothing-can-arrive early return turns two tests red, and the run takes the full 30-second
bound instead of finishing instantly -- which is the behaviour the early return exists to remove.
**Call sites converted:** the two unbounded spins in
[append.rs](examples/epoch_log/append.rs) and [fault_injection.rs](tests/fault_injection.rs), and the
test-only `pop_within` helper, which is now a thin panicking wrapper over the public API rather than a
fourth copy of the loop. The panic is the only test-specific part left: a test wants the name of what it
waited for, a consumer wants an `Option` it can act on.
## Moved 2026-09-21 02:23:34 -04:00 -- M21.3: the epoch trigger, and a review claim the measurement disproved
### <a id="m213"></a>M21.3 -- Key the epoch commit off a completed append rather than off the counter, so the trigger cannot fire on a pass that appended nothing. *(completed 2026-09-21 02:23:34 -04:00)*
The match in [main.rs](examples/epoch_log/main.rs) now yields a `closed_an_epoch` value that every arm
must produce, rather than a `appended % EPOCH_SIZE == 0` test written after it. Closing an epoch is a
fact about an append that landed, and making each arm answer the question keeps that local -- a new arm
added later cannot fall through into a commit.
**The item predicted this was a latent bug. It is not, and the measurement is what settled it.**
The retry path was instrumented to report when the old shape would have committed, and run at
`EPOCH_SIZE` of 6, 8, 12, 16 and 24. It fired **zero** times -- including at every value past `SLOTS`,
which both the item and finding `C-3` predicted would arm it.
The reason is an invariant three blocks from the trigger: `appended % EPOCH_SIZE == 0` is true at exactly
two moments -- before the first append, and immediately after a commit -- and the arena is empty at both,
because the commit waits for a covering flush that retires every outstanding write. An append is
therefore never refused at an epoch boundary, at any constants.
**So this is a coupling change, not a bug fix**, and the distinction is the useful part. The old trigger
was safe because of something nothing stated, three blocks away; the new one cannot fire because of where
it is written. The second survives a reader who changes the commit path. The first is what made the
question take a measurement to answer at all.
Swept the claim rather than only fixing the code: finding `C-3` in
[DESIGN-SESSION-2026-09-19-epoch-log-review.md](design-sessions/DESIGN-SESSION-2026-09-19-epoch-log-review.md)
carried the same wrong prediction and now carries the correction beside it. The review lesson recorded
there is the narrow one: "unreachable today, armed tomorrow" is a claim about a program's reachable
states, and reading the code is not how to settle one.
## Moved 2026-09-21 16:06:19 -04:00 -- M21.4: what a failed commit means, and the sample's first tests
### <a id="m214"></a>M21.4 -- State what a failed commit does to durable_through, and bind it with tests in both directions. *(completed 2026-09-21 16:06:19 -04:00)*
The doc on `Committer::claim` said "A failed commit advances nothing", which reads as a permanent verdict.
It is not. Every commit here is a **covering** flush, so commit *N+1* reaches epoch *N*'s writes -- queued
before it -- and observing *N+1* makes *N* durable after all. What makes a record durable is a flush that
covered it, not the identity of the flush named for its epoch. The doc now says that, and says what a
caller must not read into a failure: not "epoch *N* is lost", but "not yet".
**The sample had no tests at all.** Examples are not test targets by default, so `cargo test` compiled
this one and ran nothing. Binding the claim meant adding `test = true` to the `[[example]]` entry first;
that is the change that makes any of the sample's policy testable, not just this item's part of it.
**The failure path is unreachable by running the sample**, because a flush against a healthy temp file
does not fail. The crate's fault-injection seam is the only way in, so four of the six tests are gated on
`fault-injection` and the other two run by default. That follows the precedent and the reasoning already
written down in [fault_injection.rs](tests/fault_injection.rs), including that CI's
`cargo test --workspace --all-features` job is what stops gated tests from being tests that never run.
**An assumption caught by asserting it.** The test first asserted that an injected `ERROR_ACCESS_DENIED`
would surface as `io::ErrorKind::PermissionDenied`. It surfaces as `Other`: the crate preserves the
HRESULT in an `IoRingError` rather than classifying it. Corrected to assert the Win32 code, which is the
assertion `tests/fault_injection.rs` already makes one layer down.
**Sabotage-verified in both directions, and the two produce different failure sets** -- which is what
shows the tests discriminate rather than all keying on one fact:
- `is_durable` returning `true` unconditionally: **4 of 6 fail**, caught by the assertions that an epoch
is *not* yet durable.
- `is_durable` requiring an exact match with the watermark: **2 of 6 fail**, caught by the covering and
monotonicity tests.
**Leverage from earlier in this milestone:** `commit_and_pop` is three lines because `M21.2` published
`IoRing::pop_within`. Without it every test here would have carried its own bounded wait, which is the
duplication `M21.2` existed to remove.
## Moved 2026-09-21 18:18:52 -04:00 -- M21.5: every unbounded wait in the crate, not the two that were named
### <a id="m215"></a>M21.5 -- Give the harness's wait loops a bound, and collapse the hand-written waits onto the bounded pop. *(completed 2026-09-21 18:18:52 -04:00)*
`await_flush` and `await_writes` in [strategy.rs](examples/epoch_log/strategy.rs) blocked in
`submit_and_wait` for `WAIT_MS`, ignored that it had returned without a completion, and went round
again forever. Both now carry a deadline and raise `TimedOut`, which is the policy
[event_loop.rs](examples/epoch_log/event_loop.rs) already documented: a measurement harness that hangs
reports nothing, which is strictly worse than one that fails.
**The item named two loops. A census found four, and two more of a related shape.** Counted by command
over every `.rs` outside `target/` and the spikes:
- [strategy.rs](examples/epoch_log/strategy.rs) `await_flush` and `await_writes` -- the two named.
- [failure_paths.rs](tests/failure_paths.rs) and [kernel_span.rs](tests/kernel_span.rs), each with a
helper **called `await_one`**, byte-identical to the other, neither named by the item.
- [batch/tests.rs](src/batch/tests.rs), where two registration waits were a single `try_pop` -- the
flake shape `pop_within` documents -- in a file whose *third* such wait already used the helper.
One predicate, three sites, half-converted, which is FAIL FAST rule 1 exactly.
**Sabotage-verified, and it revealed the milestone compounding.** Making `classify` stop filing flush
results leaves `await_flush` looking for a completion that is never recorded -- an unbounded loop would
hang forever. It failed with `timed out after 30s waiting for a commit's flush`, and it did so in **two
seconds**, not thirty: `pop_within`'s nothing-can-arrive early return (`M21.2`) answers immediately once
the ring is quiesced. The bound is what makes the failure possible; the early return is what makes it
quick.
A `remaining` helper and a single `timed_out` constructor keep the two waits from describing the same
condition two ways, and `WAIT` is derived from `WAIT_MS` rather than written twice.
Factoring `Lane::classify` out of `Lane::drain` is what let the bounded waits file a completion they
blocked for without a second copy of the claim-then-check logic.
## Moved 2026-09-21 21:15:12 -04:00 -- M21.6: the API review's four defects
### <a id="m216"></a>M21.6 -- Fix the four defects an independent review of the M21.2 surface found: the timeout mapping, its victim in `run_down`, the INFINITE collision, and the test hole that hid all of them. *(completed 2026-09-21 21:15:12 -04:00)*
Queued and closed in one item because the first two share a root cause and the fourth is the reason
neither was caught. The surface is unreleased and on a branch, so none of this is a breaking change.
**The defect.** `SubmitIoRing` reports an expired wait as `ERROR_TIMEOUT` (`0x800705B4`), a *failure*
HRESULT. `RingWait::block` passed it through `check`, so `pop_within` returned `Err` on every real
timeout and never the `Ok(None)` it documents. All six converted call sites treat `Err` as fatal and
`Ok(None)` as the timeout signal, so the `ErrorKind::TimedOut` mapping they document was dead code on
the only path that produces it.
**Its second victim, pre-existing.** `run_down` polls in 50 ms steps with the same `check`, so any
operation slower than 50 ms made rundown return `Err` with work still outstanding -- and `Drop` then
asserted and called `CloseIoRing` anyway, which is precisely the "the kernel may still be writing
through a token's buffer" hazard the function exists to prevent. One `wait_outcome` helper now
classifies a wait-only `SubmitIoRing` result for both callers, so they cannot disagree again.
The rundown loop is deliberately left unbounded. Every SQE that queues produces exactly one completion
(M10.2), so it terminates; blocking until that holds is the safe failure, and closing the ring early is
not.
**The `INFINITE` collision.** A finite bound above ~49.7 days saturated onto `u32::MAX`, which is Win32's
`INFINITE` -- and `CompletionWait` explicitly invites implementations built on `WaitForMultipleObjects`.
Clamped to `MAX_WAIT_MS` (one below), and the trait now states that `timeout_ms` is never zero and never
`INFINITE`, so it can be passed straight to a Win32 wait.
**The contract gap that would have propagated it.** `CompletionWait` never said how to report an expired
wait, and every Win32 wait reports one as an error -- so any third-party implementation forwarding its
underlying result would have reproduced the defect exactly. The trait now says an expired bound is
`Ok(())`, and says why.
**Why no test caught it, which is the finding worth keeping.** The M21.2 tests drive the loop with a wait
that never enters the kernel. Deterministic, and it leaves the Win32 interaction untested: replacing
`RingWait::block`'s entire body with an unconditional error left **the whole suite green**.
Closing that needed an operation still pending when the bound expires, and three attempts failed before
one worked -- each measured, not assumed:
| Attempt | Result |
|---|---|
| Buffered read, up to 256 MiB | completion already poppable, 3-5 us |
| Flush over 512 MiB of dirty cache | 3 us -- the lazy writer had already written it back |
| Unbuffered read, 256 MiB | 3 us -- **a handle without `FILE_FLAG_OVERLAPPED` is synchronous**, so the read completes inline during submit |
| Unbuffered **and** overlapped, 64 MiB+ | genuinely pending; the bound expires |
That third row is the general finding: **the crate's existing tests all use synchronous handles**, so
ring operations complete inline during submit and asynchronous completion is never exercised. That is why
a counting waiter over the existing flush pattern was reached in 0 of 50 trials.
[tests/bounded_pop.rs](tests/bounded_pop.rs) now covers it with five tests over a 128 MiB unbuffered,
overlapped read. Each asserts `outstanding() > 0` alongside the expected answer, so a machine fast enough
to finish the read early fails loudly instead of passing vacuously. Nothing asserts an upper bound on
elapsed time: Windows' default timer resolution is ~15.6 ms, so a 5 ms bound routinely takes 14-19 ms.
**Sabotage-verified, twice.** Reverting the timeout mapping turns **all five** red. Reproducing the
review's original mutation -- `block` always failing -- now turns three red, where before it turned none.
The review's fifth finding, that `check-borrow-surface.ps1` is blind to trait methods and to borrows in
parameter position, is queued as `M21+.1` rather than fixed here: it is a process gap, not a runtime
defect, and widening the check obliges a borrow-question answer for every entry it newly reports.
## Moved 2026-09-21 21:23:24 -04:00 -- M21+.1: the borrow-surface check learns two shapes
### <a id="m21plus1"></a>M21+.1 -- Teach [check-borrow-surface.ps1](../../tools/check-borrow-surface.ps1) the two shapes it was blind to: methods of a \pub trait\, and borrows in parameter position. *(completed 2026-09-21 21:23:24 -04:00)*
The check inspected only the text after the last `->` on lines matching `pub fn`. Trait items are
declared `fn`, not `pub fn`, so nothing inside any `pub trait` was ever examined; and a borrow-carrying
type in *parameter* position was invisible in any function. `CompletionWait::wait` is both at once.
**Four entries appeared, and one of them predates the widening by months.**
`IoRingErrorExt::as_ioring_error` returns `Option<&IoRingError>` and had simply never been inventoried --
the blind spot made concrete rather than a new risk. The other three are `CompletionWait::wait`,
`Batch::new` and `EventDelivery::new`, the last two carrying an explicit lifetime in a parameter.
The borrow question is answered for all four in
[DESIGN-NOTES.md](DESIGN-NOTES.md#borrow-surface-audit-m21plus1), before the inventory was regenerated,
as [DESIGN-INSTRUCTIONS.md](DESIGN-INSTRUCTIONS.md) requires. None is a hole.
**A plain `&T` parameter is deliberately not reported.** Reporting every method that borrows something
would list the whole crate and mean nothing. Only an *explicit lifetime* in parameter position counts --
the borrow-carrying wrapper whose lifetime the crate chose, not a reference the caller lent us. That is a
heuristic, and the archived audit says so: it would miss a `&dyn Trait` parameter carrying no named
lifetime.
**Verified by five probes, restored afterwards** -- and the negative control is the one that matters,
because a check that fires on everything is as useless as one that fires on nothing:
| Probe | Expected | Result |
|---|---|---|
| `pub fn` returning `&[u8]` (control, the old rule) | caught | caught |
| `pub trait` method returning `&[u8]` | **now caught** | caught |
| `pub fn` taking `&mut RingWait<->` | **now caught** | caught |
| `pub fn` taking a plain `&Completion` | **silent** | silent, exit 0 |
| `pub trait` with a default body plus a borrow-returning sibling | only the sibling | only the sibling |
**The probes found a latent bug in the checker itself**, which is the argument for running them rather
than reasoning about the regex. A one-line body -- `pub fn f() -> &[u8] { &[] }` -- never satisfied the
"line ends with `{`" test, so the signature accumulator ran past the end of the file. The old script did
not crash on it only because it never indexed the lines again afterwards; it silently swallowed the
following lines instead. Signature termination is now "the accumulated text contains a `{`", and the
return type is truncated at that brace.
Also swept while here: the script header and its failure message both said **three** shipped defects of
this shape and listed D-35, D-36, D-43. It is four, and has been since D-45.
[M19.3](COMPLETED-CHECKLIST.md) swept that count through `DESIGN-INSTRUCTIONS.md` and missed this file,
which is the restatement-drift pattern landing on the very tool built to stop a different one.
## Moved 2026-09-21 22:08:52 -04:00 -- M22+.1: a pending operation that owes nothing to a device
### <a id="m22plus1"></a>M22+.1 -- Make [bounded_pop.rs](tests/bounded_pop.rs) independent of how fast a device is, by reading from an overlapped pipe nobody has written to. *(completed 2026-09-21 22:08:52 -04:00)*
**Queued and completed within the hour, and the queueing was the error.** It was filed as `M22+.1` with
an entry in `UNRESOLVED-TEST-FAILURES.md` on the grounds that the fix did not belong in a push of
finished milestones. That is a scheduling preference, not a blocker, and the repository's PRIME
DIRECTIVE is explicit that only a genuine blocking factor justifies deferral. The mechanism was
understood when it was filed; the two open questions were each one probe away.
**Both probes answered, and neither was safe to assume.** `IoRing` does accept a pipe handle for
`read_raw`; and `pop_within(20ms)` against an unwritten overlapped pipe returns `Ok(None)` with
`outstanding == 1`. `Win32_System_Pipes` was added to the dev-dependency feature set; `PIPE_ACCESS_INBOUND`
is not re-exported where the module name suggests, so it is a named local constant, as
`FILE_FLAG_NO_BUFFERING` already was in this file.
**The substance of the change is the question the test asks.** A 128 MiB unbuffered read asks "will this
device take longer than 5 ms?" -- a question about someone else's hardware, which may answer differently
on two runs of the same machine. A pipe with no writer asks nothing: the read is pending because no byte
exists to satisfy it, and it completes exactly when the test writes one.
Where a delay is still needed -- `run_down` polls in 50 ms steps, so forcing it to observe an expired poll
means releasing the read later than that -- it comes from a `thread::sleep`, whose guarantee runs the safe
way round: a sleep may overshoot, never undershoot. No assertion depends on an operation *finishing*
within any bound.
**Verified:** 25 consecutive runs of the target and 3 full `--all-features` suite runs, all green; both
feature configurations green. **Both sabotages still bite exactly as before the rewrite** -- reverting the
timeout mapping turns all 5 red, making `RingWait::block` always fail turns 3 red -- which is the
assertion that matters, because a deterministic test that had lost its discriminating power would be a
worse outcome than the flake.
Also 11x faster (0.20s against 2.26s), with no 128 MiB fixtures.
The `UNRESOLVED-TEST-FAILURES.md` entry moved to
[RESOLVED-TEST-FAILURES.md](RESOLVED-TEST-FAILURES.md) in this commit, per the append-only rule.
## Moved 2026-09-22 15:54:53 -04:00 -- M22.1: batching the appends, and the confound it was meant to test
### <a id="m221"></a>M22.1 -- Batch an epoch's appends into one submission in both append paths, and measure whether the per-record submission cost was flattening the strategy comparison. *(completed 2026-09-22 15:54:53 -04:00)*
`Appender::append_batch` and `Lane::append_batch` replace the single-record pushes. Both compose as
many records as there are free arena slots and submit once, so the arena rather than the caller's
list decides the batch size -- which is what keeps the two halves of an append honest, since a slot
is composed into only while the kernel is not reading it. With eight slots, an epoch of 64 records
goes from 64 submissions to 8.
**The teaching defect is the smaller half.** A sample whose job is to teach `Batch` was paying one
`SubmitIoRing` per record, which is the one thing `Batch` exists to avoid.
**The measurement was the point, and it returned a negative result.** Finding `E-1` raised the
possibility that the per-record cost was a term every strategy paid equally, and therefore a shared
constant capable of flattening the three-way comparison into "indistinguishable" without that being
true. Twenty runs -- ten each side, taken in one sitting by stashing the change so both sets came
from the same machine and build -- are kept in
[measurements/2026-09-22-append-batching/](measurements/2026-09-22-append-batching/).
Throughput did not move in a way that can be distinguished from noise: median records/sec shifted by
1-5% while a single strategy's run-to-run range spans 1.17x to 1.57x. The cross-strategy spread did
not shrink, and stayed at or below one strategy's own range -- which is the sample's own stated test.
**So `E-1`'s hypothesis is not supported, and the existing conclusion survives a confound raised
specifically against it.**
What did move is commit p50, and it is the one figure here that separates: for `covering-flush` the
ten before-values and ten after-values barely overlap. That is the expected shape rather than a
surprise -- batching removes seven of every eight submissions from the append path, shortening the
interval between the last append and the flush being reached. Throughput is bound by the device
flush and does not move; latency is not, and does.
The figures live in the capture and are **linked** from `strategy.rs` and from `M20.6` rather than
pasted into either, per the rule that a measurement has one home.
**`M21.3`'s property was preserved deliberately.** Batching changes the append loop, and the naive
rewrite would have reintroduced a commit trigger reachable on a pass that appended nothing. The loop
`continue`s when zero records are accepted, so the epoch check is reachable only after progress --
the same structural guarantee, stated in the same place.
**Unblocks `M20.6`**, whose remaining question is the part no number speaks to: whether alternating
rings earns its cost on correctness and blast-radius grounds.
### <a id="m222"></a>M22.2 -- Collapse the two free-slot implementations to one, derived from the arena's own outstanding counts rather than tracked beside them. The item called both correct; one was not -- the tracked free list leaked a slot on every refused append. *(completed 2026-09-22 16:21:23 -04:00)*
The sample had two answers to "which arena slots are free". `Appender` asked the arena, filtering on
`outstanding(slot) == Some(0)`. `Lane` kept a `Vec<u32>` and maintained it by hand. Both now call one
`free_slots` in [append.rs](examples/epoch_log/append.rs); `Lane`'s field, its initialiser, and its
push-on-claim are gone.
**The item's premise was wrong in the direction that mattered.** It said "both are correct and the cost
difference is nil ... the duplication is the defect, because the two *can* drift". They had already
drifted. The free list took its slot *before* composing into it, so an append refused between the two --
a record too long for a slot is the reachable path -- returned with the slot popped and no operation ever
issued. The arena considered that slot quiet forever; the list never offered it again. `SLOTS` such
refusals and the harness reports a full arena while the kernel holds nothing.
So this was not a tidying exercise with a correctness footnote: **deriving the fact removed a live bug**,
and it removed it by construction rather than by fixing the copy -- a slot nothing was pushed against
never stopped being free.
**Verified by sabotage, in both directions.**
- *Does each caller really bind to the one definition?* Breaking `free_slots` to offer busy slots failed
the `Lane` path with the arena's own refusal (`buffer 0 still has 1 operation(s) outstanding`), while
the appender ran clean -- so that run proved only half of it. A second sabotage (`take(0)`) starved the
appender, which then failed before the strategy section was reached. Both halves bind; one sabotage was
not enough to show it, which is the point of running the second.
- *Was the leak real, or argued from the source?* Re-injecting the free list and failing eight appends
left the lane reporting **0** of 8 slots free with the arena entirely idle.
**What the new tests do not catch, stated in the tests.** [strategy/tests.rs](examples/epoch_log/strategy/tests.rs)
asserts the property from the arena's side, so a re-introduced free list would *not* fail it -- under the
re-injection above it passed, and only a temporary assertion against the list itself went red. What keeps
a second definition from returning is that there is one function and both callers call it. Claiming the
test covers that would be the cosmetic binding the repository's own rules warn about, so the module says
so plainly instead.
Two harness defects were themselves caught by sabotage discipline and are worth recording, because both
produce a *false green*: a `.Replace` that matched nothing reported success and ran an unmodified tree,
and a PowerShell helper that logged with `Write-Output` returned its log line into the patched text. Per-site
match-count assertions caught the first; the second surfaced as a run with no output at all. The repository
already requires the match-count check for exactly this reason.
**Raised a layer-placement question rather than answering it silently:** `outstanding()`'s rustdoc names
this use case, and both in-repo consumers hand-rolled it anyway. Queued as `M22+.2` with its blocker named
(a public API addition needs a lib test, and that test opens a real ring -- the `M24` pile).
### <a id="m223"></a>M22.3 -- Give the registered arena a stated placement: the epoch-log arena is placed on the NUMA node its own log file's volume reports, and the allocator moved into the library as `NumaBuffer` rather than being copied a second time. The sample says plainly that the placement cannot pay at this workload. *(completed 2026-09-22 17:41:01 -04:00)*
The item offered a choice -- adopt `ring_copy`'s allocator, or write down why a durability sample makes no
locality decision. Both halves turned out to be needed, and a third thing fell out of doing them.
**The decision.** [placement.rs](examples/epoch_log/placement.rs) asks the log's own handle through
`FSCTL_QUERY_VOLUME_NUMA_INFO` -- the documented call [What is not reachable](DESIGN-NOTES.md) already
established, which takes a file or directory handle directly and needs no device-tree walk -- and the arena
is allocated preferring whatever comes back. A volume that names no node yields no preference and the log
runs on; the report line says which happened and why.
**No benefit is claimed, and the sample says so in its own output.** The arena is eight slots of four
kilobytes against a workload bound by a per-epoch device flush costing hundreds of microseconds, which
`M22.1` measured directly. So this demonstrates how the decision is made and reported, not that it was
worth making; `examples/ring_copy` remains where placement meets a load that could show it. The report is
also qualified by `GetNumaHighestNodeNumber`, because on a one-node machine "placed on node 0" is true and
misleading -- the line instead says the choice was never available. Two unit tests hold that in **both**
directions: the disclaimer must appear for a single-node machine and must **not** appear for a multi-node
one, since a disclaimer that shows up everywhere trains a reader to ignore it.
**The allocator moved into the library ([D-51](DESIGN-NOTES.md#d-51)), by the engineer's call on a
question raised before any code was written.** The crate's front page names this allocation as the
highest-leverage locality decision available and then supplied nothing, so the first consumer wrote it in a
sample and this item was about to write the second. `git mv` carried the history; `ring_copy` lost its
local module and binds to the library type.
**That move surfaced a packaging defect that would have reached a consumer.** `cargo check --all-targets`
unifies dev-dependency features into the build, so the library's newly-required `Win32_System_Memory` was
being supplied by the dev-dependency list and the lib target compiled clean. `cargo doc` -- which does not
get dev-dependencies -- failed immediately, and `cargo check --lib` confirmed it: **anyone depending on
this crate alone would not have compiled it.** The manifest now carries the feature on the library
dependency, and the stale comment asserting "the library itself needs none of them" is corrected rather
than left to mislead.
**Verified by sabotage, in both directions.** Forcing the FSCTL to report a node the machine does not have
failed the run at arena allocation with `ERROR_INVALID_PARAMETER`, which is what proves the queried node
actually reaches the allocator rather than being reported decoratively. Forcing the FSCTL to fail produced
the `Unplaced` line, the stated reason, and a log that still kept its contract -- the path that will not
otherwise execute on a machine where the query succeeds.
**Swept the claim rather than the one site.** `VirtualAllocExNuma` and the arena's old `Vec` allocation
were stated in six places; [README.md](README.md), [lib.rs](src/lib.rs), two DESIGN-NOTES sections and the
manifest comment were updated, and the design-session and archive copies were left alone as historical
record. `M23.2` was **narrowed** in the same pass: its option (a) is no longer "should a sample do this"
but the residual library question, because the sample half is now done.
Gate: fmt, clippy, the full suite (14 new `NumaBuffer` tests, 9 new placement tests), `cargo doc` clean,
lib-only and `--no-default-features` builds, the borrow-surface, encoding and publishable checks, and the
example end to end.
### <a id="m203"></a>M20.3 -- Make `ring_copy`'s degraded-fallback path observable in a test, asserting both that an absent relation degrades and that a present one does not. *(completed 2026-09-22 20:11:18 -04:00)*
The whole-machine fallback in `Policy::select` is the branch every zero-relation machine takes -- the
shape [D-48](DESIGN-NOTES.md#d-48) records as ordinary rather than exotic -- and it cannot be reached
by *running* the sample on a machine that reports its relations. A synthetic topology reaches it.
Fifteen tests in [examples/ring_copy/policy/tests.rs](examples/ring_copy/policy/tests.rs).
**The item's reason for demanding both halves was verified rather than trusted.** It argued that a
test of the absent case alone "would pass against a function that always degrades". Sabotaging
`select` to degrade unconditionally showed exactly that: `a_policy_whose_relation_is_absent_...`
**still passed**, while four present-case tests failed. The reverse sabotage -- never degrade -- failed
five absent-case tests. Neither half is redundant, and that is now a measured statement.
One test, `degrading_unconditionally_would_fail_a_test_here`, exists to put that dependency in code
rather than in a comment, so a future edit that deletes the present-case coverage has something named
to delete.
**Done without waiting on `SH-4.12`, and the coupling was narrowed rather than ignored.** The recorded
callout said that item "rewrites the selection arm this test would assert against". That is true only
of a test asserting through `ByL3`. The fallback tail is shared by all five policies and is not what
`SH-4.12` changes -- it changes which domains `ByL3` matches -- so exercising it through `ByNode` and
`ByPackage` pins nothing. Both checklists now say so, and `SH-4.12` inherited the one assertion that is
genuinely its own: `ByL3`'s degradation condition, under whichever rule replaces the `level: 3` match.
Beyond the two halves, the cases cover what the fallback must get right and what it must not claim: a
memory domain with no processors is not a usable node; degradation is per-policy rather than a property
of the machine; `Single` returns the whole machine **undegraded**, because degrading is a statement
about not getting what was asked for and `Single` asked for exactly this; the fallback covers every
online processor, excludes reserved-but-offline slots, and spans processor groups; and it carries no
observations, because nothing observed it.
The synthetic memory domain uses `Observed::NotObserved` for its size rather than `Known(0)`, which the
type's own documentation calls the variant "a hand-written description leaves behind". `Known(0)` would
have asserted a measurement nobody made -- in a test whose subject is honest reporting.
`ring_copy` was auto-discovered and therefore **not a test target**, so `cargo test` would have compiled
these and run nothing. It now has an explicit `[[example]]` entry with `test = true`, the same reason
`epoch_log` has one.
### <a id="m201"></a>M20.1 -- Restate the cache heuristic as "the outermost cache level that actually partitions the machine", sweep every restatement, and replace the consumer that bound to the level number. *(completed 2026-09-22 20:27:15 -04:00)*
**Done together with `SH-4.12`, because they are one change.** That coupling was real, unlike `M20.3`'s:
this item's sweep reaches `policy.rs`'s doc comments, and rewriting those to describe the new rule while
the code still filtered `level: 3` is exactly the contradiction the blast-radius convention exists to
prevent. Splitting them would have produced a commit whose documentation lied.
**The item's evidence was a shipping ARM part with no L3. Measuring the consumer found a second shape,
on this workspace's own development machine, that nobody had anticipated.** It reports an L3 spanning
**all 16 processors** above a real **8-way L2** partition. So the old filter did not fail the way the
item assumed:
| | domains selected | reported degraded? |
|---|---|---|
| old `level: 3` filter | **1**, mask `0xffff` | **no** |
| `outermost_partitioning_cache()` | **8**, at L2 | no |
The old code *matched something*, so it did not degrade -- it reported success while collapsing an
eight-domain machine to a single ring. A silent wrong answer, not a visible fallback, and live on the
machine this repository is developed on rather than on hardware nobody here owns.
Three consumers restated the rule, not the two the items named. `ring_copy`'s `policy.rs` and the prose
were known; `examples/l3_domains.rs` also hardcoded `cache.level == 3` and was named after the
assumption. It is now `examples/cache_domains.rs`, asks the same primitive, and reports the level it
found -- `git mv` kept its history.
**`byl3` and `l3` are rejected rather than aliased.** They named a rule the sample no longer implements;
mapping them onto `ByCache` would let a script keep asking for L3 and keep believing it got L3, which on
an L2-partitioned machine is a wrong answer delivered quietly. An unknown policy prints the usage line,
which is a question rather than a wrong answer.
Five tests were added to the file `M20.3` created two hours earlier -- the assertion `SH-4.12` had
inherited. Re-injecting the `level: 3` filter fails three of them, including the one that pins the
measured shape above. The swept sites: `DESIGN-NOTES.md` (the heuristic section, the sizing note, the
policy list, D-27's pointer, and D-48's own "restating the rule is M20.1" reference, which was itself a
restatement that would have gone stale), `README.md`, `src/lib.rs`, both examples, and both checklists.
### <a id="m241"></a>M24.1 -- Settle whether a co-tested fake escapes the mock objection. Answered: the fake was the wrong instrument. *(completed 2026-09-22 21:18:30 -04:00)*
The item predicted two outcomes and both held -- a wrong **accounting** model was caught by the shared
suite, a wrong **Windows belief** slipped through. Two further cases changed the answer.
**Case 3, which the item did not predict, is the argument *for* co-testing.** Run an assertion written
from the *wrong* belief against both peers: the kernel goes red and refutes us, the fake goes green and
confirms us. That is the manufactured-evidence mechanism made visible, and also the escape -- a
mock-only world never performs that experiment.
**Case 4 invalidated the line the first three cases suggested.** Raised in review: kernel behaviour is an
observation at a point in time, not objective truth, and we must not over-index on a record of how it
runs. Demonstrated: "after submitting, the completion is already queued" reads like a contract and gave
**opposite answers on two handles of the same API**. So the axis is not "accounting versus Windows
behaviour" -- it is **our specified contract versus the platform's incidental behaviour**.
**Case 5 replaced the technique.** Also from review: model what the platform is *permitted* to do and
let a seed pick a resolution, so the assertions are about this crate rather than about the kernel. That
dissolves the mock objection instead of working around it, because there is no belief to be wrong about.
A minimal resolver broke a FIFO-assuming consumer under 189 of 200 seeds -- and passed under the other
11, which is the point: a fixed fake reports whichever single answer it encoded.
**Two apparatus failures in one session, both caught, both the same shape.** The first spike draft ran
each condition once and printed a verdict -- exactly `D-47`'s error; rewritten to 500 trials it
immediately found a condition that pends about 1% of the time and would have been called "never". The
case-4 harness used a bare flush, which completes inline on every handle, so it could not discriminate
until it was rebuilt around sector-aligned writes. Both are recorded because the repository's rule is
that an instrument nobody has shown can go red is not evidence.
Outcome: the rejection **stands** with its scope sharpened ([D-52](DESIGN-NOTES.md#d-52)), `M24.4` is
**withdrawn**, `M24` becomes unconditional, and the technique that actually answers the question is
`M26`. The apparatus is kept as
[kernel-response-space-probe.rs](design-sessions/kernel-response-space-probe.rs); the reasoning is
[DESIGN-SESSION-2026-09-22-kernel-response-space.md](design-sessions/DESIGN-SESSION-2026-09-22-kernel-response-space.md).
### <a id="m242"></a>M24.2 -- Extract the handle-free accounting into its own type, composed by `IoRing`. *(completed 2026-09-22 21:29:07 -04:00)*
The item named five fields as handle-free and five as carrying kernel state. **Checked before acting,
and it was exactly right** -- `ring_id`, `next_user_data`, `outstanding`, `registered_files` and
`registered_buffers` against `handle`, `version`, `supported_ops`, `registered_buffer_infos` and
`completion_event`. Nine methods touch only the first five; they moved with the fields, and `IoRing`
delegates. `RingId` moved too, since the identity counter is part of the ledger rather than of the
handle. The public surface did not move.
**19 hermetic tests, and they are the first in `src/` for which [D-49](DESIGN-NOTES.md#d-49)'s
complaint does not apply.** They open no ring, because every rule they check is this crate's own
specification: an identity is never reused, a refused reservation costs nothing, the counters saturate
rather than wrap, the two registration indices are independent, and two ledgers never share an
identity. The last of those used to need two live kernel objects.
**The tests found an off-by-one in their own author's assumptions.** Two of them asserted that the
last identity handed out is `usize::MAX`, and failed: `checked_add` runs *before* the value is
returned, so a reservation made at `usize::MAX` fails rather than handing it out, and the identity
space is `0..=usize::MAX - 1`. Not a defect -- one value out of 2^64, and "fails rather than wraps" is
the property that matters -- but invisible from the source, so
`the_last_identity_is_max_minus_one_not_max` records it rather than leaving the next reader to make
the same wrong assumption.
**Sabotage, four ways, all caught:** a no-op `record_completion` (2 red), a recycled identity (6 red),
`wrapping_sub` in place of `saturating_sub` (1 red), and the buffer-registration count advancing the
file counter (4 red). The recycled-identity sabotage first produced *no output at all* rather than a
red suite -- an ambiguous-integer compile error -- which is the silent-failure shape the repository's
rules warn about, and was rerun with an explicit type before being believed.
**The payoff was deliberately not taken here.** The item's motivation said most of the ring-opening
lib tests "become hermetic *in place*". That is 71 tests across four files, which is a different item
wearing this one's name; it is queued as `M24.7` with the measured per-file census, and with a warning
that the census must be recounted because the first attempt at it produced false positives by matching
`to_string()`.
### <a id="m247"></a>M24.7 -- Convert the lib tests that construct a ring only to exercise bookkeeping. *(completed 2026-09-22 21:43:29 -04:00)*
**The recount the item demanded was right to demand.** Its figure of 71 was a per-*file*
`IoRing::new` count. A per-*test* census gives **61**, and the difference is not rounding -- a file
with 40 tests and 35 constructions has five tests that never touch a ring.
**61 -> 52.** Two changes, one structural and one an excision.
**`Token::new` now takes the ring's ledger rather than the ring.** It only ever used
`reserve_user_data()` and `ring_id()`, both bookkeeping, so the wide parameter was the only reason
`token`'s tests opened a ring at all -- **all seven of them, to mint a token and nothing else.** The
ring was actively a liability there: none of those tests ever submitted, so `Drop`'s run-down would
wait for completions that were never coming, and a `settle` helper existed purely to stop teardown
hanging. The helper is gone with the hazard it worked around. `token` is now 7 hermetic, 0 opening.
**Two `ring` tests were removed rather than converted**, being duplicates of what `M24.2`'s hermetic
tests now cover: `reserve_user_data_increments_outstanding_and_never_repeats_an_id` and
`record_completion_saturates_rather_than_underflowing`. Deleting them loses no delegation coverage --
`run_down_returns_once_a_recorded_completion_zeroes_the_count` already drives reserve, `outstanding`
and `record_completion` through `IoRing`, and must keep a ring for its own sake.
**The remaining 52 are not convertible, and the reason is structural rather than effort.** Recorded
here so the next reader does not re-derive it:
- `event_delivery` (6) needs a real ring and the thread pool. There is nothing to narrow.
- `ring`'s injected-failure cluster **looks** convertible by name and is not. It uses a real
completion on purpose -- one test says so in an assertion message, "the flush really did succeed,
or this test proves nothing" -- because `with_injected_failure` *transforms* a real completion, and
fabricating one is precisely the unsoundness the seam exists to avoid.
- `batch` (13) needs a `Batch`, which needs the handle for its `Build*` calls. Narrowing that
parameter is not possible the way `Token`'s was; it becomes reachable only under `M26.2`'s FFI
seam, which is a far larger change.
So **relocation, not conversion, is the remedy for the rest**, which is `M24.3` -- updated with this
finding and with a warning to recount its own stale figure of 25.
### <a id="m243"></a>M24.3 -- Relocate the lib tests that open a ring but use only public API into `tests/`. *(completed 2026-09-22 22:43:39 -04:00)*
**52 -> 41.** Eleven tests moved into `tests/ring_lifecycle.rs` (new), `bounded_pop.rs`,
`event_delivery.rs`, `registration.rs` and `submission_lifecycle.rs`. Totals conserved: 162 lib + 68
integration before, 151 + 79 after.
**The item predicted 25 and that was never achievable.** Its figure predated two milestones of test
growth, and more importantly it assumed the constraint was *which* tests had been looked at rather
than what they reach. `M24.7` had already found the structural version of this; relocation hits the
same wall from the other side.
**A regex census got the classification wrong, and the compiler caught it.** The scan excluded
`pop_within` from the blocking list because it is a public method on `IoRing` -- but `ring.rs` also
has a `#[cfg(test)] pub(crate) fn pop_within(ring, what)` free function, and
`windows_refuses_an_empty_buffer_registration` uses *that*. It was moved, failed to compile, and was
returned. A second miss was structural rather than nominal: the scan only looked at `fn` definitions,
so it did not see that `a_supplied_wait_is_not_consulted_when_nothing_can_arrive` depends on a
`RecordingWait` **struct** shared with eight other call sites. That one was caught before moving, by
a second pass that looked for local `struct`/`const` definitions too.
The method that worked was **moving the candidates and letting the compiler rule**, rather than
trusting the census. Two milestones running, a crude census has produced false classifications here;
the compiler produced none.
`HugeBuffer` and `NULL_FILE` travelled with the two tests that used them, having no other call sites.
**`M24.5` needed re-planning as a result, and that is the more consequential outcome.** It assumed
that after `M24.2` and `M24.3` the lib tests would "construct no ring at all", so a zero-check would
do. Forty-one remain and none is movable, so a zero-check would fail on day one and could only be
satisfied by deleting real coverage. The item now asks what the rung should actually assert -- a
ratchet, an allow-list, or nothing until `M26.2` -- rather than presuming the answer.
### <a id="m245"></a>M24.5 -- Put the rule on a rung, so it cannot regress. *(completed 2026-09-22 23:44:37 -04:00)*
**The rule the item assumed was false, and that is the decision this item really made**
([D-53](DESIGN-NOTES.md#d-53)). It expected a zero-check -- "the lib tests should construct no ring at
all" -- which would have failed on day one and could only ever be satisfied by deleting real coverage.
Forty-one remain and none is movable without `M26.2`.
So the rung is an **inventory**: [RING-OPENING-LIB-TESTS.txt](RING-OPENING-LIB-TESTS.txt), regenerated
from source by [check-ring-tests.ps1](../../tools/check-ring-tests.ps1), failing when the two
disagree. Deliberately the same mechanism as the borrow-surface check, so there is nothing new to
explain. Per-test rather than per-file, because two thirds of the 41 live in `ring/tests.rs` and a
file-level allow-list would let exactly that file grow. An inventory rather than a count, because
add-one-remove-one nets to zero and a bare number is derived data nobody can check by reading.
**The bidirectional verification found a defect in the guard, which is the whole reason the rule
demands it.** Direction 3 -- a test that reaches a ring only through a helper -- reported the expected
test *and an innocent one*. The body extraction ended at the next `#[`, so a plain helper defined
after the last test in a file was swallowed into that test's body, and a helper containing
`IoRing::new` made the test above it look ring-opening. The body now ends at a column-0 `}`, which
`cargo fmt` guarantees is a function end. Had only the "must fire" direction been run, the check would
have shipped with a false positive that fires on innocent changes -- the fastest way to train people
to ignore it.
Four directions verified after the fix: fires on a direct `IoRing::new`, fires on a ring reached only
through a helper, stays silent on a new hermetic test, and reports removals as progress needing only
regeneration. Wired into CI as its own job beside `borrow-surface`; needs no toolchain.
### <a id="m246"></a>M24.6 -- Sweep what this milestone makes false. *(completed 2026-09-23 11:49:58 -04:00)*
The item named three sites. **Two were false alarms and the third was false for a different and more
serious reason than the item gave** -- which is the argument for running the census rather than
editing the named list.
**Named, and genuinely stale: [D-49](DESIGN-NOTES.md#d-49).** Its "63 of 131" is the figure `M24`
*started* from; measured after, **41 of 151 open a ring and 110 do not**. It also still said the mock
rejection "stands until `M24.1` settles it" and that the remedy choice was "gated on one unresolved
question", both of which `M24.1` closed. Corrected, with pointers to [D-52](DESIGN-NOTES.md#d-52) and
[D-53](DESIGN-NOTES.md#d-53).
**Named, false alarm: the testing-strategy section.** The item expected it to be stale because it was
"written when every lib test opened a ring". Reading it, nothing in it turns on hermeticity -- it
classifies *defect populations* and *techniques*, and `M24` added or removed neither. Its "all five
techniques" framing is also correctly left alone: the resolver is a sixth *when `M26` builds it*, and
`M26.6` already owns that edit. Claiming six today would be the opposite error.
**Named, false alarm: `M21.6`'s archive entry.** The item said its "a wait that never enters the
kernel" clause "stops being the notable exception once the suite is hermetic". Hermeticity does not
bear on that sentence, and the archive is append-only history describing what was true when written.
**Named, and false -- but not because of `M24`: `F-13`.** Its headline says "Every fixture in this
crate's tests, examples and samples opens its handle that way [synchronous]". Three do not:
`flush_barrier.rs`, `handover.rs` and `flush_barrier_stress.rs` open
`FILE_FLAG_OVERLAPPED | FILE_FLAG_NO_BUFFERING`, dated 2026-08-28, 08-29 and 09-06 -- **weeks before
F-13 was recorded on 09-21**. So it was false when written, not made false by this milestone.
That matters because the entry's carry-forward escalated from the false half: it says every claim
about ordering, draining, the completion event and the barrier "was measured against operations that
may have completed inline" and names D-19, D-23, D-24 and D-47 for re-reading. But D-23, D-24 and
D-47 were measured by `flush_barrier.rs` -- one of the three overlapped fixtures. The entry even
hedged correctly ("the drain-ordering spike used `NO_BUFFERING` ... so it is probably fine") and then
checked only the spike, not the tests sharing its shape. A dated correction was added rather than a
rewrite, so the record of what was believed survives.
**Unnamed, and found by the sweep: [bounded_pop.rs](tests/bounded_pop.rs) named the wrong gap.** It
said "every other test of `pop_within` drives the loop with a wait that never enters the kernel".
Several do enter it through `SubmitWait`. The real gap is narrower and more interesting: the tests
using the kernel wait drive operations that *complete*, and the one test that lets a bound expire
fakes both halves -- a `RecordingWait` instead of the kernel and a bare `reserve_user_data` instead of
a pending operation. **No test had a real operation pending when a real bound expired**, which is
exactly the state the `ERROR_TIMEOUT` path needs. Corrected in place.
**Unnamed, and found by the sweep: this milestone's own header** still carried the 63-of-131 opening
figure and a "recount before starting" caution that had been acted on.
**The transferable part.** Three of the five corrections were over-generalisations from a single
observation -- one fixture becoming "every fixture", one wait shape becoming "every other test". Each
was a census away from being right, and each then had an alarm built on top of it. That is the same
shape as the spike that ran one trial per condition, in the same crate, two days earlier.
### <a id="m206"></a>M20.6 -- Re-evaluate `CommitStrategy::AlternatingRings` and the epoch-log benchmark's conclusion against D-47. *(completed 2026-09-23 12:06:40 -04:00)*
**Question 1 -- does alternating rings still earn its cost? Answered on its stated grounds: no.**
`S-2` argued that because a covering flush reaches every operation outstanding on its ring, two rings
bound what a commit's barrier can be dragged into. That is structurally false for this sample and
needs no measurement: `RegisteredBuffers::get_mut` refuses a busy slot and there are `SLOTS` slots, so
at most `SLOTS` appends are outstanding on a ring **by construction** -- and each alternating lane
registers its own arena of the same size. The arena bounds the blast radius, not the ring topology.
Probing agreed (8 and 8); the argument does not rest on it and holds whatever the platform does about
pending. The argument survives against genuinely unrelated traffic from another component; this sample
has none.
**The strategy is deliberately NOT removed.** The item said that if it no longer earns its place, that
is an API change to a published example -- and the temptation was to make it. What two rings could
*also* buy is **overlap**, and overlap is precisely what this harness cannot exhibit: its handle is
synchronous, so a ring operation completes inline during submit and nothing is ever outstanding across
a submit boundary. Deleting a strategy on the strength of a measurement that could not have shown it
working would be the same error as the measurements this item exists to correct. `M25.5` answers it on
a harness where operations genuinely pend.
**Question 2 -- re-read, re-run, or annotate the numbers?** The investigation concluded "none of
those": the column measures deferral rather than a commit, and prose cannot fix a measurement.
**That conclusion was right about the measurement and wrong about the output.** Leaving a column
labelled `commit p50` in a published sample until `M25` lands is shipping a false claim for the sake
of a purist position on annotation. The column is now `ack lag`, with a caveat line naming the p99 = 0
blocking measurement, and pointing at `M25`.
**The rustdoc already knew, which is the finding worth keeping.** `Outcome::commit_latencies` already
said the figure is "not device flush time", that deferral inflates it, and that alternating rings
"reports the highest latency of the three while matching them on throughput". A previous pass had
diagnosed the artifact correctly **and only in the rustdoc** -- the printed output never got the same
treatment, and the M20.6 investigation re-derived from scratch what was already written one file away.
What the investigation genuinely added is the extent: blocking is not merely a component of the figure,
it is **0 us at p99**, so the number is entirely deferral; and underneath that, no pipeline exists at
all.
**Swept the mechanism, not just the label.** The explanation "the strategies differ about how long the
flush itself waits and the extra host round trip, and those land in the tens" appears in the module
docs, in a `main.rs` code comment, and in the printed summary line. It is wrong in the same way at all
three: those differences cannot occur on a synchronous handle. The three are indistinguishable because
they do the same serialized work -- both readings give the same ranking and only one is true. All three
corrected, plus the "a real log keeps appending while a commit is outstanding" claim, which describes
a state this program has never reached.
### <a id="m206-correction"></a>M20.6 -- correction, same day *(recorded 2026-09-23 13:03:35 -04:00)*
**The entry above declared `AlternatingRings`' blast-radius justification "dead on structural grounds".
That over-reached, and the over-reach is the kind this repository now has a rule against** -- see
OPTION INTEGRITY in the repository instructions, added by this correction.
What the structural argument actually establishes is that **this harness** cannot exhibit a
blast-radius difference, because each lane registers its own arena of `SLOTS` slots and the arena is
the limiter rather than the ring topology. That is a statement about the apparatus. Generalising it to
"the justification is dead" converted a fact about one sample's configuration into a verdict on a
design option.
**It also contradicted the crate's own recorded position.** [D-27](DESIGN-NOTES.md#d-27) is this
crate's decision that one ring per thread is userspace's proxy for one ring per CPU, and records the
hardware reason: NVMe queue pairs are per-CPU with each pair's completion interrupt routed by its own
vector. Two rings on two pinned threads *is* that architecture. Declaring a multi-ring strategy's
justification dead on the strength of one sample's arena sizing sits directly against a decision the
crate already made on stronger grounds.
The conditions under which alternating rings would pay are now written at
`CommitStrategy::AlternatingRings`, and they are ordinary rather than exotic: a ring shared with any
other component, arenas sized asymmetrically from the lanes, real overlap (where the same covered
count is not the same wait), and per-CPU queue affinity. The sample's job is restated as giving a
consumer the means to answer this on their own hardware, not handing them a verdict from ours.
Nothing about the measurement corrections in the entry above changes: the ack-lag relabel, the p99 = 0
blocking finding, and the swept mechanism claim all stand. What changed is the conclusion drawn from
them.
### <a id="m231"></a>M23.1 -- State in the epoch-log contract that the barrier is ring-wide while the flush names a file, so one ring per log is a precondition of the cost model. *(completed 2026-09-23 17:03:35 -04:00)*
*Queued from finding `S-1` of the 2026-09-19 epoch-log review.*
**The item was right that the contract was silent, and wrong about what it was silent on.**
`S-1` reasoned from [D-47](DESIGN-NOTES.md#d-47)'s surviving half -- the barrier reaches every
operation outstanding on the ring, not only the current submission batch -- and concluded that
"one ring per log" is a precondition of the sample's *durability contract*.
[contract.rs](examples/epoch_log/contract.rs), written before the code precisely so it would state
preconditions, said nothing about it.
**Writing it found two scopes conflated, and the first draft shipped the conflation.**
`IOSQE_FLAGS_DRAIN_PRECEDING_OPS` is a flag on the **ring**; `BuildIoRingFlushFile` names a
**file**. So the barrier bounds what a commit *waits for* and the flush bounds what it *makes
durable*, and completion is not durability -- a claim the same file already made three paragraphs
earlier, about a record's own write. The corrected reading: a shared ring does **not** endanger the
guarantee, which the flush's own file target secures. It endangers the **cost model**, because the
barrier waits for unrelated traffic unconditionally.
**Three commits, because the first two were not right.** `d845bb28` added the section, an
assumption, a non-guarantee, `Clause::ALL` -- replacing a hand-written variant list in `main.rs`
that would have printed one section short had a fourth clause ever been added -- five tests over the
report's own properties, and the crate's first [sabotage.json](sabotage.json): five injected defects
caught, plus a control that rewords a statement and survives, so the guards are sensitive to the
report degrading without being bound to the contract's wording. `4adea675` corrected the
conflation. `48dc99d3` right-sized what the correction had grown into -- a title giving the device
equal billing with the ring, plus a bullet and a milestone pointer about multi-device reach, in a
sample that runs one log file on one ring.
**The conflation was caught by a question, not by the gate**, which stayed green across all three:
every test passed, the sabotage sweep reported all six cases as declared, and the two contradictory
sentences sat a screen apart in one file. The tests check that the report *prints* correctly and
deliberately not what it *says*, so nothing built here could have found it.
**Two things this work left elsewhere.** The primitive-level half moved to the library under
[D-54](DESIGN-NOTES.md#d-54): the barrier/flush scope distinction is a fact about one flush, so it
belongs on `FlushCoverage` rather than only in a sample's contract. And
[checkpoint.rs](examples/epoch_log/checkpoint.rs) gained the reciprocal note -- it already took its
own ring, for an unrelated *delivery* reason (D-21), so the structure was right twice over with only
one reason written down. Both sites now point at each other, so collapsing the rings cannot look
harmless from either end.
### <a id="m232"></a>M23.2 -- Decide how a caller arrives at a NUMA node: `win-numa-sys` offers declaring and discovering, and refuses the shortcut that does both at once. *(completed 2026-09-23 19:57:34 -04:00)*
*Queued from finding `S-3` of the 2026-09-19 epoch-log review.*
**The item was narrowed three times before it was answered, and the last narrowing moved it out of
this crate entirely.** `M22.3` settled its sample half by making the allocator library surface.
[D-54](DESIGN-NOTES.md#d-54) removed its other half -- sharding by backing device needs the concept
of a set of operations that commit together, which this crate does not have -- and handed that to
the durability layer, where it was sharpened from device identity to *flush equivalence*. Then
`win-numa-sys` was created and `NumaBuffer` moved into it, so "this crate" in the item text stopped
naming the crate that had to answer.
**What remained was answered by building that crate, so this item's deliverable was the recorded
decision rather than code.** It is
[N-D-1](../win-numa-sys/DESIGN-NOTES.md#n-d-1): a caller may *declare* a node
(`NumaBuffer::new`), *discover* one (`volume_numa_node`), or *qualify* what a discovered answer is
worth (`highest_numa_node`); what is refused is a `NumaBuffer::for_file` that would query and
allocate in one step.
Four reasons for the refusal, of which the first is the one that generalises: such a call **hides
the answer**, and on a single-node machine "placed on the node the volume named" and "no preference"
are the same allocation, so a caller could not tell whether the query found anything. It also fuses
two failure domains, withholds an answer useful beyond one buffer, and saves exactly one line,
since `NumaBuffer::new(len, volume_numa_node(h).ok())` already type-checks.
**A fifth argument was dropped rather than kept, and the decision says so.** When this was first
argued, a `for_file` constructor would have dragged `Win32_System_Ioctl` into a crate that
otherwise touched only memory. That was true of `windows-ioring-sys` and is not true of
`win-numa-sys`, where the query already lives. Recording a void argument as void is cheaper than
having someone re-make it.
**Neither path is speculative.** `examples/epoch_log` discovers from its log file's volume;
`examples/ring_copy` declares a node it computed from the processor topology. Both were already
written against this shape before the decision recorded it.
[D-8](DESIGN-NOTES.md#d-8) is intact, which was the item's stated constraint: locality stays the
consumer's decision, and the crate supplies a fact and an allocator rather than a choice.
### <a id="m233"></a>M23.3 -- Decide what this crate offers for holding a token between push and completion: the ring owns the inventory, `IoRing` becomes generic, and the break is accepted. *(completed 2026-09-23 23:09:13 -04:00)*
*Recorded as [D-55](DESIGN-NOTES.md#d-55). Implementation is `M28`. The exploration, including
everything it falsified, is in
[DESIGN-SESSION-2026-09-23-pending-inventory.md](design-sessions/DESIGN-SESSION-2026-09-23-pending-inventory.md).*
**The item was decided against a different argument than the one it was written on.** It argued
from duplication -- nine sites keeping the same map. A census found ~12 sites of which only a
third keep the map described, so duplication was both mis-counted and the wrong frame. The
mechanism is that this crate **mandates** the construct and does not provide it:
`Batch::write` returns a `Token`, `IoRing::pop_within` returns a `Completion`, and nothing
connects them but caller-supplied storage -- which the crate's own rustdoc instructs callers to
build, twice. That follows from [D-4](DESIGN-NOTES.md#d-4) splitting ring-side counting from
caller-side identity, which is right; what was missing is the half it left to prose.
**A working spike was built and is why the decision is informed rather than argued.**
`Pending<T, X>` fits a real consumer -- converting `append.rs` removed its `InFlight` struct and
its hand-driven oracle calls -- and sabotage established that removing its drop guard or cutting
its oracle wiring is caught. A test driving a failed write through the injection seam turned
`M22.2`'s ordering defect from undetected into caught by assertion.
**But the spike also showed why offering it beside the ring is not enough.** Nothing forces a
minted token into it, so the ring-to-inventory drift survives -- and `Pending::checked()` owning
an oracle made the converted consumer's existing `RingContract` a decoy, passing
`assert_quiescent()` vacuously with nothing in the suite catching it. A generic `IoRing<T>`
owning the map removes the class, because the consumer never holds a token to lose.
**The objection that had ruled that out was false, and checking it was what settled the item.**
Per-ring monomorphisation holds for every real consumer; `tests/generated_sequences.rs` already
carries eight token types on one ring behind a closed `enum Held` with no runtime type check;
and [D-4](DESIGN-NOTES.md#d-4) rules type erasure out in as many words. The dismissal had
contradicted a decision already on the books, in the opposite direction from the one it assumed.
**Two findings the decision rests on that were not in the item.** `RingContract` never prunes --
one retained entry per operation for the process's life, undocumented, invisible in a sample that
appends 24 records -- which rules out checking by default and is `M28.2`. And `epoch_log`'s
commits are tokenless because a flush has no buffer and a *borrowed* `RawHandle` leaves its token
nothing to guard, which is `M28.5` and may dissolve when `M25.3` changes how the log is opened.
**What it refuses**, unchanged by the shape: batching, ordering and which slot to pick stay caller
questions. The sharper refusal is that the inventory does not decide whether a caller is checked.
### <a id="m234"></a>M23.4 -- A failing test that left registered buffers outstanding aborted the process instead of reporting; the drop guards now stay silent during unwind. *(completed 2026-09-23 23:16:48 -04:00)*
`RegisteredBuffers::drop` refused to free while an operation was outstanding (M5.3, correctly) and
said so with a bare `debug_assert!(false, ...)`. Nothing checked `std::thread::panicking()`, so a
test that failed *because* a slot leaked panicked, unwound, dropped the arena, panicked a second
time inside `Drop`, and aborted -- replacing its own assertion message with
`STATUS_STACK_BUFFER_OVERRUN`. The detection was never weakened; what the abort destroyed was the
diagnosis.
**The item named one site and there were three.** It said "the fix is presumably the same one line
here", and a sweep of every `impl Drop` in the crate found the assert it named plus **two more** in
`IoRing::drop` -- the rundown failure and the `CloseIoRing` failure. That second impl is the worse
of the two: a ring is dropped on the way out of almost every failing test in this crate, so an
unguarded assert there converts a readable failure into a crash in the *common* case rather than a
rare one. The reported site was a sample of the population, which is what CONTRACT INTEGRITY rule 3
says to expect.
**The sweep also produced two false positives worth naming**, because the pattern that produced
them is the obvious one to reach for. A grep for `impl.*Drop for` matched cargo-mutants-style
comment text (`<impl Drop for RegisteredBuffers>::drop -> ()`) inside test files, which read as two
further unguarded sites. Anchoring the pattern at line start reduced eight candidate impls to the
three real asserts. A loose grep over a crate that documents its own mutants will find its
documentation.
**`Pending::drop` was already correct** and is what the fix copies -- it returns early when
`std::thread::panicking()`, which is why the M23.3 spike never exhibited this.
**Verified by sabotage, as the item required, and the verdict alone would not have shown it.** The
`M22.2 regression` case in [sabotage.json](sabotage.json) was `caught` before the fix and `caught`
after; what changed is that it ended in `exit 101` -- a clean `FAILED` naming
`a_failed_write_still_releases_its_arena_slot` and its message -- instead of `exit -1073740791`.
That case's `why` text, which had documented the abort as expected behaviour, now carries the
post-fix failure mode and says a regression in *either* direction (no longer caught, or caught but
crashing) is visible there. The full sweep stayed at 9-of-9 as declared with the `CONTROL` still
surviving.
**Only one of the guards has a test that depends on it firing, and the sabotage that established
that also falsified the first draft of this entry.** Suppressing both guards unconditionally
(`true` in place of `std::thread::panicking()`) turned
`batch::tests::dropping_a_registration_with_work_outstanding_is_refused` red, which is the check
that the fix *narrowed* the guard rather than removing it -- that test drops deliberately, not
during an unwind, so `thread::panicking()` is false and the assert still fires. But
`ring::tests::dropping_a_ring_actually_runs_its_drop_body` stayed green under the same sabotage.
It is not a `#[should_panic]` test and it does not reach either assert; this entry had claimed it
did, on the strength of its name, until the sabotage said otherwise.
**`IoRing::drop`'s two asserts are therefore unreachable from any test on a healthy host**, and
that is recorded at the definition rather than left to be rediscovered. `run_down` fails only when
`SubmitIoRing` or `PopIoCompletion` returns a kernel error HRESULT, and `CloseIoRing` fails only
when the kernel refuses the close; the crate's `fault-injection` seam sits at the
*completion-result* level (`Completion::with_injected_failure`) and cannot produce either. Reaching
them needs a seam over the raw HRESULTs, which is `M23.5` -- spawned rather than assumed, per the
move-or-spawn rule, because "no test can reach it" is a blocker to name and not a reason to check
the box and move on. The fix still lands there on its merits: it is precisely the `IoRing` case
that turns a readable failure into a crash most often, since a ring is dropped on the way out of
almost every failing test in this crate.
> **The paragraph above was wrong about the remedy, and [M23.5](#m235) overturned it the same
> evening.** Both asserts are reachable, no seam was built, and no production line changed: the
> kernel rejects a *null* ring handle cleanly, and `ring::tests` is a child module that can put one
> in the field. What the paragraph got right is the finding that prompted it -- the guards were
> genuinely uncovered. It is left standing rather than rewritten because the archive is history,
> and because the error in it is instructive: it priced a seam it never checked was necessary.
### <a id="m235"></a>M23.5 -- Both asserts in `IoRing::drop` are now reached by tests, and the seam the item priced turned out not to be needed. *(completed 2026-09-23 23:20:27 -04:00)*
M23.4 left both asserts in `IoRing::drop` uncovered: suppressing them entirely left every test in
the crate green. This item asked whether to build a fault-injection seam over the raw HRESULTs the
ring's Win32 calls return, or to accept the asserts as documented-unreachable. **Neither. The item's
premise was false**, and one probe falsified it.
**What the probe measured.** `CloseIoRing(null)` and `SubmitIoRing(null, ..)` both return
`0x80070006` -- `HRESULT_FROM_WIN32(ERROR_INVALID_HANDLE)` -- a clean refusal. `CloseIoRing` on a
plausible-looking `0xDEAD_0000` raises `STATUS_ACCESS_VIOLATION`. So a ring handle is **a pointer
the kernel dereferences, not an index into a handle table**, and null is the one bad value that is
refused rather than followed. That asymmetry is the whole finding, and it is recorded at the
definition and in both tests, because a later cleanup that "tidies" the null into a non-null
sentinel converts two passing tests into a process crash.
**Why no seam was needed.** `mod tests` is a *child* of `ring`, so it already sees the private
`handle` and `accounting` fields -- a child module can see its ancestors' private items. A
`#[cfg(test)]` constructor, `IoRing::refused_by_the_kernel`, assembles a whole `IoRing` around a
null handle, and the tests let the real `Drop` body run against it. Whether `run_down` submits at
all is what selects between the two asserts, since it loops only while something is outstanding.
Production code was not touched: the blast radius the item worried about was zero, because the
change is entirely in test-only code.
**The D-49 ring-test gate improved the design, which is what it is for.** The first working version
opened a real ring, closed it by hand, and put a null in the field -- and
[tools/check-ring-tests.ps1](../../tools/check-ring-tests.ps1) flagged two `ADDED` entries and
asked its standing question: *does this test need the kernel, or only a ring-shaped thing?* Only
the latter. `RingVersion::V1` is a public const and `OpSupport` derives `Default`, so every one of
`IoRing`'s six fields is constructible without opening anything. Answering the gate rather than
re-baselining it removed the real ring, the hand-close, two `unsafe` blocks and their safety
arguments from both tests, and left the ring-opening population unchanged at 41. These tests need
the kernel only to *refuse* them, and refusing costs no ring.
**Three sabotages recorded in [sabotage.json](sabotage.json), not one.** Suppressing the rundown
guard leaves the close test green and vice versa, so the two asserts are independent conditions and
a single case would have declared the pair covered while half of it was not. The third sabotage is
of the *test* rather than the code: removing the reservation makes the ring fall through to the
close, and the panic message becomes `CloseIoRing failed: 0x80070006` against an expected substring
of `IoRing rundown failed before close`. That is what shows `should_panic`'s `expected` string is
load-bearing in selecting the assert rather than decorative. The full sweep is 12-of-12 as declared
with the `CONTROL` still surviving.
**The methodological point, which is the same one this crate keeps paying for.** The item was
written an hour earlier, by me, and it reasoned from the shape of the code to "this needs a seam"
without ever asking the kernel what it does with a bad handle. It then priced that seam's blast
radius and proposed accepting a permanent coverage gap as the alternative. Both options were
answers to a question that a single `eprintln!` dissolved. The repository's standing instruction is
never to report that something cannot be done on the basis of reading it; that applies to "no test
can reach this" exactly as it applies to "this will not compile".
**A note on release builds, swept but deliberately not changed.** These are `#[should_panic]` tests
over `debug_assert!`, so they would fail under `cargo test --release`, where the assert compiles
out. That is a pre-existing property of the crate --
`batch::tests::dropping_a_registration_with_work_outstanding_is_refused` has the same shape and is
ungated -- and no CI job runs tests in release. Matching the existing precedent was preferred over
introducing a `cfg(debug_assertions)` gate on two of the three, which would have left the crate
inconsistent with itself. If release-mode testing is ever added, all three need the gate together.
### <a id="m251"></a>M25.1 + M25.2 -- Records gained a fixed sector stride with a zeroed block tail, and replay learned to walk it. *(completed 2026-09-23 23:36:23 -04:00)*
**These two items could not land separately, and that is a defect in how they were written rather
than a discovery about the code.** A strided writer and an unstrided reader do not describe the same
file. The checklist sequenced M25.1 (writer) before M25.2 (reader), so the tree between them holds a
log nothing can read. They are recorded here as one entry, citing both IDs, per the checklist rule
for acknowledged coupling; the alternative -- restructuring them into independent items -- is not
available, because the writer and the reader of one format are not independent.
**What the coupling actually cost was nearly a silent break.** With M25.1 applied alone, the log was
unreadable: replay advanced by a record's own length, landed in a zeroed block tail, decoded
`NeverWritten`, and reported every record after the first as a missing durable record. And **all 21
of the example's tests passed anyway.** The only thing that caught it was
`cargo run --example epoch_log`, which asserts and exits 101 -- and no CI job runs the sample. That
is the more important finding of the two, and it is queued as `M25.1b` rather than left in this
entry, because a finding recorded only in an archive is a finding nobody is obliged to act on.
**Three end-to-end tests were added to close the specific hole**, each binding one writer to the
real reader through a real file: `records_land_one_per_stride_and_replay_walks_them_back` over the
log's own `Appender`, `a_run_lays_its_records_out_one_per_stride` over the harness's `Lane`, and
`a_reused_slot_does_not_write_the_previous_records_tail` over the zeroing. The harness test exists
because a sabotage said it had to: reverting `Lane` to a packed layout was **caught by nothing**
until it was written.
**The zeroing is not observable through replay, and the comment says so rather than inventing a
failure mode for it.** Replay decodes only at block starts and takes a record's extent from its own
header, so a stale fragment past a short record's end is never read. A first draft of the comment
claimed a stale fragment "would be decoded as a record", which is false for exactly that reason.
What zeroing actually prevents is the log carrying fragments of unrelated records -- a hygiene
defect in a format whose purpose is reconstructing what happened after a crash -- so the test
asserts on the file's bytes rather than on a replay outcome.
**Two duplications were collapsed rather than converted twice.** The sample has two writers over one
on-disk format, and both carried a packed layout *and* a verbatim copy of the comment justifying it.
Converting each in place would have turned one duplicated decision into one duplicated rule, so the
stride moved to `record`, beside the format it describes, and the whole composition -- encode, then
zero the remainder -- became `record::encode_block`, which both writers call. That makes half the
drift unrepresentable rather than merely tested for.
**`Decoded::total_len` became a derived `extent()` rather than a silenced warning.** Once replay
advanced by the stride, the field's only consumer was a test, and it was exactly
`HEADER_LEN + payload.len()` -- a stored copy of a fact `payload` already carried. The dead-code
warning was the signal; the fix was to delete the copy, not to `allow` it.
**Replay now confines each decode to its own block.** Previously `decode` received the rest of the
file, so a corrupted `payload_len` was bounded only by the file's length and a record could claim
bytes belonging to its successors -- caught, but by the checksum happening to fail rather than
structurally. The stride is what makes a block boundary exist to confine it to.
**The cost is reported, not described.** M25.1 asked for the write amplification to be "a real cost
to state rather than hide", and the first draft stated it as a ratio in a doc comment -- a
hand-maintained copy of a number the program can compute. The sample now measures and prints it
(`layout: N bytes of records in M bytes of file`) and the doc comment points at that line instead of
restating it. No conclusion is drawn about whether the ratio is acceptable, because that depends
entirely on a caller's record size.
**A `const` assertion carries the sector rule**, verified load-bearing in both directions: a stride
of 4000 fails the build with `error[E0080]: evaluation panicked: RECORD_STRIDE must be a whole
number of sectors`, and 4096 builds clean. That is the build rung rather than a test, which matters
because the failure it prevents -- `ERROR_INVALID_PARAMETER` from a `NO_BUFFERING` write in M25.3 --
would otherwise appear only on 4K-native storage, on somebody else's machine.
**Four sabotages recorded, not one.** The writers' offset advances are separate facts at separate
sites: measured, reverting the harness lane leaves every appender test green and vice versa, so a
single case would have declared the pair covered while half of it was not. The full sweep is
16-of-16 as declared with the `CONTROL` still surviving. Note what the appender case does *not*
establish: packing its offsets makes records overlap inside a block, so two guards fire at once for
two different reasons -- the narrower evidence that the stride itself is what is caught is the
reader case, which moves only one number.
**All three replay paths report the same numbers as before the change**, which is the check that the
layout moved and the contract did not: 24 durable records verified, 3 tail records tolerated, the
torn tail still stopping at `Truncated`, and the negative control still catching a corrupted byte.
The torn-tail simulation needed its arithmetic rewritten to keep meaning that -- it trimmed a fixed
count of bytes off the end of the file, which after striding lands in the final record's zero
padding and tears nothing at all. It now derives the cut from the last record's own block, which
also survives M25.3's pre-allocation.
### <a id="m253"></a>M25.3 -- The log and every strategy file are pre-allocated and opened `NO_BUFFERING | OVERLAPPED`. *(completed 2026-09-24 12:45:57 -04:00)*
Two steps in `logfile::create_preallocated`, neither interchangeable with the other: write the
extent with an ordinary handle and drop it, then reopen `OPEN_EXISTING` with both flags. This is
[the spike](design-sessions/spikes/write-pending-spike.rs)'s condition D, the only one of four it
measured as behaving differently from a buffered handle.
**What this buys is an opportunity, not a guarantee, and nothing here claims otherwise.** Per the
standing constraint on M25, Windows specifies nothing about when a ring operation completes relative
to `SubmitIoRing`. The log is correct either way; what changes is whether a commit is separately
*measurable*, which is M25.4's problem and M25.5's to read.
**The sweep found four live sites, and the item named two of them.** It predicted two statements in
[strategy.rs](examples/epoch_log/strategy.rs) reasoning from a synchronous handle. There were also
two in [main.rs](examples/epoch_log/main.rs) -- one an internal comment, one **printed to the user**
as part of the strategy comparison's own narrative. All four were corrected the same way: the
historical finding is preserved in the past tense, since it was true when written and is how M20.6
reached its conclusion, and what follows is that the cause has been removed *without* asserting the
consequence. Whether the strategies are now distinguishable is not settled by changing a flag.
**The third site the item named no longer exists, and that is the right outcome rather than a
miss.** `placement.rs`'s `volume_numa_node` documented that it could not use
`windows-overlapped-io-sys`'s typed `BlockingEndpoint::ioctl` partly because the handle was
synchronous. Earlier this session, `947b252b` moved that function into `win-numa-sys`, and the
comment went with the move -- correctly, because `win-numa-sys` depends only on `windows-sys` and
has no occasion to explain why it is not using a crate it does not reference. The item was written
before that move; its prediction of "a narrowed comment, not a refactor" was answered by the
comment's home changing.
**The `NO_BUFFERING` alignment rule could not be tested the obvious way, and finding that out is
what produced the better test.** The first attempt wrote through [`std::io::Write`] and failed on
the *aligned* write: `write_all` issues `WriteFile` with a null `OVERLAPPED`, which an asynchronous
handle refuses however well-aligned the transfer is. That is now its own test -- it is the only
property of the handle's *mode* reachable from here, since `GetFileInformationByHandleEx` does not
report it and the ring works on synchronous and asynchronous handles alike. The alignment rule is
tested through a real ring instead, which is also how production reaches this handle, with both
directions asserted: an aligned write accepted and an unaligned one refused. Each test has a control
using an ordinary handle, so the refusals are attributable to the flags rather than to anything else
about the file.
**A blind spot is recorded in [sabotage.json](sabotage.json) as a declared survivor rather than left
invisible.** Replacing the zero-fill with `set_len` is a **real regression that nothing here
detects**: both produce a file of the right size whose bytes read back as zero, because reads past
the valid data length are answered with zeros the filesystem synthesises without touching the disk.
Only the zero-fill advances that valid data length -- which is the thing that decides whether a
later write is extending. A `set_len` extent silently returns the log to the configuration measured
as behaving like a buffered handle. The only user-mode way to read a valid-data length back is
`FSCTL_QUERY_FILE_REGIONS`, and adding it to a sample purely to check a property the sample does not
otherwise use was judged machinery for its own sake. Recording it as `expect: "survives"` means a
future change that makes it observable will show up as a discrepancy in the sweep.
> **Corrected 2026-09-24, the same day, after review challenged the claim rather than the code.**
> The paragraph above asserts from documentation that `set_len` is a regression, and the reasoning
> it gives is the wrong mechanism. Two measurements settled it:
> [2026-09-24-set-len-zero-fill-cost/](measurements/2026-09-24-set-len-zero-fill-cost/README.md)
> shows the zeroing cost is **identical** for a sequential writer, so that is not the reason; and
> [2026-09-24-set-len-vs-zero-fill/](measurements/2026-09-24-set-len-vs-zero-fill/README.md) shows
> the zero-filled extent pends at a median of 471/500 against `set_len`'s 268/500, which is. The
> blind spot is real and the conclusion survives; the argument for it did not. The second capture
> also corrects this entry's own framing of the spike, which repeated "only the pre-written extent
> pended" from a single run that does not replicate.
**The harness caught a stale case in its own manifest, which is worth more than the case was.** The
`M25.1: the appender packs its record offsets` sabotage stopped compiling, because M25.1 ended by
removing the `total` binding its patch referenced in order to clear an unused-variable warning --
so a sabotage that was correct when written was broken by a later edit in the same session. The
harness reported `MANIFEST DOES NOT COMPILE (tests never ran)` rather than scoring it as caught or
survived. **A manifest is a restatement site like any other**, and has to be swept when the code it
patches moves; nothing else would have noticed.
**One assertion had to change because pre-allocation made it vacuous.** The strategy harness
compared `outcome.bytes` against the file's length to check its own accounting. A pre-allocated file
spans its whole extent from the moment it is created, whatever was written into it, so that equality
would have held just as well for a run that wrote nothing. It now checks the accounting against the
layout rule -- one block per record -- and separately that the extent covers what was written.
**Deliberately left buffered, and said so at the definition.** The checkpoint file's records are
sixteen bytes from a `Vec` at offset 0, which satisfies none of `NO_BUFFERING`'s three alignment
rules; the retired segment is written once with `std::fs::write` and never goes through a ring. The
control plane's correctness comes from its covering flush, not from how its bytes are cached.
**The log is pre-allocated with slack rather than to its exact record count.** A real write-ahead log
pre-allocates ahead of its writer, because an append that reaches the end of the extent becomes an
extending write again. It also means a clean log now ends in zeros rather than at EOF, so replay
stops with `NeverWritten` -- the path M25.2 taught it to tolerate, now actually exercised by the
sample rather than left for a reader of a real log to meet first.
### <a id="m251b"></a>M25.1b -- The sample's own verification now runs under `cargo test`, and `main` itself under CI. *(completed 2026-09-24 15:20:02 -04:00)*
The item asked where this belonged: a CI job running the binary, or a `#[test]` calling the sample's
functions. **Both, because they carry different facts**, and the split follows the FAIL FAST ladder
rather than splitting the difference.
**Everything the sample asserts is now a test.** `run_log` and `verify` are ordinary functions over
a generic `Report`, so `tests.rs` drives the real log against a real ring and a real file and then
runs the real verifier -- the same code path `main` takes. That is not a proxy for running the
sample; it *is* running it, minus the strategy comparison. It costs nothing measurable: the example
suite still finishes in well under a second.
**The comparison's strongest assertion had no test, and now does.** `compare_strategies` requires
all three strategies to write **byte-identical** logs -- replay checks a log against itself, where
this checks the three against each other, so a dropped record or a wrong offset in any one shows up
as a difference from the other two. `M25.1`'s harness test runs `CoveringFlush` alone, so this was
the one assertion only `cargo run` could reach. `every_strategy_writes_the_same_log` runs all three
at two epochs of three records instead of thirty-two of sixty-four. The expectation survives the
ring count because a record's offset is its position in the global sequence times the stride, and a
strategy decides which *ring* submits a write, never where it lands.
**CI runs the binary for what a test cannot reach**: `main` itself -- its path setup, its error
plumbing, its exit code. This is a published example a consumer runs, so a panic on startup is
exactly the failure worth catching, and it is the rung that fits because nothing smaller executes
`main`. Release, where the sample takes about a second.
**The sabotage found a real hole, and it was in the thing this item exists to protect.** Reverting
`M25.2`'s torn-tail cut to the old file-length form was **survived** by the new end-to-end test.
With the extent pre-allocated, trimming a fixed count of bytes off the file lands in the slack, so
every record stays whole -- and `is_clean()` and the durable count both pass while the case
demonstrates the opposite of what it claims. A verifier that had quietly stopped verifying.
What makes that worth recording is where the hazard already was: **written out in full, in the
comment directly above the cut**, since `M25.2`. Describing it caught nothing. Two assertions now
carry it -- that `tail_stopped` is `Truncated` specifically, not merely present, since a cut past
the last record reports `NeverWritten` and would mean the case had stopped tearing; and that a tail
record was actually lost. Prose is not a rung, stated once more by a file that had the prose.
**What each guard is load-bearing for**, measured:
| sabotage | end-to-end test | M25.1's layout tests |
|---|---|---|
| replay walks by extent (the `M25.1` defect) | red | red |
| torn cut taken from the file length | **red** | green |
| negative control moved out of the durable region | **red** | green |
So the end-to-end test would have caught the defect that motivated the item, and it is the only
thing covering two failures that make the sample's evidence vacuous rather than wrong. Both are
recorded in [sabotage.json](sabotage.json).
**What is deliberately still only in CI**: the strategy comparison end to end. Running it in a test
would multiply the suite's cost to re-check a layout and a replay that `M25.1` already covers, and
what the full run adds beyond the invariant above is a *measurement* -- which is not a contract this
crate may assert.
### <a id="m254"></a>M25.4 -- The commit is measured as submit / blocking / deferral, so the flush's own cost and the deferral window cannot be confused again. *(completed 2026-09-24 16:32:56 -04:00)*
`CommitTiming` replaces the single `Duration` the harness used to publish. The three parts sum to
that old number, and `flush()` is `submit + blocking` -- the deferral excluded, which is the whole
point.
**The split satisfies M25's standing constraint by construction rather than by assumption.** Windows
specifies nothing about when a ring operation completes relative to `SubmitIoRing`, so the harness
must be meaningful either way: if the flush completes inline the device round trip lands in `submit`
and `blocking` is zero; if it pends, `submit` is short and the wait appears in `blocking`. Nothing
has to know which case it got.
**The strategies were always distinguishable on the commit, and the blended number hid it
completely.** They now separate by roughly sixfold on the flush, where the old figure ranked them in
the opposite order -- alternating-rings reported the *worst* commit latency while being the fastest,
because its deferral is about twice the others'. That is not a new finding so much as `M20.6`'s
finding finally visible in the program's own output. Figures are not quoted here; the sample prints
them and `M25.5` is where they are read.
**`blocking` reads zero for all three, and the output says plainly that this proves nothing.** A
zero beside a large deferral is ambiguous: the operation may have completed inline, or it may have
pended and finished while the program was busy elsewhere. Those are indistinguishable from here.
Recording that is the point -- reading `blocking` alone would be the same error as before in the
opposite direction, and the temptation is real now that `M25.3` has given the handle the shape the
spike measured as pending.
**The settle logic became one function rather than two copies.** Both sites -- the loop's and the
drain after it -- applied the same rule about where deferral ends and blocking begins, and a rule
stated twice can be half-corrected. `settle` states it once.
**A sabotage found the guard that the obvious assertions miss.** Asserting the identity
`flush() == submit + blocking` catches deferral being folded back in, and is **survived** by a part
that is never measured at all: replacing the deferral measurement with zero satisfies every identity
while making the decomposition a rename. The test therefore also requires some sample of `deferral`
and of `submit` to be non-zero -- phrased as "some sample" rather than a lower bound on a duration,
because this harness defers by construction, so a run in which nothing deferred means the clock is
not running rather than that the machine was fast. `blocking` deliberately gets no such guard, since
zero is a legitimate and frequently observed reading for it.
Three cases in [sabotage.json](sabotage.json): the fold, and the two parts that can silently read
zero. They are listed separately because `submit` and `deferral` are measured at different sites and
one can be lost without the other.
**What this does not do** is assert any value. Which part carries the cost is a property of the
machine and the handle, not of this crate.
### <a id="m255"></a>M25.5 -- The comparison was re-run over fifteen runs and `M20.6` answered: no other ground found for `AlternatingRings`, and the accounting defect that would have inverted the reading was fixed first. *(completed 2026-09-24 17:19:59 -04:00)*
The capture is [measurements/2026-09-24-commit-decomposed/](measurements/2026-09-24-commit-decomposed/README.md)
and the figures live there rather than here.
**The measurement had a defect that had to be found before it could answer anything.** `M25.4`'s
first numbers showed `HostSequenced` committing roughly **six times cheaper** than the other two.
That was where the clock started: it waits for every write in userspace before pushing an unordered
flush, and the commit clock began at the *submit*, so its host round trip fell outside every
measured part. The cost had not gone anywhere; nothing was looking at it.
A fourth part, `prepare`, now covers whatever a strategy must do before its flush can be pushed.
With it, `HostSequenced` is within noise of the others rather than six times cheaper. **A reader of
the uncorrected figures would have drawn the opposite of the right conclusion** -- which is the same
failure mode `M20.6` was opened to fix, one layer down, found by reading the very numbers the fix
for it produced.
**What fifteen runs show.** The three are **not distinguishable** on throughput or on total commit
cost: medians within a few percent, every range overlapping every other, and run-to-run spread
within a single strategy larger than the spread across them -- which is the condition the sample's
own output tells a reader to check. What *is* structural, and never inverts across fifteen runs, is
where each spends its commit: `HostSequenced` in `prepare` and almost nothing in `submit`, the
covering strategies the reverse. That difference is invisible in any blended number, which is what
`M25.4` was for.
**The open question, answered as far as this can answer it.** `AlternatingRings` shows no advantage
this harness can measure -- its throughput and commit medians sit inside the others' ranges, its
deferral is consistently about twice theirs, and the single worst commit p99 in the capture is its
outlier -- against a doubled arena registration it pays for the life of the run.
**That is not a finding against the strategy**, and the entry says so where a reader will meet it.
The ground it was built on is blast radius, and `M20.6` established that this harness **cannot
exhibit that difference at all**, because each lane registers its own arena of the same size so the
per-ring bound is identical by construction. What this capture could answer is whether some *other*
ground appears, and none did. Whether that changes the strategy's status is the engineer's decision;
the conditions under which it would pay are already written in
[strategy.rs](examples/epoch_log/strategy.rs).
**`block` reads zero at the median in every run**, and the capture records that this establishes
nothing: an operation that pended and finished during a deferral of several milliseconds is
indistinguishable from one that completed inline.
**The harness caught stale manifest patches for the third time today.** Adding `prepare` changed
both `flush()` and the deferred tuple, invalidating two `M25.4` cases that had been correct when
written hours earlier. Each time the code moved, the manifest's patches stopped applying and only
`run-sabotage.ps1` noticed -- `MANIFEST STALE: pattern found 0 times`. The note that a manifest is a
restatement site like any other is now load-bearing three times over, which is enough to call it a
standing hazard rather than an incident.
### <a id="m256"></a>M25.6 -- Swept what this milestone made false, and recorded the two findings as `D-56` and `D-57`. *(completed 2026-09-24 18:03:24 -04:00)*
**Four named sites, and the sweep found two more.** The item listed the "keeps appending while a
commit is outstanding" rationale, `strategy.rs`'s "what the measurement found" section, the `M22.1`
capture's commit-p50 claim, and any DESIGN-NOTES text calling the sample's I/O buffered. Grepping
the falsified *claims* rather than the listed files also turned up the spike's premise that
`epoch_log` "writes variable-length records at packed offsets", and a second copy of the "two orders
of magnitude" mechanism inside the `M22.1` capture's `Settles` paragraph. As usual the reported
sites were a sample of the population.
**One named site turned out not to exist.** No DESIGN-NOTES text describes the sample's I/O as
buffered. The three near-matches are about other things -- `D-40` is a cached *read* in the handover
tests, `D-49` is the unit suite's hermeticity, and the note that a ring handle "does not need
`FILE_FLAG_OVERLAPPED`" is a fact about rings that `M25` did not touch. Recorded because a sweep
that quietly finds nothing at a named site is indistinguishable from one that did not look.
**The `M22.1` correction is the sharpest of them, and it is not that the number was wrong.** That
capture reported a commit-p50 reduction as "the one finding here that separates", beside a
throughput result reported as unmoved. The reduction is real and its stated mechanism is correct as
written -- batching shortened the interval between the last append and the flush being reached. What
is wrong is the label: `M20.6` established that figure was **entirely deferral**, so it measured the
*append path* getting faster. And that makes it not an independent finding at all. Both lines are
the same fact seen twice -- the appends got faster, the run is flush-bound, so the change appears in
the metric that is not flush-bound and not in the one that is. Reporting them as two results
overstates the evidence by exactly one result.
**One figure is now measured and is not what it said.** The old explanation had the strategies
differing by amounts "two orders of magnitude below" the flush, "in the tens" of microseconds.
`M25.5` measured hundreds. The conclusion is unchanged, because they remain smaller than the
run-to-run spread -- but "below the noise" and "two orders of magnitude below the flush" are
different claims and only the first held, so both copies of the stronger one were corrected rather
than left standing beside a note.
**`strategy.rs`'s top section was restructured rather than annotated.** It had accumulated a true
current claim, a superseded mechanism, and two correction sections underneath, so a reader met the
false explanation first and the correction several paragraphs later. It now states what is measured,
links the capture instead of quoting figures, and keeps both superseded explanations compactly below
under a heading that says they are superseded -- which is what CONTRACT INTEGRITY asks for and what
the file was violating.
**The spike's prediction is left in the past tense rather than deleted**, because the reasoning is
the reusable part: it said that if only the pre-written condition pends, "the harness fix is not a
flag change -- it is a change to the log's on-disk format." That is exactly what happened, and it is
now `D-57`.
**Two decisions recorded.** `D-56`: a benchmark that defers its await measures the deferral, and the
number survived three rounds of correction because every round re-read the conclusion instead of the
instrument -- with the generalisation that "the conclusion still holds" is not evidence that the
instrument does. `D-57`: a flag whose requirements reach into the caller's data layout is not a flag
change, and costing it as one underestimates it by the size of a format migration.
### <a id="m257"></a>M25.7 -- Replay keeps its slice for a reason about failure vocabulary, recorded as `D-58`; the second multi-megabyte buffer became a digest. *(completed 2026-09-24 19:07:31 -04:00)*
The item offered two acceptable answers -- stream the verifier, or stay legible -- and required only
that an 8 MiB `fs::read` not sit unremarked in a teaching sample.
**The decision is to keep `replay(&[u8])`, and the reason is not simplicity.** The walk is strictly
forward one block at a time and never looks back, so it genuinely has no need of the whole file, and
a real log is larger than memory -- which makes reading the whole file the wrong reflex to teach at
exactly the point a reader is learning to verify one. That argument is real and it lost to a
stronger one.
**`replay` returns an `Outcome`, not a `Result`.** Every way it can end is a statement about the
log: verified, tolerated, or a `Violation`. A streaming reader introduces a third kind of ending --
`io::Error` -- into the one component whose entire job is to distinguish *the log broke its promise*
from *the log kept it*. Those want different responses from a caller, and a signature returning both
through one channel invites precisely the conflation this file exists to prevent: an unreadable file
reported as a missing durable record. **So the streaming version is a different interface, not a
smaller allocation** -- which is what the item suspected, and the suspicion is what turned out to
decide it.
The cost of declining it is stated where it is paid rather than hidden: 140 KiB at the log's own
`fs::read`, 8 MiB per strategy at the harness's, each with a comment saying so. `D-58` records the
decision and what a consumer building a real verifier should want instead -- the streaming shape,
*with* the two failure kinds kept apart inside it.
**What was reducible without touching that interface was reduced.** The cross-strategy comparison
held a whole reference log in memory for the length of the comparison, so two multi-megabyte buffers
were alive at once. It now keeps a 32-bit digest, which halves the peak and loses nothing a reader
had: the assertion could already only say *that* two logs differed, never where.
**The digest is a weaker check than the byte comparison it replaced**, and the weakening is guarded
rather than assumed away. Two different logs can in principle share a digest where two byte arrays
cannot share their bytes, so `record/tests.rs` -- a test module `record.rs` did not previously have
-- pins that a flipped byte, a dropped record, and a trailing zeroed block each change it. The
sabotage confirms the separation is real: a digest folding only the length still distinguishes logs
of different sizes, so the two length-based tests stay green and only the flipped-byte one fails.
What none of them establish, and the definition says so, is that no two logs collide.
**The two 64 KiB sites were left with a note rather than churned.** `RETIRED_LEN` is exactly 64 KiB
-- at the threshold this repository treats as the point to ask the question, not past it -- so both
the fill that writes it and the read that checks it are within the rule. The note says what a reader
growing that segment should do: the write has the same shape as `logfile`'s zero-fill, and the check
is a fold that never needs the bytes all at once.
## Moved 2026-09-24 19:17:17 -04:00 -- M25: the epoch-log sample's I/O became a shape where a commit is observable
The milestone's eight items are archived individually above; what follows is the context the
section carried, kept because it records what M20.6 found and the constraint every item was
held to.
## M25 -- Make the epoch-log sample's I/O a shape where a commit is observable
Queued by the `M20.6` investigation, which found three things the item did not anticipate.
**The harness measures the wrong quantity.** Decomposing its commit latency into *deferral* (flush
pushed -> harness next looked) and *blocking* (time actually waiting) gave blocking p50 **and p99 of
0 us for all three strategies**. The published `commit p50/p99/max` column is entirely deferral: it
reports how long the next epoch's appends took, not anything about the commit.
**There is no pipeline to measure.** The commit's `SubmitIoRing` took 289-555 us and returned with
all 9 completions already queued. The handle has no `FILE_FLAG_OVERLAPPED`, so the batch ran inline,
and the comment in [strategy.rs](examples/epoch_log/strategy.rs) reading "a real log keeps appending
while a commit is outstanding" describes something that cannot happen there.
**`AlternatingRings`' blast-radius claim is answered structurally, and needs no run.**
`RegisteredBuffers::get_mut` refuses a slot with an operation outstanding and there are `SLOTS`
slots, so at most `SLOTS` appends are outstanding on a ring **by construction** -- and each
alternating lane registers its own arena of the same size. The per-ring bound is identical either
way. Measured at 8 and 8, but the argument does not rest on the measurement, and it holds whatever
the platform does about pending.
[write-pending-spike.rs](design-sessions/spikes/write-pending-spike.rs) then established which
configurations pend at all. `FILE_FLAG_OVERLAPPED` alone changed nothing (0/500). Only
`NO_BUFFERING` over a **pre-written extent** pended reliably, and its submit p50 fell from ~500 us to
116 us -- the flush's cost leaving the submit path is what makes a commit separately observable for
the first time.
> **Corrected 2026-09-24, after `M25.3` landed: the paragraph above overstates what replicates.**
> Sixteen runs with a fifth condition added are in
> [measurements/2026-09-24-set-len-vs-zero-fill/](measurements/2026-09-24-set-len-vs-zero-fill/README.md).
> What holds is that a **buffered** handle essentially never pends while every `NO_BUFFERING` one
> pends in most runs. What does not hold is "only the pre-written extent pended": the extending
> condition has a median of 268/500 over those runs. The zero-filled extent is still the best of
> the five -- median 471/500, floor 121 against 1 -- so `M25.3`'s choice stands, but as a
> difference of degree rather than of kind. The single-run reading came from a pair of numbers the
> spike's own header already warned was unstable. `M25.4` and `M25.5` must be read with that
> variance in mind rather than against the original framing.
**A standing constraint on every item below.** That 500/500 is an observation, not a contract:
Windows specifies nothing about when a ring operation completes relative to `SubmitIoRing`. So the
sample may *adopt* this shape -- it is what real write-ahead logs do, and it is the only shape where
the measurement means anything -- but **nothing here may depend on an operation pending.** Every item
must leave the log correct if the platform completes inline tomorrow.
- [x] **M25.1** -- Records gained a fixed sector stride with a zeroed block tail, in both writers. -> [completed 2026-09-23](COMPLETED-CHECKLIST.md#m251)
- [x] **M25.2** -- Replay walks by the stride and confines each decode to its own block. Landed with `M25.1`: a strided writer and an unstrided reader cannot coexist. -> [completed 2026-09-23](COMPLETED-CHECKLIST.md#m251)
- [x] **M25.1b** -- The sample's own verification now runs under `cargo test`, and `main` itself under CI. -> [completed 2026-09-24](COMPLETED-CHECKLIST.md#m251b)
- [x] **M25.3** -- The log and every strategy file are pre-allocated and opened `NO_BUFFERING | OVERLAPPED`. -> [completed 2026-09-24](COMPLETED-CHECKLIST.md#m253)
- [x] **M25.4** -- The commit is measured as submit / blocking / deferral, so the flush's own cost and the deferral window cannot be confused again. -> [completed 2026-09-24](COMPLETED-CHECKLIST.md#m254)
- [x] **M25.5** -- The comparison was re-run over fifteen runs and `M20.6` answered: no other ground found for `AlternatingRings`, and the accounting defect that would have inverted the reading was fixed first. -> [completed 2026-09-24](COMPLETED-CHECKLIST.md#m255)
- [x] **M25.6** -- Swept what this milestone made false, and recorded the two findings as `D-56` and `D-57`. -> [completed 2026-09-24](COMPLETED-CHECKLIST.md#m256)
- [x] **M25.7** -- Replay keeps its slice for a reason about failure vocabulary, recorded as `D-58`; the second multi-megabyte buffer became a digest. -> [completed 2026-09-24](COMPLETED-CHECKLIST.md#m257)
### <a id="m261"></a>M26.1 -- The permitted space is specified in [RESPONSE-SPACE.md](RESPONSE-SPACE.md) as eleven cited clauses, and recorded as `D-59`. *(completed 2026-09-24 19:30:19 -04:00)*
**Eleven clauses: seven permissions and four constraints**, each with an ID, a source, and a
provenance tag. The IDs exist so `M26.3`'s resolver, `M26.4`'s properties and `M26.6`'s kernel tests
can cite a clause rather than restate it -- and so a clause no code cites is visible as
unimplemented.
**The provenance tag is what makes it a specification rather than a recording.** Every clause is
`Observed`, `Over-provision`, or `Decided`, and where a clause is wider than its own observation the
two parts are split so they can be argued separately. `RS-P-1` is the clearest case: that an
operation may complete inline or pend is measured, but that operations within one batch resolve
**independently** is not, and the space permits it anyway -- because a consumer depending on them
resolving together depends on something Windows never promised.
**The call the item demanded, made rather than defaulted: `RS-C-4` constrains the resolver to
honour the drain half of `DRAIN_PRECEDING_OPS`.** [D-47](DESIGN-NOTES.md#d-47) measured roughly
4,500 trials without a single violation; the drain is what this crate's durability story rests on;
and a resolver permitted to break it would require every consumer to re-verify durability some other
way, which is to say it would make the primitive useless. The cost is stated plainly: a Windows that
broke the drain would not be caught by the resolver at all. That is why `M26.6` gained a line
requiring at least one kernel test to exercise the clause -- otherwise the one constraint the space
takes on faith is untested in both halves at once.
**The hold-back half stays unconstrained**, since `D-24` claimed it and `D-47` withdrew it. `RS-P-2`
applies in full to anything queued after a drained flush, which is the defect class that campaign
found.
**Three citations were checked and one was wrong.** The draft attributed `M22.2`'s defect to the
checkpoint control plane; it was on the *append* path, and the checkpoint module merely documents
the same case. Both now appear, distinguished. The other two -- `pop_within`'s "promises nothing
about poppability" and `D-47`'s trial count -- were verified against the files rather than recalled.
**A working artifact was found rather than assumed missing.**
[kernel-response-space-probe.rs](design-sessions/kernel-response-space-probe.rs) already contains a
seeded `Resolver` exercising `RS-P-2`, which broke a FIFO-assuming consumer under 189 of 200 seeds.
`M26.3` now points at it as a starting point, with the note that the probe marks itself throwaway --
so promoting it is a deliberate decision rather than a default.
**Four things are listed as deliberately undecided** -- rates, partial transfers, failure-code sets,
and timing -- so that a later reader can tell an omission from a choice. Rates in particular are
excluded on principle: a space carrying observed probabilities would be the recording this milestone
exists to avoid.
## Moved 2026-09-24 20:36:46 -04:00 -- M26.2: the kernel-call seam
### <a id="m262"></a>M26.2 -- Build the seam that makes the `windows-sys` calls indirect, so `M26.3`'s resolver can answer them. *(completed 2026-09-24 20:36:46 -04:00)*
The shape is recorded as [D-60](DESIGN-NOTES.md#d-60); what follows is what the work found.
Eight calls became indirect -- `SubmitIoRing`, `PopIoRingCompletion`, and the six `Build*` entry
points the crate uses -- across 27 call sites in [batch.rs](src/batch.rs) and [ring.rs](src/ring.rs).
Each now goes through a `pub(crate) unsafe fn` in [sys.rs](src/sys.rs) that is `#[inline(always)]`
and dispatches through a `through_seam!` macro. With the `kernel-seam` feature off, the macro
expands to the bare FFI call and nothing else; with it on, the call first asks the installed
responder.
**The shape was decided by a constraint already on the books, not by taste.** The obvious
alternative -- parameterising the ring as `IoRing<K>` over a kernel -- is unavailable because
[D-55](DESIGN-NOTES.md#d-55) has already spent `IoRing`'s type parameter on `M28.3`'s token
inventory. A kernel generic would publish `IoRing<T, K>`, which is a two-parameter public type on a
shipped crate, and the second parameter exists only so the crate can test itself. Module
indirection costs the public surface nothing.
**The responder is thread-local, for the reason `DROP_RUNS` is.** `cargo test` runs tests as
threads in one process ([DESIGN-NOTES.md](DESIGN-NOTES.md) records this as the reason this
workspace is not on nextest), so a process-global responder would let one test answer another
test's kernel calls. `with()` uses `try_borrow_mut` rather than `borrow_mut`, so a re-entrant call
from inside a responder falls through to the kernel instead of panicking; `Installed::drop` uses
`try_with`, so teardown during TLS destruction cannot abort the process ([M23.4](#m234)).
**Five lifecycle calls were deliberately left direct** -- `CreateIoRing`, `CloseIoRing`,
`GetIoRingInfo`, `IsIoRingOpSupported`, `SetIoRingCompletionEvent`. `M26` is justified by the
*response space*: what the kernel may answer to submitted work. Routing ring construction and
teardown through the seam as well would be hermeticity for its own sake, and hermeticity is
[M24](#m242)'s subject, not this one. The line is recorded so a later reader can tell a boundary
from an oversight.
**The seam's transparency is measured, not argued.** The full suite passes with the feature off
(153 lib tests) and on (159 -- the six new ones), and the crate builds clean with zero warnings in
five configurations: default, `--all-features`, `--no-default-features`, `--features kernel-seam`,
and release. The sabotage case *`M26.2: the seam consults the responder but ignores its answer`*
turns `an_installed_responder_answers_instead_of_the_kernel` red, which is what shows the
consultation is load-bearing rather than decorative -- a seam that asks and discards would pass
every other test in the crate. The full sweep is 28-of-28 as declared, with the `CONTROL` and the
`set_len` blind spot both still surviving.
**Two type signatures were wrong on the first attempt and the compiler caught both**, which is
worth recording because they are the kind of thing a hand-written trait gets wrong silently if it
is ever allowed to diverge: `BuildIoRingWriteFile`'s caching flag is `i32`, not `u32`, and
`BuildIoRingRegisterFileHandles` takes `*const *mut c_void`, not `*const isize`. The `real`
submodule re-exports the `windows-sys` items so the trait's default methods call them directly,
which is what keeps the two in step -- there is one spelling of each signature, not two.
**`kernel-seam` crossed with `--no-default-features` is a published configuration nothing built.**
The gate multiplies with `threadpool`: the workspace `--all-features` steps build the seam only
alongside the threadpool, and the existing `ioring-no-threadpool` job built the no-threadpool path
only with the seam off. Two steps were added to that job rather than a new job, on the same
argument that bought the job in the first place. Verified locally before committing: clippy clean
and 155 lib tests green in that combination.
**A borrow-surface row was owed and was three items late.** `./tools/check-borrow-surface.ps1`
failed on `Pending::contract -> Option<&RingContract>`, added by `M23.3`'s spike, because that
change did not run the gate -- so it reported on the next run instead, against unrelated work. The
row is now in [DESIGN-NOTES.md](DESIGN-NOTES.md)'s audit table: `RingContract` is a pure
observation record owning no handle, buffer, or registration index, so there is nothing the kernel
could invalidate, and the borrow is a plain `&self` borrow that blocks submission through that
`Pending` for its duration. The lateness is recorded in the row itself.
**A tooling mistake destroyed two source files and is worth the warning.** A PowerShell
`.Replace()` bound the wrong overload and rewrote [batch.rs](src/batch.rs) and [ring.rs](src/ring.rs)
one character per line. `git checkout --` recovered both, and the conversion was redone with
`[regex]::Replace` anchored on `(?<![\w:])Name\(` so it matched call positions only. This is the
fifth time in this session that driving a source edit through PowerShell string handling has
corrupted a file.
## Moved 2026-09-24 21:15:16 -04:00 -- M26.3: the response-space resolver
### <a id="m263"></a>M26.3 -- Build the resolver over the space `M26.1` specifies, bound to its clause IDs and seeded on its own axis. *(completed 2026-09-24 21:15:16 -04:00)*
The shape is recorded as [D-61](DESIGN-NOTES.md#d-61); what follows is what the work found.
**The resolver is in [resolver.rs](src/sys/resolver.rs), implementing `M26.2`'s `Responses`.** It
answers the eight submission-path calls itself, so operations the kernel never received still flow
through this crate's ordinary accounting. Every freedom cites the `RS-P-n` permitting it and every
restriction cites the `RS-C-n` requiring it, which is what lets a reader check the resolver against
[RESPONSE-SPACE.md](RESPONSE-SPACE.md) mechanically rather than by reading both and hoping.
**The asymmetry is the design.** `ResolverConfig` has a switch per permission and none for any
constraint. Narrowing a freedom is how a test isolates another -- a test about ordering does not
want arbitrary operation failures on top -- while a knob relaxing a constraint would let a test
assert against a platform that cannot exist. The default is the widest point, so a test that does
not choose gets every freedom and fails loudly under one it did not handle. A test asserts the
count: seven fields, seven `RS-P-n`, and it reads them off `Debug` so a field added without a clause
fails there rather than passing unnoticed.
**Where a permission and a constraint collide, the constraint wins, and that had to be decided
rather than discovered.** `RS-C-4` holds a drain-flagged operation back even on a tick where
`RS-P-1`'s coin said complete it now; `RS-C-1` forces a post an operation's coin kept deferring.
Two bounds exist solely to make `RS-C-1` finite -- per-operation deferrals, and consecutive declined
submits -- and both are properties of the resolver rather than of the space, which carries no rates
deliberately.
**A submit resolves; a pop only rescues.** The split is not tidiness. A pop that flipped coins would
resolve a consumer's work on its first `try_pop`, so "the operation pended" -- the thing `RS-P-1`
exists to let a test observe -- would be unobservable to exactly the consumer most likely to care.
The freedom would have been implemented and untestable.
**`SetIoRingCompletionEvent` moved behind the seam, which `M26.2` had left it outside of.** It looks
like lifecycle and is not: it is how a completion becomes *observable*, so `RS-P-6` is a clause
about that call. A resolver unable to make it would not satisfy `RS-P-6` vacuously -- it would never
signal at all, parking every [`EventDelivery`](src/event_delivery.rs) consumer rather than testing
one. The checklist item had authorised exactly this ("if a clause turns out to need ... extending
the seam is part of this item"), and this is the clause that needed it.
**`RS-C-4` is decided by position, and the invariant that makes that sound is now asserted.** The
pool is held in build order, so the set queued before `pool[i]` is exactly `pool[..i]` and a barrier
is eligible only as the oldest unresolved operation. That reduction is the whole of the constraint's
implementation and it holds only while the pool stays sorted -- an edit that sorted or reshuffled it
would relax `RS-C-4` to nothing while every line around it still read as though it applied. A
`debug_assert!` now says so at the point of use, rather than a comment saying so nearby.
**The first contact with a real ring found a live defect, queued as `M26.8` rather than fixed
here.** A submit declined under `RS-P-7` propagates out of `IoRing::run_down` as an error with
`outstanding() > 0`, after which `Drop` asserts and calls `CloseIoRing` anyway -- `M21.6`'s hazard,
reachable again through a different `HRESULT`. Measured by narrowing one permission at a time: 32
seeds pass with `may_fail_submits` off, seed `0x1A` fails at `0x80070008` with it on. It is queued
rather than corrected because `run_down`'s own documentation argues that blocking is the safe
failure mode while "no hang" is one of the properties `M26.4` is about to write, and the two pull
opposite ways -- a decision, not a correction. The narrowing is declared in the test that takes it,
and the current behaviour is pinned by its own test so that whichever way `M26.8` is settled, a test
has to change.
**The integration test is where it is for the gate's own reason.** `check-ring-tests.ps1` asks
whether a test needs the kernel or only a ring-shaped thing; a resolver test needs a real ring
because [D-60](DESIGN-NOTES.md#d-60) deliberately left lifecycle real, which makes it an
operating-system boundary and therefore `tests/`. The ring-opening lib population is unchanged at
41.
**Sabotage found a defect in the tests, which is what it is for.** `RS-C-3`'s case survived: the
test asserted that nothing pops *right now*, which a resolver that had wrongly made staged
operations eligible also satisfies, because the starvation rescue posts only what has run out of
deferrals and a fresh operation has not. The test now polls past the bound and asserts the counters,
so it checks the claim -- unsubmitted work is never eligible -- rather than the symptom. Eight cases
added, one of them a declared blind spot; the full sweep is 36-of-36 as declared with both blind
spots and the `CONTROL` surviving.
**The harness caught stale patches a sixth time, and the cause was new.** Four cases reported
`pattern found 0 times` while every line of each pattern was present individually. The cause is that
the built-in file-creation tool writes **CRLF** on Windows, so the multi-line patterns could not
match a file whose line endings were not LF -- and single-line patterns matched fine, which is why
three of the eight cases passed and hid it. The repository's own instructions warn about this tool;
`M26.2`'s files escaped it only because git normalised them on commit before that sweep ran. The
three new files were converted to LF before proceeding.
## Moved 2026-09-24 21:48:36 -04:00 -- M26.4: the properties that must hold under every resolution
### <a id="m264"></a>M26.4 -- Write the properties that must hold under every resolution, with `RingContract` as the definition rather than a second copy. *(completed 2026-09-24 21:48:36 -04:00)*
The shape is recorded as [D-62](DESIGN-NOTES.md#d-62); what follows is what the work found.
**Five properties, in
[properties_under_every_resolution.rs](tests/properties_under_every_resolution.rs).** Conservation
(P-1), no hang (P-2), `pop_within` honours its bound (P-3), `outstanding` is accurate (P-4), no
use-after-free (P-5) -- driven over generated plans, each under its own resolution drawn from
`M26.3`'s resolver at the widest point in the space.
**Only two of the five needed anything new, and that is the item's main point.** P-1 is already
this crate's own oracle, so the harness reports to [`RingContract`](src/contract.rs) and asks it for
the verdict. P-4 follows the same rule rather than counting for itself: the expected outstanding
count is read back out of the contract through its own `Outstanding` violation, because a counter in
the harness would be a third party to the disagreement and, when the two disagreed, the harness is
what would get "fixed". P-5 is [`windows_guard_alloc::GuardAlloc`], already the established
detector.
**Two weaknesses are declared in the file rather than papered over.** `pop_within`'s upper bound is
nearly free under an ordinary resolution, because the resolver answers a wait immediately and the
call rarely approaches its deadline -- so the non-vacuous case needs a resolution in which *nothing
completes during the window*. That is supplied by a separate degenerate responder, and what it is
matters: not an `RS-C-1` violation, since no finite observation can distinguish "eventually" from
"never", but the prefix of a satisfying resolution in which the eventually has not happened yet.
And P-5 covers this crate's memory handling rather than the kernel's, since under a resolver nothing
external writes into a buffer at all; the kernel-side half stays with
[generated_sequences.rs](tests/generated_sequences.rs), against a real ring.
**A third seed axis, kept separate.** This file carries the plan seed, the resolver seed and the
guard allocator's. They are independent on purpose -- pinning the plan alone reproduces the same
operations against different resolutions, which is what a suspicious plan calls for -- and a failure
prints all three, because only all three replay the whole run.
**The harness's own first defect was treating a declined submit as a property failure.** Under
`RS-P-7` a submit may be declined, and `pop_within` surfaces that as an `Err` -- which is *within
its documented contract*, since it says it returns any error from `SubmitIoRing`. A correct consumer
retries; a harness that panicked was asserting a contract the crate never offered. It now retries,
and recognises a refusal by **asking the resolver** whether its decline counter moved rather than by
matching an `HRESULT`, because a hard-coded code here would be a second copy of a choice the
resolver owns. Retrying is bounded by P-2's budget in every caller, so a resolution that declined
forever is still caught.
**That finding extended `M26.8` rather than creating a second item, and the extension is about the
document.** `RS-P-7` is written as a *consequence* clause -- "if `SubmitIoRing` fails, operations
already built remain queued" -- citing `D-5`, which establishes the no-rewind consequence and
nothing about submits failing spontaneously. `M26.3`'s resolver read it as a permission, and the
space nowhere states that a submit may fail at all. That is a gap rather than a decision, since
submits demonstrably can fail, so `M26.8` now has to settle both halves together.
**Coverage counters are asserted, not printed, and that is what makes a green run evidence.** All
five properties are satisfied trivially by a run that does nothing: an empty plan, a resolution that
completes everything inside its submit, or a harness that quietly stopped reporting would each pass
every assertion. The counters are what separate "the properties held" from "nothing reached the
states they are about", and one sabotage exists purely to show they are load-bearing. Their
thresholds were set from a measured spread across eight fresh seeds rather than guessed; the
declined-submit count is the thinnest signal and its threshold is only "more than none" for that
reason, since tightening it would buy a flake rather than a guarantee.
**The integration test is where it is for the gate's own reason**, as in `M26.3`: it needs a real
ring, because `D-60` deliberately left lifecycle real. The ring-opening lib population is unchanged
at 41.
**A sabotage case was written, measured, and removed as unsound.** It disabled the P-1 verdict
check, and it survived -- correctly, because disabling an assertion that does not fire on a green
baseline cannot fail anything. What actually establishes that the verdict is read is the
identity-reuse case: measured, the suite fails carrying the oracle's own wording, so the path from
the crate through `RingContract` to a red test is traversed end to end. That evidence now lives in
that case's reasoning, and the removal is recorded here because "sabotage a check" is an appealing
and empty move worth recognising next time. Three cases added, all scoped to this test target on
purpose -- these mutations are caught by many tests in the crate, and "something went red" would not
have shown that *these* properties are the ones watching. Full sweep 39-of-39 as declared.
## Moved 2026-09-24 22:13:02 -04:00 -- M26.5: calibrating the resolver
### <a id="m265"></a>M26.5 -- Re-inject the two historical defects and confirm the resolver turns red. *(completed 2026-09-24 22:13:02 -04:00)*
The shape is recorded as [D-63](DESIGN-NOTES.md#d-63); what follows is what the work found.
**The two defects are calibrated differently because they live in different places**, and noticing
that was most of the item. `M21.6`'s -- an expired wait reported as a failure -- was in this crate,
so re-injecting it means mutating the crate, which a test cannot do; it is a case in
[sabotage.json](sabotage.json). [D-47](DESIGN-NOTES.md#d-47)'s -- a consumer believing a covering
flush holds back what follows -- is in a **consumer**, so there is nothing here to mutate and the
defective consumer had to be written out. It is, in
[calibration.rs](tests/calibration.rs), as a `HoldBackBeliever` that records when its assumption
fails; the test asserts that the resolver breaks it.
**What the calibration file adds on the `M21.6` side is the precondition, not the detection.** A
sabotage of code the suite never executes is caught for some unrelated reason or not at all, and
either way measures nothing -- so there is a test asserting that `RS-P-4` actually reaches
`pop_within`, read off the resolver's own counter rather than inferred from a timing, because "the
call took a while" is not evidence that a wait expired.
**Both directions were verified by execution before being written into the manifest.** Reverting
`wait_outcome`'s `WAIT_EXPIRED` arm fails the calibration naming the seed and `0x800705B4`. Making
the resolver enforce the hold-back fails it with "no seed of 64 broke a consumer that assumes a
covering flush holds back what follows it".
**The second of those runs the opposite way to every other sabotage in this crate**, and the
manifest says so: it does not break the code, it makes the **instrument** go narrow. That is the
`M17.4` failure mode -- a suite sampling the right state while being insensitive to the defect
living in it -- and nothing detects it except a test that demands the sensitivity.
**The two suites' sensitivities were measured and are not equivalent.** `M21.6`'s defect is caught
by the calibration *and* by `M26.4`'s property suite; the latter only because that suite
distinguishes a declined submit from a genuine error, so the detection is deliberate rather than
lucky. A narrowed resolver is caught by the calibration **alone**, and the property suite correctly
stays green -- a narrower resolution is still a valid one and conservation holds under it. That
measured gap is the argument for a calibration file rather than a calibration assertion bolted onto
the property suite, and it is an argument from data rather than from taste.
**The D-47 calibration breaks its believer on every seed, which needs saying rather than
celebrating.** `D-47` measured the overtake on real hardware at well under one trial in a hundred;
the resolver does it constantly. That is not the resolver being unfaithful:
[RESPONSE-SPACE.md](RESPONSE-SPACE.md) carries no rates deliberately, because a resolver
reproducing an observed frequency would be a model of Windows and therefore the trap
[D-52](DESIGN-NOTES.md#d-52) was opened to escape. A defect class that is rare on hardware is
precisely the one a rate-free resolver earns its keep on. The figures are printed by the tests so a
reader can judge the instrument's strength rather than take "more than none" on trust.
**Seeds here are fixed rather than clock-derived**, unlike the generated suites. A calibration that
could sometimes fail to demonstrate its own sensitivity would be the exact failure it exists to
prevent, arriving as a flake.
## Moved 2026-09-24 23:54:45 -04:00 -- M26.6: the kernel tests' new job
### <a id="m266"></a>M26.6 -- Point the kernel tests at confirming reality stays inside the declared space. *(completed 2026-09-24 23:54:45 -04:00)*
The shape is recorded as [D-65](DESIGN-NOTES.md#d-65); what follows is what the work found.
**A hole was found exactly where the split is load-bearing.**
[flush_barrier.rs](tests/flush_barrier.rs) already asserted `RS-C-4` against a real ring -- but the
assertion sat behind an early return taken whenever its control could not discriminate. On such a
machine the constraint was untested on **both** sides at once: the resolver is forbidden to produce
a violation, and the only test that would notice had skipped. That is precisely the condition the
item was written to prevent, and it was live.
**The fix separates two questions one gate had been answering together.** *Is the barrier doing
work* is a comparative claim and genuinely meaningless when the control shows no reordering -- the
covering case would match a control that did nothing. *Did the kernel stay inside `RS-C-4`* is a
conformance question, where skipping can only ever hide a violation; observing none is weak
evidence when nothing could have reordered, but observing one is a finding on any machine. The
conformance assertion now runs everywhere and only the comparative claim is withheld, with the
output saying `PARTIAL` rather than `SKIP` so the difference is visible in a log.
**The census was green and useless on its first build, and only trying to make it go red found
that.** It searched each file for the clause ID anywhere in its text. Removing `RS-C-4`'s check
from the only test performing it did **not** turn it red, because
[generated_sequences.rs](tests/generated_sequences.rs) mentioned that clause solely to *disclaim*
it -- "`RS-C-4` is flush_barrier's" -- and under a substring search a disclaimer reads exactly like
a claim. This repository had already recorded that trap once, for a probe whose only mention of a
tag was a comment, and it was walked into again anyway.
**So a claim is now a structured marker.** `CONFIRMS:` for a constraint checked against a real
kernel, `EXERCISES:` for a permission the resolver takes, each naming a clause and nothing else --
a hedged `CONFIRMS: RS-C-4 eventually` does not parse as a claim, and that case is asserted. The
rebuilt census was verified to go red in three directions before being trusted: a removed marker, a
marker naming a clause the document does not declare, and a prose mention standing in for a marker.
**Two blind spots are declared rather than left invisible.** A census over source proves a clause is
*claimed*, never that the file's assertion still runs or still means anything -- comparing a
constant against itself leaves the marker in place and the suite green. And dropping the covering
flag does **not** fire the `RS-C-4` assertion on every machine: measured here, a device stack that
orders a flush behind its file's outstanding writes by itself produces no violation to see, so that
sabotage demonstrates nothing portable. The assertion's conformance value -- reporting a violation
if one occurs -- is separate from its sensitivity, and only the first is claimed.
**The sweep found a stale site from two milestones back.**
[D-60](DESIGN-NOTES.md#d-60) still listed `SetIoRingCompletionEvent` among the five lifecycle calls
left outside the seam, which `M26.3` had moved behind it. The correction is recorded in that row
rather than only in `D-61`, since a reader arriving there would otherwise take the superseded list
as current.
**The "five techniques" framing becomes six, and the sixth is different in kind.** The other five
check this crate against its own stated contract and cannot tell you that contract is wrong -- which
is what happened in the two most expensive defects. The resolver checks the crate against a written
specification of what the platform may do, and the kernel tests check the platform against that same
specification. `M26`'s row also joins defect population A, on the observation that the kernel's
*response* is a precondition and was never varied either.
## Moved 2026-09-25 10:48:58 -04:00 -- M26.7: auditing the suite for frozen observations
### <a id="m267"></a>M26.7 -- Audit the existing suite for assertions that are frozen observations rather than contracts. *(completed 2026-09-25 10:48:58 -04:00)*
The shape is recorded as [D-66](DESIGN-NOTES.md#d-66); what follows is what the work found.
**The census came from a command, and it was not the file anyone guessed.** The item named
[flush_barrier.rs](tests/flush_barrier.rs) as the obvious candidate and said plainly that the
candidate was a guess. It was: that file turned out to be one of the better-behaved ones, since its
transfer assertion is explicitly framed as its own precondition and its ordering counter is
*reported* rather than asserted. The census started from the whole assertion population, narrowed to
assertions in tests that touch a real ring -- reusing `RING-OPENING-LIB-TESTS.txt` as the classifier
rather than inventing a second one -- and then to four candidate shapes.
**One class, 31 assertions wide, across five files.** `try_pop()` straight after `submit_and_wait`
with the `Option` unwrapped, asserting the kernel had **already** queued the completion. That is
`D-52`'s demonstrated failure exactly -- an assertion that gives opposite answers on two handles of
the same API -- and this crate's own documentation denies it: a submit-side wait's return "promises
nothing about poppability". `RESPONSE-SPACE.md` states it as `RS-P-5`. They passed for the reason
[D-40](DESIGN-NOTES.md#d-40) measured: a buffered read completes inside the submit in 80 of 80
attempts, while an unbuffered one genuinely pends.
**Restated as the contract `D-52` prescribed** -- ask for the completion within a bound this crate
chooses -- which holds on every handle rather than on the one a test happens to open. The same sweep
found an unbounded `try_pop` spin loop in `registration.rs`, which is the shape `pop_within` was
introduced to replace, and it went the same way.
**Nothing catches a regression by running, and that is the part worth keeping.** Reverting a site
leaves its test green on any machine where the observation is true, which is precisely why 31 of
them survived years of review. So the guard is a **census that refuses the shape at the source**,
plus a resolver-driven test that makes the pending case reachable on demand and shows `try_pop`
failing where `pop_within` succeeds. One of the three sabotages exists to make that point: reverting
a restated site is caught by the census and by nothing else, and a reader who checked by re-running
the test would find it green and conclude the change was cosmetic.
**A second, narrower class was found and deliberately not settled.** Five assertions require a
complete transfer. `Completion::result` promises only "the transferred byte count", and the space
lists partial transfers as *deliberately undecided* with the instruction to decide them before a
resolver relies on either answer. So the suite silently answers a question the specification leaves
open -- and the two are not yet in conflict only because the resolver reports `Information: 0` and
no resolver-driven test reads a transfer count. Queued as `M26.10`. One of the five is already in
the honest form and is left alone.
**What the audit did not find is worth stating too.** The `outstanding() == 0` assertions after a
rundown are the crate's own contract, not observations. `flush_barrier_stress.rs`'s assertion that
reordering *happened* is a control verifying its own precondition, with a message explaining why --
the correct pattern. And the handover tests rest on `D-40`'s synchronous-completion measurement but
verify that precondition rather than assuming it, which is what a frozen observation fails to do.
## Moved 2026-09-25 12:34:02 -04:00 -- M26.8: retry policy belongs to the caller
### <a id="m268"></a>M26.8 -- Decide what `IoRing::run_down` should do when a submit it makes is refused. *(completed 2026-09-25 12:34:02 -04:00)*
The shape is recorded as [D-67](DESIGN-NOTES.md#d-67); what follows is what the work found.
**The item was framed as a decision and was mostly a reading.** It asked which way to resolve a
tension between "blocking is the safe failure mode" and "no hang". The tension was real but the
question had an authority nobody had consulted: `SubmitIoRing`'s reference page. The correction that
made the difference came from the engineer -- **no amount of measurement constitutes a contract** --
and it was needed, because the session had spent the previous hour measuring and had drawn a
confident conclusion from it.
**What the documentation settles.** A return-value row gives `IORING_E_WAIT_TIMEOUT` the meaning
*"All operations were submitted without error and the subsequent wait timed out"*, and the Remarks
add *"If this function returns an error other than IORING_E_WAIT_TIMEOUT, then all entries remain in
the submission queue"* and that a per-entry failure arrives as a completion rather than as a submit
failure.
**A measurement had been read backwards, and the documentation is what caught it.** A probe showed
an operation completing after a submit reported `E_INVALIDARG`, which was taken to mean the work had
gone in despite the error. It had not: the entry stayed in the submission queue exactly as
documented, and what pushed it through was `pop_within`'s *own* internal submit. The observation was
right and the attribution was wrong -- which is precisely the failure mode a contract prevents and a
measurement invites.
**`Batch::submit_and_wait` carried `M21.6`'s defect** at the one site that sweep did not reach.
`do_submit` passed a timed-out wait to `check`, so a fully successful submission was reported as an
error. The damage is worse than a wrong sign: because any *other* error means the entries are still
queued, an `Err` was ambiguous between "your buffers are free" and "the kernel still owns them",
which is [D-5](DESIGN-NOTES.md#d-5)'s hazard with the sign hidden.
**`run_down` was the only waiting API in this crate shaped wrongly**, and the engineer named the
principle that identifies it: *never implement a retry policy ourselves -- the caller keeps their
own backoff, counts before giving up, and whatever else they want.* Waiting in segments inside a
period the caller supplied is fine; `run_down` had the segments and no such period, which made "how
long to keep trying" this crate's policy. [`IoRing::run_down_within`](src/ring.rs) is the primitive
the caller bounds; `run_down` is now that with an unbounded period, which is a choice made by
calling it. Checked against the rest of the surface rather than assumed: `pop_within` and
`submit_and_wait` were already correctly shaped, so `run_down` really was the sole outlier.
**An error from rundown is no longer terminal, and did not need to be.** The entries remain queued,
the ring is resumable, and the documentation is what makes that safe to say. The one thing a caller
must not do after an error is drop the ring.
**`RS-P-7` and `RS-P-3` moved from inference to citation.** `M26.1` wrote `RS-P-7` as a consequence
clause -- "if a submit fails, entries remain queued" -- citing a decision that established the
consequence and nothing about submits failing at all, and `M26.3`'s resolver read it as a permission
regardless. The space now carries a **`Documented`** tag that explicitly outranks `Observed`, since
a measurement describes one run of one build.
**The manifest drifted and the harness caught it, for the seventh time this session.** Renaming
`WAIT_EXPIRED` to `IORING_E_WAIT_TIMEOUT` -- so the constant carries the documented name rather than
a derivation -- left `M26.5`'s sabotage patching text that no longer existed. The sweep that
followed found four more live sites; the archive was left alone. A per-case uniqueness check now
runs before the sweep, which would have caught this in seconds rather than in a five-minute run.
## Moved 2026-09-25 14:19:35 -04:00 -- M26.9, the event_delivery stall
### <a id="m269"></a>M26.9 -- The `event_delivery` stall: the wait was armed after the event was signalled, which `SetThreadpoolWait` forbids. *(completed 2026-09-25 14:19:35 -04:00)*
Cause, fix and before/after figures are in [DESIGN-NOTES.md](DESIGN-NOTES.md) -> D-68; the
investigation as it stood when the cause was found is in
[RESOLVED-TEST-FAILURES.md](RESOLVED-TEST-FAILURES.md). The item as it read when it closed:
- [x] **M26.9** -- Find and fix the intermittent `Timeout` in
[event_delivery.rs](tests/event_delivery.rs)'s two threadpool-delivery tests, recorded in
[UNRESOLVED-TEST-FAILURES.md](UNRESOLVED-TEST-FAILURES.md).
**Why this is not merely a flaky test to re-run.** Measured at 1 failure in 80 runs of the
compiled binary. The sabotage harness runs the whole suite once per case and the manifest holds
41, so that rate gives roughly a **40% chance of a corrupted sweep** -- and the corruption falsely
reports `caught`, which is a sabotage the suite did not catch being recorded as a clean bill of
health. The harness is this repository's mechanism for keeping earlier guarantees checked; a 40%
chance of a silent false pass undermines every conclusion drawn from it.
**What has already been ruled out, so it is not re-tried:** ring-resource pressure (zero failures
after roughly 18,000 ring create/close cycles), the widened seeded sweeps (zero after repeated
property-suite and calibration runs), and CPU starvation (zero under a concurrent `cargo build`
saturating the machine).
**Narrowed 2026-09-25 to a minimal reproducer, and the investigation now leaves this crate.**
Measured: 0 failures in 1000 serial runs against 7 in 1000 parallel; every occurrence identical,
with **both** delivery tests failing together and `callbacks run: 0` -- the pool never invokes the
callback at all, for either ring, and nothing arrives ten seconds later. The two delivery tests
alone do not reproduce it (0 in 1000); a third test has to be co-running, and the two that trigger
it both create an `EventDelivery` over a ring with nothing outstanding and drop it promptly.
Full figures, the reproducer, and what remains unestablished are in
[UNRESOLVED-TEST-FAILURES.md](UNRESOLVED-TEST-FAILURES.md).
**The next step is in [`windows-threadpool-sys`](../windows-threadpool-sys/src/wait.rs)**, not
here: both waits are registered on the default process threadpool, and one object's lifecycle
appears to stop other, unrelated armed waits from ever firing. Per the repository's mono-repo bug
policy, the fix belongs in that layer. `ThreadpoolWait`'s `Drop` has been read and only touches
its own object, so the mechanism is **not yet established** -- do not start from a guess about it.
**Narrowed further the same day, with a configurable trace.** The default pool is **not** wedged:
a probe at the moment of failure runs a plain work item and a brand-new armed wait, and measured,
both ran. The stalled waits were created and armed -- the trace shows it -- and the trampoline
never fires for any of them. **The stall is permanent by design**: the completion event is edge
triggered ([D-19](DESIGN-NOTES.md#d-19)), a stalled ring's queue never returns to empty, so the
setup signal is the only wakeup that ring will ever receive and losing it once ends delivery for
good. The open question is now narrow -- why an armed wait does not observe a signal raised just
before it was armed -- and a plausible mechanism is recorded in
[UNRESOLVED-TEST-FAILURES.md](UNRESOLVED-TEST-FAILURES.md) **as a hypothesis with no evidence
behind it**, together with the experiment that would settle it.
**The trace is compiled out unless `--features trace` is on**, and narrowed at run time by
`WINDOWS_THREADPOOL_TRACE`, because the instrument for a timing-dependent fault must not change
the schedule it measures. The flake still reproduces with it on, which was checked first.
**A cheaper interim mitigation exists and is a separate decision:** the harness could treat a
failure in these two tests as *inconclusive* rather than as `caught`, which would stop the false
clean bills without pretending the behaviour is understood.
## Moved 2026-09-25 15:02:00 -04:00 -- M26.10, the partial-transfer decision
### <a id="m2610"></a>M26.10 -- Decided: a completion may report fewer bytes than requested, so `RS-P-8` permits it and the caller owns the remainder. *(completed 2026-09-25 15:02:00 -04:00)*
The decision and its two-layer shape are in [DESIGN-NOTES.md](DESIGN-NOTES.md) -> D-69; the clause
itself is `RS-P-8` in [RESPONSE-SPACE.md](RESPONSE-SPACE.md). The durability half it spawned is
`M26.11`. The item as it read when it closed:
- [x] **M26.10** -- Decide whether a completion may report **fewer bytes than requested**, which
[RESPONSE-SPACE.md](RESPONSE-SPACE.md) currently lists as deliberately undecided. Found by
`M26.7`'s audit, and queued rather than settled there because the space's own instruction is to
"decide it before a resolver relies on either answer" -- which is a call about what this crate
tolerates, not a correction.
**The observation.** Five kernel assertions require a full transfer (`assert_eq!(transferred,
LEN)`), so the suite already answers the question by assuming one. `Completion::result` promises
only "the transferred byte count" and never a complete one, so nothing in the crate backs that
assumption up. The two are not in conflict today only because `M26.3`'s resolver reports
`Information: 0` for every operation and no resolver-driven test reads a transfer count -- so a
resolver and a kernel test would disagree about the same field and nothing would notice.
**Note one of the five is already right and should be left alone.**
[flush_barrier.rs](tests/flush_barrier.rs) checks the transfer explicitly as its *own
precondition* -- a short write would make every count in that test meaningless -- and says so.
That is the honest form of the assertion whichever way this is decided.
**What deciding it costs.** Permitting partial transfers widens what every consumer must handle
and would make the resolver able to produce them, which the properties in
[properties_under_every_resolution.rs](tests/properties_under_every_resolution.rs) would then
have to survive. Requiring complete transfers is a `Decided` constraint of the kind `RS-C-1`
already is, and would need a `CONFIRMS:` marker on whichever kernel test carries it -- the census
added in `M26.6` will then hold the two halves together.
## Moved 2026-09-25 16:21:00 -04:00 -- M26.11, the epoch log's handle requirements
### <a id="m2611"></a>M26.11 -- The epoch log states the capabilities it requires of a handle; the transfer requirement is checked, the durability requirement is a caller warranty. *(completed 2026-09-25 16:21:00 -04:00)*
The contract gained a fourth clause, `Requires`, beside `Guarantees`,
`DoesNotGuarantee` and `Assumes` -- the growth the module's own `Clause::ALL`
doc had anticipated, and its exhaustive `heading()` match is what forced the
edit. The decision not to gate the handle is [D-70](DESIGN-NOTES.md#d-70). The
item as it read when it closed:
- [x] **M26.11** -- Document the capability requirements [examples/epoch_log](examples/epoch_log) places on
the handle it is given, and let an unmet one surface at the operation that needs it. **No pre-flight
handle check** ([D-70](DESIGN-NOTES.md#d-70)).
**Why no check, which is the part worth not relitigating.** `M26.10` established that the ring permits a
short transfer because it never asks what kind of handle it was given ([RS-P-8](RESPONSE-SPACE.md),
[D-69](DESIGN-NOTES.md#d-69)), and the obvious next move is for the log to police what the ring does
not. It was examined and rejected: the check points the wrong way. A console handle, a pipe or a closed
handle is caught by `GetFileType`, but every one of those already fails loudly at the first positioned
write -- the check buys a better message. A RAM disk, a remote share, or a volume whose write cache is
not power-protected succeeds at every API call and silently fails to be durable, and no probe catches
the last of those at all. So the gate guards the failures that were already loud and misses every
failure that is silent, while implying a validation that did not happen.
**What to write instead.** The log's own contract, stated by the log rather than inherited from the
ring, listing what the handle must support: positioned I/O at explicit offsets; `FILE_FLAG_OVERLAPPED`;
`FILE_FLAG_NO_BUFFERING` together with the sector-aligned buffer, offset and length it requires; a
preallocated extent; that a successful write of `N` bytes transfers `N`; and that a completed flush
reaches stable media.
**Mark the last two for what they are.** The transfer requirement is the `RS-P-8` narrowing this log
earns by constraining its input -- it is checkable, and the log already compares transferred against
requested, so a mismatch is a contract violation to report loudly rather than a case to absorb. The
durability requirement is **a warranty the caller gives**, not a property this code can verify;
say so in those words, because a contract that merely sounds confident about it is how a silent
failure gets built on.
**Guard what is guardable, and do not pretend about the rest.** The transferred-against-requested
comparison is a real assertion and gets a sabotage case: suppress the comparison and the suite must go
red. There is no guard for the durability warranty, and the item is complete with that stated rather
than papered over. If the contract carries runnable examples they are compiled as doctests, per the
repository's rule that prose containing code must compile.
## Moved 2026-09-25 16:24:00 -04:00 -- M26 complete, all eleven items
Every item was archived individually as it closed, so what migrates here is the milestone's own
framing -- why a resolver over a permitted space was worth building, and the standing constraint
that the space must be wider than anything observed. The per-item stubs are dropped, carrying
nothing the entries above do not already hold.
## M26 -- Test against the space of kernel responses, not one observation of it
Queued by
[DESIGN-SESSION-2026-09-22-kernel-response-space.md](design-sessions/DESIGN-SESSION-2026-09-22-kernel-response-space.md),
which set out to answer `M24.1` and found a different technique instead.
**The idea.** A fake that models *what Windows does* freezes one run's testimony. A **resolver**
models what Windows is *permitted* to do, and a seed picks one resolution out of that space: which
operations finish inside `SubmitIoRing` and which pend, in what order completions are posted, which
fail. The assertions are then about **us** -- does this crate behave correctly under that resolution
-- and never about the kernel. There is no belief to be wrong about, which is why this dissolves the
mock objection rather than working around it.
**Justified by what it catches, not by hermeticity.** `M24` reaches a hermetic lib suite without it,
so this milestone has to earn its place on the defect class it detects: code that is brittle to
platform variation *inside* the permitted space. Nothing in the toolkit that preceded it detected
that -- [DESIGN-NOTES.md](DESIGN-NOTES.md#what-none-of-them-cover) records that the five techniques
which existed before `M26` all check this crate against *its own stated contract*. The resolver is
the sixth, added by this milestone.
**The standing constraint, inherited from the session.** The permitted space must be **wider than
anything observed**, and must not be derived from observation -- deriving it from what we have seen
closes the trap again. It is a deliberate specification of what we will tolerate, and therefore a
reviewable artifact rather than a recording.
## Moved 2026-09-25 20:47:00 -04:00 -- M28.1, what a caller receives
### <a id="m281"></a>M28.1 -- Decided: a push returns a `Copy` identity that owns nothing, and the ring returns the payload at its own pop -- `Token` is split, not moved. *(completed 2026-09-25 20:47:00 -04:00)*
The decision is [D-71](DESIGN-NOTES.md#d-71). The item asked between two shapes and the census
answer was both, because `Token` is an ownership guard and an identity welded together and only
the first is what `M28` moves. The item as it read when it closed:
- [x] **M28.1** -- **Decide what a caller receives, before writing any of it.** If the ring owns
the token then `Batch::write` can no longer hand one back, and the shape of what replaces it is
the whole design: an identity the caller matches later, or a claim that returns `(T, X)`
directly from the ring. The second makes drift impossible and is the point of the break; the
first is a smaller change that may not be worth breaking for. Settle it with the
`Token::claim_if` safety argument in hand, since that is what currently makes a mismatched
completion unclaimable.
## Moved 2026-09-25 21:12:00 -04:00 -- M28.2, bounding the contract oracle
### <a id="m282"></a>M28.2 -- `RingContract` is bounded by operations in flight: terminal entries are retired, and a capped history keeps a duplicate distinguishable from an unrecognised completion. *(completed 2026-09-25 21:12:00 -04:00)*
The decision, including why naive pruning was rejected as worse than the leak, is
[D-72](DESIGN-NOTES.md#d-72). The item as it read when it closed:
- [x] **M28.2** -- **Bound `RingContract` before anything depends on it more heavily.**
`operations: HashMap<usize, State>` is never pruned -- `observe_claim` marks an entry
`Completed` and keeps it -- so the oracle retains one entry per operation for the process's
life. Undocumented, and not visible in the sample because it appends 24 records. A long-running
consumer following the crate's own recommendation leaks. This blocks any design that checks by
default, which is why it is here rather than filed separately: `M23.3` reached for always-on
checking and this is what ruled it out. Decide whether completed entries are dropped, whether
`check_quiescent` needs them, and document the answer either way.
## Moved 2026-09-26 00:26:05 -04:00 -- M28.4.1d.2, the ring-owned inventory migration
### <a id="m2841d2"></a>M28.4.1d.2 -- Convert every consumer that can be converted before the token API is retired. *(completed 2026-09-26 00:26:05 -04:00)*
**Converted** (each its own commit, full gate green at every one):
`bounded_pop.rs`, `flush_barrier.rs`, `flush_barrier_stress.rs`, `calibration.rs`,
`completion_event.rs`, `event_delivery.rs`, `fault_injection.rs`, `handover.rs`,
`resolver_over_a_real_ring.rs`, `submission_lifecycle.rs`, `kernel_span.rs`,
`registration.rs`, `model_a_delivery.rs`, `model_b_multiplexed.rs`,
`epoch_log/logfile/tests.rs`, `ring_copy/engine.rs`, and the convertible half of
`failure_paths.rs`.
`registration.rs` was far smaller than its 27 `claim_if` sites suggested: most are
`PendingBufferRegistration` / `PendingFileRegistration` handles, which are a different
mechanism from `Token` and not in M28.4.1's push set. Eight were real token sites.
**Re-planned after execution: the rest of d.2 cannot precede d.3.** Four files remain,
and every one of them is blocked on `M28.4.1d.3` rather than on conversion effort --
converting them first would mean writing assertions against an API in the same commit
that another item deletes it. This is a genuine sequencing discovery, not a deferral
for convenience. Three are blocked specifically by [D-74](DESIGN-NOTES.md#d-74),
because their subject matter *is* what it removes; the fourth is blocked by the
retirement of `Pending<T, X>`, which is the same item but not the same reason:
- `generated_sequences.rs` and `properties_under_every_resolution.rs` both sample a
**claim-or-drop axis** as a generated dimension, with their own coverage assertions
(`coverage.deliberate_drops > 0`, `coverage.deliberate_leaks`). Converting them
*removes a dimension from the generated space*, which is a change to what the oracle
covers and belongs in the commit that retires `observe_deliberate_leak`.
- `failure_paths.rs`'s leak-ordering test asserts `Violation::LeakedToken` directly. It
is not convertible at all -- it is d.3's to delete, alongside the variant.
- `epoch_log/append.rs` uses the crate's `Pending<T, X>`, which d.3 retires, and raises
a question d.3 must answer first: it takes `&mut IoRing` rather than owning the ring,
so an appender cannot reach the inventory without the ring's payload type reaching
its own signature. **That is the first consumer to feel `D-73`'s type parameters at
an API boundary**, and it should be decided rather than discovered.
**Correction, same day**: an earlier version of this list also called
`epoch_log/checkpoint.rs` and `epoch_log/strategy.rs` blocked -- and omitted
`strategy.rs` from d.3's list while doing so, which would have lost it entirely.
Neither was blocked: both own their rings and use the plain sidecar pattern, and
neither samples the claim-or-drop axis that couples the others to `D-74`. The
over-broad claim came from reading `checkpoint.rs`'s own `Pending` struct as the
crate's `Pending<T, X>`; they are unrelated types with the same name. Both are now
converted. The lesson is narrow and worth keeping: *"blocked" is a claim about a
specific dependency, and naming that dependency is what makes it checkable* -- the
three files that really are blocked each name the item that unblocks them.
**Two API gaps the conversion found, both now closed**: `flush_raw_owned` did not exist
(eleven token pushes had ten owned counterparts), and `pop_within` had no reclaiming
form. A third is a real defect, fixed: `HeldCompletion` collapsed `Option<(T, X)>` via
`entry.payload.map(..)`, which discarded the sidecar of every *bufferless* operation --
so a flush could never say which group of writes it belonged to. It is
`Option<(Option<T>, X)>` now: the outer option is whether the ring stowed anything, the
inner one whether what it stowed included a buffer.
## Moved 2026-09-26 00:31:44 -04:00 -- M28.4.1d.2b, the cost of a single payload type
### <a id="m2841d2b"></a>M28.4.1d.2b -- Record what a single payload type costs a consumer holding heterogeneous buffers, and decide whether anything is owed. *(completed 2026-09-26 00:31:44 -04:00)*
Record what a single payload type costs a consumer holding
heterogeneous buffers, and decide whether anything is owed.
Found converting [epoch_log/logfile/tests.rs](examples/epoch_log/logfile/tests.rs): one
test wrote a `NumaBuffer` and a `Vec<u8>` through the same ring. A ring holds one payload
type ([D-73](DESIGN-NOTES.md#d-73)) and [D-4](DESIGN-NOTES.md#d-4) forbids erasing it, so
that consumer's choices are an enum payload implementing `IoBuf`, or a ring per buffer
type. The test took a ring each, which was free there because its two writes were already
sequential -- but that will not generally be true, and a consumer multiplexing buffer
types over one ring has no cheap answer today.
This is not a request to relax `D-73`; the seal is load-bearing and the single type is
what makes the payload come back without a cast. It is a request to **state the
consequence where a consumer will meet it** rather than leaving them to discover it from
a type error, and to decide whether the crate should offer an `IoBuf` enum helper or
simply document the two options. Gated on `M28.4.1d.2` finishing, so the full shape of
the problem is visible first.
**Decided: document, build nothing.** The two ways round a mixed payload -- a ring per
buffer type, or an enum payload carrying its own `unsafe impl IoBuf` -- are now stated on
`IoRing::with_inventory`, where a consumer meets the parameter rather than a type error,
and the decision with its evidence is recorded in [DESIGN-NOTES.md](DESIGN-NOTES.md#d-73).
The premise this item was written on did not survive the rest of the conversion. It asserted
that a consumer multiplexing buffer types over one ring "will not generally" find a ring
each free. Of the seventeen ring types the conversion introduced, one carried two buffer
types through what had been a single ring, and its writes were already sequential. The
prediction was made from a single observation and tested against sixteen more.
## Moved 2026-09-26 10:26:56 -04:00 -- M28.4.1d.3, the commit that ended the break
### <a id="m2841d3"></a>M28.4.1d.3 -- Retire the token API and apply D-74. *(completed 2026-09-26 10:26:56 -04:00)*
**Also carries the three consumers d.2 could not convert** (see
`M28.4.1d.2`): `generated_sequences.rs` and `properties_under_every_resolution.rs`
(each samples claim-or-drop as a generated dimension, with a coverage assertion that
the axis was exercised), and `failure_paths.rs`'s leak-ordering test (asserts
`Violation::LeakedToken` -- deleted here, not converted). Each is blocked on this item
rather than on effort, because what they assert is what this item deletes.
**Decide `epoch_log/append.rs`'s boundary question before converting it**: it takes
`&mut IoRing` rather than owning the ring, so reaching the inventory would put the
ring's payload type into the appender's own signature. It is also the last user of the
crate's `Pending<T, X>`, so retiring that type and answering this question are the same
piece of work.
Retire every `Token`-returning push -- there is now exactly one per `*_owned` form,
since `flush_raw_owned` closed the last gap -- along with `Token` itself and
`Pending<T, X>`; apply [D-74](DESIGN-NOTES.md#d-74) in the same commit -- drop
`Violation::LeakedToken`, `State::Leaked`, `observe_claim` and
`observe_deliberate_leak`, and collapse the two push states.
**Dropping `observe_claim` reaches past the three unconverted files.** `M28.4.1d.2`
left its calls in place in [handover.rs](tests/handover.rs),
[kernel_span.rs](tests/kernel_span.rs),
[submission_lifecycle.rs](tests/submission_lifecycle.rs) and the converted half of
[failure_paths.rs](tests/failure_paths.rs) -- deliberately, because the oracle still
modelled a claim while both token models were live, and a converted test that stopped
reporting one would have looked like a leak. Those calls have nothing left to report
once the pop is the claim, so they go with the API rather than being rewritten.
[contract.rs](src/contract.rs) also carries a doctest that calls it, and
[pending.rs](src/pending.rs) calls both -- the latter disappears with `Pending<T, X>`. Leaving them would describe
a hazard the API no longer has, which is worse than a gap: a reader would go looking for
the way to leak a token and not find one. **This is the commit that ends the break.** **Convert all of them or
none**: converting a few relocates the duplication rather than removing it, which is the
lesson `win-numa-sys` recorded the same day when it moved one `VirtualAllocExNuma` and
left the other. This is the step that ends with one token model.
**Done.** The token API is gone: `pending.rs` deleted, `Token` stripped from
`token.rs` (`OperationId` survives, with its own tests -- deleting the file would have
left the surviving type untested), ten `Token`-returning pushes and the private
`finish_push` removed from `batch.rs`, and `try_pop`/`pop_within` now name the one
reclaiming pop each. D-74 applied in full.
**The count in this item was wrong, and it was my own correction that broke it.** An
earlier revision said "ten"; `M28.4.1d.2` "corrected" it to eleven from a command that
counted `io::Result<Token<...>>` occurrences. Eleven was the occurrence count; **ten** was
the push count, because the eleventh was the private `finish_push` helper. The original
was right. The rule that a census must come from a command holds -- but a command is only
as good as what it is pointed at, and a return type is not a push.
**Two guards were disturbed by the rename, and one of them had gone blind.**
[check-ring-tests.ps1](../../tools/check-ring-tests.ps1) detected ring-opening tests by
grepping `IoRing::new`, so every ring moved onto `with_inventory` vanished from its
inventory -- reported as five REMOVED entries, which the script's own text calls
"progress". It was a guard silently stopping guarding. Fixed to match the constructor
*name* (which also catches a type alias like `PipeRing::with_inventory`), and the fix
immediately surfaced two ring-opening tests the old pattern had never matched at all.
[check-borrow-surface.ps1](../../tools/check-borrow-surface.ps1) reported the one genuine
change, a removal, answered in [DESIGN-NOTES.md](DESIGN-NOTES.md#borrow-surface-audit-m2841d3).
**The documentation sweep was larger than the code change.** Retiring `Token` broke 73
intra-doc links and left 35 prose references describing a model that no longer exists --
including the README's headline example, which is the first thing a consumer reads. All
rewritten. The census in [response_space_census.rs](tests/response_space_census.rs) also
caught two unbounded `try_pop` spins that the rename made visible to it; both are now
bounded pops.
## Moved 2026-09-26 11:54:28 -04:00 -- M28.4.2, sabotaging the inventory
### <a id="m2842"></a>M28.4.2 -- Sabotage the inventory: a push that does not record, and a pop that does not retire, must both turn the suite red. *(completed 2026-09-26 11:54:28 -04:00)*
Sabotage the inventory: a push that does not record, and a pop that does
not retire, must both turn the suite red.
The pop half now has a worked precedent to follow rather than invent: `M28.4.1d.2`
made both public pops retire their entry, and verified it by re-injecting the
stranding -- five `registration.rs` tests went red, and all eighteen passed once it
was restored. Record that as a case in `sabotage.json` rather than leaving it as a
command that was run once and discarded.
**Five cases added, two repaired, three removed.** The manifest is the artifact; what
follows is what running it taught.
**The sweep found that `M28.4.1d.3` had broken the manifest, and nobody noticed because
nobody ran it.** Two cases patched `src/pending.rs`, which that commit deleted, so the
harness aborted before testing anything. A third re-injected the `M22.2` ordering defect
-- checking a write's result before claiming -- which is no longer expressible. Two more
were silently stale: `M23.5`'s rundown guard had been re-indented, and `M26.7`'s
restated-pop site now destructures a tuple, so both matched zero times. That is exactly
the decay the harness exists to catch, arriving in the harness itself.
**One case had to be rewritten because it was caught for the wrong reason.** The
push-does-not-record sabotage originally dropped the entry, which frees the buffer while
the kernel writes into it: the suite failed with `STATUS_HEAP_CORRUPTION` (0xC0000374).
That scores as `caught` because the process died, not because a test noticed, and
whether it dies at all depends on allocator behaviour -- the repository's own cargo-mutants
guidance says to treat a crash-caught result as uncovered. The patch now *leaks* the entry
instead. The ring still records nothing, `ring.held()` and every `held.expect(..)` go
red by assertion, and the two defects are separated: this case asserts the ring failed to
**record**, which is a claim a test can make, rather than that a freed buffer is unsound,
which no sabotage can establish reliably.
**The two pops are asserted separately on purpose.** They have separate bodies, and a fix
to one has already failed to reach the other: the pair existed as `try_pop`/`try_pop_held`
and `pop_within`/`pop_within_held` precisely because the reclaiming behaviour was added
one at a time, and `pop_within` went a full milestone without it.
**Two cases moved rather than died.** `Pending<T, X>`'s "push stops telling the oracle"
guarantee survived its type -- `Appender` keeps a `RingContract` and must observe both
ends -- so it is relocated to the appender, alongside a new completion-side case that
replaces the retired `M22.2` ordering one.
## Moved 2026-09-26 13:05:21 -04:00 -- M28.5, the tokenless push
### <a id="m285"></a>M28.5 -- Answer the tokenless push. *(completed 2026-09-26 13:05:21 -04:00)*
**Answer the tokenless push.** `flush_raw` returns a bare `usize` and
`epoch_log`'s commit path depends on it, because a flush has no buffer and a *borrowed*
`RawHandle` gives its token nothing to guard. An inventory the ring owns has to say what it
does with operations that have no token -- `RingContract` already models them separately with
`observe_tokenless_push`. Note this may dissolve rather than need solving: if the sample owned
a `SharedFile` instead of passing a `RawHandle` it could use the safe `flush` and get a token,
which `M25.3` reopens anyway by changing how the log is opened.
**Most of the question had already dissolved, but not the way this item guessed.** It
expected the sample to stop passing a `RawHandle`. What actually happened is that
`D-74` collapsed the two push states, which left `RingContract::observe_tokenless_push`
**byte-identical** to `observe_push`, with a doc describing a leak violation that no
longer exists. Two names for one behaviour is what `FAIL FAST` rule 1 is about, so the
tokenless one retired and its eight call sites moved.
**What genuinely remained was the meaning of the outer `None` from a pop**, and the
public doc was wrong about it: it said `None` meant a contract violation, when a
`_raw` flush legitimately produces one. Recorded as [D-75](DESIGN-NOTES.md#d-75). The
answer is that the ring cannot separate the two causes and does not try -- the caller
can, because it chose the push. Closing the gap was considered and rejected: an entry for
every raw push needs an `X` the caller never supplied, which is the residue `D-73`
named, and the `_owned` forms already serve a caller who wants one.
**The contract is now asserted rather than only documented**, bidirectionally on one ring,
because a test showing only one side would pass against a ring that answered the same way
every time. Two things came out of verifying it by sabotage. Patching `try_pop` left the
test green -- it pops with `pop_within` -- which is the same separation `M28.4.2`
asserts with two cases rather than one. And the first manifest case was crash-caught
(STATUS_STACK_BUFFER_OVERRUN): manufacturing a payload needs an `X` from nowhere, and
`std::mem::zeroed()` on a ring whose sidecar is `&'static str` is a null reference.
The case is arithmetic now, and fails by assertion on every ring.
## Moved 2026-09-26 13:30:33 -04:00 -- M28.6, sweeping what the break made false
### <a id="m286"></a>M28.6 -- Sweep what the break makes false. *(completed 2026-09-26 13:30:33 -04:00)*
**Sweep what the break makes false**, including the README's ring examples, the
`D-4` detail section, and every rustdoc that tells a caller to match a completion against a
held token -- `Completion::user_data` and `IoRing::push_raw` both do, and they are the evidence
D-55 rests on, so they are the first things the change invalidates.
**The named targets were mostly already done, and the sweep's value was the rest.**
`M28.4.1d.3` had fixed the README example and reworded `Completion::user_data`; what
remained of the list was `D-4`, `D-55`, and a nuance the item got slightly wrong.
**`D-4`**: its load-bearing half -- no slab entry, no box, **no type erasure** -- is
unchanged and is what `D-55` and `D-73` both rest on. What moved is who holds the
buffer, and the forget-on-failure mechanism did not disappear so much as become the
crate's: `Drop for IoRing` forgets the inventory for exactly the reason a dropped token
used to forget its value. Amended in place rather than superseded.
**`D-55`**: landed, with two of its own assertions corrected. Its evidence was that "its
own rustdoc twice instructs a caller to match it against a held `Token`" -- but one of
those two sites was `IoRing::push_raw`, which *still* says to match by hand, and is
right to: a raw push creates no entry, which is `D-75`'s subject. The defect was in the
tokened path; the raw seam only looked the same. And `generated_sequences.rs` no longer
carries the eight-token-type enum that was the strongest evidence the erasure objection
was false -- the argument stands, the artifact is gone, and a reader sent to look for it
should know before going.
**Six live false claims in the examples**, none of them on the item's list: `append.rs`
and `strategy.rs` still told a reader that claiming a token returns its arena slot,
`model_b_multiplexed.rs` twice said the kernel writes through buffers "those tokens
own", `checkpoint.rs` documented a map field that no longer holds a token, `main.rs`
asserted "every token was claimed", and `append/tests.rs` pointed at a library test
`M28.4.1d.3` had deleted.
**Three defects this sweep found in `M28.4.1d.3`'s own prose sweep**, which is the part
worth remembering. That sweep replaced 35 `Token` references by table-driven
substitution, and three of the replacements were wrong in ways no gate could catch:
`buf.rs` was left with a sentence broken mid-clause (`-- a` followed by `The ring
has no Drop...`), and two `ring.rs` sites lost their backticks or kept a stale
"every claim path", because a PowerShell double-quoted replacement string eats backticks.
**Prose does not compile, and rustdoc only checks links.** A substitution sweep over
documentation needs its output read, not just its exit code.
`MUTATION-SURVIVORS.md` and the `M18.1` borrow-surface table are dated captures and
were **annotated, not edited** -- the record of what was measured then is the thing worth
keeping, but a reader should not learn from a compile error that an item is gone.
## Moved 2026-09-26 16:39:03 -04:00 -- M28.7, decided against
### <a id="m287"></a>M28.7 -- Decide whether the ring should check conservation itself. *(completed 2026-09-26 16:39:03 -04:00)*
**Decide whether the ring should check conservation itself, rather than a
caller driving `RingContract`.** Raised by [D-74](DESIGN-NOTES.md#d-74) and deliberately not
taken there. Once the inventory is the only push path, a pop already knows whether the identity
was stowed, and `held()`/`outstanding()` are both the ring's own numbers -- so
`UnexpectedCompletion`, `DuplicateCompletion` and `Outstanding` are all answerable without a
caller reporting anything.
**Why it is a decision and not a cleanup.** [DESIGN-SESSION-2026-09-23-pending-inventory.md](design-sessions/DESIGN-SESSION-2026-09-23-pending-inventory.md) recorded the structural
complaint that an oracle's "value depends on being driven correctly by the very code it checks",
and this would answer it. But `RingContract` is deliberately *not* wired into `Batch` ([its own
rustdoc](src/contract.rs) says why): a ring driven through `push_raw` bypasses this crate's
bookkeeping entirely, so an internal hook would cover less than it appears to, and a consumer
validating its own harness needs to drive the same rules from outside. Moving the checking
inward trades that away. Gated on `M28.4.1d`.
**Decided: no, and it schedules no work.** Recorded as [D-76](DESIGN-NOTES.md#d-76).
`RingContract` is unchanged.
**Of the four violations, the ring can answer one.** `Outstanding` it already does --
`run_down` waits on that number and `Drop` debug-asserts it.
`UnexpectedCompletion` it **cannot**, because [D-75](DESIGN-NOTES.md#d-75) -- decided in
this same milestone -- established that a `_raw` push creates no entry, so "the ring
holds nothing for this identity" is legitimate rather than a violation.
`DuplicateCompletion` needs a finished-identity history the ring does not keep.
`BufferStillInUse` reads a count living in `RegisteredBuffers`, which the *caller*
owns.
**The decisive argument was the failure mode, not the coverage.** An internal check
reporting `UnexpectedCompletion` for an entry-less completion would fire on every
`_raw` push -- 13 call sites across 8 files, including the sample's commit path. Not a
check that covers less than it appears to; a check that is wrong about correct code.
**Two corrections to the reasoning as this item stated it.** Its `push_raw` argument
rested on a seam with **zero** call sites; the real bypass is `flush_raw`/`cancel_raw`.
And the structural complaint it would have answered is in this tree almost entirely a
*test* concern: of 37 `RingContract` instances, 36 are tests and one is a consumer.
**What reopens it:** `M23.3` had ruled an always-on checked inventory out because the
oracle grew without bound, and `M28.2` removed that obstacle -- so memory is no longer
the reason. If `_raw` pushes ever gain inventory entries, `UnexpectedCompletion`
becomes answerable and the question is live again.
**Also fixed while here:** three links to `src/pending.rs`, which `M28.4.1d.3`
deleted -- this item's own, and one each in `D-71` and `D-74`. All now point at the
design session that holds the material.
## Moved 2026-09-26 17:24:22 -04:00 -- M28.8, the duplication closed properly
### <a id="m288"></a>M28.8 -- Give `RingContract` an in-flight count, and delete the counter that duplicates it. *(completed 2026-09-26 17:24:22 -04:00)*
**Give `RingContract` an in-flight count, and delete the counter that
duplicates it.** [`Appender`](examples/epoch_log/append.rs) keeps `outstanding: usize`
next to its `RingContract`, incremented at every push and decremented at every
completion -- two hand-driven lines at each of two sites. The oracle already knows that
number: it is what [`RingContract::check_quiescent`](src/contract.rs) computes
`Violation::Outstanding` from. The counter exists only because the oracle has no
accessor for it, and [main.rs](examples/epoch_log/main.rs) needs `in_flight() > 0` for
its two drain loops.
Add `RingContract::in_flight()`, have `Appender::in_flight` read it, delete
`Appender::outstanding`. One source of truth. Sabotage the accessor to confirm the
drain loops genuinely depend on it rather than terminating for another reason.
**Why this is worth an item.** It is the residue of the complaint that started
[D-55](DESIGN-NOTES.md#d-55): an oracle driven *beside* a consumer's own record of the
same event, "a restatement in the repository's own terms, and one that can drift in both
directions" -- see
[DESIGN-SESSION-2026-09-23-pending-inventory.md](design-sessions/DESIGN-SESSION-2026-09-23-pending-inventory.md).
`M23.3`'s `Pending<T, X>` had closed it, by making one call drive both the map and the
oracle. **`M28.4.1d.3` retired `Pending<T, X>` and re-opened it in a smaller form**, and
that went unnoticed because a `usize` counter looks nothing like the map it replaced.
This is not [M28.7](COMPLETED-CHECKLIST.md#m287) in another guise: nothing moves into
the ring, so [D-75](DESIGN-NOTES.md#d-75)'s objection does not apply.
**Two hand-written copies, not one.** The appender''s counter was the known one. Looking
for it found a second in
[properties_under_every_resolution.rs](tests/properties_under_every_resolution.rs):
`contract_outstanding` counted `Violation::Outstanding` out of `check_quiescent()`,
allocating a `Vec` to ask a map its length. Its own doc said "derived from the contract
rather than counted here" -- the right instinct, implemented by hand because the type
offered no accessor. Both now call `RingContract::in_flight`.
**The verification this item specified was aimed at the wrong place, and measuring it
said so.** The item asked to confirm `epoch_log`''s drain loops depend on the accessor.
They do not. `main.rs` loops on `appender.in_flight() > 0 || committer.in_flight() > 0`
and the committer keeps its own map, so the second operand holds the loop open while each
pass drains both producers -- the sample runs clean with the accessor hard-coded to zero.
That is exactly the "terminating for another reason" the item warned about, found by
doing the check rather than assuming it.
What does catch it is `P-4`, which compares `IoRing::outstanding()` against the
contract''s own count -- the agreement this accessor asserts. Recorded as a case in
[sabotage.json](sabotage.json) so it is re-run rather than discarded, with the
wrong-place finding written into its `why`.
**The appender''s half of that loop condition is not dead**, only unexercised by the
sample''s path: it would matter with appends outstanding and no commit pending. Left
alone rather than "simplified".
## Moved 2026-09-26 19:11:20 -04:00 -- M26.12, which turned out not to be a race at all
### <a id="m2612"></a>M26.12 -- Find why a signal raised just after `wait.arm` can be lost, and fix it. *(completed 2026-09-26 19:11:20 -04:00)*
**The item's premise was false, and measuring it said so.** It asked for the mechanism of a
wakeup lost in a window after arming. There is no window. Isolating `SetThreadpoolWait` from
the ring entirely lost no wakeups in arm-then-signal order, none with the 50 ms pause, and
none in the signal-then-arm order the API documents against; a real ring holding a backlog
lost none with no pool, and none under a wait armed exactly as `EventDelivery::new` arms one.
Full figures in [RESOLVED-TEST-FAILURES.md](RESOLVED-TEST-FAILURES.md).
**The cause was the narrow finding review reported on PR #108**, which the investigation had
set aside as too small. `attach_completion_event_unsignalled` reported the setup signal "still
owed" only when that call had performed the attachment, so a caller that attached earlier and
consumed that signal handed over a non-empty queue with no wakeup pending -- and D-19's edge
cannot re-arm without the queue first returning to empty. Deterministic, not rare. The
reproducer's own report had said so all along: `callbacks run: 0` is a signal never raised.
**Two claims in the record were wrong in the same direction**, and both had been treated as
measurement: that signalling unconditionally did not fix it, and that a 50 ms sleep did.
Signalling unconditionally passes 30 of 30. The likely cause of the bad measurement was
reproduced by accident during this work -- a stale test binary, after a file restore that
preserved mtime let cargo skip the rebuild. Recorded as reconstruction, not finding.
**Fixed** by deleting the flag: attach and signal stay separate so a caller can arm in
between, but both `IoRing::completion_event` and `EventDelivery::new` raise the signal
unconditionally. Recorded as [D-77](DESIGN-NOTES.md#d-77), which also corrects the one clause
of [D-68](DESIGN-NOTES.md#d-68) the measurements contradict -- D-68's ordering stands, since
it binds to what `SetThreadpoolWait` documents rather than to what this host tolerates.
**Guarded** by two sabotage cases in [sabotage.json](sabotage.json) and by a new test,
`a_repeat_call_signals_again_once_the_earlier_signal_has_been_consumed`, which is the only
test that consumes both earlier signals and so the only one that can observe the ring-level
half -- confirmed by sabotage to be the single test that fails when the old behaviour returns.
**Left open:** `M26.9`'s intermittent stall was real and its fix is kept, but the mechanism
D-68 offered for it does not survive these measurements. Queued as `M26.13`.
## Moved 2026-09-26 21:55:20 -04:00 -- M26.13.1, the trace coverage the stall capture lacked
### <a id="m26131"></a>M26.13.1 -- Paired entry/exit records added to all five pool trampolines, the re-arm, and the test's post-mortem path; a sabotage sweep confirms each is load-bearing. *(completed 2026-09-26 21:55:20 -04:00)*
**What the item asked for, and what was added.** `M26.13`'s capture had no records at all
between 0.0023s and 5.0087s, so the stall itself was unobserved. The item named four gaps in
[windows-threadpool-sys](../windows-threadpool-sys/src/wait.rs) and one in this crate's test,
and all five are closed:
1. the wait trampoline's **exit** (`trampoline-left`), paired with the entry that existed;
2. `rearm_reporting`'s two ends -- `rearm-entered` recorded *before* the suppression lock it
can block on, then `rearm-left` or `rearm-suppressed`, so a re-arm parked on that mutex is
distinguishable from one that never started;
3. work-item callbacks in [work.rs](../windows-threadpool-sys/src/work.rs), both ends, plus
`created` and `submitted` -- without the submit, a `trampoline-entered` has nothing to be
late relative to;
4. `ThreadpoolWait::drop`'s stages, confirmed rather than changed: the interval from
`drop-drained` to the handle's own close holds exactly `CloseThreadpoolWait`, the context
free, and the `target` field drop that runs the instant the body returns. `drop-begin` and
`drop-closed` now carry the target handle, because nothing after `drop-closed` can;
5. the post-mortem in [event_delivery.rs](tests/event_delivery.rs), under a `postmortem`
target: the `recv_timeout` expiry, the `outstanding()` call **bracketed** (it takes the ring
mutex the delivery callback drains under, so an `outstanding-begin` with no
`outstanding-read` is itself a finding), the liveness probe's phases, and the second wait.
**Extended past the four the item listed**, because the item's own title says *every*
pool-invoked functor and there are five: the `io`, `timer`, and `timer-periodic` trampolines
now record both ends too. A reader of a capture can then take silence to mean no functor ran,
rather than no *traced* functor ran.
**Verified by sabotage, not by reading.** Each of the thirteen new library call sites was
deleted in turn and the guard re-run: all thirteen `caught`, and a control case (deleting the
unasserted `suppress-and-disarm` record) `survives`, so the guard is sensitive to these call
sites rather than merely sensitive. The post-mortem records were verified by making the
expiry happen for real -- `DELIVERY_BOUND` cut to one nanosecond -- which exercises the whole
failure path rather than a proxy for it, and every `postmortem` record appeared in the
captured report.
**The guard is `every_pool_trampoline_records_both_of_its_ends`** in
[trace/tests.rs](../windows-threadpool-sys/src/trace/tests.rs), with the `io` half asserted
inside the existing `pending_read_completes_through_the_callback`, which already owns the only
overlapped-read exercise in that crate. It asserts **only** when `WINDOWS_THREADPOOL_TRACE` has
narrowed the trace, because the filter is read once per process and cached, so a test cannot
set it without racing every other test in the binary; the test says so rather than leaving a
reader to infer it from a green run. Run it as `$env:WINDOWS_THREADPOOL_TRACE = '*'` with
`cargo test -p windows-threadpool-sys --features trace`.
**Not recorded in a sabotage manifest.** [tools/run-sabotage.ps1](../../tools/run-sabotage.ps1)
has no way to set an environment variable for a case's test command, and this guard is
env-gated by construction, so a manifest entry would run in a build where the test returns
having asserted nothing. The sweep above was therefore run by hand and is not persisted; closing
that gap means either env support in the harness or a CI job that builds with `--features trace`,
and both are decisions to raise rather than take in passing.
**Left to M26.13.** The report's "it arrived, N past the bound" wording is still measured from
the start of the post-mortem; `second-wait-begin` / `second-wait-ended` now make that interval
readable in the trace, but the wording fix is M26.13's own, not this item's.
## Moved 2026-09-26 22:13:01 -04:00 -- M26.13.2, in which the buffer question turned out to be two questions
### <a id="m26132"></a>M26.13.2 -- The buffer is per-process, so the population was never "the full suite"; the capturing binary emits 89 records, the threadpool crate's own 14061, and an overflow now announces itself. *(completed 2026-09-26 22:13:01 -04:00)*
**The item's premise was wrong in a way that mattered.** It asked for one or two full-suite
runs, looking for the `... earlier record(s) dropped` line. That line appears only in a
`dump()`, and `dump()` is called from exactly one place in this workspace --
[event_delivery.rs](tests/event_delivery.rs)'s `trace_section`, on a failing delivery. A
passing full-suite run produces no dump, so the prescribed check could not have answered the
question it was asked to answer, whatever the answer turned out to be.
**The buffer is also per-process, and every test binary is its own process.** So "the full
suite" is not one population, it is one per binary, and only the binary that takes a capture
can carry a truncated one. Measured with the trace narrowed to everything, the two that record
at all are four orders of magnitude apart:
| Binary | Records offered | Against a capacity of 8192 |
|---|---|---|
| `windows-ioring-sys`' `event_delivery` (7 tests, the only one that captures) | 89 | never fills |
| `windows-threadpool-sys`' lib tests (224 tests) | 14061 | 5869 evicted -- the first 42% of the run |
**So the answer to the question as asked is that no capture was ever at risk**, and the answer
to the question underneath it is that the buffer was nonetheless losing most of a traced run
of the crate that owns it -- the half nobody could see, because nothing there dumps.
**Three changes, in the order they matter.**
1. **An overflow now announces itself to stderr, once, as it happens.** This is what makes the
check re-runnable by anyone rather than a measurement that rots: a run that never dumps is
now still told that its capture no longer reaches back to the start. It is the one place
this module formats on the traced path, and it is reached only after `CAPACITY` records have
already been taken. Both suites were re-run under it afterwards with zero announcements.
2. **`CAPACITY` raised from 8192 to 65536**, which is 4.6x the larger measured population
rather than a round number. The buffer is allocated only when something is actually
recorded, so a build carrying the feature with the environment unset still pays nothing;
a run that asks for a trace pays about 4.7 MB.
3. **The eviction policy was lifted out of the `trace` feature gate** into a `Buffer` type
tested at a capacity of four. It now runs in every developer's `cargo test` and in CI, with
no feature flag and no environment variable -- where the rest of this facility's guards
need both. Filling the *real* buffer in a test was never an option: it would evict every
record the rest of the suite had just taken.
**Verified by sabotage.** Eight behavioural mutations of the policy -- evict newest instead of
oldest, stop counting losses, announce every eviction, announce none, let a clear leave the
announcement spent or the count standing, drop the zero-capacity guard, let a full buffer stop
recording -- were each injected in turn and all eight caught. A control that loosens `>=` to
`==` survives, which is correct: the buffer is never allowed past its capacity, so the two
forms cannot differ.
**A side effect worth naming.** `M26.13.1`'s guard reads the global dump inside the very
binary that was overflowing. It asserts immediately after its own exercise, so the newest
records were always its own -- but the hazard was real and is now removed outright rather than
left improbable, because that binary no longer wraps.
**Not covered.** The glue in `record` -- that it passes `CAPACITY` and forwards the push's
answer to the announcement -- has no test. Reaching it needs 65536 real records through the
global buffer, which is the exact thing the `Buffer` split exists to avoid. The decision it
carries is tested; the five lines that carry it are not.
## Moved 2026-09-26 22:34:12 -04:00 -- M26.13.3, which found what ends the stall and moved the search
### <a id="m26133"></a>M26.13.3 -- The stall ends when a work item is queued to the pool, at no other time, and the five-second coincidence is the probe's timing rather than a timer. *(completed 2026-09-26 22:34:12 -04:00)*
**The question the item set** was what record immediately precedes `trampoline-entered` at the
five-second mark. In all nine captures taken for this item it is **`work submitted`**, from the
pool-liveness probe the delivery test runs after giving up, a few hundred microseconds earlier.
**That alone would have been a correlation**, and `M26.13` had already rejected the probe as an
explanation once, on the grounds that the probe's own `wait created` is stamped *after* the first
`trampoline-entered`. It is: in all nine. The resolution is that it is the **work** half of the
probe that precedes dispatch, not the wait half -- which is why looking at the wait half ruled the
probe out.
**Two further configurations turned the correlation into a cause**, each a temporary edit to the
test that delays only what happens *after* the test has already given up:
- **A two-second quiet period** inserted before anything touches the ring or the pool. Zero records
of any kind appear inside it, in all six captures that have one, and the stall ends two seconds
later than before -- tracking the probe, not the clock. So the stall does not end on a timer, and
the test thread waking from `recv_timeout` is not what ends it either.
- **A one-second gap between `ThreadpoolWork::new` and `submit`.** The work object exists for a full
second in the same silence; dispatch follows the `submit`. So it is the queuing, not the creation.
Captures, the generated figures, and the exact edits:
[measurements/2026-09-26-what-releases-the-stall/](../windows-threadpool-sys/measurements/2026-09-26-what-releases-the-stall/README.md).
The figures there are generated from the captures rather than transcribed -- five of nine were wrong
when the table was first typed by hand, which is recorded in the README because it is the same
transcription failure this repository's instructions already warn about.
**What it cost to find:** 3 failures in 1200 runs, 3 in 1312, and 3 in 1070, at about 0.05s a run.
The rate is in the same range as the one in three hundred recorded before this trace existed.
**It moved the search, so `M26.13` was re-planned in the same commit.** That item was framed as a
ring question and its four queued experiments all added ring ingredients to a pool-only isolation.
The remaining question is a pool question -- why a queued wait callback waits for an unrelated
`SubmitThreadpoolWork` -- so those four are withdrawn as the *next* step (recorded, not deleted, and
still available if the new line dead-ends) and four pool-side experiments replace them, starting
with the pool's worker-thread supply and with whether a private pool carrying a non-zero minimum
stalls at all.
**Deliberately not concluded.** The captures do not observe the pool's own thread accounting: the
gap between `SetThreadpoolWait` returning and the trampoline being entered is inside the pool, where
this workspace's trace cannot see. The supply reading is the obvious hypothesis and it is queued as
an experiment rather than written up as a finding.
**One instruction in the item was not needed and is recorded so it is not re-derived.** It warned
against polling the completion event to see whether it was still signalled, because the event is
auto-reset and a successful poll would consume the signal and manufacture the bug. Nothing here
polled it -- the question was answered from record ordering alone -- so the `NtQueryEvent`
dependency decision it flagged stays unraised.
## Moved 2026-09-27 11:39:37 -04:00 -- M26.13.4, which moved the fault out of this crate
### <a id="m26134"></a>M26.13.4 -- Experiment 3: nothing else releases it, and nothing else dispatches either -- a wait, a timer and an I/O object armed during the stall all sit undispatched, so the fault is pool-wide and not this crate's. *(completed 2026-09-27 11:39:37 -04:00)*
**The question asked** was whether any pool poke releases the stall or only a work submit. Five
configurations, three captures each, with a work-submit control in every one so that a poke which
did nothing could not be confused with a stall that had already ended:
[measurements/2026-09-27-which-poke-releases-the-stall/](../windows-threadpool-sys/measurements/2026-09-27-which-poke-releases-the-stall/README.md).
**The answer to the question as asked:** only a work submit. A fresh wait armed and signalled, a
fresh timer due in a millisecond, and a real overlapped read that completed each leave the delivery
stalled for the full two-second observe window; the work submit releases it within microseconds.
Fifteen captures, no exceptions.
**The answer underneath it is larger, and it is why this item matters more than its own question.**
The poke's *own* callback does not run either. A wait, a timer and an I/O object created and armed
**during** the stall, with no connection to any ring, all sit undispatched for two seconds and then
run only once a work item is submitted. So the stall is not a property of the ring's wait, and not
of waits: the process thread pool is dispatching **nothing**, of any kind, and a work submit
restarts everything at once.
**`M-T1.1` is what made this readable, one day after it was queued.** The claim "the poke was
established at 7.01s and dispatched at 9.01s" rests entirely on the `created` and `armed` records
that item added to the timer and the I/O object -- `timer created` / `timer armed` at 7.016s against
`timer trampoline-entered` at 9.017s, and the same shape for `io`. Without them the capture would
have shown only a late trampoline, which is equally consistent with a poke that never armed at all.
That was the exact distinction the item was written for.
**One further observation, recorded because it bears on what is left:** when dispatch resumes, a
*single* pool thread runs everything queued, in order -- the delivery waits first, which had been
queued since about two milliseconds into the run, then the work items.
**What it does not establish.** Why the pool stops, and why a work submit is the one thing that
restarts it. That interval is inside the pool, where this workspace's trace cannot reach, and it is
`M26.13`'s remaining experiment 1. The worker-supply reading is the obvious hypothesis and is
recorded as a hypothesis, not a finding.
**`M26.13` was re-planned in the same commit**, for the second time in two days and for the same
reason both times: the measurement moved the question. Its experiment list is now three pool-side
items rather than four, and it carries a handoff noting that a remedy most likely lands in
[windows-threadpool-sys](../windows-threadpool-sys/CHECKLIST.md) or in what this crate passes it as
an environment -- not in the ring.
**Rates**, all three-failure runs: none 3 in 525, wait 3 in 1307, timer 3 in 612, io 3 in 2105,
work 3 in 859.
**No source changed.** The experiment was a temporary edit to
[event_delivery.rs](tests/event_delivery.rs)'s post-mortem, reverted after the runs and described
in the measurement README.
## Moved 2026-09-27 12:01:28 -04:00 -- M26.13.5, a null result that the queued wording would have hidden
### <a id="m26135"></a>M26.13.5 -- Experiment 2: a private pool does not stall in 12000 runs, but the thread minimum is not why -- the no-minimum arm is already clean, so the supply reading it was written to test is unsupported. *(completed 2026-09-27 12:01:28 -04:00)*
**The experiment as queued would have produced a confident wrong answer.** Its wording was: "if a
private pool with `SetThreadpoolThreadMinimum` does not stall, the supply reading is supported."
That conflates two variables -- being off the default process pool, and having a non-zero thread
minimum -- so the clean result it anticipated would have been read as confirming worker supply.
Split into four arms, it is a **null result on that hypothesis**:
| Arm | Failures | Runs |
|---|---|---|
| `default` (control) | 13 | 4000 |
| `private-min0`, no minimum set | 0 | 4000 |
| `private-min1` | 0 | 4000 |
| `private-min4` | 0 | 4000 |
`private-min0` is the arm the original wording did not call for, and it is the one that carries the
finding: a private pool with **no minimum at all** is already clean, so the minimum adds nothing and
supply cannot be tested from this direction. Figures, method and the positive control in
[measurements/2026-09-27-private-pool-does-not-stall/](../windows-threadpool-sys/measurements/2026-09-27-private-pool-does-not-stall/README.md).
**What it does establish:** the stall has only ever been seen on the default process pool. All three
`EventDelivery` objects shared one pool in every arm, including the trigger's, so the variable is
*which* pool and never shared-against-isolated -- the private arms are exactly as shared as the
default arm.
**A positive control, because this experiment's failure mode is a false negative.** A private arm
whose environment silently named no pool would have run on the default pool and reported "does not
stall" for the wrong reason. Every construction therefore asserted a non-null pool in its
environment and recorded the pointer, and the assertion was checked by sabotage: removing the
`set_pool` call makes the arm fail with the message the assertion carries.
**Sizing, so that zero means something.** Every arm is the same size, so each private arm would have
been expected to produce about as many failures as the control did, and the three together about
three times that.
**What it does not establish.** Why. "A private pool" is still compound -- a different pool object,
its own threads, and no sharing with whatever else in the process uses the default one -- and
nothing here observes pool-internal state. It is not evidence that the default pool is defective,
only that the failure has never been seen off it.
**It also does not make a remedy, and `M26.13` was re-planned to say so.** Putting deliveries on a
private pool is reachable today, since `EventDelivery::new` already takes an environment. But it
would be a workaround ahead of a diagnosis, and it raises questions this item must not answer alone:
who owns the pool, one per delivery or one shared, what it means for a caller passing their own
environment, and whether a crate should quietly move a caller's callbacks off the pool they
expected. That is now recorded as a **decision to raise, not to take**.
**No source changed.** The experiment was a temporary edit to
[event_delivery.rs](tests/event_delivery.rs), reverted after the runs and described in the
measurement README.
## Moved 2026-09-27 12:47:35 -04:00 -- M26.13.6, a reading from review, half confirmed and half refuted
### <a id="m26136"></a>M26.13.6 -- The pool has 6 threads while stalled and 8 or 9 right after the work submit, so it has no worker and makes one; runs-long does not change that, and no callback of ours is holding a thread. *(completed 2026-09-27 12:47:35 -04:00)*
**Prompted by a reading offered during review**, not by the queued plan: that the pool is not
dispatching because it has no thread, and will not grow promptly because nothing told it these
callbacks may run long. Worth testing rather than arguing about, because this workspace had already
measured what that flag does -- four threads immediately, then growth throttled to roughly one
thread per 166 ms without it, against a millisecond with it, in the root
[DESIGN-NOTES.md](../../DESIGN-NOTES.md) under "`SetThreadpoolCallbackRunsLong` is the growth
mechanism, not a hint".
**The supply half is confirmed, and it is the sharpest evidence yet.** Counting the process's
threads in the post-mortem: **6 while stalled, 8 or 9 immediately after the work submit**, in all
six captures across both arms -- and, in a follow-up that examined **every** failure rather than a
capped sample, in 14 of 14. `at_stall` was 6 every single time. The capped figure is left above
because it is what this item measured; the uncapped one is the stronger claim and lives with the
data. The pool has no worker and makes two or three the moment work is
queued to it. `M26.13` had recorded this as needing an external instrument; it does not -- a
Toolhelp snapshot in the post-mortem is enough.
**The runs-long half is refuted, twice and independently.**
- *Nothing of ours is running.* Across the 27 captures from `M26.13.3`, `M26.13.4` and `M26.13.5`,
**zero** trampolines are entered during the stall. No callback of this workspace's is inside a
closure, blocked or otherwise, so none can be occupying a worker. This is exactly the distinction
the entry/exit pairing from `M26.13.1` was built for, answered from data already committed.
- *Setting the flag changes nothing measurable.* 12 failures in 4000 with
`SetThreadpoolCallbackRunsLong` on the delivery environment, against a control that has measured
13 and 21 in two separate 4000-run measurements. 12 is inside the control's own spread and no
effect is claimed in either direction.
The two are consistent rather than contradictory: the measured effect of runs-long is on growth
*under blocking callbacks*, and there are no blocking callbacks here.
**Both arms assert their own environment before use** -- the runs-long arm that the flag bit is set
and that it is still on the default pool -- and record it in the trace, because an arm that silently
carried no flag would behave exactly like the control and report its result for the wrong reason.
The assertion was checked by sabotage: removing the `set_runs_long` call makes the arm fail with the
message it carries.
**A methodological result worth as much as the finding.** The first version took its thread count on
the *setup* path, once per `EventDelivery` construction. A Toolhelp snapshot enumerates every thread
on the system, and three per run -- taken exactly where the race happens -- cut the failure rate to
roughly one in several thousand and slowed each run by an order of magnitude. The run was abandoned,
the count moved into the post-mortem, and the control returned to its usual rate. This is the hazard
the trace facility's own module documentation is built around, arriving through a different door:
an instrument cheap enough to leave in a callback is not automatically cheap enough for a setup
path.
**What is left is one sentence**, and `M26.13` was re-planned around it: why the pool will create a
worker for a submitted work item but not for a wait, timer, or I/O callback that is already queued.
**No source changed.** The experiment was a temporary edit to
[event_delivery.rs](tests/event_delivery.rs) plus one dev-dependency feature, both reverted after
the runs and described in
[measurements/2026-09-27-the-pool-has-no-worker/](../windows-threadpool-sys/measurements/2026-09-27-the-pool-has-no-worker/README.md).
## Moved 2026-09-27 15:07:48 -04:00 -- M26.13.7, a run with every instrument on
### <a id="m26137"></a>M26.13.7 -- With call-boundary and exception tracing on, no Win32 call blocks during the stall: 707 bracketed calls across 24 captures all returned, slowest 220us. *(completed 2026-09-27 15:07:48 -04:00)*
**The buffer was checked first, not after.** The trace had roughly tripled -- this workspace's
threadpool suite went from 14061 records to 47997 with the new targets on -- leaving a third of a
buffer spare at the old capacity. Raised to 262144 before the run rather than discovering a
truncated capture afterwards. The binary that actually captures holds **153** records, identically
in all 24 captures, so it is nowhere near either figure.
**The run:** 24 failures in 4000, in the range this configuration has produced all day.
**What it rules out, measured rather than assumed:**
- **No Win32 call blocked.** Every `syscall-enter` had its `syscall-leave`, in all 24 captures.
707 calls measured; the slowest single one anywhere is 220us.
- **No pool lock contended at arming time.** `CreateThreadpoolWait` returns in 2us and
`SetThreadpoolWait` in 1us, immediately before the silence begins.
- **No exception was raised**, confirming M26.13.6's result on a second population.
Figures and captures in
[measurements/2026-09-27-no-win32-call-blocks/](../windows-threadpool-sys/measurements/2026-09-27-no-win32-call-blocks/README.md).
**One observation, offered as one.** `SubmitThreadpoolWork` is the slowest call in the table by an
order of magnitude -- median 85us against 1 to 3us for everything else -- and every one of those 48
measurements is the probe's submit, the call already known to end the stall and to gain the process
two or three threads. The cost is consistent with work happening inside it that the other calls do
not do. It is not evidence of what that work is.
**A flaky guard was found and fixed on the way.** The exception observer's test asserted that a
single raised exception was recorded. Run filtered it passed; run under the full suite with every
target on it failed, because the handler records with `try_lock` and drops rather than blocks --
which is the property that makes it safe to run on a thread already inside the trace. The test now
raises repeatedly and asserts that at least one landed, which keeps the guarantee that matters
without pretending a designed-in loss does not happen. Worth recording as a method error: the guard
had only ever been exercised in isolation, and the suite is where it had to hold.
## Moved 2026-09-27 15:27:23 -04:00 -- M26.13.8, a coincidence eliminated by changing it
### <a id="m26138"></a>M26.13.8 -- The reproducer's own 5000ms submit timeout is not the five seconds: changed to 4000ms, dispatch still resumed at five in 14 of 14. *(completed 2026-09-27 15:27:23 -04:00)*
**Raised in review and worth running.** The reproducer contains a literal `5_000`, the stall lasts
five seconds, and coincidences of that shape usually are not. `submit_and_wait`'s second argument
really is `timeout_ms`, so the units matched too.
**The experiment's design is what made it decisive**, and it came from the reviewer rather than from
here: change the constant to **4000**, not to something tiny. If it governed the stall, dispatch
would resume at about four seconds -- before the test's own `DELIVERY_BOUND` of five -- so the
delivery would arrive in time and the tests would simply stop failing.
**Result:** 14 failures in 4000 runs, squarely in the range this configuration has produced all day,
and dispatch resumed at about five seconds in **14 of 14**, never at four. Figures and captures in
[measurements/2026-09-27-the-submit-timeout-is-not-it/](../windows-threadpool-sys/measurements/2026-09-27-the-submit-timeout-is-not-it/README.md).
**Three existing observations already pointed this way**, but each was an inference where the
experiment is direct: the call returns in about two milliseconds and never consumes its timeout; the
*other* victim submits with `submit_and_wait(0, 0)` and has no such constant yet fails in lockstep;
and the stall had already been moved to seven and eight seconds by delaying the probe, which no
timer armed near t=0 could do.
**What the five seconds actually is.** Two five-second constants exist in the reproducer and only
one matters: `DELIVERY_BOUND` decides when the test gives up and runs the pool-liveness probe, and
the probe's work submit is what ends the stall. The stall lasts five seconds *because* that is when
the probe runs. No five-second constant exists in either crate's library code.
## Moved 2026-09-27 15:39:37 -04:00 -- M26.13.9, in which the instrument turned out to be repairing the fault
### <a id="m26139"></a>M26.13.9 -- It never self-releases. With the probe removed the delivery never arrives in 65s, so this is a permanent hang and the "delayed dispatch" claim was an artifact of the instrument. *(completed 2026-09-27 15:39:37 -04:00)*
**Found by a question, not by a plan.** Review asked why anything waits five seconds before
submitting the work that ends the stall. Nothing does: the probe is diagnostic code in the test's
*failure* path, and the five seconds is `DELIVERY_BOUND`, the test's own assertion deadline. The
probe runs at step 5 of a sequence whose step 4 is "the test has already failed".
**Which exposed an untested assumption.** If the probe is the only thing in the process that submits
work after the stall begins, every observation of the delivery "arriving late" was taken *after* the
repair. That had never been separated.
**Measured:** probe removed, post-mortem extended to sixty seconds. 3 failures in 1896 runs, and in
all three the delivery **never arrives** -- `callbacks run: 0`, no trampoline entered, nothing in
the trace between 2ms and 65.02s.
[measurements/2026-09-27-it-never-self-releases/](../windows-threadpool-sys/measurements/2026-09-27-it-never-self-releases/README.md).
**The correction.** This record said in four places that the failure was "not a lost wakeup" but "a
delayed dispatch that eventually delivers everything", and that `D-68` had converted a permanent
loss into a late one. All four were wrong, for the most avoidable reason there is: the instrument
was repairing the fault before the measurement was taken. `M26.9`'s original signature -- a
permanent lost wakeup -- was right all along. Corrected in
[UNRESOLVED-TEST-FAILURES.md](UNRESOLVED-TEST-FAILURES.md),
[RESOLVED-TEST-FAILURES.md](RESOLVED-TEST-FAILURES.md) and this checklist in the same commit.
**What survives unchanged.** Every result about *what releases it* kept the probe and varied
something else, so all of them stand: the stall ends on a work submit and at no other time, nothing
else dispatches during it, no Win32 call blocks, no exception is raised, and it is never seen off
the default pool. What changes is the severity and the wording -- this is a hang, not a delay.
## Moved 2026-09-27 16:11:58 -04:00 -- M26.13.10, the best hypothesis this failure has had, refuted
### <a id="m261310"></a>M26.13.10 -- Not a missed wake: re-signalling the event the wait is armed on releases nothing in 5 of 5, with every SetEvent's success recorded. *(completed 2026-09-27 16:11:58 -04:00)*
**The strongest hypothesis so far, and it came from review.** Everything about the signature fits a
lost wakeup: `callbacks run: 0`, an auto-reset event ([D-21](DESIGN-NOTES.md#d-21)) whose signal is
consumed rather than left pending, an edge that cannot re-arm without the queue returning to empty
([D-19](DESIGN-NOTES.md#d-19)), and a stall now known to be permanent. This crate has had exactly
that bug twice, in [D-68](DESIGN-NOTES.md#d-68) and [D-77](DESIGN-NOTES.md#d-77).
**The test:** if the wait is armed and healthy and merely never woken, setting that same event again
must release it -- with no work submitted. `SetEvent` is safe where a zero-timeout poll is not,
because setting an auto-reset event can only add a signal, never consume one; the poll would have
manufactured the bug it was looking for, which is why
[M26.13.3](COMPLETED-CHECKLIST.md#m26133) forbade it.
**Result: 5 of 5 releases nothing.** Three seconds pass between the signal and the delivery, and the
delivery lands at the control work submit, never at the signal.
[measurements/2026-09-27-not-a-missed-wake/](../windows-threadpool-sys/measurements/2026-09-27-not-a-missed-wake/README.md).
**The positive control is what makes that a refutation rather than a null.** A `SetEvent` that
quietly failed would look identical to a signal that did nothing, so every call's return value is
recorded. Two captures show `0` for the first handle of each pass -- the trigger's delivery, already
dropped, so its duplicate is closed. Expected, and worth seeing. **Both victims' handles returned 1
in every capture.**
**So the fault is not in the signal and not in the arming.** It is that the pool does not dispatch,
which [M26.13.4](COMPLETED-CHECKLIST.md#m26134) showed from the other direction when a fresh wait, a
fresh timer and a fresh I/O completion all failed to dispatch in the same window. None of D-19, D-68
or D-77 is implicated.
## Moved 2026-09-27 16:22:50 -04:00 -- M26.13.11, a dump, and a thesis refuted
### <a id="m261311"></a>M26.13.11 -- A dump taken while stalled shows three pool workers parked idle, so the pool is not starved of threads and M26.13.6's reading was wrong. *(completed 2026-09-27 16:22:50 -04:00)*
**Suggested in review**, along with the observation that the same hook used to inject a `SetEvent`
could inject a process dump. It could, and out of process: the post-mortem spawns
`cdb -pvr -p <pid> -c ".dump /ma <path>; qd"` at the moment of failure, before the liveness probe,
so what is captured is the faulted state rather than the repaired one. Non-invasive attach injects
no thread; `qd` leaves the process running, and it went on to produce its usual report.
**The dump shows six threads, three of them the pool's**, all parked in
`ntdll!ZwWaitForWorkViaWorkerFactory` under `ntdll!TppWorkerThread` -- the kernel's "give me
work" wait. They are idle and available.
**That corrects `M26.13.6`.** It measured 6 threads while stalled and 8 or 9 after the work submit
and read it as "the pool has no worker and makes one". The counts were right; the inference was
wrong, and it was the kind of inference a thread *count* can never support -- it cannot say what the
threads are. Recorded in
[measurements/2026-09-27-the-workers-are-there/](../windows-threadpool-sys/measurements/2026-09-27-the-workers-are-there/README.md).
**So the question is now sharper and stranger.** The pool has idle workers. The wait is armed. Its
event can be signalled successfully ([M26.13.10](COMPLETED-CHECKLIST.md#m261310)). No Win32 call
blocks ([M26.13.7](COMPLETED-CHECKLIST.md#m26137)). And no callback is handed to any of those idle
workers until a work item is submitted.
**One observation, offered as one.** Of the four ways into this pool, the three that fail during the
stall -- a wait's event signalling, a timer expiring, an I/O completing -- are all delivered **by the
kernel** into the worker factory, while the one that works, `SubmitThreadpoolWork`, is a user-mode
queue push. Whether that distinction is the mechanism is not established, and a dump cannot settle
it: it captures state, not a delivery path.
> **Corrected by [M26.13.15](COMPLETED-CHECKLIST.md#m261315).** "User-mode queue push" is wrong in
> the half that matters: `TppWorkPost` pushes in user mode and then calls
> `NtReleaseWorkerFactoryWorker`, one of only four functions in `ntdll`'s thread pool that does, and
> the three failing paths call none of them. The distinction is not user mode against kernel mode;
> it is that the work path alone **explicitly asks the factory to release a worker**.
**On querying the pool's thread count:** there is no supported Win32 API, which the root
[DESIGN-NOTES.md](../../DESIGN-NOTES.md) already recorded. What works is counting threads whose
stack or Win32 start address is `ntdll!TppWorkerThread`, either from a dump as here or at run time
through `NtQueryInformationThread`. The kernel does track live and available counts in the worker
factory object, reachable through `NtQueryInformationWorkerFactory`, but the pool exposes no handle
to it -- so that route needs the handle recovered from a dump or by enumeration.
## Moved 2026-09-27 16:35:48 -04:00 -- M26.13.12, the strongest statement yet
### <a id="m261312"></a>M26.13.12 -- A self-rearming timer armed while the pool was healthy, due inside the stall window, fires only at the release in 10 of 10 -- so the fault is dispatch, not registration. *(completed 2026-09-27 16:35:48 -04:00)*
**Designed in review:** a one-shot timer armed for four seconds whose callback does nothing but
re-arm itself for another four, to see what it does to the failure rate.
**It differs from [M26.13.4](COMPLETED-CHECKLIST.md#m26134)'s timer poke in the way that matters.**
That one created a timer *during* the stall, which leaves open the objection that a pool in this
state cannot accept new registrations. This one is armed at process start while the pool is
demonstrably healthy and holds a standing commitment, so there is no registration to fail. A passing
run exits in about fifty milliseconds, long before the four-second expiry, so the heartbeat only ever
gets a chance to fire on a run that stalls -- a clean probe rather than added load.
**Result: 10 of 10 fired about a second late, at the release, never when due.** Armed at 0.000s, due
at 4.000s, fired between 5.004s and 5.019s -- the moment the work submit woke the pool.
[measurements/2026-09-27-a-timer-armed-while-healthy-also-stops/](../windows-threadpool-sys/measurements/2026-09-27-a-timer-armed-while-healthy-also-stops/README.md).
**It is its own positive control.** The timer does fire, so the machinery works and the arming took
effect; it simply cannot fire while the pool is in this state. A bare "never fires" would have been
ambiguous between a wedged pool and a broken probe. This is not.
**What it closes.** The fault is in dispatch, not in registration, and it is not about events,
handles, waits or the ring: this is a pure kernel timer expiry, registered before anything went
wrong, and it is suppressed exactly like everything else. Together with
[M26.13.11](COMPLETED-CHECKLIST.md#m261311)'s three idle workers, the picture is a pool with threads
available, commitments registered while healthy, and nothing being dispatched to them.
**On the rate:** 10 in 4000, against 13, 21, 14, 18, 24 and 14 for the same configuration earlier
the same day. Just below that spread, and one measurement cannot separate a real reduction from
ordinary variation, so no effect is claimed either way.
## Moved 2026-09-27 16:45:53 -04:00 -- M26.13.13, the clock says it never starts
### <a id="m261313"></a>M26.13.13 -- At a 100ms period the heartbeat becomes a clock: its first expiry is already missed and no pool callback of any kind is dispatched before the release, in 18 of 18. The pool never starts. *(completed 2026-09-27 16:45:53 -04:00)*
**Same probe as [M26.13.12](COMPLETED-CHECKLIST.md#m261312), period changed from four seconds to a
hundred milliseconds**, on review's suggestion. At four seconds it could only say the timer was
late; at a hundred milliseconds it is fine-grained enough to say *when* dispatch stopped.
**Rate unaffected:** 18 in 4000, squarely in the day's range. A constantly-expiring timer does not
prevent the fault.
**The timer is armed at 0.000031s**, so due at 0.100031s and every 100ms after. In all 18 captures
it fires **zero** times before the release and **once** in total, at the release -- about fifty
consecutive expiries missed.
**The stronger statement that licenses.** Across all 18, **not one** pool callback of any kind is
dispatched before the release. So this is not a pool that runs for a while and then wedges: in a run
that stalls, the pool dispatches nothing at all from process start until a work item is submitted.
The five seconds is not a period during which it stopped working; it is a period during which it
never started.
[measurements/2026-09-27-the-pool-never-starts/](../windows-threadpool-sys/measurements/2026-09-27-the-pool-never-starts/README.md).
**Still its own positive control:** the one firing at the release proves the arming took effect, so
a wedged pool and a broken probe are not confusable.
**What it cannot separate**, and what is now queued as the next experiment: whether the pool would
never have dispatched in this process, or whether it was put into this state during setup. Nothing
in these runs needs a callback before the deliveries are armed at about 2.5ms, so there is no
earlier successful dispatch to compare against. A period of one or two milliseconds would give one.
### <a id="m261314"></a>M26.13.14 -- At the 15.625ms system tick the heartbeat misses 320 consecutive expiries and the trace window is empty end to end, in 13 of 13; the onset is bracketed to the first 15.7ms, and the sub-millisecond period item 0 asked for is not reachable this way. *(completed 2026-09-27 16:59:53 -04:00)*
**Same probe as [M26.13.13](COMPLETED-CHECKLIST.md#m261313), period changed from a hundred
milliseconds to 15.625ms** -- the default Windows timer interval, 64 ticks a second, and the finest
period a process can ask for without raising the machine's global timer resolution with
`timeBeginPeriod`. A shorter request lands on the same tick.
**Rate unaffected:** 13 in 4000, inside the day's range (13, 21, 14, 18, 24, 14, 10, 18). A timer
expiring 64 times a second does not prevent the fault.
**320 consecutive expiries missed, in every capture.** The minimum and the maximum across the 13 are
both 320. The heartbeat fires exactly once, at the release, about a tenth of a millisecond behind
the post-mortem probe's work item -- which remains the first pool callback of any kind in the
process.
**The new thing this shows: the trace window is empty.** At 100ms the claim was "no pool callback is
dispatched before the release". At the tick the instrument is fine enough for a plainer statement --
the last record before the stall (`delivery setup-signalled`, about 2.5ms) and the first record
after it (`postmortem delivery-wait-expired`, about 5.01s) are **adjacent lines in the capture**.
`records_in_between` is 0 in all 13. Since the trace brackets every Win32 call and carries a
vectored exception handler as well as every callback, the window contains no callback, no syscall
and no exception. Nothing happens in the process at all.
**A tighter bracket on the onset.** In all 13 the heartbeat's `armed` record (about 0.00004s)
precedes the trigger's first `delivery event-attached` (about 0.00015s), so the timer was registered
while the pool was not merely healthy but untouched by the trigger -- and it still never fired.
With its 15.7ms due time that brackets the onset to the interval between process start and 15.7ms;
the trigger's whole create-and-drop completes inside the first 0.2ms of it.
**It ruled out its own successor's method.** Item 0 asked for a heartbeat whose first expiry
precedes the deliveries. 15.625ms is the floor for a thread-pool timer without `timeBeginPeriod`,
and 15.7ms is still after the deliveries are armed at about 2.5ms, so no timer period reaches it
without changing the machine's timer behaviour under the measurement. Item 0 is re-planned to arm a
wait on an **already-signalled** event at process start instead: due immediately, no timer
resolution needed, and in the kernel-delivered class that fails.
[measurements/2026-09-27-at-the-system-tick-it-still-never-starts/](../windows-threadpool-sys/measurements/2026-09-27-at-the-system-tick-it-still-never-starts/README.md).
### <a id="m261315"></a>M26.13.15 -- Delaying only the `SubmitThreadpoolWork` call by up to 2000ms moves the delivery with it in 99 of 99, the released worker serves the queued wait ahead of the work that woke it, and an `ntdll` census corrects "user-mode queue push" to "the only path that calls `NtReleaseWorkerFactoryWorker`". *(completed 2026-09-27 19:43:47 -04:00)*
**Three results, from one experiment and one census.**
**1. The submit, and nothing else in the probe, is what releases it.** Prior runs showed that a work
submit releases the pool and no other poke does, but the probe that submits also creates a work
object, reads the ring's count and writes to stderr first. A sleep was inserted **between
`CreateThreadpoolWork` and `SubmitThreadpoolWork`** so the submit moves and nothing else does. Five
arms of 4000 runs, delay in {0, 250, 500, 1000, 2000}ms, 99 captures (17, 19, 23, 16, 24 -- the rate
is unmoved by the delay). Delivery-minus-**submit** stays at 0.25 to 0.54ms in every arm;
delivery-minus-**create** tracks the delay across a 2000ms span. The negative half was already on
record and was not repeated: a created-but-unsubmitted work object releases nothing
([M26.13.3](COMPLETED-CHECKLIST.md#m26133)'s 1s create/submit gap arm), and with no submit the delivery never
arrives inside 60s ([M26.13.9](COMPLETED-CHECKLIST.md#m26139)).
**2. The backlog was already queued.** In 99 of 99 the first callback dispatched anywhere in the
process is the stalled delivery's **wait**, served 29 to 67us (mean 39) *ahead of* the work item
whose submit woke the worker; one released worker then drains everything, on two threads in 91
captures and one in 8. A worker that had to be told to wake and then finds a five-second-old wait
callback ahead of the item that woke it is taking the front of an existing queue. This is the
discriminator experiment 1 asked for: **queued and unserved, not unnoticed.** The limit, stated: the
trace cannot see the completion port, so this shows the wait is *ordered ahead of* the work, not the
instant the kernel enqueued it.
**3. Correction: `SubmitThreadpoolWork` makes a syscall, and it is the interesting one.** This
record has carried "`SubmitThreadpoolWork` is a user-mode queue push" since 2026-09-26 as an
unverified label. Every one of the 189 `ntdll!Tp*`/`Tpp*` functions was disassembled and grepped.
`TppWorkPost` pushes in user mode under the pool's SRW lock and then calls
`TppAdjustRunningThreadGoalWithLock`, `NtAlertThreadByThreadId` and **`NtReleaseWorkerFactoryWorker`**
-- and exactly four functions in the whole thread pool call that last one (`TppWorkPost`,
`TpPostTask`, `TppPrepareDirectParams`, `TppWorkCallbackPrologRelease`). None of the four is on the
wait, timer or I/O path; those register for kernel delivery (`NtCreateWaitCompletionPacket`,
`NtAssociateWaitCompletionPacket`, `NtSetTimer2`) and rely on the factory releasing a worker itself.
So the distinction is not user mode against kernel mode -- the submit makes a syscall too -- it is
that **the work path is the only one that explicitly asks the factory to release a worker.** The
same call also explains the thread growth that had no explanation: the running-thread goal is raised
on that path, so the 6-to-8 rise is a side effect of the submit rather than evidence about supply.
This is static evidence about a code path, not a measurement of the fault.
[measurements/2026-09-27-the-submit-is-what-releases-it/](../windows-threadpool-sys/measurements/2026-09-27-the-submit-is-what-releases-it/README.md).
**Swept the corrected claim:** 2 sites carried the wrong wording ([M26.13.11](COMPLETED-CHECKLIST.md#m261311)'s
archive entry and its measurement README); both now carry an additive correction rather than a
rewrite, since both are dated records. 4 further sites use "kernel-delivered", which the census
confirms rather than contradicts, and were left alone.
**Experiment 1 re-planned.** Its premise ("the pool has no worker and makes one, but only for work")
is gone -- [M26.13.11](COMPLETED-CHECKLIST.md#m261311) found three parked workers and the thread
growth is now accounted for. The question becomes: the factory holds parked workers and a queued
packet and does not put them together. The next measurement is the factory's own counters via
`NtQueryInformationWorkerFactory`, read from a dump taken while stalled.
### <a id="m261316"></a>M26.13.16 -- No thread alive at the stall ever runs a callback, in 12 of 12, while a passing run serves the delivery on a pre-existing thread in 30 of 30: the parked workers are present and unused, so `M26.13.11`'s "not starved" survives but the inference drawn from it does not. *(completed 2026-09-27 20:20:01 -04:00)*
**Prompted by a question about [M26.13.15](COMPLETED-CHECKLIST.md#m261315):** was the thread that
releases the stall the one that had been waiting? No, and the two calls are different in kind --
`NtWaitForWorkViaWorkerFactory` is the park that `TppWorkerThread` blocks in, while
`NtReleaseWorkerFactoryWorker` is a post that never blocks, called by whichever thread runs
`SubmitThreadpoolWork` (here the test's own post-mortem thread, running rather than waiting). But the
question pointed at one that had not been asked: are the three parked workers the ones that
eventually serve the backlog?
**Method.** A Toolhelp thread snapshot emitting one `thread-present` record per live thread with its
`GetCurrentThreadId` value. The trace stamps every record with the same id, so the thread that later
serves the stalled wait can be tested for membership. Taken in the post-mortem for the stalled arm --
[M26.13.6](COMPLETED-CHECKLIST.md#m26136) had already measured that the same snapshot on the setup
path destroys the race -- and, for the healthy control, after the delivery is armed and before any
completion can arrive.
**Result.** Both populations hold six threads at the snapshot: main, the two test threads, and three
others.
| | threads alive at the snapshot | first delivery served by a thread that already existed |
|---|---|---|
| stalled, 12 captures | 6 | **0 of 12** |
| healthy, 30 runs | 6 | **30 of 30** |
In a passing run one of the three others serves the delivery, every time. In a stalled run none of
them ever runs anything, and the backlog is served by one or two threads that did not exist when the
stall was observed.
**What it changes.** [M26.13.11](COMPLETED-CHECKLIST.md#m261311) read its dump as "the pool is not
starved of threads". The literal claim survives -- the threads are there, parked, in both
populations. The inference does not: their presence was taken to mean supply is not the subject, and
these runs show the parked workers are **present and unused**. A free worker sat available for the
whole five seconds, in the same state as the one that serves the delivery in a passing run, and the
pool dispatched only once a *new* thread existed. So the stall is not "a queued packet waiting for a
free worker".
**What it does not establish.** Whether the new thread was created *because* the parked ones were
unusable, or merely as a side effect of the submit: `TppWorkPost` calls
`TppAdjustRunningThreadGoalWithLock` on its way to the release, so a submit raises the thread goal
whether or not a worker is idle, and this cannot separate the two. What needs no such separation is
the **wait** callback -- no submit involved, queued for five seconds -- served by a thread that did
not exist at the stall in 12 of 12 while a passing run serves it on one that did in 30 of 30. It
also does not identify the three parked threads as this pool's by direct evidence; the identical
composition across both populations, one of which serves the delivery, is the argument.
[measurements/2026-09-27-the-parked-workers-are-never-used/](../windows-threadpool-sys/measurements/2026-09-27-the-parked-workers-are-never-used/README.md).
**Swept the qualified inference:** 8 pre-existing sites mention idle workers or starvation (one
further hit, on an appender, is unrelated). Two assert the inference and carry a qualifier now -- the
timeline's ruled-out row and [M26.13.11](COMPLETED-CHECKLIST.md#m261311)'s measurement README. Two
more are this archive's own dated heading and its checklist stub, left as written because the archive
is history. The remaining four say "has idle workers and does not dispatch to them", which this
sharpens rather than contradicts.
**Experiment 1 sharpened.** The factory-counter measurement now has a second reading to separate:
whether the factory believes it has an available worker while three sit parked in
`NtWaitForWorkViaWorkerFactory`. If the counters say zero available while the stacks say three
parked, that disagreement is the fault.
### <a id="m261317"></a>M26.13.17 -- Inline ntdll hooks and a worker-factory scan: the process holds two factories, and the default pool reports zero workers while stalled against one while healthy, so `M26.13.11`'s three parked workers were the *other* factory's all along. *(completed 2026-09-27 21:46:03 -04:00)*
**Built the instrument the question needed.** `windows-threadpool-sys` gained inline hooks on
`ntdll`'s worker-factory syscall stubs. No disassembler is required, because every `ntdll` `Nt*`
entry point is the same stub: the displaced bytes are not decoded and relocated, they are
**recognised**, and the trampoline is rebuilt from the one variable in them -- the system call
number. The recognition is the safety property and is enforced, patching happens with every other
thread suspended, and the whole facility needs the `trace` feature *and*
`WINDOWS_THREADPOOL_TRACE_HOOKS` before it will touch anything.
**The hooks turned out not to be how the answer arrived**, and that is itself a finding. Installing
them took 0.15s to 0.58s, which is **after** the onset (`M26.13.14` brackets it to the first
15.7ms), so in a stalled run no hooked call had ever fired and the factory handle a hook would have
learned was exactly the handle that was missing. The handle is instead found by asking each
candidate handle whether it is a worker factory -- read-only, sub-millisecond, and it cannot disturb
what it is looking at, which matters in an investigation whose instruments have repaired the fault
before.
**The result, 12 stalled captures against 20 healthy runs, every capture in each arm identical:**
| | factory with `ThreadMaximum` 768 | factory with `ThreadMaximum` 3 |
|---|---|---|
| stalled | total **0**, waiting **0** | total 3, waiting 3 |
| healthy | total **1**, waiting 1 | total 3, waiting 3 |
**This overturns [M26.13.11](COMPLETED-CHECKLIST.md#m261311).** Its dump was accurate and its
inference was not: `TppWorkerThread` is the worker routine for *every* pool in a process, so a stack
can never say which factory a parked worker serves. The three it found answer to the second factory
-- whose maximum is three, which is why there are exactly three -- and that factory is byte-for-byte
identical in passing runs. The pool under test has **no worker**, which is what
[M26.13.6](COMPLETED-CHECKLIST.md#m26136) said from thread counts and the dump was taken to
disprove. It also explains [M26.13.16](COMPLETED-CHECKLIST.md#m261316) rather than leaving it
strange: no thread alive at the stall runs a callback because the parked ones were never candidates.
**What the factory says about itself while stalled**, beyond the zero: `Paused` false, `Shutdown`
false, `MayCreate` **true**, `ThreadMinimum` 0, `PendingWorkerCount` 0, `ReleaseCount` 0,
`LastThreadCreationStatus` 0. Nothing has told it to stop, nothing has failed, and nothing has asked
it for a worker.
**Not established:** the packet's presence in the port. These counters describe workers, not queued
completions, so "the factory holds the packet and will not act on it" and "the packet never arrived"
are both still consistent with them. `M26.13.15`'s ordering evidence argues for the first and
remains an argument. The second factory's identity is also unresolved -- a loose end rather than a
gap, since it is identical in both arms.
[measurements/2026-09-27-the-default-pool-has-no-worker-at-all/](../windows-threadpool-sys/measurements/2026-09-27-the-default-pool-has-no-worker-at-all/README.md).
**Two defects in the instrument, both caught by building the guard rather than by the guard:** the
self-test stub's first argument is an out-pointer and the first draft would have installed it as the
worker factory handle; and the information struct was truncated, which makes the query fail with
`STATUS_INFO_LENGTH_MISMATCH` and look exactly like a process with no factory in it. Both are now
sabotages in the crate's new `sabotage.json` -- 9 sabotages, 8 caught and the control survived.
**Swept the overturned claim.** Named rather than counted, because a grep for "parked" or "starved"
across this crate's documents matches mostly incidental prose and a tally of it would say nothing.
The documents that *assert* the overturned reading are four, and each now carries a correction at
the point of the assertion: [the-workers-are-there](../windows-threadpool-sys/measurements/2026-09-27-the-workers-are-there/README.md)
and [the-parked-workers-are-never-used](../windows-threadpool-sys/measurements/2026-09-27-the-parked-workers-are-never-used/README.md)
open with an overturning note, [UNRESOLVED-TEST-FAILURES.md](UNRESOLVED-TEST-FAILURES.md)'s entry is
marked at its heading, and [STALL-TIMELINE.md](../windows-threadpool-sys/STALL-TIMELINE.md)'s ruled-out row now reads "not
ruled out after all". The dated archive entries above are left as written, with the correction
carried here.
**A standing lesson, recorded in [STALL-TIMELINE.md](../windows-threadpool-sys/STALL-TIMELINE.md):** count a pool's workers by
asking each factory, never by reading stacks. A parked `TppWorkerThread` says *a* pool has a worker;
in a process with more than one factory it does not say which, and here that distinction was the
whole answer.
### <a id="m261318"></a>M26.13.18 -- Hooks installed before `main` show the pool's first worker is never created: a healthy run makes one 0.24-0.31ms after the delivery is armed, a stalled run makes none before the release in 8 of 8, and the `AlreadySignaled` race is refuted. *(completed 2026-09-27 22:37:50 -04:00)*
**From a suggestion in review**: the hooks were arriving after the tests started, so put something
in front of the tests. The better form of that is to move the *install* earlier rather than delay
the tests -- a function pointer in `.CRT$XCU`, the C runtime's static-initialiser table, runs before
`main` and therefore before the harness has created a thread. It is also cheaper than the lazy
install it replaces: patching live code requires every other thread to be stopped, and at that point
there are none, so the suspend-and-resume pass finds nothing and perturbs nothing.
**The result, with the hooks in place from process start:**
| | first worker announces `NtWorkerFactoryWorkerReady`, relative to the delivery being armed |
|---|---|
| healthy, 10 runs | **0.243 -- 0.309 ms** |
| stalled, 8 captures | **5003 -- 5017 ms**, which is the release |
A healthy run does the whole thing in a quarter of a millisecond: the event signals, the kernel
makes a worker, it announces itself, parks, is handed the queued packet at once, and runs the
callback. **A stalled run has no `ready` and no `park` on that factory at any point before the
release, in 8 of 8.** So the stall is not a worker that fails to wake, nor a packet handed to the
wrong thread, nor a callback that runs and goes missing. The thread to run it is never made.
**Refuted: the `AlreadySignaled` race**, which was the reason the wait registration was hooked.
`NtAssociateWaitCompletionPacket` sets an out-parameter when the object is already signalled at
association time; on that path the kernel queues no completion and leaves the caller to act, which
is a second delivery path taken only on a race and exactly the shape of a lost callback. It is not
what happens: the flag is **false** for every association made before the first callback, in both
arms. The only `true` anywhere is a re-arm after the release, which is the ordinary case.
**Two instrument defects fixed on the way, neither found by a test.** The suspended window
allocated -- `held.push` grew a vector while other threads were stopped, and a thread suspended
holding the allocator lock can never give it back. Both vectors are now reserved before anything is
suspended. And the install pass was paying for `CreateToolhelp32Snapshot` *per hook*; it snapshots
every thread on the machine and is then filtered to this process, about 0.12s each, which put the
last of six installs 0.73s into the process. One snapshot for the batch puts all of them at 0.132s.
**Cost:** 8 failures in 4000 with four hooks installed, inside the range this configuration produces
without them.
[measurements/2026-09-27-the-factory-never-makes-its-first-worker/](../windows-threadpool-sys/measurements/2026-09-27-the-factory-never-makes-its-first-worker/README.md).
**Still not established:** the packet's presence in the port. "The kernel queued the completion and
the factory did not act on it" and "the kernel never queued it" produce identical records here,
because every observation is of a call *ntdll* makes and neither case involves one.
**What is left is outside this workspace's reach.** A worker factory with no workers, permitted to
create one, not paused and not shut down, owes a callback and does not make the thread to run it
until `NtReleaseWorkerFactoryWorker` asks. Why is inside the kernel.
### <a id="m261319"></a>M26.13.19 -- An ETW kernel trace confirms the missing worker independently of the ntdll hooks: across 900 traced runs exactly one process has a gap over a second, the failing one, at 5008.7ms against a healthy 0.232-17.928ms. *(completed 2026-09-28 00:08:32 -04:00)*
**Why a second instrument.** Everything known about the missing worker came from hooks this
workspace plants in `ntdll` -- a good instrument, and a self-interested one, since it reports on the
very mechanism it modifies. A reader is entitled to ask whether the hooks are why the worker is
missing. This answers that with the kernel's own thread-creation record, collected through the NT
Kernel Logger's `PROC_THREAD` flag, which needs no cooperation from `ntdll` or from this crate.
**The result**, over 900 runs in one trace:
| | first pool worker created, relative to the last test thread |
|---|---|
| healthy, 899 processes | **0.232 -- 17.928 ms**, mean 1.298 |
| failing, 1 process | **5008.736 ms** |
Exactly one process in the trace has a gap of more than a second anywhere in its thread activity,
and it is the run that failed. The slowest healthy process is 17.9 ms, so the separation is complete
with nothing in between. The failing process then creates **three** workers inside 373 us, which is
consistent with the release path raising the running-thread goal on its way to
`NtReleaseWorkerFactoryWorker`, and is why it is one of only three processes in the trace to reach
ten threads.
**So the hooks were not the cause.** The kernel logged the same absence, in the same runs, through a
path the hooks do not touch -- and logged normal behaviour for 899 other processes that had those
same hooks installed.
[measurements/2026-09-27-the-kernel-agrees-no-thread-is-made/](../windows-threadpool-sys/measurements/2026-09-27-the-kernel-agrees-no-thread-is-made/README.md).
**Method note worth keeping.** Thread ids are recycled aggressively across 900 short-lived
processes, so matching the failing run to its process by any single id is wrong -- several processes
in this trace contain any given id. The match is the process holding **all seven** ids that appear
in the failing run's own user-mode trace, and only one process does.
**Elevation.** The NT Kernel Logger refuses a non-elevated session; `sudo` in Inline mode supplied
it, for `xperf` only. The reproducer itself ran unprivileged, exactly as every earlier arm, so the
conditions are unchanged. One failure in 900 is the usual rate, and the session was stopped and
merged before analysis.
**Still not settled, and no ETW will settle it**: whether the wait-completion packet reached the
port. `PROC_THREAD` records threads, not queues, and a census done the same day found no public
event anywhere for a packet arriving at a completion port or for a worker factory being activated --
1198 registered providers, `xperf -providers KF`, and all 40 `Microsoft-Windows-Kernel-*` manifests.
**Queued rather than taken:** `DISPATCHER`, which adds `ReadyThread` and would name what causes the
worker to appear in a healthy run. Left out here to keep a machine-wide trace small; it is now the
whole of M26.13 experiment 1.
### <a id="m261320"></a>M26.13.20 -- Re-verified the reproducer's premises from a clean build: the 3-test reduction matches the full binary (16 vs 18 in 4000), the trigger is necessary (0 in 4000 without it), one victim suffices at about half the rate, and "fail together, never singly" was wrong. *(completed 2026-09-28 08:27:01 -04:00)*
**Why re-check the setup at all.** Nineteen findings had accumulated on top of a reproducer that was
itself a reduction, and several of them had already corrected earlier ones. The premises were worth
re-establishing from a clean build of the committed tree rather than from the record.
**What the reproducer is, stated precisely.** Each run is one **fresh process** -- measured, not
assumed: the ETW capture in `M26.13.19` recorded 900 distinct process ids for 900 runs. Inside it
the tests run **concurrently**; the machine has 16 logical processors, `libtest` defaults to that
many threads, and the kernel trace shows the three test threads created within 105us. Outside it
there is **no deliberate load** -- the loop is serial, one process at a time. "Concurrent" in this
investigation has always meant between the tests inside one process, and saying so removes an
ambiguity that had never been written down.
**Five arms, 4000 runs each:**
| arm | what runs | failures |
|---|---|---|
| A | 3-test reproducer: 2 victims + trigger | 16 |
| B | the whole binary, all 7 tests | 18 |
| C | 2 victims, no trigger | **0** |
| D | one victim + trigger | 8 |
| A again | as A, against a cleared temp directory | 13 |
B says the **reduction is representative** -- narrowing 7 tests to 3 changed nothing. C re-confirms
from scratch that the **trigger is necessary**. D is the interesting one: a single victim still
fails, at about half the rate, which fits a race between the trigger's teardown and *a victim's*
setup (two victims, two chances to enter the state) better than a per-process failure that any
victim would report. Offered as the reading the numbers fit.
**A claim corrected.** The record said the victims "fail together, never singly". Across the 80
committed captures, 79 report both and **one reports a single victim**. It is the same phenomenon:
that capture's trace shows the pool dead for the full five seconds, and what differs is the finish
-- the victims' deadlines are a fraction of a millisecond apart, the first to expire runs the probe,
the probe releases the pool ~0.3ms later, and the second victim's deliveries land inside its own
deadline. Both are always *stalled*; they usually both *report*. One more instance of the probe
repairing the fault it measures.
**A defect found and queued as M26.14.** The tests leak their temp files -- two per run, nothing
ever deletes them, and roughly thirty thousand runs had left **307,383 files, 1.2 GB** on the
machine, since removed. Fifteen of this crate's test files share the pattern. Measured **not** to be
a confound (13 in 4000 against a cleared directory, 16 against a full one) before being queued, and
deliberately not fixed during M26.13 because adding teardown changes the reproducer while it is the
instrument of an active investigation.
[measurements/2026-09-28-re-verifying-the-premises/](../windows-threadpool-sys/measurements/2026-09-28-re-verifying-the-premises/README.md).
**Extended on review, with the question stated plainly: several different test cases per process,
never a repeat.** `libtest` runs each `#[test] fn` exactly once and this file has no
parameterisation, so the 3-test reproducer is three distinct functions -- a 6-line trigger and two
victims of 83 and 99 lines -- on three concurrent threads. No run in this investigation has ever had
fewer than two test cases in a process, because the fault needs the trigger co-running; all the
repetition is *across* roughly thirty thousand fresh processes and none within one.
**They overlap less than "concurrent" suggests, and the ordering turned out to be a constant.** The
trigger creates its delivery, arms, drops and closes it within about 0.05 ms, and the victims arm
about 2.4 ms later -- so the trigger has finished before the victims arrive. That holds in **80 of
80** failing captures, which on its own looks like a signature. It also holds in **183 of 183**
passing ones, with an indistinguishable gap distribution, so it is simply how `libtest` schedules
these three tests. Reading the failures alone would have promoted a constant to a discriminator;
the passing control is the only reason that did not happen.
### <a id="m26141"></a>M26.14.1 -- The trigger's *teardown* is what poisons: built and never dropped it gives 0 in 4000 against a live control's 11, and a ring dropped without a delivery also gives 0. *(completed 2026-09-28 12:57:52 -04:00)*
**From a review reading**: that `dropping_with_nothing_outstanding_does_not_hang` poisons the
process rather than competing with the victims for anything. Tested rather than accepted, by making
the trigger selectable so one build served five arms of 4000 runs.
| arm | what the trigger does | failures |
|---|---|---|
| `default` | ring + `EventDelivery`, **dropped** | **11** |
| `none` | nothing | 0 |
| `ring-only` | ring created and dropped, **no delivery** | 0 |
| `leak` | ring + `EventDelivery`, **never dropped** | **0** |
| `x4` | four create-and-drop cycles | **26** |
`default` was re-run in the same session, so the three zeroes are measured against a live
reproducer rather than against an earlier figure.
**`leak` is the result.** The trigger builds exactly what it always builds -- a ring, an event
attached to it, a `ThreadpoolWait` armed on that event -- and does not drop it. Zero in 4000. So the
setup does not poison; taking it back down does. `ring-only` closes the other half: a ring created
and dropped with no delivery over it is also zero, so the ring alone is innocent too. It takes the
delivery **and** its teardown.
**What that teardown is**, on the trigger's own thread and spanning 54us: a `TP_WAIT` created, armed
on the ring's completion event, then `SetThreadpoolWait(NULL)`, `WaitForThreadpoolWaitCallbacks` and
`CloseThreadpoolWait` -- all before the pool has dispatched anything, and about 2.4ms before either
victim arms its own delivery.
**`x4` gives 26 against 11**, which is 3.5 Poisson sigma below four independent chances and 2.9
above no increase. Repeating it raises the rate without quadrupling it. Offered as a direction
rather than a dose-response, and the reason is stated: four cycles take longer than one, which also
moves when the trigger finishes relative to the victims arming, so the arm changes two things at
once.
**Not established:** which part of the teardown does it -- disarm, drain and close are 12us apart
and this cannot separate them. Nor why a ring is needed when the wait is on an ordinary event.
Both are queued as M26.14.2.
[measurements/2026-09-28-the-teardown-is-what-poisons/](../windows-threadpool-sys/measurements/2026-09-28-the-teardown-is-what-poisons/README.md).
### <a id="m26142"></a>M26.14.2 -- Closing the wait too soon after disarming it is the poison: a 1ms gap between the disarm and the close, or a true drain in place of a cancel, each give 0 in 20000 against a control's 10. *(completed 2026-09-28 20:39:34 -04:00)*
**The decisive run**: three arms interleaved, 20000 each, pipes drained asynchronously, 60s bound
per run, timing balanced to 1.3% (157.7 / 155.6 / 156.3 ms per run) and no timeouts.
| arm | teardown | failures in 20000 |
|---|---|---|
| `hand-full` | disarm -> `WaitForThreadpoolWaitCallbacks(TRUE)` -> close | **10** |
| `hand-disarm-sleep-close` | disarm -> **sleep 1ms** -> close | **0** |
| `hand-disarm-drainwait-close` | disarm -> `WaitForThreadpoolWaitCallbacks(FALSE)` -> close | **0** |
Each zero has probability `exp(-10)` = 4.5e-5 under the control's rate.
**The whole decomposition reads down one column.** Nothing before the disarm matters -- never
armed then closed, armed then abandoned, and armed then closed while still armed are all 0 in
4000. The disarm alone is innocent: 0 in **15000** cumulative. The poison appears exactly when a
close follows a disarm closely (disarm -> close, 11 in 8000; disarm -> cancel-drain -> close, 10 in
20000), and putting any real gap between those two calls stops it.
**The control that makes the sleep mean something** is `hand-sleep-after`: the identical 1ms sleep
placed *after* the whole teardown, which still fails. So the sleep does not help by delaying the
trigger or by moving when it finishes relative to the victims arming -- the gap must sit between
the disarm and the close.
**Cancelling is not draining.** `WaitForThreadpoolWaitCallbacks`' `fCancelPendingCallbacks` is TRUE
in `EventDelivery`'s path and does not prevent the stall; FALSE does. One argument, 10 failures
against 0 over 20000 runs each. The arm was written expecting it might hang -- the trigger signals
the event while the wait is armed, so a callback is usually pending -- and it never did, in 20000
runs against a 60s bound.
**This is a workaround with a mechanism-shaped hint, not a diagnosis.** A sleep that makes a race
disappear is evidence of a race, not an explanation of one. Nothing here says what the close races,
nor why the worker factory is left unable to make its first worker. No threshold is established
either; 1ms was chosen as about 80x the natural 12us gap.
**Two false turns on the way, both caught by controls rather than by reasoning.** A five-arm
interleaved run came back all zeroes and was reported as void on the grounds that the harness had
broken the reproducer; re-testing the control on the historical launcher gave 4 in 4000 against the
harness's 3 in 8000 (p = 0.18), so the harness was innocent and the run was merely underpowered --
its four events had in fact landed 3 and 1 on the two unmodified-teardown arms and 0 on the three
modified ones. Separately, an apparent 5x instability in the failure rate was mostly Poisson noise
on counts of 5 to 16: a chi-square over six measurements of the unmodified teardown gives p = 0.048,
and five of the six sit inside the 95% band around a pooled 2.6 per 1000. The practical lesson
governed this run's design -- at 4000 runs a zero is strong but a rate comparison is not, which is
why the decisive arms used 20000.
[measurements/2026-09-28-closing-too-soon-after-the-disarm/](../windows-threadpool-sys/measurements/2026-09-28-closing-too-soon-after-the-disarm/README.md).
**Raised, not taken:** M26.14.3 asks whether `windows-threadpool-sys` should adopt the draining
teardown, which is a one-argument change with real semantic weight -- a drop that blocks until a
pending callback runs can deadlock a caller. M26.14.4 asks for the threshold sweep if it does not.
## <a id="moved-2026-10-01-m20-through-m24"></a>Moved 2026-10-01 21:19:21 -04:00 -- M20 through M24, and the M21+/M22+/M28+ buckets they filled
Eight milestones, every item in them complete. Each item was archived individually as it
finished, so the entries below are the stubs that pointed at those archive entries plus the
milestone prose that framed them -- which is the part that lived only here.
### M20 -- Repairs from the 2026-08-30 NUMA-sharding measurement
Queued from
[DESIGN-SESSION-2026-08-30-numa-sharded-io-execution-domains.md](../../design-sessions/DESIGN-SESSION-2026-08-30-numa-sharded-io-execution-domains.md),
which measured a shipping ARM laptop and found the L3 heuristic's justification does not hold there. These
were queued as documentation and policy repairs only, on the basis that **no defect was found in
`ring_copy`** -- `Policy::select` already degrades to a whole-machine domain and reports it, which an
initial reading of the session got wrong and the code corrected.
**Corrected 2026-09-19: that basis no longer holds, and it changes the order.** `SH-4.12` in
[CHECKLIST-ship-topology-and-queues.md](../../CHECKLIST-ship-topology-and-queues.md) later found two
defects in that same function: it selects on `DomainKind::Cache { level: 3, .. }` rather than asking
`outermost_partitioning_cache()` -- the one definition of which cache level partitions a machine, shipped
in `windows-topology-sys` 0.2.0 -- so it can produce **overlapping** ring domains where two cache kinds
report at level 3, and degrades silently on a host whose outermost partition sits at another level.
`M20.1` and `M20.3` both land on that function and that rule, so both are **coupled to `SH-4.12`** and
must follow it. `M20.2` and `M20.4` are done. `M20.6` is gated the other way, on `M22.1`. That leaves
`SH-4.12` as the only thing standing between M20 and completion.
The design questions the session opened are deliberately **not** queued here. It is still open, and its
conclusions belong to it until it converges.
- [x] **M20.1** -- Restate the cache heuristic as "the outermost cache level that actually partitions
the machine", sweep every restatement, and replace the consumer that bound to the level number.
Done together with `SH-4.12`, which is the code half of the same change.
-> [completed 2026-09-22](COMPLETED-CHECKLIST.md#m201)
- [x] **M20.2** -- Record the 2026-08-30 ARM measurement as a decision, beside the zero-NUMA-node
observation it is the sibling of.
-> [completed 2026-09-19](COMPLETED-CHECKLIST.md#m202)
- [x] **M20.3** -- Make `ring_copy`'s degraded-fallback path observable in a test, asserting both
that an absent relation degrades and that a present one does not. Done without waiting on
`SH-4.12`: the fallback tail is shared by every policy, so exercising it through `ByNode` and
`ByPackage` pins nothing that item rewrites.
-> [completed 2026-09-22](COMPLETED-CHECKLIST.md#m203)
- [x] **M20.4** -- Correct "What is not reachable" in [DESIGN-NOTES.md](DESIGN-NOTES.md): the
file-handle-to-storage-node mapping is reachable on mechanism, and the conclusion it supported now rests
on volume granularity, absence, and spanned volumes instead.
-> [completed 2026-09-19](COMPLETED-CHECKLIST.md#m204)
- [x] **M20.5** -- Dissolved by [D-47](DESIGN-NOTES.md#d-47-detail) rather than decided: the
`flush_barrier` assertion was measuring a claim the platform does not honour, so it was never a
flaky test. -> [completed 2026-09-07](COMPLETED-CHECKLIST.md#m205)
- [x] **M20.6** -- Re-evaluate `CommitStrategy::AlternatingRings` and the benchmark's conclusion
against [D-47](DESIGN-NOTES.md#d-47-detail). **The harness cannot exhibit a blast-radius
difference, which is a fact about the harness and not a finding against the strategy** -- each
lane's own arena is the limiter there. The strategy stays, with the conditions under which it
would pay written down; `M25.5` re-runs the comparison where operations genuinely pend. The
sample's output and prose are corrected so they stop claiming to measure a commit.
-> [completed 2026-09-23](COMPLETED-CHECKLIST.md#m206)
### M21 -- Epoch-log review: correctness repairs
Queued from
[DESIGN-SESSION-2026-09-19-epoch-log-review.md](design-sessions/DESIGN-SESSION-2026-09-19-epoch-log-review.md)
(findings `C-1` through `C-5`). Independent of each other; listed in ascending cost. Nothing in this
milestone was observed failing at the sample's current constants -- these are a withdrawn justification,
two hang shapes, a mis-keyed trigger, and a specification gap. (`M21.3` predicted that its trigger was
merely unreachable *today*; measuring it while implementing showed it is unreachable at any constants, so
what it corrected was the coupling rather than a latent bug. The archived entry has the numbers.)
- [x] **M21.1** -- Correct the last site that still asserts [D-24](DESIGN-NOTES.md#d-24)'s withdrawn
half: the epoch-order assertion in the epoch-log committer, whose justification cited the hold-back
claim [D-47](DESIGN-NOTES.md#d-47) removed.
-> [completed 2026-09-20](COMPLETED-CHECKLIST.md#m211)
- [x] **M21.2** -- Publish a bounded pop and the wait it is generic over, then remove the two unbounded
spins. `IoRing::pop_within` / `pop_within_with`, over a `CompletionWait` the caller supplies, because
[D-21](DESIGN-NOTES.md#d-21) means the crate cannot choose the wait for them.
-> [completed 2026-09-21](COMPLETED-CHECKLIST.md#m212)
- [x] **M21.3** -- Key the epoch commit off a completed append rather than off the counter, so the
trigger cannot fire on a pass that appended nothing. The predicted latent bug turned out to be
unreachable at any constants -- measured, not re-reasoned -- so this is a coupling change rather than
a fix.
-> [completed 2026-09-21](COMPLETED-CHECKLIST.md#m213)
- [x] **M21.4** -- State what a *failed* commit does to `durable_through`, and bind it with tests in both
directions. Required making the sample a test target at all (`test = true`), and gating the
failure-path tests on `fault-injection`, since a healthy flush cannot be made to fail.
-> [completed 2026-09-21](COMPLETED-CHECKLIST.md#m214)
- [x] **M21.5** -- Give the harness's wait loops a bound, and collapse the hand-written waits onto the
bounded pop. The item named two loops; a census found four, plus two flaky single-`try_pop` sites.
-> [completed 2026-09-21](COMPLETED-CHECKLIST.md#m215)
- [x] **M21.6** -- Fix the four defects an independent review of the `M21.2` surface found: the timeout
mapping, its victim in `run_down`, the `INFINITE` collision, and the test hole that hid all of them.
-> [completed 2026-09-21](COMPLETED-CHECKLIST.md#m216)
### M21+ -- Queued by the 2026-09-21 API review
Queued from the review of the `M21.2` surface, recorded in
[DESIGN-SESSION-2026-09-21-m21-remediation-findings.md](design-sessions/DESIGN-SESSION-2026-09-21-m21-remediation-findings.md).
Its other four findings were fixed in `M21.6`.
- [x] **M21+.1** -- Teach [check-borrow-surface.ps1](../../tools/check-borrow-surface.ps1) the two shapes
it was blind to: methods of a `pub trait`, and borrows in parameter position. Four entries appeared, one
of them predating the widening; the probes also found a latent bug in the checker itself.
-> [completed 2026-09-21](COMPLETED-CHECKLIST.md#m21plus1)
### M22 -- Epoch-log review: submission and arena
Queued from the same session (findings `E-1` through `E-3`). `M22.1` is sequenced first because `M20.6`
re-reads numbers that its change moves.
- [x] **M22.1** -- Batch an epoch's appends into one submission in both append paths, and measure
whether the per-record submission cost was flattening the strategy comparison. It was not:
throughput did not move out of the noise, though commit p50 did.
-> [completed 2026-09-22](COMPLETED-CHECKLIST.md#m221)
- [x] **M22.2** -- Collapse the two free-slot implementations to one, derived from the arena's own
outstanding counts rather than tracked beside them. The item called both correct; one was not --
the tracked free list leaked a slot on every refused append.
-> [completed 2026-09-22](COMPLETED-CHECKLIST.md#m222)
- [x] **M22.3** -- Give the registered arena a stated placement: the epoch-log arena is placed on the
NUMA node its own log file's volume reports, and the allocator moved into the library as
`NumaBuffer` rather than being copied a second time. The sample says plainly that the placement
cannot pay at this workload.
-> [completed 2026-09-22](COMPLETED-CHECKLIST.md#m223)
### M22+ -- Queued by what the M21 work left behind
- [x] **M22+.1** -- Make [bounded_pop.rs](tests/bounded_pop.rs) independent of how fast a device is, by
reading from an overlapped pipe nobody has written to. Filed and completed the same hour; the deferral
was a scheduling preference rather than a blocker.
-> [completed 2026-09-21](COMPLETED-CHECKLIST.md#m22plus1)
### M24 -- Make the unit suite hermetic
The defect and its classification are [D-49](DESIGN-NOTES.md#d-49); the remedies and their costs are
[DESIGN-SESSION-2026-09-21-hermetic-unit-tests.md](design-sessions/DESIGN-SESSION-2026-09-21-hermetic-unit-tests.md).
It began at **63 of 131 lib tests opening a real kernel ring**, so `cargo test --lib` did not mean
what its name implies, and the repository's own Quality rule already classifies an operating-system
API as an external boundary.
**Where it ended: 41 of 151 open a ring, 110 do not.** The remainder is not movable without
`M26.2`'s FFI seam, and [D-53](DESIGN-NOTES.md#d-53) records the rung that keeps it from climbing
back -- an inventory of *which* tests open a ring, since a zero-check would fail on day one and
could only be satisfied by deleting coverage.
**Sequencing is the open question, not whether. Corrected 2026-09-22: the cost of waiting is close
to zero, which is the opposite of what this paragraph first said.** It claimed that waiting
compounds, "because every milestone that adds tests adds to the pile to be migrated, and `M22` is a
testing-heavy milestone". The mechanism is real but the instance was not checked, and it is false:
**all three `M22` items touch only `examples/epoch_log/`**, and none adds a lib test.
**Unconditional as of 2026-09-22.** `M24.1` concluded and `M24.4` is withdrawn, so nothing in this
milestone waits on an evaluation any more. The hermetic goal is reached by relocation and by the
accounting extraction alone; the technique `M24.1` went looking for turned out to be a different
and larger thing, and is `M26`.
Checked across the whole queue rather than for `M22` alone, since the first claim was wrong for
want of exactly that: **no pending item outside this milestone modifies `src/**/tests.rs`.** `M22`
is example-only; `M23.1` is the *sample's* `contract.rs`, not the crate's; `M20.1` and `M20.6` are
documentation and the `ring_copy` sample; `M23.2` is a decision that may imply API later. The 63
therefore do not grow while this waits.
So sequencing turns on other things, and they point the other way:
- **`M24.2` is an internals refactor of a published crate**, and the branch carrying this work is
already 19 commits with one `feat` and three `fix` commits on it. Stacking a field-layout change
on top makes one review cover both a new public API and that refactor.
- **`M24.1` is an evaluation whose answer could invalidate `M24.4`**, so beginning the build before
it concludes risks building something the evaluation rejects.
- **`M22.1` unblocks `M20.6`**, an open question since 2026-09-07 about whether a strategy still
earns its place in a published sample -- which is a decision waiting on a measurement `M22.1`
produces.
**`M24.1` concluded (2026-09-22) and nothing here waits on it.** `M24.4` is withdrawn; `M24.2`,
`M24.3` and `M24.7` are the path to a hermetic suite and are independent of each other.
- [x] **M24.1** -- Settle whether a co-tested fake escapes the mock objection. **Answered: the fake
was the wrong instrument.** A shared suite is strong over what we specify and blind to the
platform's incidental behaviour, and an assertion about the latter is a frozen observation rather
than a contract. Superseded by the resolver in `M26`.
-> [completed 2026-09-22](COMPLETED-CHECKLIST.md#m241)
- [x] **M24.2** -- Extract the handle-free accounting into its own type, composed by `IoRing`. The
item's field split was verified exactly: five fields carry no kernel state, five do.
`Accounting` now owns them with 19 hermetic tests, and `IoRing` delegates nine methods.
-> [completed 2026-09-22](COMPLETED-CHECKLIST.md#m242)
- [x] **M24.7** -- Convert the lib tests that construct a ring only to exercise bookkeeping.
**61 -> 52**, by narrowing `Token::new` to take the ring's ledger rather than the ring. The
remaining 52 are not convertible and the reason is structural, not effort -- see the archive.
-> [completed 2026-09-22](COMPLETED-CHECKLIST.md#m247)
- [x] **M24.4** -- **Withdrawn by `M24.1` (2026-09-22).** A shared conformance suite over a
hand-written fake is superseded by the response-space resolver in `M26`, which serves the same
purpose without encoding a belief about the platform at all. Nothing is deferred by this: `M24`'s
goal is a hermetic lib suite, and `M24.2` plus `M24.3` achieve that without it.
- [x] **M24.3** -- Relocate the lib tests that open a ring but use only public API into `tests/`.
**52 -> 41.** Eleven moved; the "25" the item predicted was never achievable, and the reason is
the same structural one `M24.7` found.
-> [completed 2026-09-22](COMPLETED-CHECKLIST.md#m243)
- [x] **M24.5** -- Put the rule on a rung. An **inventory** of which lib tests open a ring
([D-53](DESIGN-NOTES.md#d-53)), not the zero-check the item assumed -- that rule is false and
could only be satisfied by deleting coverage. The guard's own bidirectional check found a defect
in the guard.
-> [completed 2026-09-22](COMPLETED-CHECKLIST.md#m245)
- [x] **M24.6** -- Sweep what this milestone makes false. Two of the three sites the item named
were false alarms; the third was false for a different and larger reason than the item gave, and
the sweep found two more it did not name.
-> [completed 2026-09-23](COMPLETED-CHECKLIST.md#m246)
### M23 -- The ring as a durability domain, and storage affinity
Queued from the 2026-09-19 epoch-log review (findings `S-1` and `S-3`). `S-2` is an addendum to `M20.6`
rather than an item here. `M23.1` and `M23.2` are done; `M23.3` is the remaining question, and it is
about this crate's own surface rather than about storage at all.
- [x] **M23.1** -- State in the epoch-log contract that the barrier is ring-wide while the flush names a file, so one ring per log is a precondition of the cost model. -> [completed 2026-09-23](COMPLETED-CHECKLIST.md#m231)
- [x] **M23.2** -- Decide how a caller arrives at a NUMA node: `win-numa-sys` offers declaring and discovering, and refuses the shortcut that does both at once. -> [completed 2026-09-23](COMPLETED-CHECKLIST.md#m232)
- [x] **M23.3** -- Decide what this crate offers for holding a token between push and completion: the ring owns the inventory, `IoRing` becomes generic, and the break is accepted. -> [completed 2026-09-23](COMPLETED-CHECKLIST.md#m233)
- [x] **M23.4** -- Drop guards that panicked during unwind aborted the process instead of reporting; they now stay silent while `std::thread::panicking()`. -> [completed 2026-09-23](COMPLETED-CHECKLIST.md#m234)
- [x] **M23.5** -- Both asserts in `IoRing::drop` are now reached by tests; the raw-HRESULT seam the item priced turned out not to be needed, because the kernel refuses a null ring handle cleanly. -> [completed 2026-09-23](COMPLETED-CHECKLIST.md#m235)
### M28+ -- Opened by the inventory
- [x] **M28.7** -- Decided against: the ring can answer 1 of 4 violations, and an internal check would be wrong about 13 live `_raw` push sites. -> [completed 2026-09-26](COMPLETED-CHECKLIST.md#m287)
- [x] **M28.8** -- `RingContract::in_flight` added; two hand-written copies deleted. The drain loops it named turned out not to depend on it -- `P-4` does. -> [completed 2026-09-26](COMPLETED-CHECKLIST.md#m288)
## <a id="moved-2026-10-01-m26143"></a>Moved 2026-10-01 21:26:40 -04:00 -- M26.14.3, the draining-teardown decision
### <a id="m26143"></a>M26.14.3 -- Taken, and taken the way this item proposed: `windows-threadpool-sys` adopted the draining teardown. *(completed 2026-10-01 21:26:40 -04:00)*
The item as it stood when the decision was taken:
> - [ ] **M26.14.3** -- **DECISION TO RAISE, not to take: adopt the draining teardown in
> `windows-threadpool-sys`?** `M26.14.2` found that `ThreadpoolWait`'s drop -- disarm, then
> `WaitForThreadpoolWaitCallbacks` with `fCancelPendingCallbacks` **TRUE**, then close -- is what
> poisons, and that passing **FALSE** instead prevents it entirely (0 in 20000 against 10).
>
> **It is a one-argument change and a real semantic one, which is why it is a decision rather than
> a fix.** Cancelling returns promptly and abandons a pending callback; draining blocks until that
> callback has actually run. A drop that waits for a callback can deadlock a caller whose callback
> needs something the dropping thread holds -- and this crate's `Drop` is not a place a caller can
> see a deadlock coming. Questions that belong to the engineer, not to this item: whether drop may
> block at all, whether the draining form should be opt-in on a builder rather than the default,
> what it means for `stop_and_drain` and `cancel_pending` which already expose both shapes, and
> whether the suppression machinery that exists to make drop safe is still needed if drop drains.
>
> **Do not ship the sleep.** The 1ms gap works equally well in the measurement and is the worse of
> the two: it is a timing constant with no principle behind it, no established threshold, and it
> would sit in a teardown path forever.
>
#### The decision, taken 2026-10-01
**Adopted.** `windows-threadpool-sys` drains rather than cancels, recorded as [Teardown drains
rather than cancels](../../DESIGN-NOTES.md#teardown-drains) and implemented as `M-T4`. Verified in
the shipped source rather than inferred from the decision: `ThreadpoolWait::drop` disarms and then
calls `WaitForThreadpoolWaitCallbacks` with **FALSE**, and the only remaining **TRUE** is the
explicitly named `try_cancel_pending_no_heal_tracking`.
Each of the four questions this item reserved for the engineer was answered, rather than left
implicit:
- **May drop block at all?** Yes, unconditionally -- rule 1 of that decision makes leaving an
object unsynchronised unacceptable, so the blocking is the point rather than a side effect.
- **Opt-in on a builder, or the default?** Draining is the default. Cancelling survives as an
explicit, differently named call (`try_cancel_pending`, renamed by `M-T6.3` to say in its name
that it is best-effort), so a caller who wants the prompt-return shape asks for it.
- **What does it mean for `stop_and_drain` and `cancel_pending`?** `M-T6.7` made `stop_and_drain`
the uniform name of the synchronous close across every type that drains; `M-T6.3` renamed the
cancelling form and backed it with a self-heal repair, because a cancellation is what can sever
the pool.
- **Is the suppression machinery still needed if drop drains?** Yes. `Drop` still raises the
re-arm suppression before draining -- without it a callback could re-arm during the drain and the
drain could return with the object armed. `M-T4.10` extracted it into a shared `RearmSuppression`
rather than retiring it.
**The sleep was not shipped**, as this item required.
**What this item did not do, and what still owes a measurement.** The decision was taken partly on
this crate's evidence -- `M26.14.2` measured a drain in place of a cancel at 0 failures in 20000
against a control's 10 -- but that was the *reproducer* with a patched teardown, not the shipped
crate. Nothing has yet re-run this crate's stall at scale against the shipped drain. Queued as
`M26.15`, and until it reports, the entry in
[UNRESOLVED-TEST-FAILURES.md](UNRESOLVED-TEST-FAILURES.md) stands.
## <a id="moved-2026-10-01-m2615"></a>Moved 2026-10-01 21:47:17 -04:00 -- M26.15, re-measuring the stall against the shipped drain
### <a id="m2615"></a>M26.15 -- Re-measured: control 12 in 4000, current 0 in 12000, and the stall's entry moved to RESOLVED-TEST-FAILURES.md. *(completed 2026-10-01 21:47:17 -04:00)*
The item as it stood, including the premise that turned out to be false:
> - [ ] **M26.15** -- **Re-run the stall at scale against the shipped draining teardown, and settle
> the unresolved entry either way.** Queued 2026-10-01, when `M26.14.3`'s decision landed.
>
> **What is and is not established.** `M26.14.2` measured a drain in place of a cancel at 0 failures
> in 20000 against a control's 10 -- but that was *this reproducer with a patched teardown*, not the
> shipped crate. `windows-threadpool-sys` has since shipped the drain for real, and nothing has run
> this crate's stall against it. The remedy is believed to work on evidence that predates the thing
> it is now a remedy *in*.
>
> **A green suite run is not that evidence, and must not be mistaken for it.** The stall's measured
> rate is on the order of 13 in 4000, so an ordinary `cargo test` passing says almost nothing: the
> arm has to be sized against the rate. Re-run the same reproducer, at the same scale and with the
> same positive control the earlier measurements used, so the result is comparable to the figures
> already on record rather than a fresh and unrelatable number.
>
> **It needs a control that can still fail.** If the shipped drain really closes it, every arm goes
> to zero -- and a measurement in which nothing can fail cannot distinguish "fixed" from "the
> reproducer stopped reproducing". Keep an arm that forces the old cancelling teardown (reachable
> through `try_cancel_pending`), and require it to still fail, or the zero proves nothing.
>
> **On success:** move the entry out of [UNRESOLVED-TEST-FAILURES.md](UNRESOLVED-TEST-FAILURES.md)
> into [RESOLVED-TEST-FAILURES.md](RESOLVED-TEST-FAILURES.md) under a dated heading, per the
> repository's rule that a resolved failure is moved rather than deleted -- and note there that this
> is the *second* time that file has carried a fix for this stall, the first being `D-68`, which
> `M26.13` overturned.
>
> **On failure:** the drain is not sufficient, which is a finding against the current explanation
> rather than against the decision, and `M26.13`'s diagnostics become live again.
>
#### Correction: this item's central premise was false
**"Nothing has re-run this crate's stall against the shipped drain" was wrong when it was
written**, on 2026-10-01, and nothing in the tree had changed to make it so -- the evidence was
already committed and was not looked for.
[2026-09-29-the-fix-on-the-real-path](../windows-threadpool-sys/measurements/2026-09-29-the-fix-on-the-real-path/README.md)
had measured exactly that: the committed drain, on the real `EventDelivery` path, at 33 failures
in 10000 reverted runs against 0 in 70000 drained. The item was written from `M26.14.2`'s
hand-rolled model and from the absence of a *later* measurement that in fact existed one directory
away.
The mistake is worth recording rather than quietly fixing, because it is the cheap kind to repeat:
the claim was about **what the repository already knows**, which is checkable by reading it, and it
was asserted from the shape of the argument instead. `measurements/` is the artifact that answers
such a question, and was not consulted before the gap was declared.
#### What the run did establish
The narrower question the item should have asked. The 2026-09-29 measurement was taken against the
`M-T4` build, and three things have changed the teardown path since: `M-T6` added the default-on
self-heal, whose repair timer **submits a work item** -- the one action `M26.13.3` found ends the
stall, so a mechanism that could mask the failure rather than leave it absent; `M-T6.3` renamed the
cancelling form; and `EventDelivery` gained its own draining `Drop` on 2026-10-01.
Measured in
[2026-10-01-the-fix-still-holds-after-the-self-heal](../windows-threadpool-sys/measurements/2026-10-01-the-fix-still-holds-after-the-self-heal/README.md):
the control reproduces at 12 in 4000, inside the range every earlier capture of that arm reported,
and the current build reports 0 in 12000. Under an unchanged rate, 12000 runs would be expected to
produce about 36 failures.
The control was run **first** and from the same tree, because a zero on its own cannot distinguish
a fix from a reproducer that has stopped reproducing. Both control patches were made with a tool
that fails on a missing anchor and read back out of the source before building, per the procedure
adopted after a silent `.Replace()` voided a 60000-run measurement.
**Left open deliberately:** the run did not instrument whether any self-heal repair fired, so "the
self-heal did not mask anything" rests on the mechanism -- a repair is only owed after a
cancellation, and the drain records none -- rather than on a measurement. A `--no-default-features`
arm would settle it, and is not queued, because nothing currently depends on the answer.
## <a id="moved-2026-10-01-m2614"></a>Moved 2026-10-01 22:03:17 -04:00 -- M26.14, the leaking temp files
### <a id="m2614"></a>M26.14 -- A self-removing `TempPath` guard, shared by 14 test files: an all-passing suite run went from 25 leaked files to 0, and a test panicking with its handle open now leaves none. *(completed 2026-10-01 22:03:17 -04:00)*
The item as it stood, including the premise that turned out to be wrong:
> - [ ] **M26.14** -- **The ioring tests leak their temp files, and nothing cleans up.** `temp_file` in
> [tests/event_delivery.rs](tests/event_delivery.rs) builds a path under the system temp directory
> and no test removes it; each run of the M26.13 reproducer leaves two behind, and roughly thirty
> thousand runs during that investigation accumulated **307,383 files, 1.2 GB** on the development
> machine before they were deleted by hand. It is not confined to that file: **15** of this crate's
> test files build paths under the temp directory and none clean up.
>
> **Measured not to be a confound** before being queued -- the reproducer gives 13 failures in 4000
> against a cleared directory and 16 against a full one, both inside its usual range
> ([measurements/2026-09-28-re-verifying-the-premises/](../windows-threadpool-sys/measurements/2026-09-28-re-verifying-the-premises/README.md)).
>
> **Deliberately not fixed during M26.13**, and the reason is the blocker rather than a preference:
> adding teardown changes the reproducer's shape while it is the instrument of an active
> investigation, and every failure rate on record would have to be re-established against the new
> one. Take it once M26.13 closes, or take it sooner as an explicit decision to re-baseline. The
> fix itself is small -- an RAII guard returned by `temp_file` that removes the path on drop, so
> that a panicking test still cleans up.
#### Correction: "none clean up" was wrong, and the correction sharpened the fix
**Twelve of the fifteen files already removed their temp files**, across 45 call sites, and
`flush_barrier_stress.rs` already carried a correct RAII guard with the hazard documented. Only
three -- `completion_event.rs`, `event_delivery.rs` and `submission_lifecycle.rs` -- had no removal
on any path.
The real defect is narrower and explains the leak better than the item's version did: **a trailing
`remove_file` does not run when the test panics**, and `M26.13` was thirty thousand runs of a
reproducer whose failing arm panics. Had the item's framing been taken at face value, the fix would
have been fifteen new cleanup calls on the path that already worked, and the panicking path -- the
one that produced the 307,383 files -- would have been left exactly as it was.
#### What was built and what it was measured against
A `TempPath` guard in `tests/common/mod.rs`, one definition, included by the 14 converted files.
Measured in
[2026-10-01-the-tests-stop-leaking-temp-files](measurements/2026-10-01-the-tests-stop-leaking-temp-files/README.md):
an all-passing suite run leaked **25** files before and **0** after.
**The panicking path was verified rather than assumed**, because a zero on a passing run does not
establish the property the guard exists for. A deliberate `panic!` was placed in a test *after its
file handle is open* -- the state in which a removal can fail outright -- and the run panicked
there, exited 101, and left no file. That site had no removal at all beforehand, so the same panic
leaked before the change.
#### Two things deliberately left alone, each for a reason
**The 45 existing `remove_file` calls stay.** They look redundant and are not: each runs at a point
the test controls, after the test has closed its own handle, which is strictly more reliable than a
drop order that depends on declaration order. The guard is the net underneath them, and a removal
that finds nothing is not an error, so the two compose.
**`flush_barrier_stress.rs` keeps its own `Fixture` guard**, which closes the handle before
deleting. Converting it for uniformity would have risked the precise defect it was written to fix --
every trial silently leaking a 32 MiB extent because the handle was held in the same struct.
#### The gate this item carried
It said to take this once `M26.13` closes, or sooner as an explicit decision to re-baseline, because
changing the reproducer while it was an active instrument would invalidate every failure rate on
record. `M26.13` is still open, so this was the second case. The re-baselining cost is now small:
`M26.15` re-established the rates earlier the same day, and the arm that still carries a rate is the
*control*, which is a patched build anyone re-running it would rebuild regardless.
## <a id="moved-2026-10-01-m2841d"></a>Moved 2026-10-01 22:25 -04:00 -- M28.4.1d, closing the inventory migration
### <a id="m2841d"></a>M28.4.1d -- Done by its sub-items; closing it was a documentation sweep, not a migration. *(completed 2026-10-01 22:25 -04:00)*
The item as it stood:
> - [ ] **M28.4.1d** -- Migrate every consumer onto the inventory, then retire the token API.
>
> **Measured before planning, and it is larger than "36 files" suggested**: 172 push call
> sites and **116 `claim_if` sites** across 35 files. `claim_if` is not a substitution --
> it is how each test *drives* its ring, so converting restructures control flow rather
> than replacing a call.
>
> **What a conversion actually does, which is why it is worth it.** The caller's
> `HashMap<usize, (sidecar, Token<..>)>` *disappears* at each site: the push carries the
> sidecar as `X`, and the pop returns `(payload, sidecar)` together. That is `D-55` paying
> off rather than a cost being paid.
>
> **Batched, and the reason that is legitimate.** Both APIs coexist today, so a
> partly-converted tree still compiles and every batch is a green commit. `M28.4.2`'s
> "convert all of them or none" governs the **shipped** state -- never two token models in
> a release -- not the path to it. The final batch is what makes that true, and nothing is
> released in between.
>
#### What was actually left, which was not migration
Every sub-item (`d.1`, `d.1b`, `d.2`, `d.2b`, `d.3`) was already complete, and the code bears that
out: `src/token.rs` defines `OperationId` and nothing else, `try_pop_held` is gone, and the 47
surviving `claim_if` sites are `PendingBufferRegistration` / `PendingFileRegistration` -- the
registration mechanism, which `d.2` already recorded as a different thing from the push set.
So the parent was an unchecked box over finished work. What closing it found instead was a sweep
`M28.6` did not complete: **documentation that still describes the retired token API in the present
tense**, which a reader would take as current.
- **`contract.rs`'s "What it checks" table still advertised the leak rule `D-74` retired** --
"every token is claimed, or deliberately leaked", sourced to `D-13`. The `Violation` enum has
exactly the four kernel-level variants `M28.4.1c` said it would keep, so the table was promising a
fifth check the oracle does not make. The paragraph beneath it told a reader to report `_raw`
pushes "or the oracle will demand a claim that cannot be made", which it no longer can. That
paragraph also sat *between* two table rows, so the table did not render as a table at all.
- **`ring.rs` carried a worked example of `token.claim_if(...)`** in an ```` ```ignore ```` block --
precisely the rot that `ignore` permits, since nothing compiles it. This repository's own rule
prefers `no_run` for that reason.
- **`ring.rs` attributed a safety argument to `Token::claim_if`**, a type that no longer exists.
- **`batch.rs` told callers four times to "claim its token"** to get a buffer back, which is now
done by popping the completion.
- **`token.rs`'s own module doc still announced `Token<T>`** as what the file defines.
**What was deliberately left alone.** Most `Token` mentions in this crate are *history* and read
correctly as such -- "the identity that survived `Token`'s retirement", "it used to say", the `D-74`
citations. Rewriting those would erase the record of a decision rather than correct a stale claim.
The test is whether a reader would act on the sentence, not whether the word appears.
**Verified by execution, not by reading**: full suite under `--all-features`, and `cargo doc` with
`-D rustdoc::broken_intra_doc_links`, which also cleared a pre-existing private-link warning in the
file being edited.
## <a id="moved-2026-10-01-m26-and-m28"></a>Moved 2026-10-01 22:34:22 -04:00 -- M26+ and M28, both fully complete
Two milestones with every item done. `M26+` is what remains of the wakeup-window review after its
stall half transferred to `windows-threadpool-sys`; `M28` is the pending-inventory break, closed
when `M28.4.1d` did. `M27` stays open in [CHECKLIST.md](CHECKLIST.md).
### M26+ -- The wakeup window review opened
- [x] **M26.12** -- Not a race: the setup signal was owed only to the call that attached the event, so a caller that attached earlier got no wakeup at all. -> [completed 2026-09-26](COMPLETED-CHECKLIST.md#m2612)
- [x] **M26.13** -- Transferred to `windows-threadpool-sys`: the fault is pool-wide, not this
crate's. The question, the timeline and the 33 measurement captures now live there as `M-T7.1`.
-> [windows-threadpool-sys/CHECKLIST.md](../windows-threadpool-sys/CHECKLIST.md)
- [x] **M26.14** -- A self-removing `TempPath` guard, shared by 14 test files: an all-passing suite
run went from 25 leaked files to 0, and a test panicking with its handle open now leaves none.
The item's "none clean up" was wrong; 12 of 15 did, just not on the panicking path. ->
[completed 2026-10-01](COMPLETED-CHECKLIST.md#m2614)
### M28 -- The ring owns the pending inventory
Queued by [D-55](DESIGN-NOTES.md#d-55), taken 2026-09-23 after the `M23.3` exploration. The break
is accepted deliberately: `IoRing` becomes generic so the inventory cannot drift from the ring,
because a consumer never holds a token to lose. The exploration and everything it falsified is in
[DESIGN-SESSION-2026-09-23-pending-inventory.md](design-sessions/DESIGN-SESSION-2026-09-23-pending-inventory.md);
`src/pending.rs` is the working spike and is the shape the internal map starts from.
**Sequenced so each step compiles.** The published crate is at 0.3.1, so this is a major bump and
every consumer names the type -- which means the migration order matters more than usual.
- [x] **M28.1** -- Decided: a push returns a `Copy` identity that owns nothing, and the ring returns the payload at its own pop -- `Token` is split, not moved. Recorded as [D-71](DESIGN-NOTES.md#d-71). -> [completed 2026-09-25](COMPLETED-CHECKLIST.md#m281)
- [x] **M28.2** -- `RingContract` is bounded by operations in flight: terminal entries are retired, and a capped history keeps a duplicate distinguishable from an unrecognised completion. Recorded as [D-72](DESIGN-NOTES.md#d-72). -> [completed 2026-09-25](COMPLETED-CHECKLIST.md#m282)
- [x] **M28.3+M28.4** -- **Make `IoRing` generic, move the inventory inside, and migrate every
consumer, as one commit.** Complete 2026-10-01, when `M28.4.1d` closed. Gated on [D-71](DESIGN-NOTES.md#d-71), which settled what a caller
receives.
**Merged deliberately, and the coupling is acknowledged rather than disguised.** The milestone
header requires each step to compile, `M28.4` requires converting all consumers or none, and
`M28.3` changes `IoRing`'s shape -- so the three cannot all hold with the items separate. The
alternative considered and rejected was landing `M28.3` additively, with an `OperationId` path
beside the existing `Token` one: it compiles at every step, but it leaves two token models live
at once and the old one still lets a consumer lose a token, which is the defect `D-55` exists to
remove. Never having both is worth one large commit.
**Carry the sidecar.** The census found two thirds of consumers keep per-operation data beside
the token, so an inventory holding only tokens serves a minority. Mixed-shape consumers use a
closed `enum`; [generated_sequences.rs](tests/generated_sequences.rs) is the worked example.
**Soundness already settled:** [`IoBuf`](src/buf.rs) is an unsafe trait whose contract requires
the address to survive a move, so the ring may hold buffers in a map.
Sequenced so the work is resumable, since it does not compile in the middle:
- [x] **M28.3.1** -- `OperationId`: `Copy`, no `Drop`, a name and not a capability (`D-71`).
- [x] **M28.3.2** -- `IoRing<T = ()>` carrying `inventory: HashMap<usize, T>`, and
`Batch<'ring, T>` with it. A default keeps a payload-free consumer from naming `()`.
- [x] **M28.3.3** -- Push stores the payload and returns an `OperationId`; pop returns the
payload with the completion. Shape settled by [D-73](DESIGN-NOTES.md#d-73):
`IoRing<T, X = ()>`, with the file guard held in a concrete internal `Held` rather than
made generic, which is sound because `FileTarget` is sealed.
**Do the `Drop` half in this step, not later.** Moving buffers into the ring removes the
protection `Token`'s leak-on-unclaimed-drop provides on the path where `run_down` fails --
which `Drop for IoRing` takes best-effort, closing anyway. The inventory must be
*forgotten* rather than dropped there, or the close frees memory the kernel may still be
writing into. Landing the inventory without this is a use-after-free, so it is one step.
The tokenless shape is `M28.5`'s and is only accommodated here, not answered.
- [x] **M28.3.4** -- Carry the parameter through `EventDelivery`, `RingScope` and the contract
wiring.
**The tree stops compiling here, as this plan said it would.** The delivery callback is
now `Fn(Completion, Option<(T, X)>)`, so ten call sites across
[event_delivery.rs](tests/event_delivery.rs), [handover.rs](tests/handover.rs),
[checkpoint.rs](examples/epoch_log/checkpoint.rs) and
[model_a_delivery.rs](examples/model_a_delivery.rs) take a one-argument closure and no
longer build. The library and its own unit tests are green; the integration and example
targets are `M28.4.1`'s to migrate. Contract wiring is untouched deliberately -- what
becomes of the oracle's leak rules is a decision `M28.4.1` carries, per
[D-73](DESIGN-NOTES.md#d-73).
- [x] **M28.4.1a** -- Restore the tree: every delivery callback takes the payload, so the
build is green again on every feature set.
**Two call sites were invisible to an ordinary sweep**, and both are worth remembering
rather than rediscovering. One lives in a **doctest**, found only because this repository
compiles its prose. The other is in [failure_paths.rs](tests/failure_paths.rs), which
compiles only under `--all-features`, so a default-feature check could not see it. A
migration sweep here has to run `--all-features` **and** `--doc` before it means anything.
- [x] **M28.4.1b** -- All ten push shapes have an inventory form: `read_raw_owned`,
`write_raw_owned`, `read_owned`, `write_owned`, `flush_owned`, `cancel_owned`, and the
four registered variants. `Held` is populated by the guarded pushes and
`Held.registration` by the registered ones, so both halves are exercised and neither
needs a dead-code marker any longer. `read_owned` is the only
entry point that stows, so "migrate the consumers" has no destination for the other ten
shapes yet -- writes, flushes, cancels, and the registered variants. Give each an
inventory form first, populating `Held` for the guarded ones, which is what retires the
`#[expect(dead_code)]` on `Held` and `FileGuard`.
- [x] **M28.4.1c** -- Decided in [D-74](DESIGN-NOTES.md#d-74): the leak rules **retire** rather
than narrow, because a push that hands back only an `OperationId` leaves nothing to drop
unclaimed. `State::Pushed` and `State::PushedTokenless` collapse with them. The
conservation they approximated becomes `held() == outstanding()`, which the ring can check
about itself. `RingContract` keeps the four claims that are about the kernel rather than
about a caller's bookkeeping.
- [x] **M28.4.1d** -- Done by its sub-items; closing it was a documentation sweep, not a
migration. Four present-tense references to the retired token API survived `M28.6`,
including a rule the `RingContract` module still advertised that `D-74` had removed. ->
[completed 2026-10-01](COMPLETED-CHECKLIST.md#m2841d)
- [x] **M28.4.1d.1** -- [bounded_pop.rs](tests/bounded_pop.rs) converted as the worked
pattern. The `Token<Vec<u8>>` threaded through `push_pending_read`, `settle` and six call
sites is gone; `PipeRing = IoRing<Vec<u8>>` holds the buffer instead. The conversion
found a real defect in rundown, which is recorded as `M28.4.1d.1b`.
- [x] **M28.4.1d.1b** -- Decided: there should be one pop, not two. **Decide what `try_pop`
and `pop_within` mean on a ring that holds payloads.** Found by converting the first file: `drain_for_rundown` popped with `try_pop`
and never reclaimed, so rundown stranded every entry it reaped. Fixed there -- rundown is
teardown, so dropping is right, and the completion is the proof that makes freeing safe.
**The same gap is open on the public paths, and the answer is that the split should not
exist.** An earlier note here offered three ways to manage a permanent distinction
between `try_pop` and `try_pop_held`. That was wrong, and the symmetry is the argument:
`try_pop_held` sits beside `try_pop` for exactly the reason `read_raw_owned` sits beside
`read_raw` -- the inventory was added *beside* the token API rather than replacing it.
Both are the same transitional duplication, and the push side is already settled as
ending with one family.
**Nothing needs a non-reclaiming pop.** Measured: the only internal callers are
`drain_for_rundown` (now reclaims), `pop_within_with`, and one other -- and once the
token pushes are gone, *every* operation has an inventory entry, including the tokenless
ones, which stow `payload: None`. There is no operation a pop could legitimately find
nothing for.
So `try_pop` becomes the reclaiming pop, `try_pop_held` disappears as a name, and
`pop_within` returns the same shape. Preserving the distinction would be a rule against
an impossible act, which [D-74](DESIGN-NOTES.md#d-74) already names as worse than no rule
at all.
**This does not gate the conversion.** Consumers migrate onto the reclaiming pop either
way; the rename lands in `M28.4.1d.3` beside the push retirement.
- [x] **M28.4.1d.2** -- Every consumer that can be converted before the token API is retired now is; three plus `append.rs` are blocked on `M28.4.1d.3`. -> [completed 2026-09-26](COMPLETED-CHECKLIST.md#m2841d2)
- [x] **M28.4.1d.2b** -- Documented the two ways round a mixed payload on `IoRing::with_inventory` and in `D-73`; no `IoBuf` enum helper built. -> [completed 2026-09-26](COMPLETED-CHECKLIST.md#m2841d2b)
- [x] **M28.4.1d.3** -- Token API retired, `D-74` applied, one reclaiming pop per shape; the break is closed. -> [completed 2026-09-26](COMPLETED-CHECKLIST.md#m2841d3)
- [x] **M28.4.2** -- Five inventory sabotages added (push, both pops, both appender halves); the sweep also found `d.3` had broken the manifest. -> [completed 2026-09-26](COMPLETED-CHECKLIST.md#m2842)
- [x] **M28.5** -- `observe_tokenless_push` retired; the outer `None` has two causes the caller distinguishes, recorded as `D-75` and asserted both ways. -> [completed 2026-09-26](COMPLETED-CHECKLIST.md#m285)
- [x] **M28.6** -- Swept what the break made false: `D-4` and `D-55` amended, six live example claims corrected, and three defects found in `d.3`'s own prose sweep. -> [completed 2026-09-26](COMPLETED-CHECKLIST.md#m286)
## <a id="moved-2026-10-01-m271"></a>Moved 2026-10-01 22:42 -04:00 -- M27.1, the realizer census
### <a id="m271"></a>M27.1 -- Census done: one gap, and it is not in this crate. *(completed 2026-10-01 22:42 -04:00)*
The item as it stood:
> - [ ] **M27.1** -- **Census what a realizer would need from this crate, against the plan vocabulary,
> and name what is missing.** A plan states which processor a domain pins to, which memory node its
> pool allocates from, how many queues of which types, and where each channel's buffer lives. Walk
> each of those to the public API that would realize it and record the gaps. `NumaBuffer`
> ([D-51](DESIGN-NOTES.md#d-51)) is one half of the pool answer and arrived this month; the ring's
> own construction takes no placement input at all. **The output is a gap list, not an API** --
> proposing surface before the plan vocabulary is settled would be binding to a draft.
>
#### The gate was examined rather than obeyed or ignored
The item carried a `CROSS-COMPONENT PREREQUISITE` on the planner's `EP-1+.5` and `EP-1+.6`, and
both are still open. They were read before proceeding, and they do not bind a census:
- **`EP-1+.5`** decides the *type* of stage 1's connectivity graph and which crate holds it.
- **`EP-1+.6`** decides that the answer is plural and what a candidate carries beyond the
arrangement.
Neither changes what a realizer needs **from this crate**: a realizer consuming one candidate or
five still supplies the same construction inputs, and the four physical facts the census walks are
enumerated in `M27.1`'s own text rather than drawn from a type that does not exist yet. The gate
binds `M27.2`, which proposes surface -- binding an API to a draft vocabulary is precisely the
failure it names -- and a gap list binds nothing.
Recorded rather than quietly decided, because the prerequisite is the engineer's and this is a
reading of it: if the intent was that *nothing* under `M27` proceeds until the vocabulary lands,
this census was taken early and the finding stands independently of when it was taken.
#### The finding, and what it was not
**One gap, and it is not in this crate** -- which is not what the item expected. `M27.1` was
written anticipating this crate would be found short, and the two things it is short of turn out to
be someone else's:
- **Thread placement** has no expression anywhere in the workspace. Measured surfaces:
`ThreadpoolPool` offers `new` / `set_min_threads` / `set_max_threads`; `CallbackEnviron` offers
`set_pool` / `clear_pool` / `set_priority` / `set_runs_long`. A domain can get its own pool with a
bounded thread count; it cannot say which processor runs it. Queued as `M-T8.1` in
[windows-threadpool-sys](../windows-threadpool-sys/CHECKLIST.md).
- **The ring's own queue memory** takes no placement input because `CreateIoRing` has no parameter
for one -- a version, two sizes, and required/advisory flag words. That is a platform limit to
state, not a gap to close.
**Two expectations in the item were corrected by looking.** It said `NumaBuffer` is "one half of the
pool answer": it is the *whole* of the caller-buffer answer, because
[numa_buffer_io.rs](src/numa_buffer_io.rs) implements `IoBuf` and `IoBufMut` for it, so a node-bound
allocation is pushed and registered like any other buffer with no new surface at all. And it said
"the ring's own construction takes no placement input at all", which is exactly right -- but the
reason is the platform, not an omission here.
## <a id="moved-2026-10-01-m27-retired"></a>Moved 2026-10-01 23:08 -04:00 -- M27.1 and M27.2, retired rather than completed
**Retired, not finished.** `M27.1` was done and `M27.2` was open; both are withdrawn because the
question they served does not belong to this crate.
### Why
The design records a **planner** and a **realizer** as distinct things: the planner experiments
against a description of the application's structure and the machine's reported characteristics,
and the realizer assembles the I/O, queue and thread topology from the planner's analysis.
[EP-D-5](../topology-planner/DESIGN-NOTES.md#ep-d-5) places the realizer **outside** this crate --
its own component, depending on `topology-model` and the runtime crates -- and it does not exist.
`windows-ioring-sys` fundamentally delivers a safe API over `IoRing`. If a realizer turns out to
need API it does not have, that is the ordinary lifecycle of a consumer asking its layer for
something, worked out when it happens by the component that discovered the need. A standing
milestone item here saying *the realizer might one day need something from us* is not work; it is
an obligation to a consumer nobody has written, and it cannot be discharged because nothing can
tell us whether it is satisfied.
**This is a blocked-on-a-missing-dependency retirement, not a "no visible client" deferral.** The
distinction matters because the second would be the anti-pattern. The dependency is named and
specific: the realizer component of `EP-D-5`. What is retired is the *speculation*, not the
capability -- nothing about this crate's API changed, and nothing stops a realizer asking later.
### What the retired work did produce, and where it went
- **The one real gap was not in this crate.** The census found that thread placement has no
expression anywhere in the workspace: `ThreadpoolPool` offers `new` / `set_min_threads` /
`set_max_threads`, and `CallbackEnviron` offers `set_pool` / `clear_pool` / `set_priority` /
`set_runs_long`. That is queued as `M-T8.1` in
[windows-threadpool-sys](../windows-threadpool-sys/CHECKLIST.md), which states the measured
surfaces itself and does not depend on the census document.
- **The census document and the synthetic-plan test are deleted.** Both existed to answer the
retired question. The test was a hand-rolled stand-in realizer living in this crate's `tests/`,
which in hindsight was the clearest signal that the question had the wrong owner.
- **Two assertions survived, because they are this crate's own contract.** Nothing asserted that a
ring reports back the exact depths it was created with -- the nearest was
`submission_queue_size > 0` -- and nothing exercised the `IoBuf` / `IoBufMut` impls for
`NumaBuffer`. Both are now in
[ring_and_buffer_contracts.rs](tests/ring_and_buffer_contracts.rs), with no plan vocabulary
anywhere in them.
- **`CreateIoRing` takes no placement parameter** -- a version, two sizes, and required/advisory
flag words. A fact about Win32 rather than a gap in this crate, recorded here because it is the
only durable thing the census learned about the ring itself.
### The items as they stood
> ## M27 -- What this crate owes the topology planner
>
> **Re-planned 2026-09-23, the same day it was written.** M27 was originally "Adaptivity: the benefit
> without the architectural commitment", and asked whether *this crate* should derive a partition for a
> consumer who expresses no preference. That was the wrong owner, and the checklist rules require
> saying so rather than quietly rewriting it. The adaptivity the
> [adoption thesis](../../DESIGN-NOTES.md#the-adoption-thesis) asks for is delivered by
> [topology-planner](../topology-planner/COMPONENT.md), which takes a dataflow description of the
> application and returns one or more suggested realizations
> ([EP-D-6](../topology-planner/DESIGN-NOTES.md#ep-d-6)). Had the original M27.1 been answered here it
> would have grown a second, weaker policy surface beside the one that component exists to provide --
> the `outermost_partitioning_cache` defect again, where a policy answer lands in a crate whose job is
> something else.
>
> > **-> CROSS-COMPONENT PREREQUISITE:** `M27.1` and `M27.2` are gated on component
> > `crates/topology-planner` -> `M1+` -> `EP-1+.5` and `EP-1+.6`, which decide the plan vocabulary
> > this crate would be realized from. See [CHECKLIST.md](../topology-planner/CHECKLIST.md).
>
> **What survives here is the realization end, not the policy end.** The planner emits a plan; the
> outward adapter realizes it as buffers, rings and threads
> ([EP-D-5](../topology-planner/DESIGN-NOTES.md#ep-d-5)). That adapter is a separate crate, but it can
> only build what this crate exposes, and nothing has ever checked that what it exposes is sufficient.
> [D-8](DESIGN-NOTES.md#d-8) is untouched by all of this: policy stays out of this crate, and being
> *constructible from* a policy decision made elsewhere is the opposite of taking one.
>
> - [x] **M27.1** -- Census done: one gap, and it is not in this crate. Caller-buffer placement is
> already covered by `NumaBuffer`; the ring's own queue memory is a Win32 limit; what is missing is
> thread placement, which belongs a layer down. -> [completed 2026-10-01](COMPLETED-CHECKLIST.md#m271)
>
> Full census: `REALIZATION-CENSUS.md`. *(de-linked: deleted with the retirement above)*
>
> > **-> CROSS-COMPONENT HANDOFF:** the gap is queued as component `crates/windows-threadpool-sys`
> > -> `M-T8` -> `M-T8.1` (`Decide whether this crate expresses thread placement, and if so where`).
> > See [CHECKLIST.md](../windows-threadpool-sys/CHECKLIST.md).
>
> - [ ] **M27.2** -- **Still gated on the planner's `EP-1+.6`; its verification half is done.**
>
> **The verification half, done 2026-10-01.**
> `tests/realization_from_a_plan.rs` *(de-linked: deleted with the retirement above)* realizes a plan-shaped description
> against a synthetic machine using only the public API, which is what `M27.1`'s "covered" verdicts
> rested on -- they were reached by reading signatures, and now a test builds the arrangement
> instead. The stand-in plan type is local to that file, so nothing public binds to a vocabulary
> that is not settled. Sabotage-checked: swapping the two queue depths inside the realizer fails the
> depth assertion by name.
>
> **The gap-closing half has nothing to close, and that is why it stays open rather than closing.**
> The census named one gap and it is `windows-threadpool-sys`' (`M-T8.1`); the ring-memory one is a
> Win32 limit. But the census walked the four facts *as `M27.1` described them*, not a settled
> vocabulary -- so whether `EP-1+.6` names a fact nobody has walked is exactly what this item is
> waiting to find out. Closing it now would assert the vocabulary adds nothing.
>
> Unchanged when it resumes: each gap is an input a caller supplies, never a choice this crate
> makes.
>
## <a id="moved-2026-10-01-m273"></a>Moved 2026-10-01 23:14 -04:00 -- M27.3, pricing the alternatives
### <a id="m273"></a>M27.3 -- `ring_copy --compare` prices every policy on the machine in hand and reports what each cost, with no verdict. *(completed 2026-10-01 23:14 -04:00)*
The item as it stood:
> - [ ] **M27.3** -- Give a consumer the means to answer placement questions on their own hardware.
> **Not gated on the planner** -- it is the client-side half of the thesis, and it is what lets a
> developer disagree with any plan they are handed. `cache_domains.rs` now prints every cache level
> beside the heuristic's pick; the equivalent for placement is a sample that reports what a chosen
> arrangement costs and what the alternatives would have cost, on the machine in hand.
> [ring_copy](examples/ring_copy) is the natural host, being already policy-selectable. **Do not ship
> a verdict** -- report the observation and let the consumer conclude, per OPTION INTEGRITY.
#### What was built
`--compare` runs the copy under every policy and reports what each cost, with `--rounds` (default
3) controlling repetitions. Three choices are worth recording, because each was the difference
between a number and a defensible number:
- **The arms share one code path.** `run_arrangement` was extracted from `main` so the alternatives
are priced through the identical path the chosen policy takes. A comparison whose arms do not
share a path measures the difference between the arms' *code* as much as between the
arrangements.
- **The order rotates each round.** Back-to-back runs over the same file are not independent -- the
first pass warms the filesystem cache and every later one benefits. A fixed order would hand that
advantage to the same policy every time and bake it into the result. Rotating does not remove the
effect; it stops it being *attributed* to one arm.
- **Wall time per arrangement, not the sum of its domains.** The domains run concurrently, so
summing them counts the same seconds once per domain and makes a wider arrangement look slower
the more parallelism it was given.
#### No verdict, and the output says why
Per OPTION INTEGRITY, no arm is marked best. The report states what the figures include (the
filesystem cache, whatever else the machine was doing, this sample's own chunking), what they
cannot separate (a genuinely better arrangement from one that ran while the machine was quieter),
and that the choice belongs to the consumer and their workload.
The fastest/slowest/median columns exist for that last point: **where one policy's spread overlaps
another's, this run did not distinguish them**, and the reader can see that rather than being told
a winner.
A single-domain arrangement is reported as the machine saying it cannot express that arrangement --
information about the host, not a finding against the policy.
#### Observed on the development machine
Run against a 16 MiB file, 3 rounds. Reported here as a demonstration that the sample produces
intelligible output, not as a measurement of anything: one file, one machine, one moment.
`ByCache` and `ByCore` both selected 8 domains; `ByNode`, `ByPackage` and `Single` each collapsed to
one on this host. `ByCache`'s slowest round came in far below its median while `ByCore`'s rounds sat
close together -- which is the cold first pass showing up exactly where the rotation is designed to
expose it rather than hide it. A reader who took the medians alone would have missed it, which is
why the spread is printed beside them.
## <a id="moved-2026-10-01-m27"></a>Moved 2026-10-01 23:29:13 -04:00 -- M27, the last open milestone
Completes this crate's checklist. `M27.1` and `M27.2` were retired rather than finished -- the
reasoning is in the [earlier group](#moved-2026-10-01-m27-retired) -- and `M27.3` shipped
`ring_copy --compare`.
### M27 -- Placement questions a consumer can answer on their own hardware
**Re-planned 2026-10-01, and most of this milestone retired.** M27 previously asked what this crate
owes a topology realizer, and carried a census (`M27.1`) and a gap-closing item (`M27.2`) against
the plan vocabulary. Both are retired: see
[COMPLETED-CHECKLIST.md](COMPLETED-CHECKLIST.md#moved-2026-10-01-m27-retired).
**The reason is structural, not a judgement that the work is unimportant.** The realizer is a
distinct component -- [EP-D-5](../topology-planner/DESIGN-NOTES.md#ep-d-5) places it outside this
crate, depending on `topology-model` and the runtime crates -- and it does not exist. This crate
fundamentally delivers a safe API over `IoRing`. If a realizer turns out to need API it does not
have, that is the ordinary lifecycle of a dependency asking its layer for something, worked out
when it happens by the component that discovered the need. It is not a standing obligation this
crate carries on behalf of a consumer nobody has written.
[D-8](DESIGN-NOTES.md#d-8) is untouched by all of this: policy stays out of this crate.
- [x] **M27.1** -- Retired with `M27.2`; the census it produced found no gap in this crate, and its
one finding is queued where it belongs. ->
[retired 2026-10-01](COMPLETED-CHECKLIST.md#moved-2026-10-01-m27-retired)
- [x] **M27.2** -- Retired: a gap-closing item for gaps a non-existent consumer has not asked for.
-> [retired 2026-10-01](COMPLETED-CHECKLIST.md#moved-2026-10-01-m27-retired)
- [x] **M27.3** -- `ring_copy --compare` prices every policy on the machine in hand and reports
what each cost, with no verdict. -> [completed 2026-10-01](COMPLETED-CHECKLIST.md#m273)
## Moved 2026-10-04 15:27:15 -04:00 -- M-R2, the identity-keyed ledger and the unminted-completion panic
### M-R2 -- The outstanding-operation ledger, after the PR #113 review
Opened 2026-10-04. The review found that a raw push could still reach a false quiesce; the fix is
[D-78](DESIGN-NOTES.md#d-78).
- [x] **M-R2.1** -- **Track outstanding operations by identity rather than by count.** Done
2026-10-04. `Accounting` keeps the set of minted, unretired `user_data`; `record_completion`
and `cancel_reservation` take the identity; quiescence is that set being empty, for owned
and raw pushes alike. Pinned by `a_completion_that_retires_nothing_is_not_quiescence`
(both push kinds, both directions) and by the `PR #113` case in
[sabotage.json](sabotage.json), which restores count semantics and is caught.
- [x] **M-R2.2** -- **Decide whether the ring should now answer `UnexpectedCompletion`.** Done
2026-10-04, decided by the engineer and recorded as [D-79](DESIGN-NOTES.md#d-79): a
completion for an identity not in flight is a defect, traced always and raised as a panic
from `pop_raw` except during an unwind. `RingContract` stays external. Pinned by four
`kernel-seam` tests in `ring::tests` that forge the kernel's answer -- never minted,
duplicate, the minted control, and the unwind case -- and by the two `D-79` cases in
[sabotage.json](sabotage.json).
## Moved 2026-10-05 19:58:47 -04:00 -- M30.1, a refused push hands back what it was given
### <a id="m301"></a>M30.1 -- A refused owned push hands back its payload and sidecar in a `PushRefused`. *(completed 2026-10-05 19:58:47 -04:00)*
The decision is [D-80](DESIGN-NOTES.md#d-80). Points recorded only here:
- **Scope widened by one method, on the item's own wording.** `register_buffers` takes ownership
of the caller's buffers and dropped them on refusal, including on a full queue, which it
documents a retry for. It is an owned push by the item's definition, so it returns
`PushRefused<Vec<B>, ()>`. Its length checks now run before any address is taken, so a
refusal can still hand the `Vec` back whole.
- **Not covered: a registration whose completion fails.** `PendingBufferRegistration::claim_if`
still drops the buffers when the registration op completes with an error (D-18). That is not a
refused push: the kernel saw the addresses, and the claim is what proves it has finished
with them. Whether it should hand them back is a separate question, left open.
- **Not covered by a test: the `Unsupported` refusal.** Every ring on the development machine
supports every op a push requires, so no test can produce that refusal on demand. It returns
through the same `refuse!` macro as the oversized-buffer and foreign-registration refusals,
and those are tested; the "a refusal before the build drops the payload" sabotage case
patches the macro once for all of them.
- **Breaking-change reach.** No crate outside `windows-ioring-sys` calls an owned push. Inside,
`?` and `.expect(..)` absorbed nearly every call site. The ones that changed read `.kind()`
off the error, which is now `refused.error.kind()`; `ring_copy`'s `submit_one` gained an
`io::Error: From<E>` bound.
## Moved 2026-10-05 20:09:31 -04:00 -- M30 complete: M30.2, safe file registration
M30 was opened from the durable-ioring design session
([DESIGN-SESSION-2026-10-05-epoch-ring.md](../../design-sessions/DESIGN-SESSION-2026-10-05-epoch-ring.md),
"DI-2.2: what the ring crate cannot give dioring safely") to close two gaps in the safe surface,
fixed here rather than in a wrapper crate by the engineer's direction. `M30.1` is archived
[above](#m301).
### <a id="m302"></a>M30.2 -- `Batch::register_shared_files` registers files safely, with the ring holding them for its life. *(completed 2026-10-05 20:09:31 -04:00)*
The decision is [D-81](DESIGN-NOTES.md#d-81). Points recorded only here:
- **The release does not wait on the close succeeding**, although the item's target said it
"waits for the close". It still happens *after* `CloseIoRing`; it is just not conditional on
the close's result. A quiesced ring has nothing in flight that could use a handle. Gating on the
close's success would also leave the rundown gate untestable: the only ring whose rundown can
be made to fail on demand is the null-handle one, whose close always fails too.
- **The "released after the close" ordering has no test.** No observation can distinguish a
handle closed just before `CloseIoRing` from one closed just after. It is argued in D-81 and
in the comment at the release.
- **Naming left to the engineer.** M8.6's convention would give the safe method the plain name
`register_files` and rename the `unsafe` one to `register_files_raw`. That breaks every caller,
so this item stayed additive as planned.
> **-> CROSS-COMPONENT HANDOFF:** next work is in component `crates/durable-ioring` -> `DI-M2` -> `DI-2.2` (composition with `IoRing<T, X>`). See [CHECKLIST.md](../durable-ioring/CHECKLIST.md).
## Moved 2026-10-07 16:18:25 -04:00 -- M31.3: `SharedFile` gives way to `SharedHandle`
### <a id="m313"></a>M31.3 -- `SharedFile` is removed; a safe push's file is `win-shared-os-owned-handle`'s `SharedHandle`. *(completed 2026-10-07 16:18:25 -04:00)*
Planned as an alias (`pub type SharedFile = SharedHandle`); the engineer chose
removal instead, landed on `main` in #119 as a breaking change, and recorded as
[D-82](DESIGN-NOTES.md#d-82). The M30 additions that existed only on the
durable-ioring branch -- `register_shared_files`, its bookkeeping and tests, and
the `PushRefused` example -- moved to `SharedHandle` when that branch merged
`main`, and [D-81](DESIGN-NOTES.md#d-81) carries an amendment marker. The
both-directions lifetime test is
`the_rings_hold_on_a_shared_handle_ends_when_the_pop_retires_it` in
[ring_owned_inventory.rs](tests/ring_owned_inventory.rs), beside the existing
test for the other direction; a leaked guard is recorded in
[sabotage.json](sabotage.json) and was verified caught by it.
## Moved 2026-10-07 17:12:55 -04:00 -- M31.1: the delivery callback's contract is stated
### <a id="m311"></a>M31.1 -- `on_completion`'s re-entrancy and concurrency are stated on `EventDelivery::new` and pinned by a test and a sabotage. *(completed 2026-10-07 17:12:55 -04:00)*
The decision is [D-83](DESIGN-NOTES.md#d-83); the statement a consumer binds to is
`EventDelivery::new`'s "What `on_completion` may rely on". The re-entrancy test is
`a_callback_may_open_a_scope_and_submit` in
[event_delivery.rs](tests/event_delivery.rs): the first read's callback reaches the
delivery through a `Weak` and submits a second read. It deliberately does not use the
file's `recv_one`, whose post-mortem reads the ring's count under the very lock a
regression would leave held, and it leaks the delivery on panic so the deadlocked
callback fails the test instead of hanging its teardown. The sabotage that holds the
lock across the callback is in [sabotage.json](sabotage.json), and a harness run
recorded it caught by that test alone, at the bound rather than by a hang. Overlap is
stated, not tested, as the item required.
The item as it stood at completion:
- **Re-entrancy:** `on_completion` is called with the ring's lock released, so it may call
`EventDelivery::scope` and submit without deadlocking.
- **Concurrency:** two invocations may run at once on different pool threads, and completions
handed to concurrent invocations carry no order relative to each other. A consumer that needs
an order must impose it. The drain-rearm-drain shape is what produces the overlap.
- Both go in `EventDelivery::new`'s rustdoc, and in [DESIGN-NOTES.md](DESIGN-NOTES.md) as a
decision, so a later change to the delivery loop has a stated contract to answer to.
- **Tests, in both directions:** a callback that calls `scope()` and pushes completes, which a
lock held across the callback would turn into a deadlock -- run it with a bounded wait so a
regression fails rather than hangs. The concurrency statement is a permission, not a
promise, so it is asserted by documentation; do not write a test that requires overlap to be
observed.
- **Sabotage:** hold the ring lock across `on_completion` in `drain`, and confirm the
re-entrancy test fails.
## Moved 2026-10-11 10:30:59 -04:00 -- M33.1: a wake-only delivery
### <a id="m331"></a>M33.1 -- `CompletionWake`: the thread pool wakes an owner, who drains its own ring; `EventDelivery` re-expressed on it, so the delivery protocol is written once. *(completed 2026-10-11 10:30:59 -04:00)*
The decision is [D-84](DESIGN-NOTES.md#d-84), raised by durable-ioring's [DI-D-53](../durable-ioring/DESIGN-NOTES.md#di-d-53). `src/completion_wake.rs` holds the wake and the protocol moved out of `EventDelivery`; `src/completion_wake/tests.rs` pins delivery after the wake is built, the setup signal's call with nothing queued and its reach of a backlog, submitting from inside a call, and silence after the drop. `EventDelivery`'s own tests, and its two contract sabotages, are the proof the re-expression keeps its contract. The assistant's choices are queued for review as `M33.2`. The item as it stood at completion:
**`CompletionWake`: the thread pool wakes an owner, and the owner drains its own
ring** ([D-84](DESIGN-NOTES.md#d-84)). A wake built over `&mut IoRing`, calling the owner's
closure on a pool thread, holding no reference to the ring and no lock while it does, with an
optional callback environment; the protocol -- attach unsignalled, arm, raise the setup signal;
call, re-arm, call -- moved out of `EventDelivery` into it, and `EventDelivery` re-expressed on
it, so the protocol is written once. Its contract stated in rustdoc: drain to empty on every call
(D-19), a call with nothing to pop is normal, calls may overlap (D-83), one waiter per ring
(D-21), and never dropped from inside a call. Tested: a completion after the wake is delivered, a
backlog before it is delivered by the setup signal, the owner may submit from inside a call, and
nothing is called after the wake is dropped; `EventDelivery`'s own tests are the proof the
re-expression keeps its contract, and the sabotages anchored in its moved code re-anchored.
## Moved 2026-10-11 11:50:59 -04:00 -- M34.1: a registration's span check
### <a id="m341"></a>M34.1 -- `RegisteredBuffers::check_span`: the span check every push makes, asked without pushing. *(completed 2026-10-11 11:50:59 -04:00)*
Raised by durable-ioring's `DI-3.10.3.5.6.2.1`: its core holds a gated operation until the gate is durable and pushes it then, so a span the registration does not contain has to be refused when the operation is accepted, not at a push nobody is waiting on. `check_span` is `checked_span`, the private check every span push already makes, made public; it adds no rule of its own. `check_span_answers_what_a_push_against_the_span_does` in `tests/registration.rs` binds it to the push both ways, span by span -- every span it accepts a push accepts, and every span it refuses a push refuses, with the same kind -- and the two `M34.1` sabotages, one accepting every span and one refusing every span, are each caught.
## Moved 2026-10-11 18:03:39 -04:00 -- M35.1: a kernel responder registered for one ring
### <a id="m351"></a>M35.1 -- A responder registered for one ring, answering its calls on any thread, ending at the ring's drop after its rundown or at `Registration::end`. *(completed 2026-10-11 18:03:39 -04:00)*
Per [D-85](DESIGN-NOTES.md#d-85). `sys::register(&ring, responder)` registers a `Responses + Send`
in a process-wide registry keyed by the ring's `RingId`; every seam wrapper now takes the ring's id
beside its handle, and a ring's responder is consulted before a thread's. A responder is not
consulted re-entrantly for its own ring, and a panicking one neither wedges its ring nor poisons the
dispatch. `IoRing`'s `Drop` ends the registration after its rundown and before `CloseIoRing`. **One
departure from the item as written:** the item said a guard dropped early would end the
registration; instead **dropping the `Registration` ends nothing**, and `Registration::end` ends it
explicitly, handing the responder back -- because locals drop in reverse order, a drop-ending guard
would end before the ring's rundown in the ordinary `let ring; let registration;` shape, the very
hazard the form removes. Recorded in `D-85`. `Resolver` is `Send`: its stats are an `Arc<Mutex<..>>`
and its completion event a `Send` wrapper; `ResolverWatch` keeps its API. Tests: ten unit tests of
the registry in `sys/registered/tests.rs`, which open no ring, and four in
[tests/responder_per_ring.rs](tests/responder_per_ring.rs) over real rings -- a submit on another
thread answered, another ring not, the rundown answered then the registration ended, and a later
ring not answered by a dropped one's registration. Eight sabotages behave as declared, one refused
by the build, and a control survives. The item as it stood at completion:
**A responder registered for one ring** ([D-85](DESIGN-NOTES.md#d-85)), under the `kernel-seam`
feature: keyed by the ring's `RingId`, answering that ring's seam calls on whatever thread makes
them; a ring with none going to the kernel. The responder is `Send`, so `Resolver` is made `Send`:
its stats are shared through `Rc<RefCell<..>>` (`ResolverWatch`), and it keeps the completion event
as a raw pointer. A registration ends when its ring drops, after the ring's rundown, so a responder
holding operations still answers the rundown that pops them; a guard may end it earlier. The seam's
wrappers in [sys.rs](src/sys.rs) are handed a raw handle today, so they must learn which ring a call
is for. Tests, each direction asserted: a ring's responder answers its calls from a thread other
than the one that registered it; it does not answer another ring's; left alone, a registration ends
at the ring's drop, after answering the rundown; its guard dropped first ends it then; and after it
ends the ring reaches the kernel. Sabotages for each, in [sabotage.json](sabotage.json).
## Moved 2026-10-11 18:30:44 -04:00 -- M35.2, completing M35: the kernel responder registered for one ring only
### <a id="m352"></a>M35.2 -- The thread-local responder removed: every user moved to the per-ring form, `sys::install` and `Installed` deleted. *(completed 2026-10-11 18:30:44 -04:00)*
Per [D-85](DESIGN-NOTES.md#d-85), completing M35. `Resolver::scoped` is replaced by
`Resolver::register(&ring) -> ResolverWatch`, called straight after the ring is created; the one
ordering requirement the resolver still states is to be registered before the ring's first
operation, the other -- outliving the rundown -- being the registration's own now. The ten `scoped`
sites in `calibration.rs`, `properties_under_every_resolution.rs` and `resolver_over_a_real_ring.rs`
became blocks that register after creating the ring; `Stalled`, and the four forged-completion
tests, register on their ring, and those that `mem::forget` their ring end the registration by hand
first. The transparency test now asks whether a resolver registered for one ring leaks onto another.
`sys/installed.rs` became `sys/responses.rs` (`git mv`), keeping the `Responses` trait and losing
the thread-local install; its tests went, each property they held being one `sys/registered`'s tests
hold for a ring, and M35.1's precedence test went with the precedence. Sabotages: `M26.2`'s
re-anchored on the ring dispatch; M35.1's precedence entry retired; and two `M26.3` entries
re-anchored that M35.1 had broken, by rewriting the resolver lines they patch without checking the
manifest. A breaking change to the `kernel-seam` surface shipped in 0.7.0. The item as it stood at
completion:
**The thread-local responder removed** ([D-85](DESIGN-NOTES.md#d-85)): every user moved to the
per-ring form -- `Resolver::scoped`, which installs before its ring exists and so registers once the
ring is made; the property test `Stalled`; the forged-completion tests in
[ring/tests.rs](src/ring/tests.rs); and the seam's own tests in `sys/installed` -- then
`sys::install` and `Installed` deleted, a breaking change to the `kernel-seam` surface shipped in
0.7.0, committed as `feat(ioring)!`. The sabotages that pinned the thread-local form are retired or
re-anchored with it.
> **-> CROSS-COMPONENT HANDOFF:** next work is in component `crates/durable-ioring` -> `DI-M7` -> `DI-7.1` (failed submissions reached by tests). See [CHECKLIST.md](../durable-ioring/CHECKLIST.md).
## Moved 2026-10-11 22:22:15 -04:00 -- M36.1, completing M36: `CompletionWake`'s tests moved to `tests/`
### <a id="m361"></a>M36.1 -- The five `completion_wake::` lib tests moved to an integration test, and the ring-opening inventory check passing again. *(completed 2026-10-11 22:22:15 -04:00)*
Five tests `M33.1` added as lib tests opened a real ring and waited on the thread pool using only
public API, so `tools/check-ring-tests.ps1` failed on them. Moved by `git mv` to
[tests/completion_wake.rs](tests/completion_wake.rs), gated on `threadpool`; the check passes with
the inventory unchanged, and the entry is moved to
[RESOLVED-TEST-FAILURES.md](RESOLVED-TEST-FAILURES.md). One sabotage rationale named the old path
and now names the new. The item as it stood at completion:
**The five `completion_wake::` lib tests moved to an integration test**, since they use only public
API (`M24.3`), and the check passing again with the inventory unchanged. The entry moves to
[RESOLVED-TEST-FAILURES.md](RESOLVED-TEST-FAILURES.md) in the same commit.
## Moved 2026-10-11 23:20:20 -04:00 -- M37.1, completing M37: a safe responder that fails a ring's submits
### <a id="m371"></a>M37.1 -- `sys::submit_faults()`: a safe responder that fails a ring's submits on demand, for a consumer that forbids `unsafe`. *(completed 2026-10-11 23:20:20 -04:00)*
Per [D-86](DESIGN-NOTES.md#d-86). `SubmitFaults` and `SubmitFaultControl` in `sys/submit_faults.rs`,
under both `kernel-seam` and `fault-injection`; `InjectedFailure::as_hresult` became crate-visible
for it, and `submit_faults` compiles away when either feature is missing. A failing submit returns
without reaching the kernel, so entries stay queued; its out-parameter is left untouched, the real
call's being undocumented. `failures()` reports each failed call's thread, so a test can show a
failure was answered off its own thread. Tests: eleven unit tests that call the responder directly
and open no ring, a doctest that runs the usage, and one real-ring test in
[tests/submit_faults.rs](tests/submit_faults.rs) showing a failed submit leaves the entry queued,
the kernel never sees it, a pop still reaches the kernel, and the next submit runs it. Seven
sabotages caught and a control survives. Not provided, by decision: a safe general `Responses`. The
item as it stood at completion:
**`sys::submit_faults()`**, under both `kernel-seam` and `fault-injection`: a `SubmitFaults`
responder, for `sys::register`, that fails the ring's submits on demand and forwards every other
call to the kernel, and a `SubmitFaultControl` handle -- `Clone`, `Send` and `Sync` -- that arms it
from any thread: `fail_next(count, failure)`, `fail_until_cleared(failure)` and `clear()`, the
failure an `InjectedFailure`, which is a failure by construction and is checked when it is armed. A
failing submit returns the failure without reaching the kernel, so the entries stay queued, as
`SubmitIoRing` documents ([RS-P-7](RESPONSE-SPACE.md)); `failures()` reports the thread that made
each failed call, so a test can show a failure was answered on a thread other than its own. A submit
that waits is failed too, so a ring dropped with the fault armed and operations outstanding cannot
run down: the fault is cleared first. Tests, each direction asserted: armed, a submit fails with the
injected code and a clear stops it; a count fails exactly that many; the thread of each failure is
reported; a failed submit leaves the entry queued on a real ring, and the next submit runs it.
Sabotages for each, in [sabotage.json](sabotage.json). A decision row records why the responder is
safe rather than implemented by its consumers.
> **-> CROSS-COMPONENT HANDOFF:** next work is in component `crates/durable-ioring` -> `DI-M7` -> `DI-7.1` (failed submissions reached by tests). See [CHECKLIST.md](../durable-ioring/CHECKLIST.md).
## Moved 2026-10-12 12:02:28 -04:00 -- M38.1, completing M38: a safe responder that fails any seam call, a completion's result, or stalls submits
### <a id="m381"></a>M38.1 -- `sys::call_faults()`: a safe responder that fails any seam call, a popped completion's result, or stalls submits, with `submit_faults()` its submit-only name. *(completed 2026-10-12 12:02:28 -04:00)*
Per [D-87](DESIGN-NOTES.md#d-87). `CallFaults`, `CallFaultControl` and `Call` in `sys/call_faults.rs`
(`submit_faults.rs` renamed, its history following), under both `kernel-seam` and `fault-injection`;
`submit_faults()` now returns the same responder with a control narrowed to submits, so durable-ioring's
`DI-7.1` tests compile unchanged. A stalled submit never reaches the kernel until the stall ends, after
which submits run what is queued, so a ring dropped mid-stall still runs down; withholding completions
was rejected because `SubmitIoRing(wait)` then fails with `E_INVALIDARG`. Tests: unit tests that call
the responder directly and open no ring, two running doctests, and seven real-ring tests in
[tests/call_faults.rs](tests/call_faults.rs): a refused file-registration build and a retry that
registers, a refused buffer-registration build handing the buffers back, a failed completion keeping its
identity with the accounting balanced, a cleared completion plan, a failed pop reported once, a stalled
submit leaving the write unrun until the stall ends, and a ring dropped during a stall running down.
Twelve new sabotages and the seven M37.1 ones re-anchored, all caught, and the control survives. The
sweep found one hole: the failed completion was the first popped, and zeroing its identity named the
same operation, so the test now fails the last of three in flight. The item as it stood at completion:
**`sys::call_faults()`**, under both `kernel-seam` and `fault-injection`: a
`CallFaults` responder, for `sys::register`, and a `CallFaultControl` handle, `Clone + Send + Sync`,
that arms it from any thread. **A call fails**: `fail_next(call, count, failure)` and
`fail_until_cleared(call, failure)`, for every call the seam carries -- `Call::Submit`, `Pop`,
`BuildRead`, `BuildWrite`, `BuildFlush`, `BuildCancel`, `BuildRegisterFiles`,
`BuildRegisterBuffers` and `SetCompletionEvent` -- with an `InjectedFailure`, returned without
reaching the kernel; `failures(call)` reports the thread of each. **A completion fails**:
`fail_next_completions(count, failure)` pops the real completion and rewrites its result code,
the identity and the rest untouched, as `Completion::with_injected_failure` does for a popped
one. **A submit stalls**: `stall_submits(duration)` makes a submit return without reaching the
kernel -- success with nothing submitted if it does not wait, `IORING_E_WAIT_TIMEOUT` after its
wait or what is left of the stall if it does -- until `duration` has passed since the first it
stalled, after which submits reach the kernel and run everything queued: the platform's
permission to leave an operation pending (RS-P-1), so a bounded wait for its completion
expires, and a ring dropped meanwhile still runs down once the stall ends. `clear()` ends all of
it. `submit_faults()` stays, its control narrowed to submits over the same responder, so
there is one implementation and two ways to name it. Tests, each direction asserted: every call
fails when armed and only that call; counts; the thread of each failure; a completion's result
failed exactly that many times with its identity intact; a stalled submit leaving an operation
unrun until the stall ends, then running it; and a refused `BuildRegisterFiles`,
`BuildRegisterBuffers` and `SetCompletionEvent` on a real ring. Sabotages for each, in
[sabotage.json](sabotage.json). A decision row records the widening of `D-86`.
> **-> CROSS-COMPONENT HANDOFF:** next work is in component `crates/durable-ioring` -> `DI-M7` -> `DI-7.2` (construction's registration paths reached by tests). See [CHECKLIST.md](../durable-ioring/CHECKLIST.md).
## Moved 2026-10-12 15:05:39 -04:00 -- M39.1: rundown reaps before it waits
### <a id="m391"></a>M39.1 -- `run_down_within` reaps the completion queue before its submit-wait, so a ring whose queue is full runs down. *(completed 2026-10-12 15:05:39 -04:00)*
Found, not planned: durable-ioring's blocking-push example filled a pumped instance's queues and
dropped it, and the ring's rundown failed with `IORING_E_COMPLETION_QUEUE_TOO_FULL`, tripping the
debug assertion in `Drop`. Per [D-88](DESIGN-NOTES.md#d-88). `SubmitIoRing` with a wait reports that
error when the completion queue is full whatever it waits for, and an entry whose submit failed for
the same want of room stays queued behind the completions; rundown waited and then reaped, so it
could never get the room it needed. It now reaps once the bound has said to wait, and re-checks
quiescence, before the submit-wait -- not before the bound check, since a zero bound makes no kernel
call and the unit tests of a ring whose handle the kernel refuses pin that. Reproduced first, by
[tests/run_down_full_queue.rs](tests/run_down_full_queue.rs): a ring whose queue is full with an entry
still queued runs down, runs down within a bound, and drops cleanly, all three failing before the
change. A first attempt, which reaped before the quiescence check, broke four unit tests over the
null-handle ring and was moved. One sabotage is added, and the existing `M26.8` entry re-anchored,
its pattern now occurring twice.
## Moved 2026-10-13 23:22:30 -04:00 -- M40.1: a batch left queued
### <a id="m401"></a>M40.1 -- `Batch::leave_queued`: close a batch without submitting what it queued. *(completed 2026-10-13 23:22:30 -04:00)*
Decision [D-90](DESIGN-NOTES.md#d-90). A batch submits on drop ([D-5](DESIGN-NOTES.md#d-5)), which reports
nothing and submits once per batch; durable-ioring's threadless backend wants to queue what it is
pushed, ring the doorbell once for a batch, and see a failure of it. `leave_queued` marks the batch as
already submitted so the drop does not, leaving the entries queued with their buffers held by the ring,
as a failed `Batch::submit` leaves them. Tested in
[submission_lifecycle.rs](tests/submission_lifecycle.rs): a read left queued does not complete, polled
with `try_pop` since `pop_within`'s own wait is a submit, until a later submit issues it. One sabotage
entry, caught: without the mark the drop submits.
## Moved 2026-10-14 17:17:20 -04:00 -- M41.1: a ring typed by whether it can hold nothing for an operation
### <a id="m411"></a>M41.1 -- `IoRing<T, X, K>` with `K` `Open` or `Owned`, and `IoRing::into_owned`: a ring that cannot have an operation in flight it holds nothing for. *(completed 2026-10-14 17:17:20 -04:00)*
Decision [D-91](DESIGN-NOTES.md#d-91). A registration, the `_raw` flush and cancel and `push_raw` queue an
operation the ring holds nothing for, so a pop can return a completion with no entry -- legitimate for a
caller who chose it, and a branch every other caller wrote and could not reach. A ring now has a kind, and
an `Owned` one has no call that queues such an operation and pops what it held without the case of nothing.
- **Built**: [kind.rs](src/ring/kind.rs) (`Open`, `Owned`, the sealed `RingKind` with its `Popped` type,
and `AllowsRaw`); the kind parameter on `IoRing` and `Batch`, defaulted so `IoRing<T, X>` is still the
open ring; `where K: AllowsRaw` on the five `Batch` methods and `push_raw`; `into_owned`, which refuses
while `outstanding()` exceeds `held()` and hands the ring back; `CompletionWake::new` and the responder
registrations take either kind.
- **Tested** in [owned_ring.rs](tests/owned_ring.rs) and by `compile_fail` doctests on `AllowsRaw` whose
pinned codes were checked under `RUSTC_BOOTSTRAP=1`; the sabotage entries are caught, and the panic an owned
ring makes for a completion it held nothing for is a declared blind spot.
- **Borrow-surface audit**: `Batch::new` and `CompletionWake::new` re-asked, unchanged.