Struct CompletionRetryConfig 

pub struct CompletionRetryConfig { /* private fields */ }

Backoff ladder for the process-exit completion retry.

Separate from SignalDeliveryConfig because the two policies answer different questions. Signal delivery is a bounded enqueue ladder: it asks “has the process body materialized yet”, gives up after max_enqueue_attempts, and its millisecond ceiling is sized for a wait measured in scheduler ticks. A completion retry is unbounded — there is no attempt count at which abandoning a finished run’s terminal event becomes the right answer, so only the epoch close ends it — and it sleeps between durable store round-trips against a store that is already failing.

Borrowing the first for the second is how a value chosen for one job silently governs another. There is no attempt count here on purpose; what there is, is an interval, and it is now stated in one place where it can be changed.

The fields are private, and that is the whole gate

🔴 A value of this type is a ladder that CAN CLIMB. That is not a convention a caller is asked to honour, it is the only thing Self::try_new returns — and try_new plus Default are the only ways to obtain one. An earlier revision left the fields pub and put the check in EngineBuilder::build, which meant the invariant held for engines built through that builder and nowhere else: crate::RuntimeHandle::new is pub and reads completion_retry straight out of a RuntimeConfig a caller assembled by hand. There is now no second door to hold shut, because there is no way to name a degenerate ladder at all.

🔴 If this type ever gains serde::Deserialize — and the server-config lane (docs/design/aion-authoring/BRIEF-ENGINE-CONFIG-SERVER-SURFACE.md) is the lane that will want it — a derived impl reintroduces exactly the hole this removed, because a derive writes the fields directly. Deserialization must go through Self::try_new, not around it.

The closure is held by these two examples, not by the paragraph above

Privacy is what makes the constructor the only door, and nothing in a unit test can observe privacy — a test that could name a degenerate ladder would not compile, so it would be a NON-RUN rather than a failure. These two doctests are the mechanism. They run from OUTSIDE the crate, which is the embedder’s position, and they are identical but for the one line under test: if the fields are ever made pub again the second stops failing and cargo test says so.

The control — the constructor is reachable and the snippet around it is sound, so the refusal below cannot be an artefact of a broken example:

use aion::CompletionRetryConfig;
use std::time::Duration;

let ladder = CompletionRetryConfig::try_new(
    Duration::from_millis(1),
    Duration::from_secs(30),
)?;
assert_eq!(ladder.initial_backoff(), Duration::from_millis(1));

The closure — the same snippet with the constructor call replaced by a struct literal naming a ladder that cannot climb:

use aion::CompletionRetryConfig;
use std::time::Duration;

let ladder = CompletionRetryConfig {
    initial_backoff: Duration::ZERO,
    max_backoff: Duration::ZERO,
};
assert_eq!(ladder.initial_backoff(), Duration::ZERO);

🔴 The bound on that second example, measured rather than assumed. Making both fields pub again turns it red — the observed failure is rustdoc’s “Test compiled successfully, but it’s marked compile_fail” — so it does hold the closure. What it cannot do is verify WHY the compile failed: compile_fail passes when the snippet fails for any reason at all. An earlier revision of this paragraph claimed a compile_fail,E0451 annotation pinned the reason; that was measured and is FALSE on this toolchain — a snippet edited to fail on an unresolved import still passed with the code attached, so the annotation was decoration reading as a gate and has been removed rather than left to mislead.

What stands in its place is the shared shape of the two examples. They differ in one expression, so a rename of the type, the constructor or the accessor breaks the CONTROL as well, and the control is an ordinary doctest that must compile and run. The residue is narrow and named: a hand-edit that breaks only the failing copy would go unnoticed.

Implementations

impl CompletionRetryConfig

pub fn try_new( initial_backoff: Duration, max_backoff: Duration, ) -> Result<Self, InvalidCompletionRetryLadder>

Create an explicit completion-retry backoff ladder, or refuse one that cannot climb.

This is the only public constructor, and it is fallible for the reason stated on the type: the ladder governs a retry with no attempt budget, so an interval that cannot climb is not a fast retry, it is an unterminated hot loop against a store that is already unwell.

Any non-zero interval the caller chooses is accepted as given. The only pairs refused are the ones that contradict themselves.

Errors

InvalidCompletionRetryLadder::ZeroInitialBackoff when the floor is zero — 0 * 2 == 0, so the ladder can never leave it.

InvalidCompletionRetryLadder::CeilingBelowFloor when the ceiling is below the floor. sleep_backoff assigns the ceiling whenever doubling passes it, so such a ladder ratchets DOWN as the outage lengthens, which is the opposite of what a backoff is for; a zero ceiling is the extreme of that case and reaches zero on the first advance. A ceiling EQUAL to the floor is legitimate — that is a fixed interval — and is accepted.

pub const fn initial_backoff(self) -> Duration

Sleep before the second attempt. Never zero.

pub const fn max_backoff(self) -> Duration

Upper bound the exponential ladder settles at. Never below Self::initial_backoff.

Trait Implementations

impl Clone for CompletionRetryConfig

fn clone(&self) -> CompletionRetryConfig

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Copy for CompletionRetryConfig

impl Debug for CompletionRetryConfig

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for CompletionRetryConfig

fn default() -> Self

1 ms initial, 30 s ceiling, and no attempt budget — ruled, with reasoning, on 2026-08-06.

Full record and the argument behind every clause: docs/design/aion-authoring/RULING-COMPLETION-RETRY-BOUND-2026-08-06.md. The reasoning is reproduced here because the numbers without it read as arbitrary defaults, and it was exactly that appearance — a ladder that looked inherited and was actually introduced — that made a ruling necessary.

Where the numbers came from, and what was NOT inherited with them

1 ms / 8 ms were SignalDeliveryConfig’s values, kept verbatim when this knob was split out of that struct so no intermediate revision altered behaviour. 🔴 That is not the same as “inherited”: nothing at HEAD retries a completion at all, so measured against the tree this lands on the ladder is a value this change INTRODUCES.

And the bound was not borrowed with the numbers. SignalDeliveryConfig pairs its ladder with max_enqueue_attempts: 8 — it gives up. At an 8 ms ceiling with no budget this retry was roughly 125 attempts per second, indefinitely, against a store that is by definition unwell, all on the single-worker engine-task executor. That is a runaway which also starves everything else on that executor.

The store cost of that runaway is one to six history reads per attempt, depending on how far the attempt gets before it dies, and both magnitudes below are derived from that range rather than from a single number. 🔴 Two earlier revisions of this paragraph gave a single number and then a one-to-three range; both were arrived at by reading the first branch and stopping. The list below is the enumeration, and the rule it exists to serve is that a cost is counted along every reachable path, not along the one that comes to mind.

• One, on the fail-fast path. handle_process_exit_attempt’s first fallible call past the epoch gate is the history read (lifecycle/completion.rs:382), so an attempt that fails there has issued one and only one.

• Two or three, on the recorded-then-bookkeeping-failed path. Past the append, upsert_workflow_visibility reads history (lifecycle/visibility.rs:25, skipped only for TimedOut) and reconcile_terminal_registry reads it again (lifecycle/completion.rs:546).

• Four or five, on the ContinuedAsNew branch — which is the ordinary continue-as-new completion, not an edge case, so under a sick store EVERY attempt of that run’s loop pays it. Past the two or three above, start_continuation_replacement hands off to open_successor_generation, which reads history ONCE under the predecessor’s recorder lock (lifecycle/continuation.rs) — that one read serves the already-started check, the terminal check, and the deadline the batch retires — and the successor’s own visibility upsert reads it again (lifecycle/visibility.rs:25).

  It was SIX before aion#213, because the successor was started through start_workflow_with_options, which read the head a third time in workflow_identity — the read that seeded the second recorder for the same history. The two generation-boundary projection reads the recorder now performs (durability/recorder/generation.rs) are non-fatal and cannot fail an attempt, so they are not counted here.

Every one of those can fail StoreError::Backend, which completion_retry’s store_error_is_transient classifies retryable, so the attempt spends its reads and the next one starts over from the top. That the deepest of them is reachable is not inferred: the same module’s TerminalWriterHeld classification is justified by naming this exact chain — start_continuation_replacement → open_successor_generation → registry.rekey_generation — which sits past all of them.

🔴 THE O5 PROOF GUARDS THE FIRST NUMBER AND CANNOT SEE ANY OF THE OTHERS. Its fixture makes the FIRST read fail, so the loop under it never reaches the post-append reads at all; its attempt-count assertion is invariant to how many reads a bookkeeping or continuation failure costs. It is a real guard on the fail-fast cost and no guard whatsoever on the rest, and an earlier revision of this paragraph offered it as a guard on both. That is also the shape crate::store_faults::FlakyStore::fail_reads_after exists to reach — a budget that can skip past the append — so the deeper numbers are testable; they are simply not tested by O5.

So: 125 attempts per second is 125 reads per second on the fail-fast path, and up to 750 on the continue-as-new one.

The ceiling: 30 s

At 30 s a stuck retry costs two attempts a minute, and therefore two to twelve reads a minute by the range above, while the 1 ms start still recovers instantly from a transient blip. 30 s is the value already ruled for DR-001’s R4, so it is precedent rather than a fresh invention.

No attempt budget, and that is DELIBERATE

🔴 Read this before “fixing” the missing budget. A completion is truth trying to land in the record. SignalDeliveryConfig’s budget is sound BECAUSE delivery is re-drivable — something upstream tries again. A completion has no re-driver. If this loop gives up, the completion does not land late, it never lands, and the workflow wedges silently — converting a transient store outage into permanent silent data loss. That is a worse failure than the runaway a budget would prevent, and the runaway is already cured by the ceiling. The epoch close stays the only terminator.

🔴 The condition on ever adding one: whoever wants a budget owes the design a place to PUT the abandoned completion first. Until that place exists, giving up is a silent fallback, and this repository does not ship those.

The price of no-budget is loudness

A retry loop ruled unbounded must never be invisible, so lifecycle::completion_retry states itself at warn once the ladder reaches this ceiling, carrying attempt count, elapsed time and the last error classification. The rate is one line per ceiling-interval attempt — which is a CONSEQUENCE of the ceiling, not a throttle. There is no separate rate limiter and there must not be one; that would be an invented cap.

Still not reachable from aion server

Grepping aion-server and aion-cli for completion_retry / CompletionRetryConfig returns nothing — nor for signal_delivery / SignalDeliveryConfig, which is the same shape. Only an embedder calling crate::EngineBuilder can change either. That gap is a separate ruled lane (docs/design/aion-authoring/BRIEF-ENGINE-CONFIG-SERVER-SURFACE.md), deliberately not a rider on this change. Until it lands, this default is the shipped answer and it cannot be answered by configuration.

Why this does not go through CompletionRetryConfig::try_new

Default::default cannot fail, and this module owns the private fields, so the literal is written directly rather than unwrapping a Result — this crate does not unwrap in library code, and a fallback on error would be a silent one. That makes these two literals the only values in the program that reach the ladder without passing the constructor, so the_default_ladder_is_one_the_constructor_would_accept puts them back through it. The gap is closed by that test, not by care.

impl Eq for CompletionRetryConfig

impl PartialEq for CompletionRetryConfig

fn eq(&self, other: &CompletionRetryConfig) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl StructuralPartialEq for CompletionRetryConfig