Skip to main content

Module breaker

Module breaker 

Source
Expand description

The circuit breaker for remote-effect steps — retry’s cross-run sibling.

retry remembers failures within one step of one run; when the remote is genuinely down, every new run still walks into it, burns its retry budget, and adds load to a dependency that needs the opposite. A breaker remembers across runs: after failures consecutive failures the circuit OPENS and further attempts fail immediately — no connection, no timeout wait — until cooldown has passed, when exactly ONE attempt is let through as a probe. The probe’s outcome decides: success closes the circuit, failure re-opens it for another cooldown.

Declared per step (breaker: {failures: 5, cooldown: 60s}) on the remote-effect kinds (http, mcp.tool, a2a.send, a2a.delegate). State is durable — keyed by workflow + the step’s UNSCOPED id, so every fan-out iteration (each[0].call, each[1].call…) shares one breaker, because they share one dependency — and per instance: two replicas keep independent breakers, which is the honest scope for state that is really a local observation about a remote.

This file is the pure state machine; the reactor owns WHEN it is consulted (single-writer, so there are no races to reason about). A fast-fail is reported through the normal step-failure path with OPEN_ERR as its error prefix — which composes: retry on the step turns into a bounded poll of the breaker, and on_error: continue + a switch on the error text is a fallback route.

Structs§

Config
Parsed breaker: declaration.

Enums§

Gate
What the gate decides before an attempt is dispatched.
Transition
A state transition worth one log line (transitions only — a breaker that logged every guarded call would be its own kind of load).

Constants§

OPEN_ERR
The error prefix a breaker fast-fail carries. The recorder skips errors with this prefix — a refusal to call is not evidence about the remote.

Functions§

gate
Consult (and update, for the probe claim) the breaker before an attempt.
key
The durable key: workflow + the UNSCOPED step id (each[0].callcall), so fan-out iterations share the breaker of the dependency they share.
record
Record an attempt’s outcome. ok is the step’s terminal disposition for this attempt; fast-fails (the OPEN_ERR prefix) must not reach here.