Expand description
keel-core: the real Keel kernel, Tier 1 scope (architecture spec §4).
One Engine per process runs every intercepted call through its
target’s layer chain — cache → rate → breaker → timeout → retry — driven
by the shared typed policy model (keel_core_api::policy). Waits are
real tokio::time sleeps: production sleeps wall-clock time, and the
conformance harness runs under start_paused where the same sleeps
advance virtual time deterministically. The engine passes the identical
scenario corpus as keel-core-stub — that equivalence is the whole point
of the conformance suite.
Beyond the stub, the real engine:
- enforces
timeoutwithtokio::time::timeoutaround each attempt (a policy-layer timeout terminates asKEEL-E011) - applies schedule
jitter(equal jitter: uniform in[w/2, w]) - is
&self-concurrent: interior state behind a mutex, never held across an await
Every call and attempt is also emitted as a tracing span
(keel.call / keel.attempt, architecture spec §4.5), with breaker
transitions and cache hits as debug events. Spans cost effectively
nothing when no subscriber is active. Enabling the optional otel
feature adds [otel::init_otlp] to export those spans over OTLP.
Independently of tracing, the events sink streams a live NDJSON
feed of attempts/backoffs/breaker transitions to .keel/events/ (for
keel tail), and mints the trace refs Tier 1 failure messages carry.
Tier 2 durable flows (FlowManager) build on the Tier 1 engine and the
keel_journal persistence layer; the FFI facade wraps this async surface
(later slices) for the PyO3/napi bridges.
Modules§
- events
- Tier 1 live event sink: an append-only NDJSON feed of what the engine is
doing right now — attempt starts/failures, backoff waits, breaker
open/half-open/close, rate-limit queueing, cache hits/misses — written
per run to
.keel/events/<run>.ndjsonforkeel tail/keel traceto follow (dx-spec §6, invariant 4). Non-contract: the line format is versioned ("v": 1on every line) but lives outsidecontracts/; only Keel’s own tooling reads it, and no daemon is involved — readers tail the file (dx invariant 3).
Structs§
- Engine
- The Keel kernel, Tier 1 scope. One per process;
&self-concurrent. - Flow
Config - Tuning for a
FlowManager: lease lifetime and the flow-level attempt cap (spec §4.3 leases; the cap turns a poison flowdead). - Flow
Descriptor - How a flow is identified and fenced. Identity is
(entrypoint, args_hash, explicit_key?)(spec §4.3);code_hashfences replay across deploys (spec §4.4). - Flow
Handle - A single running flow. Owns the step cursor (
seq), so it is used by one task via&mut; the manager stays&self-concurrent for many flows. - Flow
Manager - The Tier 2 flow manager: opens and resumes durable flows over a
Journal, running each step’s effect through the Tier 1Engine. One per process,&self-concurrent; hands out a&mutFlowHandleper running flow.
Traits§
- Discovery
Recorder - The discovery-recording surface the engine depends on: a single method, so
the engine can hold a
DiscoveryStoretype-erased regardless of theClockit was opened with. Implemented for everyDiscoveryStore<C>.