Expand description
headgate core — ports, envelope, and the state machine. No I/O lives here.
The thesis: dequeue is an admission decision, not a fetch. See ARCHITECTURE.md architecture thesis.
Structs§
- Admission
Explain - admission policy the answer to “why is this job not running” — the question this design creates and the endpoint no predecessor needs, because no predecessor has a gate.
- Admission
Unit - batch-shaped admission an admission unit: ordinarily one job, occasionally a group admitted as one decision. v0.1 always returns units of size 1, but the CONTRACT is group-shaped now because batched execution changes the gate’s accounting in four places (token spend, fairness quantum, concurrency reservation, crash attribution) and retrofitting that means reopening the atomic claim after it has traffic. Token spend and deficit charge count unit SIZE, never row count.
- Admit
Request - AllErrors
AreFailures - The default: every error is a real failure.
- Bulk
Request - control API contract a bulk mutation as data: created by the API, executed by a duty in bounded batches, polled by the caller. An empty selector is rejected at the boundary.
- Caps
- Checkpoint
- Progress within a single job. Persisted with the lease renewal that is already happening, so a mid-step crash does not lose it — River’s default writes this only after the worker returns, which is the one case it is needed.
- Claim
- Concurrency
Limit Config - Envelope
- History
Bucket - JobFilter
- JobOutput
- The latest versioned opaque output persisted while a fenced attempt was running.
fenceidentifies the attempt that wrote it;updated_at_msis stamped by the store clock, never by the worker. - JobPage
- JobProgress
- The latest progress accepted from a fenced running attempt.
fenceidentifies the writer andupdated_at_msalways comes from the store clock. - JobResult
- Versioned opaque bytes recorded atomically with successful completion.
- JobSummary
- Lease
Ref - Identifies one claimed job for
ack/renew.admitwrites ONE lease_id for every job claimed in the same call, andfencecounts per job — so (lease_id, fence) alone is ambiguous: two jobs on their first claim in one call are both fence=1. The job id selects the row; lease_id + fence still gate the write (lease fencing) so a superseded holder is rejected, never silently no-opped. - Noop
Telemetry - Operation
Status - Partition
State - Progress
Update - A portable operator-facing progress update.
currentandtotalare exact units, not a floating-point percentage; applications that naturally report percentages usetotal = 100. The optional message is deliberately small because the console polls this value while a job is running—it is status, not another log channel. - Quarantine
Entry - Queue
Stats - Quiet
Group Metrics - Rate
Class Config - Rate
Class State - Reclaimed
- A job the lease reclaimer swept.
quarantinedtells the caller which counter and event to emit — eviction and quarantine are never silent (retention and eviction contract). - Schedule
- A periodic entry. Durable in the store (surveyed policy behavior), never in a leader’s memory. The
store treats
specas opaque; tick computation lives caller-side so every backend stays spec-agnostic. - Schedule
Event - State
Counts - bounded-count contract/bounded live-control contract counts come from a BOUNDED scan, never O(queue depth): past the
threshold,
approximateis set instead of paying for exactness. - Task
Options - Trace
Context - A parsed
traceparent(plus the unparsedtracestate). - Transition
Ctx - Worker
Meta
Enums§
- Blocked
By - Codec
Error - Event
- Events emitted through the telemetry facade.
- Lifecycle
Event - The rows of conformance/state_machine.yaml that are driven by the lifecycle — sweeps
and operator actions — rather than by a worker’s ack. Kept beside
transitionso the yaml cross-check covers the whole table. - Missed
Policy - surveyed policy behavior what happens to periodic runs missed during downtime. Nobody in the surveyed field backfills; River can skip a tick entirely across a leader election.
- Outcome
- lifecycle state machine Exhaustive on purpose: adding a variant without handling it is a compile error, which is how a commented-out transition becomes impossible rather than silent.
- Resume
- Saturation
Strategy - surveyed policy behavior the action the atomic gate takes when a partition has reached its configured concurrency ceiling. String values are the wire/storage contract across all backends.
- Schedule
Event Outcome - One durable scheduler enqueue attempt. Stores retain only the newest
SCHEDULE_EVENT_LIMITrecords per schedule, so operator inspection is bounded. - State
- Store
Error
Constants§
- MAX_
OPAQUE_ SCHEMA_ VERSION - Largest opaque result/output schema version portable across every backend.
- MAX_
PROGRESS_ MESSAGE_ BYTES - MAX_
PROGRESS_ VALUE - JSON numbers consumed by the shared browser console must remain exact too; this is
JavaScript’s
Number.MAX_SAFE_INTEGER, narrower than the SQL BIGINT columns. - SCHEDULE_
EVENT_ LIMIT - TRACEPARENT
- The RESERVED envelope header carrying W3C Trace Context’s
traceparent. - TRACESTATE
- The RESERVED envelope header carrying W3C Trace Context’s
tracestate. Opaque: headgate never parses, validates, or truncates it — it round-trips the bytes. - UNIQUE_
REPLACE_ ALL - UNIQUE_
REPLACE_ MAX_ ATTEMPTS - UNIQUE_
REPLACE_ PAYLOAD - UNIQUE_
REPLACE_ PRIORITY - UNIQUE_
REPLACE_ SCHEDULED_ AT
Traits§
- Checkpoint
Inspect - Clock
- IdGen
- Inspect
- control plane the control API’s store surface. Separate from
Storethe wayTransactionalis (runtime capability boundary): a backend that cannot answer these does not have them. Every read here is bounded — no method may be O(queue depth) (invariant 6). - IsFailure
- failure classification Does this error consume an attempt? asynq’s
Config.IsFailuregeneralizes what surveyed policy behavior adopted as the one-offOutcome::RateLimited: returning false re-queues the job WITHOUT incrementingattemptand without polluting queue failure statistics. Upstream rate limits, planned maintenance windows, and “not my turn yet” all belong here rather than burning a retry budget that exists for real failures. - Notifying
- push wakeups push wakeup: sub-poll-interval latency when the store can signal new work. A missed or spurious notification costs LATENCY, never correctness — the poll fallback always stands (River’s layered-fetch lesson).
- Output
Inspect - Output
Store - Progress
Inspect - Progress
Store - Result
Inspect - Result
Store - Store
- The whole port. Coarse on purpose — the admission decision must stay atomic inside the store, so a fine-grained port would force the gate back into the worker.
- Task
- A unit of work.
TYPEis wire state — changing it strands enqueued jobs. - Telemetry
- Transactional
- TxHandle
- A caller-owned store transaction. Adapters downcast to their own concrete handle via
as_anyand reject a foreign one — the compile-time path (transactional API) is generic and never hits this; thedynpath needs the runtime check.
Functions§
- canonical_
tags - Canonical storage order for tags. Validation bounds the set before this allocates.
- check_
kind_ collisions - typed dispatch Every kind and alias must be globally unique across the registry, or dispatch is
ambiguous. Checked once at startup rather than discovered one job at a time. Every
name — TYPE and alias alike — must also pass
validate_kind: an alias is a dispatch key that jobs are enqueued under during a rename, so a rule that skipped aliases would let the rename introduce exactly the kind the rule exists to forbid. - effective_
unique_ key - Versioned, collision-free uniqueness namespace. Including the kind is the safe default; the explicit exclude flag uses a distinct namespace so scoped and unscoped jobs can never alias accidentally.
- effective_
weight - The rate-budget estimate every backend persists and charges. Proto3 scalar omission, old producers, and Rust/Go zero-value struct literals all arrive as zero, so zero is the compatibility sentinel for the documented default of one. Public APIs still reject an explicitly supplied zero because a zero-cost job should be reported as actual usage, not used to bypass admission.
- enqueue_
queue - The queue an envelope actually lands in. Every backend defaults an empty queue to
defaulton write, so the idempotent enqueue identity id comparison must normalize the same way or a replay that omitted the queue would read as a conflict against its own row. - fingerprint
- content fingerprinting the fingerprint algorithm, specified in ARCHITECTURE.md and nowhere else:
lowercase_hex(SHA256(u32_le(len(kind)) || kind || u32_le(len(payload)) || payload)[0..16]). Length-prefixed so (“a”,“bc”) and (“ab”,“c”) cannot collide; truncated to 128 bits because a collision over-quarantines. Derived CLIENT-SIDE at enqueue when the caller does not supply one; stores pass the value through untouched. - group_
admission_ claims - Turn the flat, atomically-claimed result into deterministic handler units. Grouping happens only after the store has charged every row, so N members consume N units of rate, fairness, and concurrency capacity. It changes dispatch shape, never policy.
- lifecycle_
transition Nonemeans the event is not valid in that state — terminal states never auto-transition, and e.g.operator_releaseonly applies toquarantined.- noisy_
partition_ keys - Classify noisy neighbours from observed in-flight skew (tenant fairness/backlog metrics).
- parse_
traceparent - Parse a W3C
traceparentvalue:00-{32 lowercase hex}-{16 lowercase hex}-{2 hex}. - same_
job_ content - idempotent enqueue identity does the row that already owns this id hold the SAME job?
- trace_
context - The dispatch-time read: pull
TRACEPARENTout of an envelope’s headers and parse it, attachingTRACESTATEverbatim.Nonewhen the header is absent OR invalid — the two are deliberately indistinguishable to callers (seeparse_traceparent). - transition
- lifecycle state machine The transition table, mirroring conformance/state_machine.yaml row for row.
yaml_and_code_agree_row_for_rowin the tests parses that file and cross-checks every transition, so a row commented out THERE is a failing test HERE — and an unhandledOutcomevariant here is a compile error. Both languages check against the same file so they cannot drift. - validate_
enqueue - The boundary validation every backend’s
enqueueruns before it writes anything — ONE function so the rule cannot drift between four adapters, and the layer is the store because the API and the harnesses callStore::enqueuedirectly, never through the runtime. Batch-level: a repeated id WITHIN one batch is anIdConflicton every backend rather than a constraint error from whichever row the database reached first. - validate_
kind - The one kind-format rule (typed dispatch), enforced identically at handler registration, at enqueue in every backend, and at the HTTP API.
- validate_
progress