Skip to main content

Crate cellos_sink_dlq

Crate cellos_sink_dlq 

Source
Expand description

Dead Letter Queue (DLQ) EventSink wrapper — P3-03.

Wraps any Arc<dyn EventSink> and, on a primary emit() failure, persists the failed CloudEventV1 as a JSON line to a file under an operator-nominated directory. The wrapper is a no-op unless the CELLOS_DLQ_DIR environment variable is set to a non-empty, existing-and-writable directory at composition time.

§Wire format

One file is written per failed event. The file name is:

<timestamp-millis>-<event-id-sanitized>.jsonl

where <timestamp-millis> is the Unix epoch milliseconds at the moment of the failure (monotonically increasing within a single process barring clock skew) and <event-id-sanitized> replaces any character outside [A-Za-z0-9._-] with _ so attacker-controlled or unusual CloudEvent IDs cannot escape the DLQ directory or collide with shell globs.

Each file contains exactly one JSON object on a single line:

{"event": <CloudEventV1>, "error": "<primary sink error message>"}

Using one file per event (rather than a rolling JSONL) keeps writes append- free and tolerates concurrent emitters without locking, at the cost of one inode per failure. Operators expecting high failure rates should rotate or compact ${CELLOS_DLQ_DIR} out of band.

§Failure semantics

On primary emit() failure, the wrapper:

  1. Best-effort writes the failed event to the DLQ.
  2. Logs a warning with the event id and primary error (and DLQ-write error if the persistence itself failed).
  3. Returns Ok(()) to the caller — the DLQ has assumed responsibility for the event, so callers should treat the emit as “delivered to the operator’s recovery channel”. If the DLQ write also fails, the original primary error is propagated unchanged so the supervisor still sees a failure rather than silently dropping the event.

§Activation

CELLOS_DLQ_DIR=/var/lib/cellos/dlq

Disabled by default. When unset, empty, or pointing at a path that does not exist or is not writable, DlqSink::from_env returns the inner sink unwrapped (identity).

Structs§

DlqSink
EventSink wrapper that persists failed CloudEvents to a DLQ directory.