Expand description
Read a DLQ location (a local JSONL file, a directory of *.jsonl, or a
glob) back into DLQ envelopes, and expose it as a Source so faucet dlq replay can feed the unwrapped original payloads through the normal
pipeline path.
The line-parsing core (classify_line) is pure and unit-tested; file
IO (scan_files) is a thin shim over it. A DLQ location may contain
arbitrary lines (blank lines, non-faucet output), so parsing is
tolerant: unparseable and non-envelope lines are skipped and counted,
never fatal.
Structs§
- DlqDecryptor
- Optional decryption for DLQ lines sealed at rest by the jsonl sink’s
encryption:block (#207). The default carries no keys: sealed lines are still detected (and counted asLineOutcome::Undecryptable) so an encrypted DLQ inspected without a key reports what it is instead of “all lines malformed”. - DlqReader
Source - A
Sourceover a DLQ location that yields the unwrapped original payloads (optionally filtered by reason), so a replay run feeds them through the referenced config’s transforms / quality / contract / sink. - Scan
Result - Envelopes read from a DLQ location plus the tolerant-parse tallies.
- Source
Override - A pre-built source attached to a single
ExpandedNodeso the executor runs it instead of building one from the connector registry. Used only byfaucet dlq replay, which runs exactly one invocation, so the source is taken once.Cloneshares the same cell (cloning the node does not duplicate the source).
Enums§
- Line
Outcome - Outcome of classifying a single line of a DLQ location.
Functions§
- classify_
line - Classify one raw line. Pure — no IO. Blank lines are ignored; anything else is either an envelope, malformed JSON, valid-but-not-an-envelope, or an (un)decryptable sealed line.
- classify_
line_ with classify_linewith decryption support: sealed lines are decrypted throughdecbefore classification.- expand_
location - Expand a DLQ location into the concrete local files to read.
- reason_
matches - Whether an envelope matches an optional reason filter.
Nonematches everything; a legacy envelope with noreasonfield never matches an explicit filter. - scan_
files - Read and classify every line of every file, collecting envelopes and tallies. Blank lines are ignored; malformed / non-envelope lines are counted but never abort the scan.