Expand description
Append-only download-receipt log for the node runtime (issue #248).
A node enqueues one structured DownloadReceipt per served-and-paid blob
at the voucher-acceptance point (see
crate::handlers::client::ClientHandler); the disk write happens off the
delivery path in a background writer (see the Seam note below, #803). The
log gives the operator an on-disk, off-chain record of which blobs were
delivered and paid for — enabling revenue audit, cross-restart double-spend
triage, and dispute evidence without relying solely on on-chain state.
§Seam
There are two layers. ReceiptLog is the disk boundary — a small trait
(mirroring the trait+impl pattern of
decdn_incentive::store::PoolStateStore / crate::channel_store)
whose runtime impl is the disk-backed JsonlReceiptLog, appending one JSON
object per line to <data_dir>/download_receipts.jsonl (JSON Lines).
ReceiptSink is the hot-path boundary: the paid-delivery path does not
call ReceiptLog::append directly — it enqueues through the non-blocking
ReceiptSink::record, and a single background spawn_receipt_writer
task owns the JsonlReceiptLog and performs the actual append off the
delivery path (#803), so a slow or full disk can never back-pressure
delivery. Both seams take an in-memory fake in unit/loopback tests.
§Durability
Each ReceiptLog::append writes one line straight to the file — one
write_all, which lands the bytes in the OS page cache — and does not
fsync per record. The receipt log is an audit artifact, not the
voucher-replay guard. That guard is the lane watermark in
crate::channel_store::PersistentPoolStateStore, which
advances in memory the moment a voucher or preimage is accepted, reaches
disk on the runtime’s periodic lane flush
(payment.voucher_commit_interval_ms, 5 s by default), and is flushed again
before every on-chain redemption.
Neither side fsyncs on the delivery path, so after a crash the two tails
can disagree, in either direction, and neither direction is guaranteed.
The audit log runs ahead when the background writer has already written a
line whose lane advance is still buffered — a restart then finds a receipt
whose watermark the lane store no longer holds. It runs behind when the
crash catches receipts still queued in RECEIPT_LOG_CAPACITY (that queue
is process memory and dies with the process, and only a graceful shutdown
drains it), or when a hard crash drops page-cache lines the 5 s lane flush
had already fsynced past.
Both directions are safe. Ahead costs revenue, not correctness: the node forfeits at most one flush interval of frontier (value it has not yet redeemed), which ADR 003 §Off-chain voucher state persistence accepts. Behind is a gap in the audit record and nothing more.
Replay protection rests on the redeemed watermark instead, which the
pre-redemption flush floors. That floor holds wherever the node can afford
to wait for it: the periodic sweep and the redeem-hint path skip their
submit when the flush fails, while the shutdown sweep redeems anyway,
because forfeiting the claim across the stop is worse than a bounded
re-serve risk (see crate::payment_settlement). Skipping the
per-line fsync keeps the cost off the hot delivery path (ADR 003).
The append is also crash-atomic at the line level only in the usual POSIX sense (a torn final line is possible after a hard crash); JSONL readers MUST tolerate a truncated trailing line.
§Rotation (issue #802)
Left unbounded the log grows for the entire process lifetime and across
restarts (O_APPEND), eventually exhausting data_dir. JsonlReceiptLog
therefore takes a RotationPolicy (max_file_bytes, retained_files):
once the live file would exceed max_file_bytes it is rotated to a numbered
backup (download_receipts.jsonl.1, .2, …, oldest = highest), a fresh
live file is opened, and any backup beyond retained_files is deleted. This
bounds data_dir to roughly (retained_files + 1) * max_file_bytes.
retained_files == 0 keeps no backups — the live file is truncated in place
on rotation. Rotation is best-effort: a rename/open failure mid-rotation
never leaves the log without a writable handle and never aborts paid
delivery (the background writer treats an append error as non-fatal, and
delivery is decoupled from it by the bounded queue, #803), it only trims or
skips a backup that cycle.
Structs§
- Channel
Receipt Sink - Production
ReceiptSink: enqueues to the background writer over a bounded channel. Mirrors the redeem-hint sink (#751): aFullchannel drops the receipt and counts it (decdn_receipt_writes_dropped_total); aClosedchannel — the writer stopped during shutdown — is silently ignored (the tail drain has already run, or is bounded by the shutdown deadline). - Download
Receipt - One durable record of a served-and-paid blob (issue #248).
- Jsonl
Receipt Log - JSON-Lines, append-only
ReceiptLogbacked by a single rotating file underdata_dir. - Noop
Receipt Log - A
ReceiptLogthat drops every receipt. Used as the runtime fallback whenJsonlReceiptLog::openfails at bring-up: the audit log is best-effort, so the node serves paid delivery without it rather than refusing to start (the replay guard lives in the separate lane store, which the periodic flush mirrors to disk). The operator gets oneerror!at startup naming the open failure. - RawReceipt
- One receipt as the paid-delivery hot path produces it: raw typed identifiers, no rendering and no clock read.
- Rotation
Policy - Size-based rotation policy for
JsonlReceiptLog(issue #802).
Constants§
- RECEIPT_
LOG_ CAPACITY - Capacity of the receipt-writer queue. Receipts are roughly one per voucher
interval (~1 MiB served), so this absorbs a large burst of voucher
acceptances while a disk stall (a slow or full
data_dir, the realistic end-state of #802) is worked off, without ever back-pressuring paid delivery. On overflow the audit receipt is dropped (counted viaMetrics::receipt_write_dropped) rather than the payment stalling — the payment already advanced the lane watermark. Rotation (#802) bounds the log on disk; this bounds it in memory.
Traits§
- Receipt
Log - Append-only sink for
DownloadReceipts. - Receipt
Sink - Non-blocking enqueue boundary for
RawReceipts on the paid-delivery hot path (#803).
Functions§
- spawn_
receipt_ writer - Spawn the single background task that owns the on-disk
ReceiptLogand drains the receipt queue, returning theReceiptSinkthe delivery path enqueues through plus theStopHandlethat stops the task at graceful shutdown.