Skip to main content

Module receipt_log

Module receipt_log 

Source
Expand description

Append-only download-receipt log for the node runtime (issue #248).

A node enqueues one structured DownloadReceipt per served-and-paid blob at the voucher-acceptance point (see crate::handlers::client::ClientHandler); the disk write happens off the delivery path in a background writer (see the Seam note below, #803). The log gives the operator an on-disk, off-chain record of which blobs were delivered and paid for — enabling revenue audit, cross-restart double-spend triage, and dispute evidence without relying solely on on-chain state.

§Seam

There are two layers. ReceiptLog is the disk boundary — a small trait (mirroring the trait+impl pattern of decdn_incentive::store::PoolStateStore / crate::channel_store) whose runtime impl is the disk-backed JsonlReceiptLog, appending one JSON object per line to <data_dir>/download_receipts.jsonl (JSON Lines). ReceiptSink is the hot-path boundary: the paid-delivery path does not call ReceiptLog::append directly — it enqueues through the non-blocking ReceiptSink::record, and a single background spawn_receipt_writer task owns the JsonlReceiptLog and performs the actual append off the delivery path (#803), so a slow or full disk can never back-pressure delivery. Both seams take an in-memory fake in unit/loopback tests.

§Durability

Each ReceiptLog::append writes one line straight to the file — one write_all, which lands the bytes in the OS page cache — and does not fsync per record. The receipt log is an audit artifact, not the voucher-replay guard. That guard is the lane watermark in crate::channel_store::PersistentPoolStateStore, which advances in memory the moment a voucher or preimage is accepted, reaches disk on the runtime’s periodic lane flush (payment.voucher_commit_interval_ms, 5 s by default), and is flushed again before every on-chain redemption.

Neither side fsyncs on the delivery path, so after a crash the two tails can disagree, in either direction, and neither direction is guaranteed. The audit log runs ahead when the background writer has already written a line whose lane advance is still buffered — a restart then finds a receipt whose watermark the lane store no longer holds. It runs behind when the crash catches receipts still queued in RECEIPT_LOG_CAPACITY (that queue is process memory and dies with the process, and only a graceful shutdown drains it), or when a hard crash drops page-cache lines the 5 s lane flush had already fsynced past.

Both directions are safe. Ahead costs revenue, not correctness: the node forfeits at most one flush interval of frontier (value it has not yet redeemed), which ADR 003 §Off-chain voucher state persistence accepts. Behind is a gap in the audit record and nothing more.

Replay protection rests on the redeemed watermark instead, which the pre-redemption flush floors. That floor holds wherever the node can afford to wait for it: the periodic sweep and the redeem-hint path skip their submit when the flush fails, while the shutdown sweep redeems anyway, because forfeiting the claim across the stop is worse than a bounded re-serve risk (see crate::payment_settlement). Skipping the per-line fsync keeps the cost off the hot delivery path (ADR 003).

The append is also crash-atomic at the line level only in the usual POSIX sense (a torn final line is possible after a hard crash); JSONL readers MUST tolerate a truncated trailing line.

§Rotation (issue #802)

Left unbounded the log grows for the entire process lifetime and across restarts (O_APPEND), eventually exhausting data_dir. JsonlReceiptLog therefore takes a RotationPolicy (max_file_bytes, retained_files): once the live file would exceed max_file_bytes it is rotated to a numbered backup (download_receipts.jsonl.1, .2, …, oldest = highest), a fresh live file is opened, and any backup beyond retained_files is deleted. This bounds data_dir to roughly (retained_files + 1) * max_file_bytes. retained_files == 0 keeps no backups — the live file is truncated in place on rotation. Rotation is best-effort: a rename/open failure mid-rotation never leaves the log without a writable handle and never aborts paid delivery (the background writer treats an append error as non-fatal, and delivery is decoupled from it by the bounded queue, #803), it only trims or skips a backup that cycle.

Structs§

ChannelReceiptSink
Production ReceiptSink: enqueues to the background writer over a bounded channel. Mirrors the redeem-hint sink (#751): a Full channel drops the receipt and counts it (decdn_receipt_writes_dropped_total); a Closed channel — the writer stopped during shutdown — is silently ignored (the tail drain has already run, or is bounded by the shutdown deadline).
DownloadReceipt
One durable record of a served-and-paid blob (issue #248).
JsonlReceiptLog
JSON-Lines, append-only ReceiptLog backed by a single rotating file under data_dir.
NoopReceiptLog
A ReceiptLog that drops every receipt. Used as the runtime fallback when JsonlReceiptLog::open fails at bring-up: the audit log is best-effort, so the node serves paid delivery without it rather than refusing to start (the replay guard lives in the separate lane store, which the periodic flush mirrors to disk). The operator gets one error! at startup naming the open failure.
RawReceipt
One receipt as the paid-delivery hot path produces it: raw typed identifiers, no rendering and no clock read.
RotationPolicy
Size-based rotation policy for JsonlReceiptLog (issue #802).

Constants§

RECEIPT_LOG_CAPACITY
Capacity of the receipt-writer queue. Receipts are roughly one per voucher interval (~1 MiB served), so this absorbs a large burst of voucher acceptances while a disk stall (a slow or full data_dir, the realistic end-state of #802) is worked off, without ever back-pressuring paid delivery. On overflow the audit receipt is dropped (counted via Metrics::receipt_write_dropped) rather than the payment stalling — the payment already advanced the lane watermark. Rotation (#802) bounds the log on disk; this bounds it in memory.

Traits§

ReceiptLog
Append-only sink for DownloadReceipts.
ReceiptSink
Non-blocking enqueue boundary for RawReceipts on the paid-delivery hot path (#803).

Functions§

spawn_receipt_writer
Spawn the single background task that owns the on-disk ReceiptLog and drains the receipt queue, returning the ReceiptSink the delivery path enqueues through plus the StopHandle that stops the task at graceful shutdown.