Skip to main content

Module distill_queue

spool

Module distill_queue 

Source
Expand description

Distill-pending queue — append-only JSONL log under <cwd>/.spool/distill-pending.queue that Stop / PreCompact hooks drain into candidate / accepted memory records.

§Why a separate file (vs. piggybacking on the ledger)?

  • The ledger is the lifecycle truth source. Half-cooked signals (raw tool outputs, partial prompts) MUST NOT pollute it.
  • The queue is intentionally lossy under pressure (LRU 100 by default) — old signals dropped silently. The ledger never drops.
  • The queue file is project-local; deleting .spool/ on a fresh clone safely wipes accumulated signals without touching the shared vault ledger.

§Concurrency model

Multiple Claude Code instances may run hooks against the same .spool/ (e.g. user opens two terminals in the same repo). We use fs2::FileExt::lock_exclusive (POSIX flock) to serialize all mutating operations. flock is advisory + per-fd, sufficient for single-host concurrency. We do NOT support cross-host network FS concurrency (out of scope per ADR-0001 single-user assumption).

§File format

One JSON object per line. Each entry has the shape produced by [hook_runtime::post_tool_use::DistillSignalEnvelope]:

{"recorded_at": <unix>, "tool_name": <opt>, "cwd": <abs>, "payload": <opt>}

§Operations

  • append: tail-add a single envelope; if total lines exceed the LRU cap after append, rewrite the file keeping only the most recent N entries.
  • drain_all: read & truncate to empty (Stop hook consumes queue).
  • peek_all: read without truncating (debugging / doctor).

Structs§

DistillSignal
One signal envelope written by the post-tool-use hook.

Constants§

DEFAULT_LRU_CAP
DEFAULT_QUEUE_FILE_NAME

Functions§

append
Append a single signal envelope to the queue, then enforce the LRU cap. Atomic w.r.t. concurrent appenders thanks to flock.
drain_all
Read all lines and atomically truncate the queue to empty. The caller (typically Stop hook) processes the returned signals into the lifecycle ledger.
peek_all
Read all lines without modifying the queue. Useful for doctor / debugging.
queue_path
Resolve the queue file path under <runtime_dir>/distill-pending.queue.