Skip to main content

Module output

Module output 

Source
Expand description

Task 26 — bounded tool-output delta channels with an explicit backpressure/coalescing policy (spec §8.4).

High-volume tool output used to ride an UNBOUNDED mpsc queue from the producing tool to a detached forwarding task; a fast producer with a slow consumer could therefore grow RSS without limit. This module replaces that path with:

  • a bounded channel (DELTA_CHANNEL_CAPACITY chunks, each retained chunk capped at DELTA_MAX_CHUNK_BYTES);
  • an explicit coalescing overflow buffer (at most DELTA_COALESCE_CAP_BYTES bytes): when the channel is full, chunks merge into one pending byte-capped buffer instead of queueing without bound;
  • an explicit drop policy beyond the coalesce cap, with exact dropped/coalesced byte and chunk counters (OutputCounters);
  • a UI-preview budget enforced by the forwarding task at production time via agent_core::BoundedText — the downstream (unbounded) UI event queue can never receive more than the budget from one tool call;
  • cancellation that terminates the forwarding task and thereby closes the channel, releasing producers: a send after close is a counted drop, never a block.

Policy choice (documented, deliberate): producers are NEVER blocked on the UI consumer. The delta lane is a preview lane — full fidelity for model history is owned by the tool’s own bounded result buffer — so the bounded policy here is coalesce-then-drop with exact accounting rather than producer backpressure, which would let a stalled UI consumer stall tool execution and cancellation.

Memory invariant: bytes retained in flight are bounded by DELTA_CHANNEL_CAPACITY * DELTA_MAX_CHUNK_BYTES + DELTA_COALESCE_CAP_BYTES per tool call, and always equal produced - forwarded - dropped in the counters (coalesced bytes are informational overlap, not a third bucket).

Structs§

BoundedOutput
Model-history production result. text is always a UTF-8 prefix at most the configured model-history budget; the original is represented only by exact byte accounting, never retained in full.
DeltaReceiver
Consumer half: drains the bounded channel and the coalescing overflow buffer in production order.
DeltaSender
Producer half handed to tools as ToolChannels::tx_delta. send never blocks and never fails: full channels coalesce, closed channels drop — both exactly counted.
OutputBudgets
Independent per-call output byte budgets (spec §8.4). The UI-preview lane (streamed deltas + the final ToolResult event) and the model-history lane (the tool_result block content) are bounded SEPARATELY: tightening one never widens or narrows the other. Both are enforced through agent_core::BoundedText at the point the bounded text is produced — never by materializing the full output first.
OutputCounters
Exact per-call output accounting. All methods are lock-free and cheap; metadata only (byte/chunk counts, never content).
OutputCountersSnapshot
Point-in-time copy of OutputCounters.
OutputHandle
Cloneable observation handle for one tool call’s independently bounded outputs and exact counters.
SpillHandle
Cloneable read handle onto a call’s spill artifact state.
SpillPolicy
Opt-in spill policy: capture the FULL produced delta stream (before any cap/coalesce/drop decision) into a private (0600, T4 helpers) artifact under dir, so bounded previews never mean lost data.
SpillReport
Point-in-time spill outcome. Metadata only: path, byte count, failure.
ToolDeltaChannel
One bounded delta channel pair plus its shared output handle.

Constants§

DEFAULT_UI_PREVIEW_BYTES
Default per-tool-call UI-preview byte budget enforced by the forwarder. Matches the historical max_tool_buffer bound (256 KiB) so bounded tool results remain visually identical.
DELTA_CHANNEL_CAPACITY
Maximum chunks queued in the bounded delta channel.
DELTA_COALESCE_CAP_BYTES
Byte cap of the coalescing overflow buffer used while the channel is full.
DELTA_MAX_CHUNK_BYTES
Per-chunk retention cap: a single produced chunk larger than this keeps only a UTF-8-safe prefix (the cut tail is a counted drop).

Functions§

active_ui_forwarder_count
Number of UI forwarding tasks currently alive in this process.
bounded_preview
Bound a final result string for the UI event lane, appending a metadata-only truncation marker when anything was cut. UTF-8-safe and greedy via agent_core::BoundedText.
delta_channel
Construct a bounded delta channel with fresh counters and default budgets.
delta_channel_with
Backward-compatible constructor for callers selecting only spill policy.
delta_channel_with_budgets
Construct one output producer with independent UI/model-history budgets.
spawn_ui_forwarder
Spawn the forwarding task for one tool call: drains the bounded channel, enforces the UI-preview byte budget at production time (UTF-8-safe via BoundedText), and terminates on cancellation or channel close. On termination the receiver drops, closing the channel and RELEASING every producer (their sends become counted drops).