Struct BenchmarkChannel 

pub struct BenchmarkChannel { /* private fields */ }

Headless channel that feeds pre-loaded prompts and captures agent responses.

Used by the bench runner to drive the agent loop without a real terminal or network connection. recv drains the prompt queue; send and flush_chunks accumulate responses into an internal list. Tool outputs are captured separately via tool_outputs().

Usage

use zeph_bench::BenchmarkChannel;

let prompts = vec!["What year did WWII end?".into()];
let channel = BenchmarkChannel::new(prompts);
assert_eq!(channel.total(), 1);

After the agent loop completes, call into_responses to consume the channel and retrieve all captured responses:

let channel = BenchmarkChannel::new(vec!["question".into()]);
// ... run agent loop ...
let responses = channel.into_responses();

Implementations

impl BenchmarkChannel

pub fn new(prompts: Vec<String>) -> Self

Create a new channel pre-loaded with prompts.

Prompts are fed to the agent one at a time in order via recv. The channel returns Ok(None) once all prompts have been drained.

Examples

use zeph_bench::BenchmarkChannel;

let ch = BenchmarkChannel::new(vec!["hello".into(), "world".into()]);
assert_eq!(ch.total(), 2);

pub fn from_turns(turns: Vec<Turn>) -> Self

Create a channel from a multi-turn scenario history.

User turns are fed to the agent in order via recv. Assistant turns are pre-seeded into responses so that evaluators and Phase 2 scoring have access to the captured prior context.

Note

If turns contains no crate::scenario::Role::User turns, total returns 0 and the channel cannot serve as a prompt source. The bench runner rejects this with BenchError::InvalidFormat — callers must ensure at least one user turn is present.

Examples

use zeph_bench::BenchmarkChannel;
use zeph_bench::scenario::{Role, Turn};

let turns = vec![
    Turn { role: Role::User, content: "Hello".into() },
    Turn { role: Role::Assistant, content: "Hi there".into() },
    Turn { role: Role::User, content: "What year?".into() },
];
let ch = BenchmarkChannel::from_turns(turns);
assert_eq!(ch.total(), 2); // two user turns
assert_eq!(ch.responses().len(), 1); // one seeded assistant turn

pub fn total(&self) -> usize

Total number of prompts this channel was initialised with.

Examples

use zeph_bench::BenchmarkChannel;

let ch = BenchmarkChannel::new(vec!["a".into(), "b".into(), "c".into()]);
assert_eq!(ch.total(), 3);

pub fn into_responses(self) -> Vec<CapturedResponse>

Consume the channel and return all CapturedResponses collected so far.

Call this after the agent loop exits to retrieve every response in prompt order.

Examples

use zeph_bench::BenchmarkChannel;

let ch = BenchmarkChannel::new(vec!["question".into()]);
// ... run agent ...
let responses = ch.into_responses();

pub fn responses(&self) -> &[CapturedResponse]

Borrow the captured responses without consuming the channel.

Examples

use zeph_bench::BenchmarkChannel;

let ch = BenchmarkChannel::new(vec![]);
assert!(ch.responses().is_empty());

pub fn tool_outputs(&self) -> &[ToolOutputEvent]

Borrow the tool output events captured during the agent run.

Events are appended by send_tool_output and are available for Phase 2 evaluation (#4234) after the agent loop exits.

Examples

use zeph_bench::BenchmarkChannel;

let ch = BenchmarkChannel::new(vec![]);
assert!(ch.tool_outputs().is_empty());

Trait Implementations

impl Channel for BenchmarkChannel

async fn recv(&mut self) -> Result<Option<ChannelMessage>, ChannelError>

Receive the next message. Returns None on EOF or shutdown.

fn supports_exit(&self) -> bool

Whether /exit and /quit commands should terminate the agent loop.

async fn send(&mut self, text: &str) -> Result<(), ChannelError>

Send a text response.

async fn send_chunk(&mut self, chunk: &str) -> Result<(), ChannelError>

Send a partial chunk of streaming response.

async fn flush_chunks(&mut self) -> Result<(), ChannelError>

Flush any buffered chunks.

async fn send_usage( &mut self, input_tokens: u64, output_tokens: u64, context_window: u64, ) -> Result<(), ChannelError>

Send token usage after an LLM call. No-op by default.

async fn send_tool_output( &mut self, event: ToolOutputEvent, ) -> Result<(), ChannelError>

Send a complete tool output with optional diff and filter stats atomically.

fn try_recv(&mut self) -> Option<ChannelMessage>

Non-blocking receive. Returns None if no message is immediately available.

fn send_typing( &mut self, ) -> impl Future<Output = Result<(), ChannelError>> + Send

Send a typing indicator. No-op by default.

fn send_status( &mut self, _text: &str, ) -> impl Future<Output = Result<(), ChannelError>> + Send

Send a status label (shown as spinner text in TUI). No-op by default.

fn send_thinking_chunk( &mut self, _chunk: &str, ) -> impl Future<Output = Result<(), ChannelError>> + Send

Send a thinking/reasoning token chunk. No-op by default.

fn send_queue_count( &mut self, _count: usize, ) -> impl Future<Output = Result<(), ChannelError>> + Send

Notify channel of queued message count. No-op by default.

fn send_context_estimate( &mut self, _tokens: usize, ) -> impl Future<Output = Result<(), ChannelError>> + Send

Send the projected context token count to the channel after context assembly.

fn send_diff( &mut self, _diff: DiffData, _tool_call_id: &str, ) -> impl Future<Output = Result<(), ChannelError>> + Send

Send diff data for a tool result. No-op by default (TUI overrides).

fn send_tool_start( &mut self, _event: ToolStartEvent, ) -> impl Future<Output = Result<(), ChannelError>> + Send

Announce that a tool call is starting.

fn confirm( &mut self, _prompt: &str, ) -> impl Future<Output = Result<bool, ChannelError>> + Send

Request user confirmation for a destructive action. Returns true if confirmed. Default: auto-confirm (for headless/test scenarios).

fn elicit( &mut self, _request: ElicitationRequest, ) -> impl Future<Output = Result<ElicitationResponse, ChannelError>> + Send

Request structured input from the user for an MCP elicitation.

fn send_stop_hint( &mut self, _hint: StopHint, ) -> impl Future<Output = Result<(), ChannelError>> + Send

Signal the non-default stop reason to the consumer before flushing.

fn notify_foreground_subagent_started( &mut self, _id: &str, _name: &str, ) -> impl Future<Output = Result<(), ChannelError>> + Send

Notify channel that a foreground subagent has started. No-op by default.

fn notify_foreground_subagent_completed( &mut self, _id: &str, _name: &str, _success: bool, ) -> impl Future<Output = Result<(), ChannelError>> + Send

Notify channel that a foreground subagent has completed. No-op by default.