Struct SpeculativeGenerationBatchOutput 

pub struct SpeculativeGenerationBatchOutput { /* private fields */ }

Completed speculative requests plus aggregate fair-scheduler telemetry.

Implementations

impl SpeculativeGenerationBatchOutput

pub fn new( requests: Vec<SpeculativeGenerationOutput>, scheduler: SpeculativeSchedulerStats, ) -> Self

Creates a completed batch in stable submission order.

pub fn requests(&self) -> &[SpeculativeGenerationOutput]

Per-request results in submission order.

pub fn into_requests(self) -> Vec<SpeculativeGenerationOutput>

Consumes the batch and returns its request results.

pub const fn scheduler(&self) -> &SpeculativeSchedulerStats

Aggregate scheduler telemetry.

pub fn push_request(&mut self, request: SpeculativeGenerationOutput)

Appends a result while adapting another backend-neutral execution path.

pub fn clear_requests(&mut self)

Clears adapted request results while retaining scheduler telemetry.