Struct SpeculationCache 

pub struct SpeculationCache { /* private fields */ }

Manages speculative execution of read-only tools. Spawns tokio tasks to pre-fetch results for predicted tool calls.

Implementations

impl SpeculationCache

pub fn new(max_concurrent: usize) -> Self

pub async fn get( &self, tool_name: &str, params: &Value, ) -> Option<SpeculativeResult>

Check if we have a cached result for the given tool + params.

pub async fn insert( &self, tool_name: &str, params: &Value, result: SpeculativeResult, )

Store a speculative result.

pub async fn clear(&self)

Clear all cached speculative results (called at turn completion).

pub async fn size(&self) -> usize

Number of cached results.

pub fn can_speculate(&self) -> bool

Whether we can spawn another speculative task.

pub fn start_speculation(&self) -> bool

Try to reserve a speculation slot. Returns true if the slot was acquired.

pub fn reserve_slot(&self) -> Option<SpeculationSlotGuard>

Reserve a speculation slot and get an RAII guard that releases it on drop.

Prefer this over start/end pairs to guarantee cleanup on cancellation/panic.

pub fn end_speculation(&self)

Release a speculation slot.

Saturates at zero: calling this when no slots are active is a no-op rather than wrapping usize to MAX.

pub fn active_count(&self) -> usize

Trait Implementations

impl Debug for SpeculationCache

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.