Struct SpeculativeCache 

pub struct SpeculativeCache { /* private fields */ }

Cache for in-flight speculative handles, bounded by max_in_flight.

Thread-safe; all operations hold a short parking_lot::Mutex lock. The inner Arc<Mutex<CacheInner>> is shared with the background TTL sweeper so both operate on the same handle set (C2: no separate empty instance in the sweeper).

Implementations

impl SpeculativeCache

pub fn new(max_in_flight: usize) -> Self

Create a new cache with the given capacity.

pub fn shared_inner(&self) -> Arc<Mutex<CacheInner>>

Return a cloned Arc to the inner mutex so it can be shared with a sweeper task.

The sweeper calls SpeculativeCache::sweep_expired_inner on the shared Arc instead of constructing a second SpeculativeCache that would have separate storage.

pub fn sweep_expired_inner(inner: &Mutex<CacheInner>)

Cancel and remove all handles whose TTL deadline has passed, operating on a raw Arc.

Intended for use by the sweeper task, which holds only the Arc (not a full SpeculativeCache wrapper).

pub fn insert(&self, handle: SpeculativeHandle)

Insert a new handle. If at capacity, evicts and cancels the oldest.

If a handle with the same key already exists it is replaced and explicitly cancelled so the underlying tokio task does not keep running (C4: no silent drop).

pub fn take_match( &self, tool_id: &ToolName, args_hash: &Hash, ) -> Option<SpeculativeHandle>

Find and remove a handle matching tool_id + args_hash.

pub fn cancel_by_tool_id(&self, tool_id: &ToolName)

Remove and cancel the first handle whose tool_id matches, if any.

Used when the args hash is not known (e.g., on tool-id mismatch at dispatch time).

pub fn sweep_expired(&self)

Cancel and remove all handles whose TTL deadline has passed.

Called by the sweeper task every 5 s.

pub fn cancel_all(&self)

Cancel and remove all remaining handles (called at turn boundary).

pub fn len(&self) -> usize

Number of in-flight handles.

pub fn is_empty(&self) -> bool

True when the cache is empty.