pub struct SpeculationEngine { /* private fields */ }Expand description
Per-turn speculation engine. One instance per SessionPipeline;
holds the JoinSet and the host budget. Drop = shutdown.
Implementations§
Source§impl SpeculationEngine
impl SpeculationEngine
Sourcepub fn new(
config: EnrichmentConfig,
dispatcher: Arc<dyn PrefetchDispatcher>,
) -> Self
pub fn new( config: EnrichmentConfig, dispatcher: Arc<dyn PrefetchDispatcher>, ) -> Self
Build a fresh engine bound to a dispatcher. Settings come
from config.enrichment.
Sourcepub fn with_per_host_cap(self, cap: u32) -> Self
pub fn with_per_host_cap(self, cap: u32) -> Self
Build with an explicit per-host cap. Useful for tests that want
cap = 1 to force serialisation.
Sourcepub fn is_enabled(&self) -> bool
pub fn is_enabled(&self) -> bool
true when enrichment.enabled = false — host should skip
all dispatch and not even build a plan.
Sourcepub fn timeout(&self) -> Duration
pub fn timeout(&self) -> Duration
Returns the configured wall-clock budget for one wait_within call.
Sourcepub fn pending(&self) -> usize
pub fn pending(&self) -> usize
Number of currently-spawned tasks (incl. pending ones from previous turns that have not been collected yet).
Sourcepub async fn dispatch(
&mut self,
requests: Vec<PrefetchRequest>,
) -> Vec<PrefetchOutcome>
pub async fn dispatch( &mut self, requests: Vec<PrefetchRequest>, ) -> Vec<PrefetchOutcome>
Schedule requests honouring max_parallel_prefetches and
per-host caps. Requests rejected by either gate are reported
as Skipped outcomes and not spawned.
Returns the immediately-known outcomes (skips). Settled /
failed outcomes come back from Self::wait_within.
Sourcepub async fn wait_within(&mut self) -> Vec<PrefetchOutcome>
pub async fn wait_within(&mut self) -> Vec<PrefetchOutcome>
Wait up to prefetch_timeout_ms collecting outcomes for tasks
that complete in time. Tasks still pending stay in the
JoinSet — their results arrive on the next wait_within (or
get cancelled by shutdown).
The timeout is a global deadline for the whole call, not
a per-task budget — N slow prefetches finishing one-by-one
just under the threshold can no longer stall the response
for N × prefetch_timeout_ms.
Returns Settled for every task that returned Ok(body),
Failed for every Err(...). Skipped outcomes from the most
recent dispatch call are not echoed here — the host already
has them.
Sourcepub async fn drain_pending(&mut self) -> Vec<PrefetchOutcome>
pub async fn drain_pending(&mut self) -> Vec<PrefetchOutcome>
Best-effort drain of completed tasks without blocking.
Returns outcomes for every task that has already finished,
leaves still-pending tasks alone. Lets the host call this on
the next turn (or before each dispatch) to recover bodies
that landed after the previous wait_within timed out, so
they can still be written into the dedup cache instead of
being silently lost on the next shutdown.