Skip to main content

Module prefetch_adapter

Module prefetch_adapter 

Source
Expand description

Paper 3 — production PrefetchDispatcher adapter.

Bridges SpeculationEngine to McpServer::execute_for_prefetch so the speculation path runs against the same routing / transparent-proxy / fallback machinery as the main flow. The adapter holds an Arc<McpServer> (server is consumed once at startup), and forwards every dispatch through the server’s routing engine.

Failure modes are converted to PrefetchError variants so the engine can count them as wasted prefetches without surfacing them to the LLM stream.

Wiring contract:

  1. Build Arc<McpServer> once at startup.
  2. Construct McpPrefetchDispatcher::new(Arc::clone(&server)).
  3. session_pipeline.with_speculation(Arc::new(dispatcher)).await.
  4. From now on, SessionPipeline::speculate_after will dispatch real tools/call requests through the server, results land in the dedup cache, and the LLM’s organic call collapses to L0.

See paper-3-tool-aware-enrichment.md §Race-strategy for the end-to-end timing of how prefetch results meet the LLM’s main response.

Structs§

McpPrefetchDispatcher
Production PrefetchDispatcher backed by an Arc<McpServer>.