Expand description
Paper 3 — production PrefetchDispatcher adapter.
Bridges SpeculationEngine to McpServer::execute_for_prefetch
so the speculation path runs against the same routing /
transparent-proxy / fallback machinery as the main flow. The
adapter holds an Arc<McpServer> (server is consumed once at
startup), and forwards every dispatch through the server’s
routing engine.
Failure modes are converted to PrefetchError variants so the
engine can count them as wasted prefetches without surfacing
them to the LLM stream.
Wiring contract:
- Build
Arc<McpServer>once at startup. - Construct
McpPrefetchDispatcher::new(Arc::clone(&server)). session_pipeline.with_speculation(Arc::new(dispatcher)).await.- From now on,
SessionPipeline::speculate_afterwill dispatch realtools/callrequests through the server, results land in the dedup cache, and the LLM’s organic call collapses to L0.
See paper-3-tool-aware-enrichment.md §Race-strategy for the
end-to-end timing of how prefetch results meet the LLM’s main
response.
Structs§
- McpPrefetch
Dispatcher - Production
PrefetchDispatcherbacked by anArc<McpServer>.