Skip to main content

Module pool

Module pool 

Source
Expand description

Persistent downstream MCP session pool (#1078).

Without pooling, every ctx_tools list/call reopened a connection — spawning a fresh child process and re-running the MCP initialize handshake — so each tool call paid the full spawn+handshake latency. This pool keeps one live ClientService per distinct wiring and reuses it across calls:

  • keyed by the resolved transport (same command/args/env/caps/url → same session), so two different servers never share a child,
  • idle-evicted: a session unused for IDLE_TTL is dropped on the next access (closing the child’s stdin → the server exits), swept opportunistically on every acquire,
  • liveness-checked: acquire also drops any session whose transport has closed (the child exited/crashed) before handing one out, so a request is never sent into a dead pipe — and callers never have to blindly re-send a request to recover (which could double-execute a non-idempotent tool).

The map lock is a std::sync::Mutex held only for short, await-free critical sections (the slow open() runs outside the lock), which keeps clear callable from the synchronous config/catalog paths.

Functions§

acquire
A live session for transport, reusing a pooled one when present + fresh, or opening (and caching) a new one. Sweeps idle sessions on the way in.
clear
Drop every pooled session (closing all children). Called when the gateway wiring changes (install/remove/revoke → super::catalog::invalidate).
evict
Drop the pooled session for key (e.g. after a transport-level failure), so the next acquire reopens a fresh one.
key
Stable identity for a resolved transport: same wiring → same key → same pooled session. Derived from the transport’s Debug form so every field (command/args/env/binary pin/capabilities, or url/headers) is captured.
len
Number of live pooled sessions (test/diagnostic helper).