Module instance_pool 

Pre-instantiated instance pool (roadmap feature #5, T37 implementation).

Pre-spawns N instances per (tenant_id, module_hash) tuple and draws from a per-tuple crossbeam_channel on acquire. Closes the Instance::new_async cost on the warm invocation path. The pooling allocator + MPK backend already exists (see EngineConfig::PoolingMpk); this module sits on top of it, addressing the instantiation-path cost that the memory backend does not.

Channel layout

Each entry in InstancePool::pools is a (Sender, Receiver) pair bounded at warm_instances_per_tuple (floored at 1 — a capacity-0 crossbeam_channel is rendezvous-shaped and would deadlock the release-to-channel path). The first InstancePool::acquire call for a (tenant, module_hash) key spawns warm_instances_per_tuple detached instances and seeds the channel with them. Subsequent acquire calls drain the channel via try_recv (wait-free, O(1)); on empty they fall through to TensorWasmExecutor::spawn_instance up to the executor’s existing max_instances ceiling.

Tenant + module isolation is enforced by the key: instance A drawn by tenant T1 from module M never lands in tenant T2’s channel, nor in T1’s channel for module M’. Both halves of the key are part of the PoolKey tuple.

Reset trade-off

On InstancePool::release, the spent instance is discarded and a fresh one is spawned from the cached wasmtime::Module to take its place in the channel. We pay for the wasmtime instantiate (Instance::new_async) but skip the Cranelift compile entirely (the module is already in the executor’s per-module LRU cache). Rationale:

• Linear memory must return to its post-start snapshot, with no guest-written bytes leaking across tenants or invocations. The cheapest way to guarantee this without a per-module bytecode-level snapshot is to drop the wasmtime::Store and re-instantiate.
• Mutable globals, table state, and any allocated WASI fds beyond stdio are scrubbed for free by the same drop.
• The compile step (Module::from_binary → Cranelift) is by far the expensive part of the spawn path on multi-MiB modules; the Instance::new_async cost on a cached module is small but non-zero (store construction, limiter wiring, start execution).

If the replacement reset exceeds ~10 ms (the deadline-aware drop threshold), the new instance is dropped rather than enqueued — cheaper to let the next acquire spawn fresh than to block invoke behind a slow reset.

Streaming spawns are not recycled

When crate::executor::SpawnConfig::streaming is set (T34 SSE /invoke-stream), the streaming context carries a one-shot mpsc::Sender whose receiver is drained for the duration of the response and then dropped. There is no way to “reset” a streaming context for a subsequent invoke; the gateway constructs a fresh channel for each call. InstancePool::release therefore drops streaming instances unconditionally — they are never returned to the warm channel.

Structs

InstancePool

Pre-instantiated instance pool. Backed by a per-tuple crossbeam_channel of pre-spawned instances; see module docs for the channel layout, reset trade-off, and streaming-spawn carve-out.

InstancePoolConfig

Pool configuration. One pool per executor.

PooledInstance

RAII wrapper around a pool-drawn instance.

Type Aliases

ModuleHash

BLAKE3 digest of a Wasm module’s bytes. Re-exported so embedders that hold module hashes outside the pool (e.g. for diagnostic logging) share the same shape.