Struct InstancePool 

pub struct InstancePool { /* private fields */ }

Pre-instantiated instance pool. Backed by a per-tuple crossbeam_channel of pre-spawned instances; see module docs for the channel layout, reset trade-off, and streaming-spawn carve-out.

Implementations

impl InstancePool

pub fn new(cfg: InstancePoolConfig) -> Self

Construct a new pool with the given config.

pub async fn acquire( &self, executor: &TensorWasmExecutor, wasm: &[u8], cfg: SpawnConfig, ) -> Result<PooledInstance, ExecError>

Acquire a pre-spawned instance or fall through to a fresh spawn.

Algorithm (T37):

1. Compute the (tenant, module_hash) key — building the entry on first observation by compiling the module (via the executor’s cached path), allocating a bounded channel of the configured size, and pre-spawning warm_instances_per_tuple detached instances into it.
2. try_recv on the per-tuple channel. On hit, register the drawn instance with the executor and return a PooledInstance handle.
3. On miss, fall through to TensorWasmExecutor::spawn_instance so the call still proceeds (admission control still applies — the executor’s max_instances cap fires here if saturated).

Streaming spawns (SpawnConfig::streaming set) bypass the warm-channel try_recv and always fall through to a fresh spawn: the streaming context is one-shot (the gateway-held receiver is drained per call), so the warm channel never holds streaming instances.

pub async fn release( &self, executor: &TensorWasmExecutor, pooled: PooledInstance, cfg: &SpawnConfig, )

Release an instance back to the pool.

The instance is detached from the executor’s registry (the slot stays charged for the moment) and dropped — its linear memory could be holding sensitive guest state, so “reset” here means “re-instantiate”, never “rewind”. A fresh replacement is then spawned from the cached wasmtime::Module and try_send’d to the channel; on full / send-error / streaming carve-out / reset deadline exceeded, the replacement is dropped and the slot is released.

pooled carries the originating module hash so the release path dispatches to the correct per-tuple channel without re-hashing (the release path no longer has the wasm bytes on hand).

pub fn warm_count(&self) -> usize

Number of currently warm (in-channel) instances across all tuples.

pub fn draws_total(&self) -> usize

Total successful pool draws (hits + misses). Monotonic counter.

pub fn hit_count(&self) -> usize

Draws that hit a warm instance in the channel. Monotonic counter.

pub fn miss_count(&self) -> usize

Draws that fell through to a fresh spawn. Monotonic counter.

pub fn drops_total(&self) -> usize

Number of pool drops (channel full, reset failure, streaming carve-out, or reset deadline exceeded). Monotonic counter.

pub fn config(&self) -> &InstancePoolConfig

Borrow the pool’s configuration.

pub fn shutdown(&self, executor: &TensorWasmExecutor)

Drain every warm channel and release its instances. Called by embedders that want to free pool-held wasmtime resources without dropping the executor (e.g. on a config reload).

Idempotent — calling on an already-empty pool is a no-op. Slot accounting on the executor is restored: every instance pulled out is dropped, and the corresponding slot is released.