Struct TensorWasmExecutor 

pub struct TensorWasmExecutor { /* private fields */ }

The async executor.

Implementations

impl TensorWasmExecutor

pub fn new(engine: Arc<TensorWasmEngine>) -> Self

Construct an executor over the given shared engine.

pub fn with_metrics( engine: Arc<TensorWasmEngine>, metrics: TensorWasmMetrics, ) -> Self

Construct an executor that publishes spawn/terminate events to the supplied TensorWasmMetrics registry. Metric handles are cheaply cloneable; pass a clone of the process-wide registry.

pub fn with_instance_pool(self, pool: Arc<InstancePool>) -> Self

Attach an InstancePool to this executor and return the modified executor.

Builder method; pairs with Self::new / Self::with_metrics. The pool itself is opt-in — embedders that do not call this method see the v0.3.5 behaviour (every spawn_instance does a fresh compile/instantiate). Calling it today wires up the v0.3.6 scaffold; the same call gets the v0.4 warm-pool path for free once that lands.

pub fn with_jit_cache(self, cache: Arc<KernelCache>) -> Self

Attach a JIT KernelCache to this executor and return the modified executor.

Builder method; pairs with Self::new / Self::with_metrics. When set, every instantiation registers the tensor-wasm:jit/host dispatch/alloc/free imports (see crate::jit_dispatch::add_jit_dispatch_to_linker) against this cache, so auto-offloaded guests link successfully. Embedders that do not call this leave the JIT surface unlinked — a guest importing it then fails to link, matching the pre-wiring behaviour.

pub fn jit_cache(&self) -> Option<&Arc<KernelCache>>

Borrow the attached JIT KernelCache, if any. Returns None for executors constructed without Self::with_jit_cache.

pub fn instance_pool(&self) -> Option<&Arc<InstancePool>>

Borrow the attached InstancePool, if any. Returns None for executors constructed without Self::with_instance_pool.

pub fn engine(&self) -> &TensorWasmEngine

Borrow the underlying engine.

pub fn live_count(&self) -> usize

Number of instances currently present in the registry — i.e. those that have been spawned and register_pooled_instance’d but not yet terminate’d. This is the size of the instances DashMap.

Contrast with Self::instances_len, which reports the admission count (the atomic counter that crate::engine::EngineConfig::max_instances is enforced against). The two can briefly diverge: instances_len is bumped before compile/instantiate in spawn_instance (with rollback on failure) and a pool can hold an admitted-but-unregistered instance in a warm channel, so instances_len() >= live_count() always holds, with equality once all in-flight spawns have either registered or rolled back. Use live_count() when you mean “how many handles can call_export resolve right now”; use instances_len() when you mean “how close are we to the admission cap”.

pub fn cached_module_count(&self) -> usize

Number of compiled modules retained in the per-executor cache. Exposed for tests and operators that want to confirm cache reuse.

pub fn module_cache_len(&self) -> usize

Current number of entries held by the bounded LRU module cache. Alias for Self::cached_module_count under the name used by the exec S-5 admission-control bound work; both delegate to the same underlying length so callers can pick whichever reads better at the call site.

pub fn tenant_instance_count(&self, tenant: TenantId) -> usize

Current per-tenant admission count for tenant, or 0 if the tenant holds no live slots. This is the counter the per-tenant fairness cap (crate::engine::EngineConfig::max_instances_per_tenant) is enforced against in spawn_instance. Exposed for tests and operators that want to confirm a tenant’s footprint; the count only moves when the per-tenant cap is configured (it is otherwise left at 0 and the map stays empty).

pub fn instances_len(&self) -> usize

Current admission count, sampled atomically. This is the counter the admission check in spawn_instance consults to decide whether a new instance fits under crate::engine::EngineConfig::max_instances.

This is NOT the registry size — see Self::live_count for the distinction. This counter includes instances that have been admitted but are not yet (or are no longer) in the registry: an in-flight spawn_instance between the admission bump and the registry insert, and pool-held warm instances that were detached from the registry but still occupy a slot. instances_len() >= live_count() always holds.

pub async fn spawn_instance( &self, cfg: SpawnConfig, wasm: &[u8], ) -> Result<InstanceId, ExecError>

Compile + instantiate a Wasm module. Returns the assigned InstanceId.

Deadline / ticker contract

If a SpawnConfig::deadline is set — or if the implicit MAX_START_FN_DURATION cap would otherwise apply (which it always does, since every spawn runs Instance::new_async) — the engine’s epoch ticker MUST be running. Without it the per-store epoch counter never advances, so neither the per-call deadline nor the start-function cap can fire, and a runaway guest would wedge the worker thread until it returned of its own accord. We refuse the spawn with ExecError::EpochTickerNotRunning instead of silently dropping the deadline contract; operators must call engine.spawn_epoch_ticker() (typically inside a Tokio runtime at startup) before serving traffic. The engine constructor auto-spawns the ticker when invoked from inside a runtime, so this only trips for sync-startup setups that forget the explicit call.

pub async fn call_export( &self, id: InstanceId, export: &str, ) -> Result<(), ExecError>

👎Deprecated since 0.3.7:

use call_export_with_args with an empty &[] for the same semantics; v0.4 removes this shim. See docs/MIGRATING-FROM-WASMTIME-WASMER.md § “Typed exports”.

Invoke export with no arguments and no return value.

This is the minimal signature needed for the 100-instance integration test. Richer signatures arrive in S17 (HTTP API) and S18 (CLI).

Concurrency note

The per-instance mutex is held across the inner call_async await point. Concurrent calls into the same instance therefore serialise — this matches wasmtime’s Store-is-not-Sync contract. Concurrent calls into different instances run in parallel. If you need pipelined invocation on a single instance, spawn additional instances over the same module bytes (the executor’s module cache makes that nearly free).

If the executor’s engine has not had spawn_epoch_ticker called and the instance was spawned with a deadline, this call will run until the wasm returns of its own accord — the deadline cannot fire without the ticker. A warning is logged the first time this combination is observed per call.

pub async fn call_export_with_args( &self, id: InstanceId, export: &str, args: &[WasmArg], ) -> Result<Value, ExecError>

Invoke export with the supplied args (which may be empty) and return the export’s result list, serialised as a JSON array.

This is the general entry point for guest-export invocation; the ()-shaped Self::call_export is a thin wrapper that discards the result. The choice of serde_json::Value for the return type keeps the executor’s public API free of any tensor-wasm-api types while still giving the HTTP transport a structured payload to forward verbatim.

When args is empty the implementation uses the typed func.typed::<(), ()>() fast path, matching the historical behaviour and keeping every existing () -> () export call branch-for-branch identical. With a non-empty args slice the dynamic func.call_async(&[Val], &mut [Val]) path runs instead; the result slice is sized from the export’s declared result arity at runtime, so an export returning (i32, i32) produces a JSON array with two numbers.

pub async fn terminate(&self, id: InstanceId) -> Result<(), ExecError>

Drop the instance, releasing its resources.

pub async fn call_export_then_terminate( &self, id: InstanceId, export: &str, ) -> Result<(), ExecError>

👎Deprecated since 0.3.7:

use call_export_with_args_then_terminate with an empty &[] for the same semantics; v0.4 removes this shim. See docs/MIGRATING-FROM-WASMTIME-WASMER.md § “Typed exports”.

Call an export, then unconditionally terminate the instance — even if the returned future is dropped mid-await (api S-20 + orphan-instance cleanup).

The previous flow (call_export + explicit terminate from the caller) leaks the instance into instances when the caller’s future is dropped by an outer cancellation (e.g. tower’s TimeoutLayer firing). The leaked entry holds the wasmtime Store and counts against max_instances, but is unreachable by id (the caller lost the handle). This wrapper installs an AutoTerminateGuard: on the normal exit paths it disarms the guard and calls terminate via the async API; on a Future-drop the guard’s Drop synchronously removes the registry entry (and frees the admission slot) so the leak window closes at the await boundary.

Known limitation: this wrapper cannot stop CPU work that is already running inside Wasmtime’s blocking compile path (Instance::new_async invokes Cranelift, which does not expose a cancellation hook). For now, the cap on crate::engine::EngineConfig::max_module_cache_entries limits the worst case to one compile per unique module. Per-store epoch cancellation interrupts wasm execution at the next epoch tick, which is what closes the actual run-time window.

pub async fn invoke( &self, cfg: SpawnConfig, wasm: &[u8], export: &str, args: &[WasmArg], ) -> Result<Value, ExecError>

High-level “invoke” entry point: spawn (or draw from the warm pool), call the export, return the result, and clean up the instance — all in a single async call.

When an InstancePool is attached via Self::with_instance_pool, this routes through InstancePool::acquire / InstancePool::release so the per-(tenant, module-hash) warm channel absorbs the compile+instantiate cost on the hot path (T37). Without a pool attached, behaviour is identical to Self::spawn_instance + Self::call_export_with_args_then_terminate — every existing caller of that pair sees no semantic change.

Mirrors the routes layer’s invoke / invoke-stream / invoke-async pattern: a single (wasm, cfg, export, args) shot, no caller-side id juggling. T34’s streaming and T36’s back-pressure / deadline-near semantics propagate unchanged — the SpawnConfig::streaming context flows through acquire, and per-call deadline re-arming happens inside call_export_with_args regardless of which path produced the instance.

Pool-side invariant: streaming spawns are NEVER recycled, even when a pool is attached — the streaming context is one-shot (the gateway’s channel receiver is drained for the duration of the SSE response and then dropped). InstancePool::release drops the instance instead of returning it to the warm channel when SpawnConfig::streaming.is_some().

pub async fn call_export_with_args_then_terminate( &self, id: InstanceId, export: &str, args: &[WasmArg], ) -> Result<Value, ExecError>

Argument-aware sibling of Self::call_export_then_terminate.

Identical lifecycle / drop-guard semantics — auto-terminates on success, failure, and Future-drop — but routes through Self::call_export_with_args so callers receive the export’s result list as a JSON array.

Trait Implementations

impl Clone for TensorWasmExecutor

fn clone(&self) -> TensorWasmExecutor

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.