Expand description
Async wrapper around the synchronous InferenceEngine.
Uses tokio::task::spawn_blocking for CPU-bound inference work,
ensuring the Tokio runtime is not blocked. Bounded concurrency is
enforced via a Semaphore to prevent resource exhaustion.
This module is not available on WASM targets (wasm32) because tokio’s
full feature set (including threads and network I/O) is not supported there.
Structs§
- Async
Inference Engine - Async inference engine with bounded concurrency.