Skip to main content

EMBED_RUNTIME_MAX_WORKER_THREADS

Constant EMBED_RUNTIME_MAX_WORKER_THREADS 

Source
pub const EMBED_RUNTIME_MAX_WORKER_THREADS: usize = 8;
Expand description

Upper bound on Tokio worker threads for the shared embedding runtime (GAP-SG-141 B2).

The runtime only drives HTTP polling, so worker threads past this point add scheduling overhead without adding throughput; the concurrency that matters is the request fan-out (--rest-concurrency, --llm-parallelism), not the reactor width.