pub fn run_chain_blocking<T, E>(
worker: &GpuWorker,
model_name: &str,
config: &Config,
hint: Option<ActivationHint>,
with_engine: impl FnOnce(&mut dyn InferenceEngine) -> Result<T, E>,
) -> ChainPrep<T, E>Expand description
Run a blocking chain operation on a specific GPU worker.
Acquires worker.model_load_lock for the full duration, binds the current
thread to worker.gpu.ordinal (so reclaim_gpu_memory debug asserts are
satisfied), ensures the model is loaded on GPU, takes the engine out of
the worker’s cache, passes it to with_engine, and restores the engine
unconditionally on both success and closure failure.
Safe to call from inside tokio::task::spawn_blocking. The calling thread
can be any thread — the ThreadGpuGuard clears the thread-local on return.
§Errors
Returns Err(anyhow::Error) from the outer Result if:
ensure_model_ready_syncfails (bad config, disk IO, load error).- The engine vanishes from the cache between ensure and take (cache race).
Returns Ok(Err(E)) if the closure itself returned an error — caller
preserves the closure’s typed error for precise HTTP status mapping.