Function with_cached_provider 

pub fn with_cached_provider<R>(
    cfg: &Embedding,
    idx_dim: usize,
    f: impl FnOnce(&dyn EmbeddingProvider) -> R,
) -> R

Run f with the configured provider, reusing the previous instance when (provider, model, dimension) is unchanged.

A long-lived process used to rebuild the provider on every call, and for the candle provider that means re-reading the tokenizer and re-faulting the model weights: measured at 127-137 ms of a recall on the eval index, paid again for every query. The MCP server is the case that matters — one process, many queries — and it was doing exactly that.

The cache is keyed by the configuration, so a different provider, model or dimension rebuilds it. A construction that falls back to the mock is cached like any other value: provider_from_config reports the fallback on stderr and returns a working provider either way, so caching it cannot hide an error. A panic inside f must not poison the cache for the next call, so the lock recovers rather than propagating.