Module lazy 

Lazy-loading embedding provider with idle eviction and LRU query cache.

Wraps FastembedProvider behind a state machine that defers model download and ONNX session initialization until the first embed call. After a configurable idle timeout, the ONNX session is dropped to release memory while keeping model weights on disk for fast reload.

use llm_kernel::embedding::{
    EmbeddingModel, LazyFastembedProvider, LazyOpts, EmbeddingProvider,
};

let provider = LazyFastembedProvider::new(
    EmbeddingModel::BGESmallENV15,
    "/path/to/cache".into(),
    LazyOpts::default(),
);
// Constructor returns instantly — no download yet
let result = provider.embed("hello world")?; // triggers lazy load

Structs

EmbeddingCache

LRU cache for query embedding vectors.

LazyFastembedProvider

Lazy-loading embedding provider backed by FastembedProvider.

LazyOpts

Configuration for LazyFastembedProvider.

Enums

ModelState

Lifecycle state of a lazy-loaded embedding model.

Functions

is_model_cached

Check if model weights exist in the HuggingFace cache directory.