Struct ModelLoader 

pub struct ModelLoader { /* private fields */ }

Model loader with lazy loading and caching

Thread Safety

ModelLoader uses Arc + RwLock internally for thread-safe access. Multiple threads can safely load and access models concurrently.

Caching

Once a model is loaded, it stays in memory until explicitly unloaded. Subsequent calls to load() with the same model type/variant return the cached session.

Implementations

impl ModelLoader

pub fn new(registry: Arc<ModelRegistry>) -> Self

Create a new model loader

Arguments

• registry - Model registry for metadata and downloads

Example

use llm_shield_models::{ModelLoader, ModelRegistry};
use std::sync::Arc;

let registry = ModelRegistry::from_file("models/registry.json")?;
let loader = ModelLoader::new(Arc::new(registry));

pub fn with_registry(registry: Arc<ModelRegistry>) -> Self

Create a new model loader (alias for new)

pub async fn load( &self, model_type: ModelType, variant: ModelVariant, ) -> Result<Arc<Mutex<Session>>>

Load a model (lazily, with caching)

If the model is already loaded, returns the cached session. Otherwise, loads the model from disk using the registry.

Arguments

• model_type - Type of model to load
• variant - Model variant (precision)

Returns

Arc to ONNX Runtime session

Example

use llm_shield_models::{ModelLoader, ModelRegistry, ModelType, ModelVariant};
use std::sync::Arc;

let session = loader.load(ModelType::PromptInjection, ModelVariant::FP16).await?;

pub async fn load_with_config( &self, config: ModelConfig, ) -> Result<Arc<Mutex<Session>>>

Load a model with custom configuration

Arguments

• config - Model configuration

Returns

Arc to ONNX Runtime session

pub async fn preload( &self, models: Vec<(ModelType, ModelVariant)>, ) -> Result<()>

Preload multiple models

Useful for warming up the cache before first use.

Arguments

• models - List of (ModelType, ModelVariant) tuples to preload

Example

use llm_shield_models::{ModelLoader, ModelRegistry, ModelType, ModelVariant};
use std::sync::Arc;

let models = vec![
    (ModelType::PromptInjection, ModelVariant::FP16),
    (ModelType::Toxicity, ModelVariant::FP16),
];
loader.preload(models).await?;

pub fn is_loaded(&self, model_type: ModelType, variant: ModelVariant) -> bool

Check if a model is loaded

pub fn unload(&self, model_type: ModelType, variant: ModelVariant)

Unload a specific model

Removes the model from cache, freeing memory.

pub fn unload_all(&self)

Unload all models

Clears the entire cache, freeing all memory.

pub fn len(&self) -> usize

Get the number of loaded models

pub fn is_empty(&self) -> bool

Check if no models are loaded

pub fn loaded_models(&self) -> Vec<(ModelType, ModelVariant)>

Get list of loaded models

pub fn model_info( &self, model_type: ModelType, variant: ModelVariant, ) -> Option<String>

Get information about a loaded model

Returns None if model is not loaded.

pub fn stats(&self) -> LoaderStats

Get loader statistics

Trait Implementations

impl Clone for ModelLoader

fn clone(&self) -> Self

Clone creates a new reference to the same underlying cache

All clones share the same loaded models and statistics.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.