Struct ModelServer 

pub struct ModelServer { /* private fields */ }

Model serving infrastructure

Implementations

impl ModelServer

pub fn new(registry: Arc<ModelRegistry>) -> Result<Self>

Create a new model server backed by an on-disk artifact repository rooted at registry’s storage path.

pub async fn load_model(&self, version_id: Uuid) -> Result<()>

Load a model version into memory from the on-disk artifact repository.

Looks up the artifact previously saved under the key version_id.to_string() and caches the reconstructed model for subsequent get_model calls. Returns an error if no artifact has been saved for this version yet.

pub async fn warm_up_model( &self, version_id: Uuid, samples: Vec<String>, ) -> Result<()>

Warm up model with sample inputs.

If the model is already loaded, actually runs encode over the samples once so its internal caches are genuinely primed; otherwise the samples are recorded for a future load.

pub async fn get_model( &self, version_id: Uuid, ) -> Result<Arc<Box<dyn EmbeddingModel>>>

Get a previously loaded model for inference. Returns an error if load_model has not been called successfully for this version.

pub async fn route_request(&self, test_id: Uuid) -> Result<Uuid>

Route request based on A/B test