Trait InferenceEngine

Source

pub trait InferenceEngine: Send + Sync {
    // Required methods
    fn generate(&mut self, req: &GenerateRequest) -> Result<GenerateResponse>;
    fn model_name(&self) -> &str;
    fn is_loaded(&self) -> bool;
    fn load(&mut self) -> Result<()>;

    // Provided methods
    fn unload(&mut self) { ... }
    fn set_on_progress(&mut self, _callback: ProgressCallback) { ... }
    fn clear_on_progress(&mut self) { ... }
    fn model_paths(&self) -> Option<&ModelPaths> { ... }
    fn as_chain_renderer(&mut self) -> Option<&mut dyn ChainStageRenderer> { ... }
}

Expand description

Trait for inference backends.

Required Methods§

Source

fn load(&mut self) -> Result<()>

Load model weights. Called automatically on first generate if not yet loaded.

Provided Methods§

Source

fn unload(&mut self)

Unload model weights to free GPU memory. The engine remains valid and can be re-loaded by calling load() or generating again.

Source

fn set_on_progress(&mut self, _callback: ProgressCallback)

Set a progress callback for receiving loading/inference status updates. Default implementation is a no-op for engines that don’t support progress.

Source

fn clear_on_progress(&mut self)

Clear any previously installed progress callback.

Source

fn model_paths(&self) -> Option<&ModelPaths>

Return the model’s resolved file paths, if available. Used by the server for pre-load memory checks on unified-memory systems.

Source

fn as_chain_renderer(&mut self) -> Option<&mut dyn ChainStageRenderer>

Returns a [ChainStageRenderer] view of this engine if the family supports chained video generation. Default is None — only LTX-2 distilled overrides this in v1.

Callers (the server chain route) invoke this once per stage to drive crate::ltx2::Ltx2ChainOrchestrator::run; engines that don’t support chaining return None and the caller responds with 422.