Trait LoadedModel 

pub trait LoadedModel: Send + Sync {
    // Required method
    fn as_any(&self) -> &dyn Any;

    // Provided methods
    fn as_chat(&self) -> Option<&dyn ChatModel> { ... }
    fn as_stream(&self) -> Option<&dyn StreamingModel> { ... }
}

A model whose weights are in memory. Engines downcast it back to their own type through as_any.

Required Methods

fn as_any(&self) -> &dyn Any

Provided Methods

fn as_chat(&self) -> Option<&dyn ChatModel>

The chat interface, for loaded LLMs.

fn as_stream(&self) -> Option<&dyn StreamingModel>

The streaming interface, for loaded speech models.

Dyn Compatibility

This trait is dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementors

impl LoadedModel for LoadedLlm

impl LoadedModel for LoadedStream