pub struct Model { /* private fields */ }Expand description
Cloneable opaque handle for one exact selected model.
One model owns an exact ModelId, an associated continuation type, and an
associated Session type. Requests do not repeat model selection. Caller models
remain concrete; supplied Model clones share their Provider lifecycle.
§Examples
use providers_kit::{Provider, ProviderBackend, ProviderConfig, ModelBackend, ModelId, config::{ProviderKind, OllamaConfig, TransportConfig}};
let provider = Provider::from_config(ProviderConfig::new(ProviderKind::Ollama(OllamaConfig::local()), TransportConfig::default()))?;
let model = provider.model(ModelId::new("exact-model")?)?;
assert_eq!(model.model_id().as_str(), "exact-model");
provider.shutdown().await?;
Trait Implementations§
Source§impl ModelBackend for Model
impl ModelBackend for Model
Source§type Continuation = GenerationContinuation
type Continuation = GenerationContinuation
Opaque model-authored state retained between generation turns.
Source§async fn generate(
&self,
request: &GenerateRequest<GenerationContinuation>,
) -> Result<GenerateResponse<GenerationContinuation>, KitError>
async fn generate( &self, request: &GenerateRequest<GenerationContinuation>, ) -> Result<GenerateResponse<GenerationContinuation>, KitError>
Runs non-streaming generation. Read more
Source§async fn generate_stream(
&self,
request: &GenerateRequest<GenerationContinuation>,
) -> Result<GenerateEventStream<GenerationContinuation>, KitError>
async fn generate_stream( &self, request: &GenerateRequest<GenerationContinuation>, ) -> Result<GenerateEventStream<GenerationContinuation>, KitError>
Opens a normalized generation stream. Read more
Source§async fn embed(&self, request: &EmbedRequest) -> Result<EmbedResponse, KitError>
async fn embed(&self, request: &EmbedRequest) -> Result<EmbedResponse, KitError>
Runs embeddings when supported. Read more
Source§async fn submit_batch(
&self,
request: &BatchSubmitRequest<GenerationContinuation>,
) -> Result<BatchJob, KitError>
async fn submit_batch( &self, request: &BatchSubmitRequest<GenerationContinuation>, ) -> Result<BatchJob, KitError>
Submits a model-bound batch when supported. Read more
Source§async fn open_session(&self, config: SessionConfig) -> Result<Session, KitError>
async fn open_session(&self, config: SessionConfig) -> Result<Session, KitError>
Opens a stateful model session. Read more
Auto Trait Implementations§
impl !RefUnwindSafe for Model
impl !UnwindSafe for Model
impl Freeze for Model
impl Send for Model
impl Sync for Model
impl Unpin for Model
impl UnsafeUnpin for Model
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more