Trait ModelBackend 

pub trait ModelBackend:
    Send
    + Sync
    + 'static {
    type Continuation: Clone + Send + Sync + 'static;
    type Session: SessionBackend;

    // Required methods
    fn model_id(&self) -> &ModelId;
    fn generate<'a>(
        &'a self,
        request: &'a GenerateRequest<Self::Continuation>,
    ) -> impl Future<Output = Result<GenerateResponse<Self::Continuation>, KitError>> + Send + 'a;
    fn generate_stream<'a>(
        &'a self,
        request: &'a GenerateRequest<Self::Continuation>,
    ) -> impl Future<Output = Result<GenerateEventStream<Self::Continuation>, KitError>> + Send + 'a;

    // Provided methods
    fn embed<'a>(
        &'a self,
        _request: &'a EmbedRequest,
    ) -> impl Future<Output = Result<EmbedResponse, KitError>> + Send + 'a { ... }
    fn submit_batch<'a>(
        &'a self,
        _request: &'a BatchSubmitRequest<Self::Continuation>,
    ) -> impl Future<Output = Result<BatchJob, KitError>> + Send + 'a { ... }
    fn open_session(
        &self,
        _config: SessionConfig,
    ) -> impl Future<Output = Result<Self::Session, KitError>> + Send + '_ { ... }
}

Public implementation contract for one exact selected model.

One model owns an exact ModelId, an associated continuation type, and an associated Session type. Requests do not repeat model selection. Caller models remain concrete; supplied Model clones share their Provider lifecycle.

Examples

use providers_kit::{Provider, ProviderBackend, ProviderConfig, ModelBackend, ModelId, config::{ProviderKind, OllamaConfig, TransportConfig}};
let provider = Provider::from_config(ProviderConfig::new(ProviderKind::Ollama(OllamaConfig::local()), TransportConfig::default()))?;
let model = provider.model(ModelId::new("exact-model")?)?;
assert_eq!(model.model_id().as_str(), "exact-model");
provider.shutdown().await?;

A sessionless caller model

Use Infallible when no Session can be constructed; its SessionBackend implementation is supplied by this crate. Keep the default open_session Unsupported result instead of fabricating a session. This deliberately limited caller model rejects generation as well; it demonstrates the associated-type choice, not an implemented model capability. See crate::ProviderBackend for an executable text-generating caller hierarchy.

use std::convert::Infallible;
use providers_kit::{ModelBackend, ModelId, KitError};
use providers_kit::generation::{GenerateEventStream, GenerateRequest, GenerateResponse};
use providers_kit::error::{ErrorKind, ErrorPhase, Operation};
struct Sessionless { id: ModelId }
impl ModelBackend for Sessionless {
    type Continuation = ();
    type Session = Infallible;
    fn model_id(&self) -> &ModelId { &self.id }
    async fn generate(&self, _: &GenerateRequest<()>) -> Result<GenerateResponse<()>, KitError> {
        Err(KitError::new(ErrorKind::Unsupported, Operation::Generate, ErrorPhase::InputValidation))
    }
    async fn generate_stream(&self, _: &GenerateRequest<()>) -> Result<GenerateEventStream<()>, KitError> {
        Err(KitError::new(ErrorKind::Unsupported, Operation::GenerateStream, ErrorPhase::InputValidation))
    }
}
let model = Sessionless { id: ModelId::new("sessionless")? };
let error = model.open_session(Default::default()).await.unwrap_err();
assert_eq!(error.kind(), ErrorKind::Unsupported);

Required Associated Types

type Continuation: Clone + Send + Sync + 'static

Opaque model-authored state retained between generation turns.

type Session: SessionBackend

Stateful child returned by this model.

Required Methods

fn model_id(&self) -> &ModelId

Exact selected model id.

Examples

use providers_kit::{Provider, ProviderBackend, ProviderConfig, ModelBackend, ModelId, config::{ProviderKind, OllamaConfig, TransportConfig}};
let provider = Provider::from_config(ProviderConfig::new(ProviderKind::Ollama(OllamaConfig::local()), TransportConfig::default()))?;
let model = provider.model(ModelId::new("exact-model")?)?;
assert_eq!(model.model_id().as_str(), "exact-model");
provider.shutdown().await?;

fn generate<'a>( &'a self, request: &'a GenerateRequest<Self::Continuation>, ) -> impl Future<Output = Result<GenerateResponse<Self::Continuation>, KitError>> + Send + 'a

Runs non-streaming generation.

This example requires a real service, valid credentials, and an exact model supporting this operation. Append the returned assistant message unchanged when continuing with tool results. Its exact associated continuation is model-owned and remains in that message.

Errors

Returns KitError for closed admission, unsupported selected-model behavior, invalid request combinations, transport/deadline failures, service rejection, or malformed response. The operation is authoritative; no capability matrix or fallback model supplies success.

Examples

use providers_kit::{Provider, ProviderBackend, ProviderConfig, config::{ApiKey, OpenAiConfig, ProviderKind, TransportConfig}};
let provider = Provider::from_config(ProviderConfig::new(ProviderKind::OpenAi(OpenAiConfig::new(ApiKey::new("replace-with-your-key")?)), TransportConfig::default()))?;
use providers_kit::{ModelBackend, ModelId, generation::{GenerateRequest, Message}};
let model = provider.model(ModelId::new("your-exact-model-id")?)?;
let request = GenerateRequest::new(vec![Message::user_text("Summarize the task")?])?;
let result = model.generate(&request).await;
let cleanup = provider.shutdown().await;
let _response = result?;
cleanup?;

fn generate_stream<'a>( &'a self, request: &'a GenerateRequest<Self::Continuation>, ) -> impl Future<Output = Result<GenerateEventStream<Self::Continuation>, KitError>> + Send + 'a

Opens a normalized generation stream.

This example requires a real service, valid credentials, and an exact model supporting this operation. The returned stream is demand-driven and bounded, with no forwarding task. Completion, error, or drop releases admission; no event follows Completed. The final response is authoritative; partial deltas do not replace it.

Errors

Returns KitError for closed admission, unsupported selected-model behavior, invalid request combinations, transport/deadline failures, service rejection, or malformed response. The operation is authoritative; no capability matrix or fallback model supplies success.

Examples

use providers_kit::{Provider, ProviderBackend, ProviderConfig, config::{ApiKey, OpenAiConfig, ProviderKind, TransportConfig}};
let provider = Provider::from_config(ProviderConfig::new(ProviderKind::OpenAi(OpenAiConfig::new(ApiKey::new("replace-with-your-key")?)), TransportConfig::default()))?;
use providers_kit::{ModelBackend, ModelId, generation::{GenerateRequest, Message}};
let model = provider.model(ModelId::new("your-exact-model-id")?)?;
let request = GenerateRequest::new(vec![Message::user_text("Summarize the task")?])?;
let result = model.generate_stream(&request).await;
let observed = async {
    use futures_util::StreamExt;
    let mut stream = result?;
    while let Some(event) = stream.next().await { let _event = event?; }
    Ok::<(), providers_kit::KitError>(())
}.await;
let cleanup = provider.shutdown().await;
observed?;
cleanup?;

Provided Methods

fn embed<'a>( &'a self, _request: &'a EmbedRequest, ) -> impl Future<Output = Result<EmbedResponse, KitError>> + Send + 'a

Runs embeddings when supported.

This example requires a real service, valid credentials, and an exact model supporting this operation.

Errors

Returns KitError for closed admission, unsupported selected-model behavior, invalid request combinations, transport/deadline failures, service rejection, or malformed response. The operation is authoritative; no capability matrix or fallback model supplies success.

Examples

use providers_kit::{Provider, ProviderBackend, ProviderConfig, config::{ApiKey, OpenAiConfig, ProviderKind, TransportConfig}};
let provider = Provider::from_config(ProviderConfig::new(ProviderKind::OpenAi(OpenAiConfig::new(ApiKey::new("replace-with-your-key")?)), TransportConfig::default()))?;
use providers_kit::{ModelBackend, ModelId, generation::{GenerateRequest, Message}};
let model = provider.model(ModelId::new("your-exact-model-id")?)?;
let result = model.embed(&providers_kit::embedding::EmbedRequest::new(vec!["document".into()])?).await;
let cleanup = provider.shutdown().await;
let _response = result?;
cleanup?;

fn submit_batch<'a>( &'a self, _request: &'a BatchSubmitRequest<Self::Continuation>, ) -> impl Future<Output = Result<BatchJob, KitError>> + Send + 'a

Submits a model-bound batch when supported.

Retain the exact returned job id to observe or cancel the remote job. Shared provider shutdown closes local resources; it does not cancel remote jobs.

This example requires a real service, valid credentials, and an exact model supporting this operation.

Errors

Returns KitError for closed admission, unsupported selected-model behavior, invalid request combinations, transport/deadline failures, service rejection, or malformed response. The operation is authoritative; no capability matrix or fallback model supplies success.

Examples

use providers_kit::{Provider, ProviderBackend, ProviderConfig, config::{ApiKey, OpenAiConfig, ProviderKind, TransportConfig}};
let provider = Provider::from_config(ProviderConfig::new(ProviderKind::OpenAi(OpenAiConfig::new(ApiKey::new("replace-with-your-key")?)), TransportConfig::default()))?;
use providers_kit::{ModelBackend, ModelId, generation::{GenerateRequest, Message}};
let model = provider.model(ModelId::new("your-exact-model-id")?)?;
let request = GenerateRequest::new(vec![Message::user_text("Summarize the task")?])?;
let result = model.submit_batch(&providers_kit::batch::BatchSubmitRequest::Inline { rows: vec![providers_kit::batch::BatchInput::new("row-1", request)?] }).await;
use providers_kit::batch::BatchStatus;
use providers_kit::error::{ErrorKind, ErrorPhase, Operation};
use std::time::Duration;
let observed = match result {
    Ok(job) => {
        let owned_id = job.id; // Persist this id in the application's job inventory.
        let settlement = tokio::time::timeout(Duration::from_secs(11 * 60), async {
            let acknowledgement = provider.cancel_batch(owned_id.clone()).await?;
            assert_eq!(acknowledgement.id, owned_id);
            loop {
                let status = provider.batch_status(owned_id.clone()).await?;
                assert_eq!(status.id, owned_id);
                match status.status {
                    BatchStatus::Queued | BatchStatus::Running =>
                        tokio::time::sleep(Duration::from_secs(5)).await,
                    terminal => break Ok::<_, providers_kit::KitError>(terminal),
                }
            }
        }).await;
        match settlement {
            Ok(Ok(terminal)) => {
                // Completed may win the race: this does not prove cancellation.
                println!("{owned_id}: {terminal:?}");
                Ok(())
            }
            Ok(Err(error)) => Err(error.with_correlation_id(owned_id)),
            Err(_) => Err(providers_kit::KitError::new(ErrorKind::Timeout,
                Operation::BatchStatus, ErrorPhase::Cleanup).with_correlation_id(owned_id)),
        }
    }
    Err(error) => Err(error),
};
let cleanup = provider.shutdown().await;
// Keep an unresolved id/error for later reconciliation. Shutdown does not
// delete remote files; see the batch module's artifact-ownership limits.
match (observed, cleanup) {
    (Err(primary), Err(cleanup)) => return Err(primary.with_supplementary(cleanup)),
    (Err(error), Ok(())) | (Ok(()), Err(error)) => return Err(error),
    (Ok(()), Ok(())) => {}
}

fn open_session( &self, _config: SessionConfig, ) -> impl Future<Output = Result<Self::Session, KitError>> + Send + '_

Opens a stateful model session.

Supplied Sessions support OpenAI only; Gemini, Anthropic and Ollama return Unsupported. Caller Models retain their own associated Session semantics. See the complete Session progression and consuming cleanup.

This example requires a real service, valid credentials, and an exact model supporting this operation.

Errors

Returns KitError for closed admission, unsupported selected-model behavior, invalid request combinations, transport/deadline failures, service rejection, or malformed response. The operation is authoritative; no capability matrix or fallback model supplies success.

Examples

use providers_kit::{Provider, ProviderBackend, ProviderConfig, config::{ApiKey, OpenAiConfig, ProviderKind, TransportConfig}};
let provider = Provider::from_config(ProviderConfig::new(ProviderKind::OpenAi(OpenAiConfig::new(ApiKey::new("replace-with-your-key")?)), TransportConfig::default()))?;
use providers_kit::{ModelBackend, ModelId, generation::{GenerateRequest, Message}};
let model = provider.model(ModelId::new("your-exact-model-id")?)?;
let result = model.open_session(providers_kit::session::SessionConfig::default()).await;
use providers_kit::SessionBackend;
let observed = match result { Ok(session) => session.close().await, Err(error) => Err(error) };
let cleanup = provider.shutdown().await;
observed?;
cleanup?;

Dyn Compatibility

This trait is not dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementors

impl ModelBackend for Model

type Continuation = GenerationContinuation

type Session = Session