Trait ModelBackend 

pub trait ModelBackend:
    Send
    + Sync
    + 'static {
    type Continuation: Clone + Send + Sync + 'static;
    type Session: SessionBackend;

    // Required methods
    fn model_id(&self) -> &ModelId;
    fn generate<'a>(
        &'a self,
        request: &'a GenerateRequest<Self::Continuation>,
    ) -> impl Future<Output = Result<GenerateResponse<Self::Continuation>, KitError>> + Send + 'a;
    fn generate_stream<'a>(
        &'a self,
        request: &'a GenerateRequest<Self::Continuation>,
    ) -> impl Future<Output = Result<GenerateEventStream<Self::Continuation>, KitError>> + Send + 'a;

    // Provided methods
    fn embed<'a>(
        &'a self,
        _request: &'a EmbedRequest,
    ) -> impl Future<Output = Result<EmbedResponse, KitError>> + Send + 'a { ... }
    fn submit_batch<'a>(
        &'a self,
        _request: &'a BatchSubmitRequest<Self::Continuation>,
    ) -> impl Future<Output = Result<BatchJob, KitError>> + Send + 'a { ... }
    fn open_session(
        &self,
        _config: SessionConfig,
    ) -> impl Future<Output = Result<Self::Session, KitError>> + Send + '_ { ... }
}

Public implementation contract for one exact selected model.

One model owns an exact ModelId, an associated continuation type, and an associated Session type. Requests do not repeat model selection. Caller models remain concrete; supplied Model clones share their Provider lifecycle.

See crate::ProviderBackend for a complete example.

A runnable caller model

This small test double accepts one expected request and returns a scripted response. It demonstrates portable consumption without credentials or a service; it does not model a provider or establish native compatibility. Unrelated operations retain their Unsupported defaults. Infallible is the associated Session because this model cannot construct one.

use std::convert::Infallible;
use futures_util::{stream, StreamExt};
use providers_kit::{KitError, ModelBackend, ModelId};
use providers_kit::error::ErrorKind;
use providers_kit::generation::{
    ContentPart, FinishReason, GenerateEvent, GenerateEventStream,
    GenerateRequest, GenerateResponse, Message, Usage,
};

struct ScriptedModel { id: ModelId, expected: GenerateRequest<()> }
impl ModelBackend for ScriptedModel {
    type Continuation = ();
    type Session = Infallible;
    fn model_id(&self) -> &ModelId { &self.id }
    async fn generate(&self, request: &GenerateRequest<()>) -> Result<GenerateResponse<()>, KitError> {
        assert_eq!(request, &self.expected, "unexpected request to the test double");
        Ok(GenerateResponse {
            model: self.id.clone(),
            message: Message::assistant_text("hello")?.with_continuation(())?,
            finish_reason: Some(FinishReason::Stop),
            citations: Vec::new(), artifacts: Vec::new(), hosted_tool_outputs: Vec::new(),
            usage: Usage::default(),
        })
    }
    async fn generate_stream(&self, request: &GenerateRequest<()>) -> Result<GenerateEventStream<()>, KitError> {
        let response = self.generate(request).await?;
        Ok(GenerateEventStream::new(stream::iter([
            Ok(GenerateEvent::Started { model: self.id.clone() }),
            Ok(GenerateEvent::ContentDelta { delta: "hello".into() }),
            Ok(GenerateEvent::Completed(Box::new(response))),
        ])))
    }
}
let request = GenerateRequest::new(vec![Message::user_text("Say hello")?])?;
let model = ScriptedModel { id: ModelId::new("scripted")?, expected: request.clone() };
let response = model.generate(&request).await?;
assert_eq!(&response.model, model.model_id());
assert!(matches!(response.message.content(), [ContentPart::Text { text }] if text == "hello"));
assert_eq!(response.message.continuation(), Some(&()));
let mut events = model.generate_stream(&request).await?;
assert!(
    matches!(events.next().await.transpose()?, Some(GenerateEvent::Started { model: id }) if &id == model.model_id())
);
assert!(
    matches!(events.next().await.transpose()?, Some(GenerateEvent::ContentDelta { delta }) if delta == "hello")
);
let Some(GenerateEvent::Completed(completed)) = events.next().await.transpose()? else {
    panic!("the scripted turn must complete");
};
assert_eq!(*completed, response);
assert!(events.next().await.is_none());
assert_eq!(
    model
        .open_session(Default::default())
        .await
        .unwrap_err()
        .kind(),
    ErrorKind::Unsupported
);

For the full Provider/Model/Session lifecycle and an opaque continuation, see crate::ProviderBackend.

Required Associated Types

type Continuation: Clone + Send + Sync + 'static

Opaque model-authored state retained between generation turns.

type Session: SessionBackend

Stateful child returned by this model.

Required Methods

fn model_id(&self) -> &ModelId

Exact selected model id.

See crate::ProviderBackend for a complete example.

fn generate<'a>( &'a self, request: &'a GenerateRequest<Self::Continuation>, ) -> impl Future<Output = Result<GenerateResponse<Self::Continuation>, KitError>> + Send + 'a

Runs non-streaming generation.

Append the returned assistant message unchanged when continuing with tool results. Its exact associated continuation is model-owned and remains in that message.

Errors

Returns KitError for closed admission, unsupported selected-model behavior, invalid request combinations, transport/deadline failures, service rejection, or malformed response. The operation is authoritative; no capability matrix or fallback model supplies success.

See crate::ModelBackend for the complete executable workflow and its prerequisites.

fn generate_stream<'a>( &'a self, request: &'a GenerateRequest<Self::Continuation>, ) -> impl Future<Output = Result<GenerateEventStream<Self::Continuation>, KitError>> + Send + 'a

Opens a normalized generation stream.

The returned stream is demand-driven and bounded, with no forwarding task. Completion, error, or drop releases admission; no event follows Completed. The final response is authoritative; partial deltas do not replace it.

Errors

Returns KitError for closed admission, unsupported selected-model behavior, invalid request combinations, transport/deadline failures, service rejection, or malformed response. The operation is authoritative; no capability matrix or fallback model supplies success.

See crate::ModelBackend for the complete executable workflow and its prerequisites.

Provided Methods

fn embed<'a>( &'a self, _request: &'a EmbedRequest, ) -> impl Future<Output = Result<EmbedResponse, KitError>> + Send + 'a

Runs embeddings when supported.

Errors

Returns KitError for closed admission, unsupported selected-model behavior, invalid request combinations, transport/deadline failures, service rejection, or malformed response. The operation is authoritative; no capability matrix or fallback model supplies success.

See crate::embedding for the complete executable workflow and its prerequisites.

fn submit_batch<'a>( &'a self, _request: &'a BatchSubmitRequest<Self::Continuation>, ) -> impl Future<Output = Result<BatchJob, KitError>> + Send + 'a

Submits a model-bound batch when supported.

Retain the exact returned job id to observe or cancel the remote job. Shared provider shutdown closes local resources; it does not cancel remote jobs.

Errors

Returns KitError for closed admission, unsupported selected-model behavior, invalid request combinations, transport/deadline failures, service rejection, or malformed response. The operation is authoritative; no capability matrix or fallback model supplies success.

This service example creates one inline batch, retains its exact id, and requests cancellation before observing terminal settlement. It requires a configured OpenAI account and an exact model supporting batch submission. Completed may win the race; an unresolved id remains caller-owned.

Examples

use std::time::Duration;
use providers_kit::{KitError, ModelBackend, ModelId, Provider, ProviderBackend, ProviderConfig};
use providers_kit::batch::{BatchInput, BatchStatus, BatchSubmitRequest};
use providers_kit::config::{ApiKey, OpenAiConfig, ProviderKind, TransportConfig};
use providers_kit::error::{ErrorKind, ErrorPhase, Operation};
use providers_kit::generation::{GenerateRequest, Message};
let provider = Provider::from_config(ProviderConfig::new(
    ProviderKind::OpenAi(OpenAiConfig::new(ApiKey::new("replace-with-your-key")?)),
    TransportConfig::default(),
))?;
let observed = async {
    let model = provider.model(ModelId::new("your-exact-model-id")?)?;
    let request = BatchSubmitRequest::Inline {
        rows: vec![BatchInput::new("row-1", GenerateRequest::new(vec![
            Message::user_text("Summarize the task")?,
        ])?)?],
    };
    let job = model.submit_batch(&request).await?;
    let owned_id = job.id;
    // Persist this id in the application's inventory before waiting. This recipe
    // prints it; errors below retain it for a later reconciliation attempt.
    println!("Owned batch to retain: {owned_id}");
    let settlement = tokio::time::timeout(Duration::from_secs(11 * 60), async {
        let acknowledgement = provider.cancel_batch(owned_id.clone()).await?;
        assert_eq!(acknowledgement.id, owned_id);
        loop {
            let job = provider.batch_status(owned_id.clone()).await?;
            assert_eq!(job.id, owned_id);
            match job.status {
                BatchStatus::Queued | BatchStatus::Running => {
                    tokio::time::sleep(Duration::from_secs(5)).await;
                }
                terminal => break Ok::<_, KitError>(terminal),
            }
        }
    }).await;
    match settlement {
        Ok(Ok(terminal)) => {
            // Completed can win the race; this does not prove cancellation.
            println!("{owned_id}: {terminal:?}");
            Ok(())
        }
        Ok(Err(error)) => Err(error.with_correlation_id(owned_id)),
        Err(_) => Err(KitError::new(
            ErrorKind::Timeout, Operation::BatchStatus, ErrorPhase::Cleanup,
        ).with_correlation_id(owned_id)),
    }
}.await;
let cleanup = provider.shutdown().await;
// Keep unresolved ids for reconciliation; shutdown does not delete remote files.
// The batch module explains the separate artifact-ownership limits.
match (observed, cleanup) {
    (Err(primary), Err(cleanup)) => Err(primary.with_supplementary(cleanup)),
    (Err(error), Ok(())) | (Ok(()), Err(error)) => Err(error),
    (Ok(()), Ok(())) => Ok(()),
}

fn open_session( &self, _config: SessionConfig, ) -> impl Future<Output = Result<Self::Session, KitError>> + Send + '_

Opens a stateful model session.

Supplied Sessions support OpenAI only; Gemini, Anthropic and Ollama return Unsupported. Caller Models retain their own associated Session semantics. See the complete Session progression and consuming cleanup.

Errors

Returns KitError for closed admission, unsupported selected-model behavior, invalid request combinations, transport/deadline failures, service rejection, or malformed response. The operation is authoritative; no capability matrix or fallback model supplies success.

See crate::session for the complete executable workflow and its prerequisites.

Dyn Compatibility

This trait is not dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementors

impl ModelBackend for Model

type Continuation = GenerationContinuation

type Session = Session