Crate providers_kit 

Portable provider resources with exact model selection.

The hierarchy is Provider → Model → Session. Import ProviderBackend, ModelBackend, and SessionBackend to call behavior on supplied handles. Caller implementations use these traits directly with their own associated children; no normalization constructor is required.

config owns credentials and transport settings; models owns exact model identity and discovery. Requests and returned values live in generation, embedding, files, batch, hosted, and session. Inspect typed failures through error and health results through runtime.

Provider and Model clones share shutdown. Sessions are unique and close by consumption. Keep the runtime driven until accepted cleanup settles.

Local construction

Creating a provider and selecting an exact model performs no service request:

use providers_kit::{Provider, ProviderBackend, ProviderConfig, ModelBackend, ModelId};
use providers_kit::config::{ProviderKind, OllamaConfig, TransportConfig};
let provider = Provider::from_config(ProviderConfig::new(
    ProviderKind::Ollama(OllamaConfig::local()), TransportConfig::default(),
))?;
let model = provider.model(ModelId::new("exact-local-model-id")?)?;
assert_eq!(model.model_id().as_str(), "exact-local-model-id");
provider.shutdown().await?;

Generation

This example requires a running Ollama service with the exact model installed. Selection does not fall back or prove availability; the operation is authoritative.

use providers_kit::{Provider, ProviderBackend, ProviderConfig, ModelBackend, ModelId};
use providers_kit::config::{ProviderKind, OllamaConfig, TransportConfig};
use providers_kit::generation::{GenerateRequest, Message};
let provider = Provider::from_config(ProviderConfig::new(
    ProviderKind::Ollama(OllamaConfig::local()), TransportConfig::default(),
))?;
let result = async {
    let model = provider.model(ModelId::new("exact-local-model-id")?)?;
    let request = GenerateRequest::new(vec![Message::user_text("Hello")?])?;
    model.generate(&request).await
}.await;
let cleanup = provider.shutdown().await;
let response = result?;
cleanup?;
// Preserve response.message unchanged for subsequent function continuation.
let _assistant_message = response.message;

Choose a workflow

Audience	Start here
Application builders	Generation and exact assistant replay, embeddings, owned files, batch lifecycle, and interactive Sessions
Custom-backend implementers	The complete local hierarchy on ProviderBackend, then the sessionless ModelBackend companion

The supplied Session dispatcher supports OpenAI only. Supplied Gemini, Anthropic and Ollama Models return Unsupported from open_session. Stateless OpenAI audio input/audio-chat models are also unsupported; this does not remove OpenAI Realtime Session support. These supplied-backend restrictions do not constrain an independently implemented SessionBackend.

The examples demonstrate API use, not current account/model availability or native qualification. Model ids are exact, and the selected operation decides whether a requested combination is supported.

Re-exports

pub use crate::config::ProviderConfig;

pub use crate::error::KitError;

pub use crate::models::Model;

pub use crate::models::ModelBackend;

pub use crate::models::ModelId;

pub use crate::session::Session;

pub use crate::session::SessionBackend;

Modules

batch

Model-bound batch submission and provider-owned lifecycle values.

config

Provider credentials, endpoints, and shared transport policy.

embedding

Portable embedding requests and results.

error

Structured failures and explicitly accessed, potentially sensitive native diagnostics.

files

Provider-owned file upload, metadata, pagination, and bounded downloads.

generation

Portable generation, messages, function continuation, and streaming values.

hosted

Closed provider-hosted request vocabulary and normalized hosted output.

models

Exact model identity, discovery values, and the opaque model handle.

runtime

Provider health observations.

session

Provider-native interactive values and the unique session handle.

Structs

Provider

Cloneable opaque provider handle.

Traits

ProviderBackend

Public implementation contract for one provider root.