1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
//! Unified Rust runtime for local and remote embedding, reranking, and generation models.
//!
//! Uni-Xervo provides a single, provider-agnostic API for loading and running ML models
//! across a wide range of backends — from local inference engines (Candle, FastEmbed,
//! mistral.rs) to remote API services (OpenAI, Gemini, Anthropic, Cohere, Mistral,
//! Voyage AI, Vertex AI, Azure OpenAI).
//!
//! # Key concepts
//!
//! - **[`ModelRuntime`](runtime::ModelRuntime)** — the central runtime that owns providers
//! and manages a catalog of model aliases.
//! - **[`ModelAliasSpec`](api::ModelAliasSpec)** — a declarative specification that maps a
//! human-readable alias (e.g. `"embed/default"`) to a concrete provider + model pair.
//! - **Providers** — pluggable backends that implement [`ModelProvider`](traits::ModelProvider).
//! Each provider advertises the tasks it supports and knows how to load models.
//! - **Traits** — [`EmbeddingModel`](traits::EmbeddingModel),
//! [`RerankerModel`](traits::RerankerModel), and
//! [`GeneratorModel`](traits::GeneratorModel) are the task-specific interfaces returned
//! by the runtime.
//!
//! # Quick start
//!
//! ```rust,no_run
//! use uni_xervo::api::{ModelAliasSpec, ModelTask};
//! use uni_xervo::runtime::ModelRuntime;
//! # #[cfg(feature = "provider-candle")]
//! use uni_xervo::provider::candle::LocalCandleProvider;
//!
//! # #[cfg(feature = "provider-candle")]
//! # async fn example() -> Result<(), Box<dyn std::error::Error>> {
//! let spec = ModelAliasSpec {
//! alias: "embed/local".into(),
//! task: ModelTask::Embed,
//! provider_id: "local/candle".into(),
//! model_id: "sentence-transformers/all-MiniLM-L6-v2".into(),
//! revision: None,
//! warmup: Default::default(),
//! required: true,
//! timeout: None,
//! load_timeout: None,
//! retry: None,
//! options: serde_json::Value::Null,
//! };
//!
//! let runtime = ModelRuntime::builder()
//! .register_provider(LocalCandleProvider::new())
//! .catalog(vec![spec])
//! .build()
//! .await?;
//!
//! let model = runtime.embedding("embed/local").await?;
//! let embeddings = model.embed(vec!["Hello, world!"]).await?;
//! # Ok(())
//! # }
//! ```