Struct ProviderRegistry 

pub struct ProviderRegistry { /* private fields */ }

Registry for managing multiple named LLM providers

The ProviderRegistry holds references to provider configurations and allows creating LLM clients for specific models or providers by name.

Implementations

impl ProviderRegistry

pub fn new() -> ProviderRegistry

Create a new empty provider registry

pub fn from_config( providers: HashMap<String, ProviderConfig>, models: HashMap<String, ModelConfig>, nvidia: Option<&NvidiaConfig>, ) -> ProviderRegistry

Create a provider registry from TOML configuration

pub fn reload_runtime_providers( &self, providers: Vec<RuntimeProviderEntry>, names: Vec<String>, )

Hot-swap the runtime provider map. Called by admin endpoints after mutating the DB so the new providers are visible immediately.

pub fn with_catalog(self, catalog: Arc<NvidiaCatalogCache>) -> ProviderRegistry

Attach a live catalog cache (used after construction for background refresh).

pub fn set_default_model(&mut self, model_name: &str)

Set the default model name

pub fn register_provider(&mut self, name: &str, config: ProviderConfig)

Register a provider configuration (legacy no-op if providers are already managed).

pub fn register_model(&mut self, name: &str, config: ModelConfig)

Register a model configuration (legacy backward-compat).

pub fn unregister_provider(&mut self, name: &str) -> Option<ProviderConfig>

Remove a provider by name.

pub fn unregister_model(&mut self, name: &str) -> Option<ModelConfig>

Remove a model by name.

pub fn get_provider(&self, name: &str) -> Option<ProviderConfig>

Get a provider configuration by name.

Runtime providers are checked first and synthesized into a ProviderConfig on the fly; legacy static configs are checked second. The return value is cloned so that the caller owns it — this is required because runtime providers are materialised from the arc-swapped map rather than stored as ProviderConfig internally.

pub fn get_provider_for_ctx( &self, ctx: &Arc<Context>, name: &str, ) -> Option<ProviderConfig>

Resolve a provider visible to the tenant derived from the context’s isolate namespace. Reads ctx.isolate_label(TypeId::of::<Llm>()) and strips a leading tenant:/user: prefix (mirroring ares_agent::resolver::user_id_from_ctx), delegating to crate-private tenant lookup with the derived tenant (None when unlabeled).

pub fn get_model(&self, name: &str) -> Option<ModelConfig>

Get a model configuration by name. Checks explicit legacy models first, then falls back to the live catalog.

pub fn provider_names(&self) -> Vec<String>

Get all provider names (legacy + runtime).

pub fn model_names(&self) -> Vec<String>

Get all model names (legacy + catalog ids)

pub async fn create_client_for_model( &self, model_name: &str, ) -> Result<Box<dyn LLMClient>, AppError>

Create an LLM client for a specific model by name.

Fleet-wide tenant resolution (tenant None).

pub async fn create_client_for_model_ctx( &self, ctx: &Arc<Context>, model_name: &str, ) -> Result<Box<dyn LLMClient>, AppError>

Create an LLM client for a specific model by name, deriving the tenant from the context’s isolate namespace so tenant-scoped runtime providers are used when the caller holds a tenant-isolated context.

pub async fn create_client_for_provider( &self, provider_name: &str, ) -> Result<Box<dyn LLMClient>, AppError>

Create an LLM client for a specific provider by name

pub async fn create_client_for_resolved_provider( &self, resolved: &ResolvedProviderConfig, ) -> Result<Box<dyn LLMClient>, AppError>

Create an LLM client for an already-resolved provider/model pair.

pub async fn create_default_client( &self, ) -> Result<Box<dyn LLMClient>, AppError>

Create an LLM client using the default model

pub fn has_model(&self, name: &str) -> bool

Check if a model exists in the registry

pub fn has_provider(&self, name: &str) -> bool

Check if a provider exists in the registry (legacy or runtime).

pub fn has_provider_for_tenant( &self, name: &str, tenant_id: Option<&str>, ) -> bool

pub fn get_model_capabilities( &self, model_name: &str, ) -> Option<ModelCapabilities>

Get capabilities for a registered model.

pub fn models_with_capabilities(&self) -> Vec<ModelWithCapabilities>

Get all models with their capabilities.

pub fn find_models( &self, requirements: &CapabilityRequirements, ) -> Vec<ModelWithCapabilities>

Find models that satisfy the given capability requirements.

pub fn find_best_model( &self, requirements: &CapabilityRequirements, ) -> Option<ModelWithCapabilities>

Find the best model for the given requirements.

pub async fn create_client_for_requirements( &self, requirements: &CapabilityRequirements, ) -> Result<Box<dyn LLMClient>, AppError>

Create an LLM client for the best model matching requirements.

pub fn find_agent_models(&self) -> Vec<ModelWithCapabilities>

Find models suitable for agent tasks (tool calling required).

pub fn find_vision_models(&self) -> Vec<ModelWithCapabilities>

Find models suitable for vision tasks.

pub fn find_coding_models(&self) -> Vec<ModelWithCapabilities>

Find models suitable for coding tasks.

pub fn find_local_models(&self) -> Vec<ModelWithCapabilities>

Find local-only models.

pub fn list_models(&self) -> Vec<ModelInfo>

List all registered models with their provider info.

pub async fn resolve_with_capability_fallback( &self, requirements: Option<CapabilityRequirements>, ) -> Result<Box<dyn LLMClient>, AppError>

Capability-aware fallback chain for Llm.

Tries create_client_for_requirements for the given requirements if Some, then falls back to create_default_client. This reuses the existing find_best_model → create_client_for_model chain and the coordinator’s fallback semantics without requiring a database.

pub async fn resolve_with_fallback( &self, tier_or_model: &str, tenant_id: &str, pool: &Pool<Postgres>, fleet_secrets: &FleetSecrets, ) -> Result<Vec<ResolvedProviderConfig>, AppError>

Resolve a model tier or model name to concrete provider/model entries, following the fallback chain stored in fleet_secrets for the primary provider.

1. Looks up tenant_model_tiers for the tenant + tier.
2. Falls back to the registry’s configured models.
3. Falls back to treating tier_or_model as a provider name.
4. Loads the primary provider’s fallback_providers from fleet secrets and appends each resolved provider with its own concrete default model.

Trait Implementations

impl Default for ProviderRegistry

fn default() -> ProviderRegistry

Returns the “default value” for a type.

impl Service for ProviderRegistry

fn name(&self) -> &'static str

fn init( &self, _ctx: &Arc<Context>, ) -> Pin<Box<dyn Future<Output = Result<Option<Box<dyn Disposable>>, CordisError>> + Send + '_>>

fn check(&self) -> bool

Availability predicate for this service instance.