Struct Llm 

pub struct Llm { /* private fields */ }

Unified LLM capability composing provider registry, catalog, factory, and pool.

Supports ctx.get::<Llm>(), ctx.isolate::<Llm> per-tenant scoping, and ctx.intercept::<ModelOverride> per-request model pinning.

Service::check delegates to the circuit breaker — when the breaker is Open, check() returns false so dependent fibers (e.g. Execute) deactivate via guarded withdrawal (Thm 63).

Implementations

impl Llm

pub fn new( provider_registry: Arc<ProviderRegistry>, pool: Arc<ClientPool>, catalog: Option<Arc<NvidiaCatalogCache>>, ) -> Llm

Create a new Llm.

pub fn with_factory(self, factory: Arc<ConfigBasedLLMFactory>) -> Llm

Attach a config-based factory (crate wiring / Execute boot).

pub fn from_client(client: Arc<dyn LLMClient>) -> Llm

Pin a client used by get_client / complete.

Intended for in-process tests and the library proof (Execute with no HTTP).

pub fn registry(&self) -> Arc<ProviderRegistry>

Handle for AgentRegistry construction in ares-agent.

Application code should use get_client.

pub fn with_breaker( provider_registry: Arc<ProviderRegistry>, catalog: Option<Arc<NvidiaCatalogCache>>, pool: Arc<ClientPool>, breaker: Breaker, ) -> Llm

Create with explicit breaker (e.g. Open for testing).

pub fn with_catalog( provider_registry: Arc<ProviderRegistry>, catalog: Arc<NvidiaCatalogCache>, pool: Arc<ClientPool>, ) -> Llm

Legacy constructor with non-optional catalog (kept for existing call-sites).

pub fn breaker(&self) -> Breaker

Get current breaker state.

pub fn trip(&self, until: DateTime<Utc>)

Transition breaker to Open with cooldown.

pub fn half_open(&self)

Transition breaker to HalfOpen.

pub fn reset(&self)

Reset breaker to Closed.

pub fn record_success(&self)

Record a successful request — closes the breaker and resets failure count.

pub fn record_failure(&self)

Record a failure — increments counter and opens breaker when threshold reached.

Threshold: Breaker::FAILURE_THRESHOLD (5). Cooldown: Breaker::COOLDOWN_SECS (30s). HalfOpen immediately re-opens on failure.

pub fn validate_model_override( &self, ctx: &Arc<Context>, ) -> Result<(), AppError>

Validate a request’s model override against its tenant policy.

Both values are resolved through the context prototype chain, so callers can compose TenantModelPolicy and ModelOverride on separate child contexts. With no policy, legacy override behavior is preserved.

pub async fn get_client( &self, ctx: &Arc<Context>, capability: CapabilityRequirements, ) -> Result<Arc<dyn LLMClient>, AppError>

Capability-aware client resolution with per-request ModelOverride pinning.

Public API: Llm::get_client(&self, ctx: &Arc<cordis::Context>, capability).

When EventsService is on ctx, runs waterfall "llm.get_client" first. Core is identity so handlers can set "deny": true or "model". If the result has a model string and no intercept ModelOverride, get_client_inner runs on ctx.with_intercept(ModelOverride { model }).

pub async fn get_client_boxed( &self, ctx: &Arc<Context>, capability: CapabilityRequirements, ) -> Result<Box<dyn LLMClient>, AppError>

Box variant for callers that prefer Box<dyn LLMClient>.

pub async fn complete( &self, ctx: &Arc<Context>, prompt: &str, ) -> Result<String, AppError>

Generate a completion, optionally through waterfall "llm.complete".

Without EventsService, this is get_client then generate. With events, handlers wrap payload {"prompt"}; core generates using payload["prompt"] and returns {"prompt", "content"}.

pub fn find_model_stub(&self, _capability: &str) -> Option<String>

Stub for capability-based model selection (delegates to registry).

pub fn list_models(&self) -> Vec<ModelInfo>

List registered models with their provider info.

pub fn has_provider_for_tenant( &self, name: &str, tenant_id: Option<&str>, ) -> bool

Check if a provider exists for the given tenant (legacy or runtime).

pub fn get_provider_for_ctx( &self, ctx: &Arc<Context>, name: &str, ) -> Option<ProviderConfig>

Resolve a provider visible to the tenant derived from ctx.

pub fn reload_runtime_providers( &self, providers: Vec<RuntimeProviderEntry>, names: Vec<String>, )

Hot-swap the runtime provider map.

Trait Implementations

impl Service for Llm

fn name(&self) -> &'static str

fn init( &self, _ctx: &Arc<Context>, ) -> Pin<Box<dyn Future<Output = Result<Option<Box<dyn Disposable>>, CordisError>> + Send + '_>>

fn check(&self) -> bool

Availability predicate for this service instance.