Struct Router 

pub struct Router { /* private fields */ }

Lazily loads checkpoints and sends each request to the right one.

Implementations

impl Router

pub fn new() -> Self

pub fn standalone_repos(self, standalone: bool) -> Self

Use each checkpoint’s own repository rather than the bundle.

pub fn default_model(self, model: ModelName) -> Self

Checkpoint used when detection is inconclusive.

pub fn auto_task_detection(self, enabled: bool) -> Self

Allow the typed-decisions checkpoint to be selected by workflow match.

pub fn max_loaded(self, max_loaded: usize) -> Self

How many checkpoints stay resident; least-recently-used is evicted.

pub fn attach(&mut self, model: ModelName, agent: Arc<Agent>)

Register an already-built agent rather than loading a second copy.

pub fn loaded(&self) -> &[ModelName]

Checkpoints currently resident, least-recently-used first.

pub fn unload(&mut self, model: Option<ModelName>)

Free one checkpoint, or all of them.

pub fn get(&mut self, model: ModelName) -> Option<Arc<Agent>>

Fetch an attached agent, if present.

pub fn route( &self, state: &Value, questions: Option<&Map<String, Value>>, model: Option<&str>, task: Option<&str>, language: Option<&str>, ) -> Result<RouteDecision>

Decide which checkpoint to use, without loading or running anything.

Precedence: explicit model > explicit task > detected workflow (opt-in) > explicit lang > detected script/language > default.

pub fn predict( &mut self, state: &Value, questions: &Map<String, Value>, ) -> Result<Prediction>

Route, then answer every question on the chosen checkpoint.

The chosen checkpoint must already be attached; see Router::attach, or the hub feature for automatic downloads.

impl Router

pub fn load( &mut self, model: ModelName, options: AgentBuilder, ) -> Result<Arc<Agent>>

Download and build a checkpoint, then keep it resident.

pub fn preload( &mut self, models: &[ModelName], options: impl Fn() -> AgentBuilder, ) -> Result<()>

Download and build several checkpoints up front.

A cold load costs seconds while detection costs microseconds, so a server that alternates languages should preload rather than rely on the LRU, which would otherwise reload on nearly every request.

Trait Implementations

impl Default for Router

fn default() -> Self

Returns the “default value” for a type.