pub struct RoutingTextModel { /* private fields */ }Expand description
Dispatches by Prompt::tier to the adapter a venture configured for
that tier, so venture code holds one Arc<dyn TextModel> and never
matches on the tier or the vendor itself (ADR 0002). The router lives
above the adapters, in core — the same place
RoutingPush does, and for the same reason
(ADR 0015).
This is the seam that lets a venture put drafting on one vendor and an independent judge on another without either module knowing.
A tier with no adapter is TextModelError::NotConfigured —
deliberately not TextModelError::Rejected: nothing is wrong with the
prompt, the venture simply did not wire that tier.
The router also enforces capability agreement (issue #665): a prompt
that carries Prompt::tools is refused with
TextModelError::Unsupported before a tier that does not report
Capability::Tools for that tier is called, so a venture can assert
once at compose time — router.supports(ModelTier::Strong, Capability::Tools) — that the tier it routed tools to can carry them.
RoutingTextModel::supports answers for the single routed tier: the
adapter wired for tier, or false when that tier is unwired. It is
deliberately not an aggregate over every wired tier: a prompt goes
to exactly one tier, so what matters is that tier’s answer, and a
strong-tier judge that cannot carry tools must not make a fast-tier
drafting model look incapable.
The same agreement covers images (issue #628): a prompt carrying a
Part::Image is refused with TextModelError::Unsupported before a
tier that does not report Capability::Images is called, and an
over-limit prompt by Prompt::check_images before the tier is looked
up at all.
Implementations§
Source§impl RoutingTextModel
impl RoutingTextModel
Sourcepub fn fast(self, model: Arc<dyn TextModel>) -> Self
pub fn fast(self, model: Arc<dyn TextModel>) -> Self
The adapter for ModelTier::Fast.
Sourcepub fn strong(self, model: Arc<dyn TextModel>) -> Self
pub fn strong(self, model: Arc<dyn TextModel>) -> Self
The adapter for ModelTier::Strong.
Trait Implementations§
Source§impl Clone for RoutingTextModel
impl Clone for RoutingTextModel
Source§impl Debug for RoutingTextModel
impl Debug for RoutingTextModel
Source§impl Default for RoutingTextModel
impl Default for RoutingTextModel
Source§impl TextModel for RoutingTextModel
impl TextModel for RoutingTextModel
Source§fn complete<'life0, 'life1, 'async_trait>(
&'life0 self,
prompt: &'life1 Prompt,
) -> Pin<Box<dyn Future<Output = Result<Completion, TextModelError>> + Send + 'async_trait>>where
Self: 'async_trait,
'life0: 'async_trait,
'life1: 'async_trait,
fn complete<'life0, 'life1, 'async_trait>(
&'life0 self,
prompt: &'life1 Prompt,
) -> Pin<Box<dyn Future<Output = Result<Completion, TextModelError>> + Send + 'async_trait>>where
Self: 'async_trait,
'life0: 'async_trait,
'life1: 'async_trait,
prompt. Read moreSource§fn supports(&self, tier: ModelTier, capability: Capability) -> bool
fn supports(&self, tier: ModelTier, capability: Capability) -> bool
capability for a prompt routed to
tier. Defaults to false: a model that has not opted in is
refused a tools-bearing prompt by RoutingTextModel and
run_tool_loop before it is ever called,
rather than sent a request it would silently drop the tools from.
An adapter that carries Capability::Tools or
Capability::Images overrides this to answer true for it. Read moreAuto Trait Implementations§
impl !RefUnwindSafe for RoutingTextModel
impl !UnwindSafe for RoutingTextModel
impl Freeze for RoutingTextModel
impl Send for RoutingTextModel
impl Sync for RoutingTextModel
impl Unpin for RoutingTextModel
impl UnsafeUnpin for RoutingTextModel
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Source§impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> CloneToUninit for Twhere
T: Clone,
Source§impl<T> Instrument for T
impl<T> Instrument for T
Source§fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
Source§fn in_current_span(self) -> Instrumented<Self> ⓘ
fn in_current_span(self) -> Instrumented<Self> ⓘ
Source§impl<M> TextModelExt for M
impl<M> TextModelExt for M
Source§fn complete_json<'life0, 'life1, 'async_trait>(
&'life0 self,
prompt: Prompt,
schema: &'life1 Schema,
) -> Pin<Box<dyn Future<Output = Result<Value, TextModelError>> + Send + 'async_trait>>where
'life0: 'async_trait,
'life1: 'async_trait,
M: 'async_trait,
fn complete_json<'life0, 'life1, 'async_trait>(
&'life0 self,
prompt: Prompt,
schema: &'life1 Schema,
) -> Pin<Box<dyn Future<Output = Result<Value, TextModelError>> + Send + 'async_trait>>where
'life0: 'async_trait,
'life1: 'async_trait,
M: 'async_trait,
Source§fn complete_as<'life0, 'async_trait, T>(
&'life0 self,
prompt: Prompt,
) -> Pin<Box<dyn Future<Output = Result<T, TextModelError>> + Send + 'async_trait>>where
'life0: 'async_trait,
T: JsonSchema + DeserializeOwned + Send + 'async_trait,
M: 'async_trait,
fn complete_as<'life0, 'async_trait, T>(
&'life0 self,
prompt: Prompt,
) -> Pin<Box<dyn Future<Output = Result<T, TextModelError>> + Send + 'async_trait>>where
'life0: 'async_trait,
T: JsonSchema + DeserializeOwned + Send + 'async_trait,
M: 'async_trait,
prompt and deserializes the conforming value into T,
using T’s own schema (schema_for) as the constraint — the
typed form of TextModelExt::complete_json. Read more