pub trait CompletionModel: WasmCompatSend + WasmCompatSync {
// Required methods
fn completion(
&self,
request: CompletionRequest,
) -> impl Future<Output = Result<CompletionResponse, CompletionError>> + WasmCompatSend;
fn stream(
&self,
request: CompletionRequest,
) -> impl Future<Output = Result<StreamingCompletionResponse, CompletionError>> + WasmCompatSend;
// Provided methods
fn completion_request(
&self,
prompt: impl Into<Message>,
) -> CompletionRequestBuilder<Self>
where Self: Sized + Clone { ... }
fn capabilities(&self) -> ProviderCapabilities { ... }
}Expand description
Trait defining a completion model that can be used to generate completion responses. This trait is meant to be implemented by the user to define a custom completion model, either from a third party provider (e.g.: OpenAI) or a local model.
Implementations return Rig’s normalized CompletionResponse and
StreamingCompletionResponse; a provider’s own wire types stay on the
provider’s side of this boundary, reachable through its inherent
raw_completion/raw_stream methods. Model construction belongs to
crate::client::completion::CompletionClient, not to this trait.
The trait demands only async service behavior — no Clone supertrait, in
the spirit of tower::Service: cloning or sharing a model is the caller’s
concern (wrap it in an Arc if needed). The Self::completion_request
convenience gates on Self: Clone individually, which every built-in
provider model satisfies.
Required Methods§
Sourcefn completion(
&self,
request: CompletionRequest,
) -> impl Future<Output = Result<CompletionResponse, CompletionError>> + WasmCompatSend
fn completion( &self, request: CompletionRequest, ) -> impl Future<Output = Result<CompletionResponse, CompletionError>> + WasmCompatSend
Generates a completion response for the given completion request.
Sourcefn stream(
&self,
request: CompletionRequest,
) -> impl Future<Output = Result<StreamingCompletionResponse, CompletionError>> + WasmCompatSend
fn stream( &self, request: CompletionRequest, ) -> impl Future<Output = Result<StreamingCompletionResponse, CompletionError>> + WasmCompatSend
Streams a completion response for the given completion request.
Provided Methods§
Sourcefn completion_request(
&self,
prompt: impl Into<Message>,
) -> CompletionRequestBuilder<Self>
fn completion_request( &self, prompt: impl Into<Message>, ) -> CompletionRequestBuilder<Self>
Generates a completion request builder for the given prompt.
Sourcefn capabilities(&self) -> ProviderCapabilities
fn capabilities(&self) -> ProviderCapabilities
Provider behavior a runtime should account for when preparing requests.
The default is conservative — see ProviderCapabilities. Override
this to declare the capabilities a provider actually supports.
Dyn Compatibility§
This trait is not dyn compatible.
In older versions of Rust, dyn compatibility was called "object safety".
Implementations on Foreign Types§
Source§impl<M> CompletionModel for Arc<M>where
M: CompletionModel + ?Sized,
A shared model is a model: Arc<M> forwards every method to M, so the
“wrap it in an Arc if needed” guidance holds through the generic APIs
(CompletionRequestBuilder, agent construction), not just at direct call
sites. Arc<M>: Clone always holds, so CompletionModel::completion_request
clones the Arc — never the model.
impl<M> CompletionModel for Arc<M>where
M: CompletionModel + ?Sized,
A shared model is a model: Arc<M> forwards every method to M, so the
“wrap it in an Arc if needed” guidance holds through the generic APIs
(CompletionRequestBuilder, agent construction), not just at direct call
sites. Arc<M>: Clone always holds, so CompletionModel::completion_request
clones the Arc — never the model.
fn completion( &self, request: CompletionRequest, ) -> impl Future<Output = Result<CompletionResponse, CompletionError>> + WasmCompatSend
fn stream( &self, request: CompletionRequest, ) -> impl Future<Output = Result<StreamingCompletionResponse, CompletionError>> + WasmCompatSend
fn capabilities(&self) -> ProviderCapabilities
Implementors§
impl CompletionModel for MockCompletionModel
impl CompletionModel for ModelHandle
A handle behaves exactly like the model it erased, with capabilities served from the snapshot captured at erasure time.
It deliberately adds no request validation of its own. Both agent surfaces
reach a model through CompletionRequestBuilder — runner.rs’s blocking
turn calls builder.send(), the streaming turn calls builder.stream() —
and the builder already runs
CompletionRequest::validate_message_content. Repeating it here would
scan the whole history a second time on every model call and buy nothing.