pub trait CompletionModel:
Clone
+ WasmCompatSend
+ WasmCompatSync {
type Response: WasmCompatSend + WasmCompatSync + Serialize + DeserializeOwned;
type StreamingResponse: Clone + Unpin + WasmCompatSend + WasmCompatSync + Serialize + DeserializeOwned + GetTokenUsage;
type Client;
// Required methods
fn make(client: &Self::Client, model: impl Into<String>) -> Self;
fn completion(
&self,
request: CompletionRequest,
) -> impl Future<Output = Result<CompletionResponse<Self::Response>, CompletionError>> + WasmCompatSend;
fn stream(
&self,
request: CompletionRequest,
) -> impl Future<Output = Result<StreamingCompletionResponse<Self::StreamingResponse>, CompletionError>> + WasmCompatSend;
// Provided method
fn completion_request(
&self,
prompt: impl Into<Message>,
) -> CompletionRequestBuilder<Self> { ... }
}Expand description
Trait defining a completion model that can be used to generate completion responses. This trait is meant to be implemented by the user to define a custom completion model, either from a third party provider (e.g.: OpenAI) or a local model.
Required Associated Types§
Sourcetype Response: WasmCompatSend + WasmCompatSync + Serialize + DeserializeOwned
type Response: WasmCompatSend + WasmCompatSync + Serialize + DeserializeOwned
The raw response type returned by the underlying completion model.
Sourcetype StreamingResponse: Clone + Unpin + WasmCompatSend + WasmCompatSync + Serialize + DeserializeOwned + GetTokenUsage
type StreamingResponse: Clone + Unpin + WasmCompatSend + WasmCompatSync + Serialize + DeserializeOwned + GetTokenUsage
The raw response type returned by the underlying completion model when streaming.
Required Methods§
Sourcefn make(client: &Self::Client, model: impl Into<String>) -> Self
fn make(client: &Self::Client, model: impl Into<String>) -> Self
Construct a model handle from a provider client and model identifier.
Sourcefn completion(
&self,
request: CompletionRequest,
) -> impl Future<Output = Result<CompletionResponse<Self::Response>, CompletionError>> + WasmCompatSend
fn completion( &self, request: CompletionRequest, ) -> impl Future<Output = Result<CompletionResponse<Self::Response>, CompletionError>> + WasmCompatSend
Generates a completion response for the given completion request.
fn stream( &self, request: CompletionRequest, ) -> impl Future<Output = Result<StreamingCompletionResponse<Self::StreamingResponse>, CompletionError>> + WasmCompatSend
Provided Methods§
Sourcefn completion_request(
&self,
prompt: impl Into<Message>,
) -> CompletionRequestBuilder<Self>
fn completion_request( &self, prompt: impl Into<Message>, ) -> CompletionRequestBuilder<Self>
Generates a completion request builder for the given prompt.
Dyn Compatibility§
This trait is not dyn compatible.
In older versions of Rust, dyn compatibility was called "object safety", so this trait is not object safe.
Implementors§
Source§impl CompletionModel for MockCompletionModel
Available on crate feature test-utils only.
impl CompletionModel for MockCompletionModel
Available on crate feature
test-utils only.