pub trait CompletionModel:
Clone
+ WasmCompatSend
+ WasmCompatSync {
type Response: WasmCompatSend + WasmCompatSync + Serialize + DeserializeOwned;
type StreamingResponse: Clone + Unpin + WasmCompatSend + WasmCompatSync + Serialize + DeserializeOwned + GetTokenUsage;
type Client;
// Required methods
fn make(client: &Self::Client, model: impl Into<String>) -> Self;
fn completion(
&self,
request: CompletionRequest,
) -> impl Future<Output = Result<CompletionResponse<Self::Response>, CompletionError>> + WasmCompatSend;
fn stream(
&self,
request: CompletionRequest,
) -> impl Future<Output = Result<StreamingCompletionResponse<Self::StreamingResponse>, CompletionError>> + WasmCompatSend;
// Provided method
fn completion_request(
&self,
prompt: impl Into<Message>,
) -> CompletionRequestBuilder<Self> { ... }
}Expand description
Trait defining a completion model that can be used to generate completion responses. This trait is meant to be implemented by the user to define a custom completion model, either from a third party provider (e.g.: OpenAI) or a local model.
Required Associated Types§
Sourcetype Response: WasmCompatSend + WasmCompatSync + Serialize + DeserializeOwned
type Response: WasmCompatSend + WasmCompatSync + Serialize + DeserializeOwned
The raw response type returned by the underlying completion model.
Sourcetype StreamingResponse: Clone + Unpin + WasmCompatSend + WasmCompatSync + Serialize + DeserializeOwned + GetTokenUsage
type StreamingResponse: Clone + Unpin + WasmCompatSend + WasmCompatSync + Serialize + DeserializeOwned + GetTokenUsage
The raw response type returned by the underlying completion model when streaming.
type Client
Required Methods§
fn make(client: &Self::Client, model: impl Into<String>) -> Self
Sourcefn completion(
&self,
request: CompletionRequest,
) -> impl Future<Output = Result<CompletionResponse<Self::Response>, CompletionError>> + WasmCompatSend
fn completion( &self, request: CompletionRequest, ) -> impl Future<Output = Result<CompletionResponse<Self::Response>, CompletionError>> + WasmCompatSend
Generates a completion response for the given completion request.
fn stream( &self, request: CompletionRequest, ) -> impl Future<Output = Result<StreamingCompletionResponse<Self::StreamingResponse>, CompletionError>> + WasmCompatSend
Provided Methods§
Sourcefn completion_request(
&self,
prompt: impl Into<Message>,
) -> CompletionRequestBuilder<Self>
fn completion_request( &self, prompt: impl Into<Message>, ) -> CompletionRequestBuilder<Self>
Generates a completion request builder for the given prompt.
Dyn Compatibility§
This trait is not dyn compatible.
In older versions of Rust, dyn compatibility was called "object safety".