pub trait CompletionModel:
Clone
+ Send
+ Sync {
type Response: Send + Sync + Serialize + DeserializeOwned;
type StreamingResponse: Clone + Unpin + Send + Sync + Serialize + DeserializeOwned + GetTokenUsage;
// Required methods
fn completion(
&self,
request: CompletionRequest,
) -> impl Future<Output = Result<CompletionResponse<Self::Response>, CompletionError>> + Send;
fn stream(
&self,
request: CompletionRequest,
) -> impl Future<Output = Result<StreamingCompletionResponse<Self::StreamingResponse>, CompletionError>> + Send;
// Provided method
fn completion_request(
&self,
prompt: impl Into<Message>,
) -> CompletionRequestBuilder<Self> { ... }
}Expand description
Trait defining a completion model that can be used to generate completion responses. This trait is meant to be implemented by the user to define a custom completion model, either from a third party provider (e.g.: OpenAI) or a local model.
Required Associated Types§
Sourcetype Response: Send + Sync + Serialize + DeserializeOwned
type Response: Send + Sync + Serialize + DeserializeOwned
The raw response type returned by the underlying completion model.
Sourcetype StreamingResponse: Clone + Unpin + Send + Sync + Serialize + DeserializeOwned + GetTokenUsage
type StreamingResponse: Clone + Unpin + Send + Sync + Serialize + DeserializeOwned + GetTokenUsage
The raw response type returned by the underlying completion model when streaming.
Required Methods§
Sourcefn completion(
&self,
request: CompletionRequest,
) -> impl Future<Output = Result<CompletionResponse<Self::Response>, CompletionError>> + Send
fn completion( &self, request: CompletionRequest, ) -> impl Future<Output = Result<CompletionResponse<Self::Response>, CompletionError>> + Send
Generates a completion response for the given completion request.
fn stream( &self, request: CompletionRequest, ) -> impl Future<Output = Result<StreamingCompletionResponse<Self::StreamingResponse>, CompletionError>> + Send
Provided Methods§
Sourcefn completion_request(
&self,
prompt: impl Into<Message>,
) -> CompletionRequestBuilder<Self>
fn completion_request( &self, prompt: impl Into<Message>, ) -> CompletionRequestBuilder<Self>
Generates a completion request builder for the given prompt.
Dyn Compatibility§
This trait is not dyn compatible.
In older versions of Rust, dyn compatibility was called "object safety", so this trait is not object safe.