Struct CompletionModel 

pub struct CompletionModel {
    pub model: String,
    pub prompt_caching: bool,
    /* private fields */
}

Fields

model: String

prompt_caching: bool

When enabled, cache checkpoints are inserted into Converse API requests to take advantage of Bedrock prompt caching. A checkpoint is placed after the system prompt and after the last message in the chat history. Disabled by default.

Implementations

impl CompletionModel

pub fn new(client: Client, model: impl Into<String>) -> Self

pub fn with_prompt_caching(self) -> Self

Enable Bedrock prompt caching for this model.

When enabled, CachePoint blocks are inserted after the serialized system content and after the final messages entry in each Converse API request. This allows Bedrock to cache and reuse repeated prompt prefixes, reducing both latency and input token costs.

This currently covers the system and messages request fields only. Some Bedrock models also support caching tools, but that is not wired up here yet.

Cacheability and token thresholds are model-specific. If the cached prefix is too short or the model does not support caching for a given field, Bedrock ignores the checkpoint. See the Bedrock prompt caching support table for current limits and field support.

Trait Implementations

impl Clone for CompletionModel

fn clone(&self) -> CompletionModel

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl CompletionModel for CompletionModel

type Response = AwsConverseOutput

The raw response type returned by the underlying completion model.

type StreamingResponse = BedrockStreamingResponse

The raw response type returned by the underlying completion model when streaming.

type Client = Client

fn make(client: &Self::Client, model: impl Into<String>) -> Self

async fn completion( &self, completion_request: CompletionRequest, ) -> Result<CompletionResponse<AwsConverseOutput>, CompletionError>

Generates a completion response for the given completion request.

async fn stream( &self, request: CompletionRequest, ) -> Result<StreamingCompletionResponse<Self::StreamingResponse>, CompletionError>

fn completion_request( &self, prompt: impl Into<Message>, ) -> CompletionRequestBuilder<Self>

Generates a completion request builder for the given prompt.