Struct CacheController 

pub struct CacheController;

Prompt Caching Controller

Manages cache control markers for messages and calculates cache savings.

Note on Cache Control Implementation

Cache control markers are typically added at the API request level by the provider implementation, not stored in the message content itself. This controller identifies which messages are eligible for caching and provides the information needed for providers to add appropriate cache control headers.

Implementations

impl CacheController

pub fn get_cache_eligibility( messages: &[Message], config: &CacheConfig, ) -> CacheEligibility

Identify messages eligible for cache control.

This method analyzes messages and returns information about which messages are eligible for caching based on the provided configuration.

Arguments

• messages - The messages to analyze
• config - Cache configuration specifying thresholds and options

Returns

CacheEligibility containing indices of cacheable messages

Cache Eligibility Rules

Messages are eligible for caching if:

1. They meet the minimum token threshold
2. They are within the most recent N messages (as configured)

Example

use aster::context::cache_controller::CacheController;
use aster::context::types::CacheConfig;

let messages = vec![/* ... */];
let config = CacheConfig::default();
let eligibility = CacheController::get_cache_eligibility(&messages, &config);
println!("Cacheable messages: {:?}", eligibility.cacheable_indices);

pub fn add_cache_control( messages: &[Message], config: &CacheConfig, ) -> (Vec<Message>, Vec<usize>)

Add cache control markers to eligible messages.

This method returns a new vector of messages with cache eligibility information. The actual cache control markers should be added by the provider when making API requests.

Arguments

• messages - The messages to potentially mark for caching
• config - Cache configuration specifying thresholds and options

Returns

A tuple of (messages, cacheable_indices) where cacheable_indices contains the indices of messages that should have cache control applied

pub fn is_cacheable(message: &Message, min_tokens: usize) -> bool

Check if a message is eligible for caching.

A message is cacheable if:

1. It has content
2. Its estimated token count meets the minimum threshold

Arguments

• message - The message to check
• min_tokens - Minimum token threshold for caching

Returns

true if the message is eligible for caching

pub fn calculate_cache_savings(usage: &TokenUsage) -> CacheSavings

Calculate cache cost savings based on token usage.

Uses Anthropic’s prompt caching pricing model:

• Cache write: 1.25x base input price
• Cache read: 0.1x base input price (90% discount)

Arguments

• usage - Token usage statistics including cache metrics

Returns

CacheSavings containing base cost, actual cost, and savings

Calculation

base_cost = input_tokens * base_price
cache_write_cost = cache_creation_tokens * (base_price * 1.25)
cache_read_cost = cache_read_tokens * (base_price * 0.1)
actual_cost = (input_tokens - cache_read_tokens) * base_price
            + cache_write_cost + cache_read_cost
savings = base_cost - actual_cost

pub fn calculate_cache_stats(usage: &TokenUsage) -> CacheStats

Calculate cache statistics from token usage.

Arguments

• usage - Token usage statistics

Returns

CacheStats with totals and hit rate

pub fn accumulate_cache_stats<'a>( usages: impl Iterator<Item = &'a TokenUsage>, ) -> CacheStats

Accumulate cache statistics from multiple usages.

Arguments

• usages - Iterator of token usage statistics

Returns

Aggregated CacheStats