Struct CacheCost 

pub struct CacheCost {
    pub uncached_input: u64,
    pub cache_reads: u64,
    pub cache_writes: u64,
    pub storage_token_hours: f64,
}

A request’s or a run’s input tokens by how they are billed, plus the token-hours its cache resources were stored. Sum the costs of a run’s calls (and of any cache resources beside them) and price the total with CacheCost::usd.

A run priced the same way on any provider, with the caller’s rates from the provider’s price list:

use rig_core::completion::{CacheCost, CacheRates, Usage};
use rig_core::providers::gemini::{AutoCache, CacheBook};

let run: CacheCost = calls.iter().map(CacheCost::from_usage).sum();

let opus = CacheRates { input: 4.0, cached_read: 0.2, cache_write: 5.0, storage_per_hour: 0.0 };
println!("${:.3}, {:.1}% saved", run.usd(&opus), run.saving(&opus) * 100.0);

// Gemini: the calls, plus the cache book's creation and storage.
let gemini = run + CacheCost::from(&book.report());
let flash = CacheRates { input: 0.75, cached_read: 0.075, cache_write: 0.75, storage_per_hour: 0.5 };
println!("${:.3}", gemini.usd(&flash));

Fields

uncached_input: u64

Input billed at the uncached price.

cache_reads: u64

Input read from a cache.

cache_writes: u64

Input written to a cache.

storage_token_hours: f64

Σ tokens × hours of stored cache resources.

Implementations

impl CacheCost

pub fn from_usage(usage: &Usage) -> Self

The billed input of one call: the input neither read from nor written to a cache is input_tokens - cached_input_tokens - cache_creation_input_tokens. An unreported counter counts as zero; reads and writes above input_tokens leave no uncached input rather than a negative one.

pub fn prompt_tokens(&self) -> u64

Every input token, however it was billed.

pub fn usd(&self, rates: &CacheRates) -> f64

Input cost in USD at rates: uncached, read, written and stored.

pub fn uncached_usd(&self, rates: &CacheRates) -> f64

Input cost in USD at rates had every prompt token been uncached.

pub fn saving(&self, rates: &CacheRates) -> f64

The share of the uncached input cost caching saved: 1 - usd / uncached_usd. Negative when caching cost more; zero without input.

Trait Implementations

impl Add for CacheCost

type Output = CacheCost

The resulting type after applying the + operator.

fn add(self, other: Self) -> Self

Performs the + operation.

impl AddAssign for CacheCost

fn add_assign(&mut self, other: Self)

Performs the += operation.

impl Clone for CacheCost

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Copy for CacheCost

impl Debug for CacheCost

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for CacheCost

fn default() -> Self

Returns the “default value” for a type.

impl<'de> Deserialize<'de> for CacheCost

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl From<&CacheReport> for CacheCost

The book’s caches as billed beside the calls: every created token as a cache write (priced at the input price) and token_hours of storage. Add it to the calls’ CacheCost::from_usage to price a run.

fn from(report: &CacheReport) -> Self

Converts to this type from the input type.

impl PartialEq for CacheCost

fn eq(&self, other: &Self) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl Serialize for CacheCost

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.

impl StructuralPartialEq for CacheCost

impl Sum for CacheCost

fn sum<I: Iterator<Item = Self>>(costs: I) -> Self

Takes an iterator and generates Self from the elements by “summing up” the items.