pub struct CacheCost {
pub uncached_input: u64,
pub cache_reads: u64,
pub cache_writes: u64,
pub storage_token_hours: f64,
}Expand description
A request’s or a run’s input tokens by how they are billed, plus the
token-hours its cache resources were stored. Sum the costs of a run’s
calls (and of any cache resources beside them) and price the total with
CacheCost::usd.
A run priced the same way on any provider, with the caller’s rates from the provider’s price list:
use rig_core::completion::{CacheCost, CacheRates, Usage};
use rig_core::providers::gemini::{AutoCache, CacheBook};
let run: CacheCost = calls.iter().map(CacheCost::from_usage).sum();
let opus = CacheRates { input: 4.0, cached_read: 0.2, cache_write: 5.0, storage_per_hour: 0.0 };
println!("${:.3}, {:.1}% saved", run.usd(&opus), run.saving(&opus) * 100.0);
// Gemini: the calls, plus the cache book's creation and storage.
let gemini = run + CacheCost::from(&book.report());
let flash = CacheRates { input: 0.75, cached_read: 0.075, cache_write: 0.75, storage_per_hour: 0.5 };
println!("${:.3}", gemini.usd(&flash));Fields§
§uncached_input: u64Input billed at the uncached price.
cache_reads: u64Input read from a cache.
cache_writes: u64Input written to a cache.
storage_token_hours: f64Σ tokens × hours of stored cache resources.
Implementations§
Source§impl CacheCost
impl CacheCost
Sourcepub fn from_usage(usage: &Usage) -> CacheCost
pub fn from_usage(usage: &Usage) -> CacheCost
The billed input of one call: the input neither read from nor written
to a cache is input_tokens - cached_input_tokens - cache_creation_input_tokens. An unreported counter counts as zero;
reads and writes above input_tokens leave no uncached input rather
than a negative one.
Sourcepub fn prompt_tokens(&self) -> u64
pub fn prompt_tokens(&self) -> u64
Every input token, however it was billed.
Sourcepub fn usd(&self, rates: &CacheRates) -> f64
pub fn usd(&self, rates: &CacheRates) -> f64
Input cost in USD at rates: uncached, read, written and stored.
Sourcepub fn uncached_usd(&self, rates: &CacheRates) -> f64
pub fn uncached_usd(&self, rates: &CacheRates) -> f64
Input cost in USD at rates had every prompt token been uncached.
Sourcepub fn saving(&self, rates: &CacheRates) -> f64
pub fn saving(&self, rates: &CacheRates) -> f64
The share of the uncached input cost caching saved: 1 - usd / uncached_usd. Negative when caching cost more; zero without input.
Trait Implementations§
Source§impl AddAssign for CacheCost
impl AddAssign for CacheCost
Source§fn add_assign(&mut self, other: CacheCost)
fn add_assign(&mut self, other: CacheCost)
+= operation. Read moreimpl Copy for CacheCost
Source§impl<'de> Deserialize<'de> for CacheCost
impl<'de> Deserialize<'de> for CacheCost
Source§fn deserialize<__D>(
__deserializer: __D,
) -> Result<CacheCost, <__D as Deserializer<'de>>::Error>where
__D: Deserializer<'de>,
fn deserialize<__D>(
__deserializer: __D,
) -> Result<CacheCost, <__D as Deserializer<'de>>::Error>where
__D: Deserializer<'de>,
Source§impl From<&CacheReport> for CacheCost
The book’s caches as billed beside the calls: every created token as a
cache write (priced at the input price) and token_hours of storage. Add
it to the calls’ CacheCost::from_usage to price a run.
impl From<&CacheReport> for CacheCost
The book’s caches as billed beside the calls: every created token as a
cache write (priced at the input price) and token_hours of storage. Add
it to the calls’ CacheCost::from_usage to price a run.