Struct stretto::AsyncCache

pub struct AsyncCache<K, V, KH = DefaultKeyBuilder, C = DefaultCoster<V>, U = DefaultUpdateValidator<V>, CB = DefaultCacheCallback<V>, S = RandomState> where
    K: Hash + Eq,
    V: Send + Sync + 'static,
    KH: KeyBuilder<K>,  {
    pub metrics: Arc<Metrics>,
    /* private fields */
}

This is supported on crate feature async only.

AsyncCache is a thread-safe async implementation of a hashmap with a TinyLFU admission policy and a Sampled LFU eviction policy. You can use the same AsyncCache instance from as many threads as you want.

Features

• Internal Mutability - Do not need to use Arc<RwLock<Cache<...>> for concurrent code, you just need Cache<...>
• Sync and Async - Stretto support async by tokio and sync by crossbeam.
  • In sync, Cache starts two extra OS level threads. One is policy thread, the other is writing thread.
  • In async, Cache starts two extra green threads. One is policy thread, the other is writing thread.
• Store policy Stretto only store the value, which means the cache does not store the key.
• High Hit Ratios - with our unique admission/eviction policy pairing, Ristretto’s performance is best in class.
  • Eviction: SampledLFU - on par with exact LRU and better performance on Search and Database traces.
  • Admission: TinyLFU - extra performance with little memory overhead (12 bits per counter).
• Fast Throughput - we use a variety of techniques for managing contention and the result is excellent throughput.
• Cost-Based Eviction - any large new item deemed valuable can evict multiple smaller items (cost could be anything).
• Fully Concurrent - you can use as many threads as you want with little throughput degradation.
• Metrics - optional performance metrics for throughput, hit ratios, and other stats.
• Simple API - just figure out your ideal CacheBuilder values and you’re off and running.

Fields

metrics: Arc<Metrics>

the metrics for the cache

Implementations

impl<K, V, KH, C, U, CB, S> AsyncCache<K, V, KH, C, U, CB, S> where
    K: Hash + Eq,
    V: Send + Sync + 'static,
    KH: KeyBuilder<K>,
    C: Coster<V>,
    U: UpdateValidator<V>,
    CB: CacheCallback<V>,
    S: BuildHasher + Clone + 'static, 

pub async fn insert(&self, key: K, val: V, cost: i64) -> bool

insert attempts to add the key-value item to the cache. If it returns false, then the insert was dropped and the key-value item isn’t added to the cache. If it returns true, there’s still a chance it could be dropped by the policy if its determined that the key-value item isn’t worth keeping, but otherwise the item will be added and other items will be evicted in order to make room.

To dynamically evaluate the items cost using the Config.Coster function, set the cost parameter to 0 and Coster will be ran when needed in order to find the items true cost.

pub async fn insert_with_ttl(
    &self,
    key: K,
    val: V,
    cost: i64,
    ttl: Duration
) -> bool

insert_with_ttl works like Set but adds a key-value pair to the cache that will expire after the specified TTL (time to live) has passed. A zero value means the value never expires, which is identical to calling insert.

pub async fn insert_if_present(&self, key: K, val: V, cost: i64) -> bool

insert_if_present is like insert, but only updates the value of an existing key. It does NOT add the key to cache if it’s absent.

pub async fn wait(&self) -> Result<(), CacheError>

wait until the previous operations finished.

pub async fn remove(&self, k: &K)

remove entry from Cache by key.

pub async fn close(&self) -> Result<(), CacheError>

close stops all threads and closes all channels.

impl<K: Hash + Eq, V: Send + Sync + 'static> AsyncCache<K, V>

pub fn new(num_counters: usize, max_cost: i64) -> Result<Self, CacheError>

Returns a Cache instance with default configruations.

pub fn builder(
    num_counters: usize,
    max_cost: i64
) -> AsyncCacheBuilder<K, V, DefaultKeyBuilder, DefaultCoster<V>, DefaultUpdateValidator<V>, DefaultCacheCallback<V>, RandomState>

Returns a Builder.

impl<K: Hash + Eq, V: Send + Sync + 'static, KH: KeyBuilder<K>> AsyncCache<K, V, KH>

pub fn new_with_key_builder(
    num_counters: usize,
    max_cost: i64,
    index: KH
) -> Result<Self, CacheError>

Returns a Cache instance with default configruations.

impl<K, V, KH, C, U, CB, S> AsyncCache<K, V, KH, C, U, CB, S> where
    K: Hash + Eq,
    V: Send + Sync + 'static,
    KH: KeyBuilder<K>,
    C: Coster<V>,
    U: UpdateValidator<V>,
    CB: CacheCallback<V>,
    S: BuildHasher + Clone + 'static, 

pub fn get(&self, key: &K) -> Option<ValueRef<'_, V, S>>

get returns a Option<ValueRef<V, SS>> (if any) representing whether the value was found or not.

pub fn get_mut(&self, key: &K) -> Option<ValueRefMut<'_, V, S>>

get_mut returns a Option<ValueRefMut<V, SS>> (if any) representing whether the value was found or not.

pub fn get_ttl(&self, key: &K) -> Option<Duration>

Returns the TTL for the specified key if the item was found and is not expired.

pub fn clear(&self) -> Result<(), CacheError>

clear the Cache.

pub fn max_cost(&self) -> i64

max_cost returns the max cost of the cache.

pub fn update_max_cost(&self, max_cost: i64)

update_max_cost updates the maxCost of an existing cache.

pub fn len(&self) -> usize

Returns the number of items in the Cache

pub fn is_empty(&self) -> bool

Returns true if the cache is empty

Trait Implementations

impl<K, V, KH, C, U, CB, S> AsRef<AsyncCache<K, V, KH, C, U, CB, S>> for AsyncCache<K, V, KH, C, U, CB, S> where
    K: Hash + Eq,
    V: Send + Sync + 'static,
    KH: KeyBuilder<K>,
    C: Coster<V>,
    U: UpdateValidator<V>,
    CB: CacheCallback<V>,
    S: BuildHasher + Clone + 'static, 

fn as_ref(&self) -> &AsyncCache<K, V, KH, C, U, CB, S>

Performs the conversion.

impl<K, V, KH, C, U, CB, S> Clone for AsyncCache<K, V, KH, C, U, CB, S> where
    K: Hash + Eq,
    V: Send + Sync + 'static,
    KH: KeyBuilder<K>,
    C: Coster<V>,
    U: UpdateValidator<V>,
    CB: CacheCallback<V>,
    S: BuildHasher + Clone + 'static, 

fn clone(&self) -> Self

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.