Skip to main content

ServerCache

Struct ServerCache 

Source
pub struct ServerCache<K, V> { /* private fields */ }
Expand description

An in-process cache of up to capacity entries, each kept for ttl after it was fetched. Declare one as a static, where it is needed; it is built on first use:

use g3_cache::ServerCache;
use std::time::Duration;

static DESCRIPTIONS: ServerCache<String, Option<String>> =
    ServerCache::new(Duration::from_secs(60 * 60), 10_000);

async fn description(media_id: String) -> Result<Option<String>, String> {
    DESCRIPTIONS
        .get_or_fetch(media_id.clone(), google_description(&media_id))
        .await
}

Each cache is its own static rather than one shared instance: every cache holds its own key and value types, and keeping unrelated answers apart means one can’t evict another.

K must include everything the answer depends on. For an answer that depends on the signed-in user, that means the user’s id: it arrives through a session extractor, not the function’s arguments, so a key built from the arguments alone would give one user’s answer to all.

The least useful entries are dropped first when full. For anything get_or_fetch and get_or_insert don’t cover, it dereferences to the underlying moka cache.

Implementations§

Source§

impl<K, V> ServerCache<K, V>
where K: Hash + Eq + Send + Sync + 'static, V: Clone + Send + Sync + 'static,

Source

pub const fn new(ttl: Duration, capacity: u64) -> Self

A cache keeping up to capacity entries for ttl each. const, so it can initialize a static directly.

Source

pub async fn get_or_fetch<E, Fut>(&self, key: K, fetch: Fut) -> Result<V, E>
where Fut: Future<Output = Result<V, E>>, E: Clone + Send + Sync + 'static,

The cached answer for key, or fetch’s. Concurrent misses on one key share a single fetch; an Err is returned to every waiter and not kept, so the next call retries.

E must be Clone, as dioxus::CapturedError is, since every waiter gets a copy. For an error type that isn’t (anyhow::Error), use try_get_with through Deref, which returns it in an Arc.

Source

pub async fn get_or_insert<Fut>(&self, key: K, fetch: Fut) -> V
where Fut: Future<Output = V>,

The cached answer for key, or fetch’s, for a fetch that can’t fail. Concurrent misses on one key share a single fetch.

Methods from Deref<Target = Cache<K, V>>§

Source

pub fn name(&self) -> Option<&str>

Returns cache’s name.

Source

pub fn policy(&self) -> Policy

Returns a read-only cache policy of this cache.

At this time, cache policy cannot be modified after cache creation. A future version may support to modify it.

Source

pub fn entry_count(&self) -> u64

Returns an approximate number of entries in this cache.

The value returned is an estimate; the actual count may differ if there are concurrent insertions or removals, or if some entries are pending removal due to expiration. This inaccuracy can be mitigated by calling run_pending_tasks first.

§Example
// Cargo.toml
//
// [dependencies]
// moka = { version = "0.12", features = ["future"] }
// tokio = { version = "1", features = ["rt-multi-thread", "macros" ] }
use moka::future::Cache;

#[tokio::main]
async fn main() {
    let cache = Cache::new(10);
    cache.insert('n', "Netherland Dwarf").await;
    cache.insert('l', "Lop Eared").await;
    cache.insert('d', "Dutch").await;

    // Ensure an entry exists.
    assert!(cache.contains_key(&'n'));

    // However, followings may print stale number zeros instead of threes.
    println!("{}", cache.entry_count());   // -> 0
    println!("{}", cache.weighted_size()); // -> 0

    // To mitigate the inaccuracy, call `run_pending_tasks` to run pending
    // internal tasks.
    cache.run_pending_tasks().await;

    // Followings will print the actual numbers.
    println!("{}", cache.entry_count());   // -> 3
    println!("{}", cache.weighted_size()); // -> 3
}
Source

pub fn weighted_size(&self) -> u64

Returns an approximate total weighted size of entries in this cache.

The value returned is an estimate; the actual size may differ if there are concurrent insertions or removals, or if some entries are pending removal due to expiration. This inaccuracy can be mitigated by calling run_pending_tasks first. See entry_count for a sample code.

Source

pub fn contains_key<Q>(&self, key: &Q) -> bool
where Q: Equivalent<K> + Hash + ?Sized,

Returns true if the cache contains a value for the key.

Unlike the get method, this method is not considered a cache read operation, so it does not update the historic popularity estimator or reset the idle timer for the key.

The key may be any borrowed form of the cache’s key type, but Hash and Eq on the borrowed form must match those for the key type.

Source

pub async fn get<Q>(&self, key: &Q) -> Option<V>
where Q: Equivalent<K> + Hash + ?Sized,

Returns a clone of the value corresponding to the key.

If you want to store values that will be expensive to clone, wrap them by std::sync::Arc before storing in a cache. Arc is a thread-safe reference-counted pointer and its clone() method is cheap.

The key may be any borrowed form of the cache’s key type, but Hash and Eq on the borrowed form must match those for the key type.

Source

pub fn entry(&self, key: K) -> OwnedKeyEntrySelector<'_, K, V, S>
where K: Hash + Eq,

Takes a key K and returns an OwnedKeyEntrySelector that can be used to select or insert an entry.

§Example
// Cargo.toml
//
// [dependencies]
// moka = { version = "0.12", features = ["future"] }
// tokio = { version = "1", features = ["rt-multi-thread", "macros" ] }

use moka::future::Cache;

#[tokio::main]
async fn main() {
    let cache: Cache<String, u32> = Cache::new(100);
    let key = "key1".to_string();

    let entry = cache.entry(key.clone()).or_insert(3).await;
    assert!(entry.is_fresh());
    assert_eq!(entry.key(), &key);
    assert_eq!(entry.into_value(), 3);

    let entry = cache.entry(key).or_insert(6).await;
    // Not fresh because the value was already in the cache.
    assert!(!entry.is_fresh());
    assert_eq!(entry.into_value(), 3);
}
Source

pub fn entry_by_ref<'a, Q>( &'a self, key: &'a Q, ) -> RefKeyEntrySelector<'a, K, Q, V, S>
where Q: Equivalent<K> + ToOwned<Owned = K> + Hash + ?Sized,

Takes a reference &Q of a key and returns an RefKeyEntrySelector that can be used to select or insert an entry.

§Example
// Cargo.toml
//
// [dependencies]
// moka = { version = "0.12", features = ["future"] }
// tokio = { version = "1", features = ["rt-multi-thread", "macros" ] }

use moka::future::Cache;

#[tokio::main]
async fn main() {
    let cache: Cache<String, u32> = Cache::new(100);
    let key = "key1".to_string();

    let entry = cache.entry_by_ref(&key).or_insert(3).await;
    assert!(entry.is_fresh());
    assert_eq!(entry.key(), &key);
    assert_eq!(entry.into_value(), 3);

    let entry = cache.entry_by_ref(&key).or_insert(6).await;
    // Not fresh because the value was already in the cache.
    assert!(!entry.is_fresh());
    assert_eq!(entry.into_value(), 3);
}
Source

pub async fn get_with(&self, key: K, init: impl Future<Output = V>) -> V

Returns a clone of the value corresponding to the key. If the value does not exist, resolve the init future and inserts the output.

§Concurrent calls on the same key

This method guarantees that concurrent calls on the same not-existing key are coalesced into one evaluation of the init future. Only one of the calls evaluates its future, and other calls wait for that future to resolve.

The following code snippet demonstrates this behavior:

// Cargo.toml
//
// [dependencies]
// moka = { version = "0.12", features = ["future"] }
// futures-util = "0.3"
// tokio = { version = "1", features = ["rt-multi-thread", "macros" ] }
use moka::future::Cache;
use std::sync::Arc;

#[tokio::main]
async fn main() {
    const TEN_MIB: usize = 10 * 1024 * 1024; // 10MiB
    let cache = Cache::new(100);

    // Spawn four async tasks.
    let tasks: Vec<_> = (0..4_u8)
        .map(|task_id| {
            let my_cache = cache.clone();
            tokio::spawn(async move {
                println!("Task {task_id} started.");

                // Insert and get the value for key1. Although all four async
                // tasks will call `get_with` at the same time, the `init`
                // async block must be resolved only once.
                let value = my_cache
                    .get_with("key1", async move {
                        println!("Task {task_id} inserting a value.");
                        Arc::new(vec![0u8; TEN_MIB])
                    })
                    .await;

                // Ensure the value exists now.
                assert_eq!(value.len(), TEN_MIB);
                assert!(my_cache.get(&"key1").await.is_some());

                println!("Task {task_id} got the value. (len: {})", value.len());
            })
        })
        .collect();

    // Run all tasks concurrently and wait for them to complete.
    futures_util::future::join_all(tasks).await;
}

A Sample Result

  • The init future (async black) was resolved exactly once by task 3.
  • Other tasks were blocked until task 3 inserted the value.
Task 0 started.
Task 3 started.
Task 1 started.
Task 2 started.
Task 3 inserting a value.
Task 3 got the value. (len: 10485760)
Task 0 got the value. (len: 10485760)
Task 1 got the value. (len: 10485760)
Task 2 got the value. (len: 10485760)
§Panics

This method panics when the init future has panicked. When it happens, only the caller whose init future panicked will get the panic (e.g. only task 3 in the above sample). If there are other calls in progress (e.g. task 0, 1 and 2 above), this method will restart and resolve one of the remaining init futures.

Source

pub async fn get_with_by_ref<Q>( &self, key: &Q, init: impl Future<Output = V>, ) -> V
where Q: Equivalent<K> + ToOwned<Owned = K> + Hash + ?Sized,

Similar to get_with, but instead of passing an owned key, you can pass a reference to the key. If the key does not exist in the cache, the key will be cloned to create new entry in the cache.

Source

pub async fn get_with_if( &self, key: K, init: impl Future<Output = V>, replace_if: impl FnMut(&V) -> bool + Send, ) -> V

👎Deprecated since 0.10.0:

Replaced with entry().or_insert_with_if()

TODO: Remove this in v0.13.0. Deprecated, replaced with entry()::or_insert_with_if()

Source

pub async fn optionally_get_with<F>(&self, key: K, init: F) -> Option<V>
where F: Future<Output = Option<V>>,

Returns a clone of the value corresponding to the key. If the value does not exist, resolves the init future, and inserts the value if Some(value) was returned. If None was returned from the future, this method does not insert a value and returns None.

§Concurrent calls on the same key

This method guarantees that concurrent calls on the same not-existing key are coalesced into one evaluation of the init future. Only one of the calls evaluates its future, and other calls wait for that future to resolve.

The following code snippet demonstrates this behavior:

// Cargo.toml
//
// [dependencies]
// moka = { version = "0.12", features = ["future"] }
// futures-util = "0.3"
// reqwest = "0.11"
// tokio = { version = "1", features = ["rt-multi-thread", "macros" ] }
use moka::future::Cache;

// This async function tries to get HTML from the given URI.
async fn get_html(task_id: u8, uri: &str) -> Option<String> {
    println!("get_html() called by task {task_id}.");
    reqwest::get(uri).await.ok()?.text().await.ok()
}

#[tokio::main]
async fn main() {
    let cache = Cache::new(100);

    // Spawn four async tasks.
    let tasks: Vec<_> = (0..4_u8)
        .map(|task_id| {
            let my_cache = cache.clone();
            tokio::spawn(async move {
                println!("Task {task_id} started.");

                // Try to insert and get the value for key1. Although
                // all four async tasks will call `try_get_with`
                // at the same time, get_html() must be called only once.
                let value = my_cache
                    .optionally_get_with(
                        "key1",
                        get_html(task_id, "https://www.rust-lang.org"),
                    ).await;

                // Ensure the value exists now.
                assert!(value.is_some());
                assert!(my_cache.get(&"key1").await.is_some());

                println!(
                    "Task {task_id} got the value. (len: {})",
                    value.unwrap().len()
                );
            })
        })
        .collect();

    // Run all tasks concurrently and wait for them to complete.
    futures_util::future::join_all(tasks).await;
}

A Sample Result

  • get_html() was called exactly once by task 2.
  • Other tasks were blocked until task 2 inserted the value.
Task 1 started.
Task 0 started.
Task 2 started.
Task 3 started.
get_html() called by task 2.
Task 2 got the value. (len: 19419)
Task 1 got the value. (len: 19419)
Task 0 got the value. (len: 19419)
Task 3 got the value. (len: 19419)
§Panics

This method panics when the init future has panicked. When it happens, only the caller whose init future panicked will get the panic (e.g. only task 2 in the above sample). If there are other calls in progress (e.g. task 0, 1 and 3 above), this method will restart and resolve one of the remaining init futures.

Source

pub async fn optionally_get_with_by_ref<F, Q>( &self, key: &Q, init: F, ) -> Option<V>
where F: Future<Output = Option<V>>, Q: Equivalent<K> + ToOwned<Owned = K> + Hash + ?Sized,

Similar to optionally_get_with, but instead of passing an owned key, you can pass a reference to the key. If the key does not exist in the cache, the key will be cloned to create new entry in the cache.

Source

pub async fn try_get_with<F, E>(&self, key: K, init: F) -> Result<V, Arc<E>>
where F: Future<Output = Result<V, E>>, E: Send + Sync + 'static,

Returns a clone of the value corresponding to the key. If the value does not exist, resolves the init future, and inserts the value if Ok(value) was returned. If Err(_) was returned from the future, this method does not insert a value and returns the Err wrapped by std::sync::Arc.

§Concurrent calls on the same key

This method guarantees that concurrent calls on the same not-existing key are coalesced into one evaluation of the init future (as long as these futures return the same error type). Only one of the calls evaluates its future, and other calls wait for that future to resolve.

The following code snippet demonstrates this behavior:

// Cargo.toml
//
// [dependencies]
// moka = { version = "0.12", features = ["future"] }
// futures-util = "0.3"
// reqwest = "0.11"
// tokio = { version = "1", features = ["rt-multi-thread", "macros" ] }
use moka::future::Cache;

// This async function tries to get HTML from the given URI.
async fn get_html(task_id: u8, uri: &str) -> Result<String, reqwest::Error> {
    println!("get_html() called by task {task_id}.");
    reqwest::get(uri).await?.text().await
}

#[tokio::main]
async fn main() {
    let cache = Cache::new(100);

    // Spawn four async tasks.
    let tasks: Vec<_> = (0..4_u8)
        .map(|task_id| {
            let my_cache = cache.clone();
            tokio::spawn(async move {
                println!("Task {task_id} started.");

                // Try to insert and get the value for key1. Although
                // all four async tasks will call `try_get_with`
                // at the same time, get_html() must be called only once.
                let value = my_cache
                    .try_get_with(
                        "key1",
                        get_html(task_id, "https://www.rust-lang.org"),
                    ).await;

                // Ensure the value exists now.
                assert!(value.is_ok());
                assert!(my_cache.get(&"key1").await.is_some());

                println!(
                    "Task {task_id} got the value. (len: {})",
                    value.unwrap().len()
                );
            })
        })
        .collect();

    // Run all tasks concurrently and wait for them to complete.
    futures_util::future::join_all(tasks).await;
}

A Sample Result

  • get_html() was called exactly once by task 2.
  • Other tasks were blocked until task 2 inserted the value.
Task 1 started.
Task 0 started.
Task 2 started.
Task 3 started.
get_html() called by task 2.
Task 2 got the value. (len: 19419)
Task 1 got the value. (len: 19419)
Task 0 got the value. (len: 19419)
Task 3 got the value. (len: 19419)
§Panics

This method panics when the init future has panicked. When it happens, only the caller whose init future panicked will get the panic (e.g. only task 2 in the above sample). If there are other calls in progress (e.g. task 0, 1 and 3 above), this method will restart and resolve one of the remaining init futures.

Source

pub async fn try_get_with_by_ref<F, E, Q>( &self, key: &Q, init: F, ) -> Result<V, Arc<E>>
where F: Future<Output = Result<V, E>>, E: Send + Sync + 'static, Q: Equivalent<K> + ToOwned<Owned = K> + Hash + ?Sized,

Similar to try_get_with, but instead of passing an owned key, you can pass a reference to the key. If the key does not exist in the cache, the key will be cloned to create new entry in the cache.

Source

pub async fn insert(&self, key: K, value: V)

Inserts a key-value pair into the cache.

If the cache has this key present, the value is updated.

Source

pub async fn invalidate<Q>(&self, key: &Q)
where Q: Equivalent<K> + Hash + ?Sized,

Discards any cached value for the key.

If you need to get the value that has been discarded, use the remove method instead.

The key may be any borrowed form of the cache’s key type, but Hash and Eq on the borrowed form must match those for the key type.

Source

pub async fn remove<Q>(&self, key: &Q) -> Option<V>
where Q: Equivalent<K> + Hash + ?Sized,

Discards any cached value for the key and returns a clone of the value.

If you do not need to get the value that has been discarded, use the invalidate method instead.

The key may be any borrowed form of the cache’s key type, but Hash and Eq on the borrowed form must match those for the key type.

Source

pub fn invalidate_all(&self)

Discards all cached values.

This method returns immediately by just setting the current time as the invalidation time. get and other retrieval methods are guaranteed not to return the entries inserted before or at the invalidation time.

The actual removal of the invalidated entries is done as a maintenance task driven by a user thread. For more details, see the Maintenance Tasks section in the crate level documentation.

Like the invalidate method, this method does not clear the historic popularity estimator of keys so that it retains the client activities of trying to retrieve an item.

Source

pub fn invalidate_entries_if<F>( &self, predicate: F, ) -> Result<String, PredicateError>
where F: Fn(&K, &V) -> bool + Send + Sync + 'static,

Discards cached values that satisfy a predicate.

invalidate_entries_if takes a closure that returns true or false. The closure is called against each cached entry inserted before or at the time when this method was called. If the closure returns true that entry will be evicted from the cache.

This method returns immediately by not actually removing the invalidated entries. Instead, it just sets the predicate to the cache with the time when this method was called. The actual removal of the invalidated entries is done as a maintenance task driven by a user thread. For more details, see the Maintenance Tasks section in the crate level documentation.

Also the get and other retrieval methods will apply the closure to a cached entry to determine if it should have been invalidated. Therefore, it is guaranteed that these methods must not return invalidated values.

Note that you must call CacheBuilder::support_invalidation_closures at the cache creation time as the cache needs to maintain additional internal data structures to support this method. Otherwise, calling this method will fail with a PredicateError::InvalidationClosuresDisabled.

Like the invalidate method, this method does not clear the historic popularity estimator of keys so that it retains the client activities of trying to retrieve an item.

Source

pub fn iter(&self) -> Iter<'_, K, V> ⓘ

Creates an iterator visiting all key-value pairs in arbitrary order. The iterator element type is (Arc<K>, V), where V is a clone of a stored value.

Iterators do not block concurrent reads and writes on the cache. An entry can be inserted to, invalidated or evicted from a cache while iterators are alive on the same cache.

Unlike the get method, visiting entries via an iterator do not update the historic popularity estimator or reset idle timers for keys.

§Guarantees

In order to allow concurrent access to the cache, iterator’s next method does not guarantee the following:

  • It does not guarantee to return a key-value pair (an entry) if its key has been inserted to the cache after the iterator was created.
    • Such an entry may or may not be returned depending on key’s hash and timing.

and the next method guarantees the followings:

  • It guarantees not to return the same entry more than once.
  • It guarantees not to return an entry if it has been removed from the cache after the iterator was created.
    • Note: An entry can be removed by following reasons:
      • Manually invalidated.
      • Expired (e.g. time-to-live).
      • Evicted as the cache capacity exceeded.
§Examples
// Cargo.toml
//
// [dependencies]
// moka = { version = "0.12", features = ["future"] }
// tokio = { version = "1", features = ["rt-multi-thread", "macros" ] }
use moka::future::Cache;

#[tokio::main]
async fn main() {
    let cache = Cache::new(100);
    cache.insert("Julia", 14).await;

    let mut iter = cache.iter();
    let (k, v) = iter.next().unwrap(); // (Arc<K>, V)
    assert_eq!(*k, "Julia");
    assert_eq!(v, 14);

    assert!(iter.next().is_none());
}
Source

pub async fn run_pending_tasks(&self)

Performs any pending maintenance operations needed by the cache.

Trait Implementations§

Source§

impl<K, V> Deref for ServerCache<K, V>
where K: Hash + Eq + Send + Sync + 'static, V: Clone + Send + Sync + 'static,

Source§

type Target = Cache<K, V>

The resulting type after dereferencing.
Source§

fn deref(&self) -> &Self::Target

Dereferences the value.

Auto Trait Implementations§

§

impl<K, V> !Freeze for ServerCache<K, V>

§

impl<K, V> !RefUnwindSafe for ServerCache<K, V>

§

impl<K, V> !UnwindSafe for ServerCache<K, V>

§

impl<K, V> Send for ServerCache<K, V>
where OnceLock<Cache<K, V>>: Send,

§

impl<K, V> Sync for ServerCache<K, V>
where OnceLock<Cache<K, V>>: Sync,

§

impl<K, V> Unpin for ServerCache<K, V>
where OnceLock<Cache<K, V>>: Unpin,

§

impl<K, V> UnsafeUnpin for ServerCache<K, V>
where OnceLock<Cache<K, V>>: UnsafeUnpin,

Blanket Implementations§

Source§

impl<T> Any for T
where T: 'static + ?Sized,

Source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
Source§

impl<T> Borrow<T> for T
where T: ?Sized,

Source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
Source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

Source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
Source§

impl<T> From<T> for T

Source§

fn from(t: T) -> T

Returns the argument unchanged.

Source§

impl<T> InitializeFromFunction<T> for T

Source§

fn initialize_from_function(f: fn() -> T) -> T

Create an instance of this type from an initialization function
Source§

impl<T> Instrument for T

Source§

fn instrument(self, span: Span) -> Instrumented<Self> ⓘ

Instruments this type with the provided Span, returning an Instrumented wrapper. Read more
Source§

fn in_current_span(self) -> Instrumented<Self> ⓘ

Instruments this type with the current Span, returning an Instrumented wrapper. Read more
Source§

impl<T, U> Into<U> for T
where U: From<T>,

Source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

Source§

impl<T> Pointable for T

Source§

const ALIGN: usize

The alignment of pointer.
Source§

type Init = T

The type for initializers.
Source§

unsafe fn init(init: <T as Pointable>::Init) -> usize

Initializes a with the given initializer. Read more
Source§

unsafe fn deref<'a>(ptr: usize) -> &'a T

Dereferences the given pointer. Read more
Source§

unsafe fn deref_mut<'a>(ptr: usize) -> &'a mut T

Mutably dereferences the given pointer. Read more
Source§

unsafe fn drop(ptr: usize)

Drops the object pointed to by the given pointer. Read more
Source§

impl<P, T> Receiver for P
where P: Deref<Target = T> + ?Sized, T: ?Sized,

Source§

type Target = T

🔬This is a nightly-only experimental API. (arbitrary_self_types)
The target type on which the method may be called.
Source§

impl<Ret> SpawnIfAsync<(), Ret> for Ret

Source§

fn spawn(self) -> Ret

Spawn the value into the dioxus runtime if it is an async block
Source§

impl<T, O> SuperFrom<T> for O
where O: From<T>,

Source§

fn super_from(input: T) -> O

Convert from a type to another type.
Source§

impl<T, O, M> SuperInto<O, M> for T
where O: SuperFrom<T, M>,

Source§

fn super_into(self) -> O

Convert from a type to another type.
Source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

Source§

type Error = !

The type returned in the event of a conversion error.
Source§

fn try_from(value: U) -> Result<T, !>

Performs the conversion.
Source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

Source§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
Source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.
Source§

impl<T> WithSubscriber for T

Source§

fn with_subscriber<S>(self, subscriber: S) -> WithDispatch<Self> ⓘ
where S: Into<Dispatch>,

Attaches the provided Subscriber to this type, returning a WithDispatch wrapper. Read more
Source§

fn with_current_subscriber(self) -> WithDispatch<Self> ⓘ

Attaches the current default Subscriber to this type, returning a WithDispatch wrapper. Read more