Quick Cache

Lightweight and high performance concurrent cache optimized for low cache overhead.

• Small overhead compared to a concurrent hash table
• Scan resistent and high hit rate caching policy
• User defined weight per item
• Scales well with the number of threads
• Doesn't use background threads
• 100% safe code in the crate
• Small dependency tree

The implementation is optimized for use cases where the cache access times and overhead can add up to be a significant cost. Features like: time to live, event listeners and others; are not current implemented in Quick Cache. If you need these features you may want to take a look at the Moka crate.

Examples

Basic usage

use quick_cache::unsync::Cache;

fn main() {
    let cache = Cache::new(5);
    cache.insert("square", "blue");
    cache.insert("circle", "black");
    assert_eq!(*cache.get(&"square").unwrap(), "blue");
    assert_eq!(*cache.get(&"circle").unwrap(), "black");
}

A cache with custom item weights. In this case according to the string length of the value.

use quick_cache::{Weighter, sync::Cache};

#[derive(Clone)]
struct StringWeighter;

impl Weighter<u64, (), String> for StringWeighter {
    fn weight(&self, _key: &u64, _qey: &(), val: &String) -> NonZeroU32 {
        NonZeroU32::new(val.len().clamp(1, u32::MAX as usize) as u32).unwrap()
    }
}

fn main() {
    let cache = Cache::with_weighter(100, 100_000, StringWeighter);
    cache.insert(1, "1".to_string());
    cache.insert(54, "54".to_string());
    cache.insert(1000, "1000".to_string());
    assert_eq!(*cache.get(&"1000").unwrap(), "1000");
}

Two keys variant (KQCache), which allows the caller to avoid making expensive keys on gets.

use quick_cache::sync::KQCache;

fn main() {
    let cache = KQCache::new(5);
    cache.insert("square".to_string(), 2022, "blue");
    cache.insert("square".to_string(), 2023, "black");
    // In a "non-versioned" cache would use a tuple (String, u32) as keys, which could force
    // the caller to clone/allocate the string in order to form the tuple key.
    // That is not the case with a versioned cache.
    assert_eq!(cache.get("square", &2022).unwrap(), "blue");
}

Benchmarks

Since this crate is performance oriented it needs some comparisons. That said, benchmarks can be misleading so take everything with a pinch of salt.

Benchmarks performed with a modified version of mokabench in a x64 Linux OS + AMD 4900HS CPU system.

Trace 1 (S3 from the Arc paper)

Trace 2 (DS1 from the Arc paper)

Trace 3 (OLTP from the Arc paper)

This is a much smaller dataset and since Moka Dash Cache and Sync Cache over-allocate several thousants of entries (internal write buffer) they are not comparable in this test.

References

• CLOCK-Pro: An Effective Improvement of the CLOCK Replacement. (PDF)
• Caffeine (source code)
• Cache2k (source code)

License

This project is licensed under the MIT license.