Crate cachet

Expand description

A composable, multi-tier caching library with stampede protection, background refresh, and structured telemetry.

§Why Multi-Tier Caching?

A single cache is a single point of failure and a capacity ceiling. Multi-tier caching layers fast, small caches in front of slower, larger ones:

L1 (primary) - an in-process memory cache: microsecond latency, bounded capacity, evicts under pressure.
L2 (fallback) - a remote or larger cache: millisecond latency, much larger capacity, survives process restarts.

On a miss in L1, cachet transparently queries L2 and optionally promotes the value back into L1 so the next request is fast again. The result is lower average latency, reduced load on the backing store, and resilience when either tier is temporarily unavailable.

§Why Background Refresh?

TTL-based expiration causes a synchronous miss every time an entry ages out: the next caller blocks while the value is recomputed. Background refresh (time-to-refresh, TTR) decouples freshness from latency:

While an entry is still within its TTR, all callers receive the cached value immediately (a “refresh hit”).
Once the TTR elapses, the next caller still receives the stale value, but a background task is spawned to pull a fresh value from the fallback tier.
Subsequent callers continue to hit the cache while the refresh happens, so latency never spikes.

Use TimeToRefresh together with a fallback tier to enable this pattern.

§Cache Stampede Protection

A cache stampede (also called a thundering herd) occurs when many concurrent requests all miss the cache at the same time - for example, after a cold start or after a popular entry expires. Every request independently computes the value, spiking load on the backing store.

cachet avoids this with request coalescing via the uniflight crate: when stampede protection is enabled, all concurrent requests for the same key are merged so that only one computes the value. The rest wait and share the result, including any error. Enable it with CacheBuilder::stampede_protection.

§Flexibility

cachet is designed to adapt to your infrastructure rather than the other way around:

Any storage backend - implement CacheTier to plug in Redis, Memcached, a database, or any other store.
Service middleware - with the service feature, any Service<CacheOperation> becomes a CacheTier, so you can compose retry, timeout, and circuit-breaker middleware around your storage using standard Tower or layered patterns.
Dynamic dispatch - the builder type-erases the storage tier into a DynamicCache<K, V>, so all builders produce the same Cache<K, V> output type regardless of the underlying storage or tier composition.
Configurable insert policy - choose whether, and under what conditions, values are inserted into a tier (InsertPolicy).
Clock injection - all time-based logic (TTL, TTR, timestamps) goes through a tick::Clock, making caches fully controllable in tests without sleeping.

§Why Use This Instead of Moka/Other Caches?

Moka (and similar crates) are excellent single-tier in-process caches. cachet builds on top of them and adds:

Feature	Moka	`cachet`
In-process memory cache	✅	✅ (via `cachet_memory`)
Multi-tier / fallback	❌	✅
Stampede protection	❌	✅
Background refresh	❌	✅
Service middleware integration	❌	✅
Structured telemetry	❌	✅
Pluggable storage backends	❌	✅
Clock injection for testing	❌	✅

If you only need a single in-process cache with no telemetry requirements, moka directly may be simpler. If you need any of the above, cachet is the right choice.

§Major Types

Type	Description
`Cache`	The user-facing cache. Wraps any `CacheTier` with `get`, `insert`, `invalidate`, `clear`, `get_or_insert`, `try_get_or_insert`, and `optionally_get_or_insert`.
`CacheBuilder`	Builder for `Cache`. Configure storage, TTL, name, telemetry, fallback, insert policy, stampede protection, and background refresh.
`CacheEntry<V>`	A value together with an optional cached-at timestamp and TTL. Returned by all `get` operations.
`CacheTier`	The core trait for storage backends. Implement this to add your own storage.
`InsertPolicy`	Decides whether a value should be inserted into a tier.
`TimeToRefresh`	Configures background refresh: how stale an entry must be before a background task refreshes it.
`Error`	The error type returned by all fallible cache operations.

§How Tiers Compose

Tiers are composed at build time using the builder:

Cache::builder::<K, V>(clock)
    .memory()                          // L1: fast in-process store
    .ttl(Duration::from_secs(30))      // entries expire from L1 after 30 s
    .fallback(                         // on L1 miss, consult L2
        Cache::builder::<K, V>(clock)
            .memory()                  // L2: a second in-process store (or a remote service)
            .ttl(Duration::from_secs(300))
    )
    .insert_policy(InsertPolicy::always())  // control when values are inserted into L1
    .time_to_refresh(TimeToRefresh::new(Duration::from_secs(20), spawner))  // refresh L1 in background
    .build()

On a get:

Check L1. If hit and not stale, return immediately.
If hit but stale (TTR elapsed), return the stale value and spawn a background task to fetch from L2 and repopulate L1.
If miss or expired (TTL elapsed), check L2. If found, optionally promote to L1, then return.
If both miss, return Ok(None).

Note: expired entries are not automatically removed from storage. The wrapper uses lazy expiration - it returns None but leaves cleanup to the storage backend (e.g. moka built-in eviction).

TO-DO add an ExpirationPolicy that would make this configurable.

Invalidation and clear are sent to all tiers concurrently.

§Companion Crates

cachet is the main entry point. The ecosystem is split into focused crates:

Crate	Purpose
`cachet_tier`	Core `CacheTier` trait, `CacheEntry`, `Error`, and `MockCache` for testing.
`cachet_memory`	In-process memory cache backed by moka (`TinyLFU` eviction).
`cachet_service`	Adapters between the `CacheTier` trait and the `layered::Service` / Tower service patterns.

You rarely need to depend on companion crates directly - cachet re-exports the most commonly used types from all of them.

§Cargo Features

Feature	Default	Description
`memory`	✅	Enables `InMemoryCache` and the `.memory()` builder method via `cachet_memory`.
`logs`	❌	Enables structured `tracing` log events for every cache operation. Subscribe via `telemetry::attributes` constants.
`service`	❌	Enables `ServiceAdapter`, `CacheServiceExt`, and `CacheOperation`/`CacheResponse` types for service middleware integration.
`serialize`	❌	Enables `.serialize()` on builders for automatic postcard serialization of keys and values to `BytesView`.
`test-util`	❌	Enables `MockCache`, frozen-clock utilities, and other test helpers.

§Examples

§Basic In-Memory Cache

use cachet::{Cache, CacheEntry};
use tick::Clock;

let clock = Clock::new_tokio();
let cache: Cache<String, i32> = Cache::builder(clock).memory().build();

cache.insert("key".to_string(), CacheEntry::new(42)).await?;
let value = cache.get("key").await?;
assert_eq!(*value.unwrap().value(), 42);

§Multi-Tier Cache with Fallback

use std::time::Duration;

use cachet::Cache;
use tick::Clock;

let clock = Clock::new_tokio();
let l2 = Cache::builder::<String, String>(clock.clone()).memory();

let cache = Cache::builder::<String, String>(clock)
    .memory()
    .ttl(Duration::from_secs(60))
    .fallback(l2)
    .build();

§Serialization Boundary

When a fallback tier operates on serialized bytes (e.g., Redis), use .serialize() to add a postcard serialization boundary. Keys and values are automatically serialized to BytesView before reaching the fallback tier, and deserialized on the way back.

use cachet::{Cache, FallbackPromotionPolicy};
use tick::Clock;

let clock = Clock::new_tokio();
let remote = Cache::builder::<bytesbuf::BytesView, bytesbuf::BytesView>(clock.clone()).memory();

let cache = Cache::builder::<String, String>(clock)
    .memory()
    .serialize()
    .fallback(remote)
    .promotion_policy(FallbackPromotionPolicy::always())
    .build();

// Keys and values are String on the outside, BytesView in the fallback tier.
cache.insert("key".to_string(), "value".to_string()).await?;

§Telemetry

Cachet provides two complementary telemetry channels:

§Tracing events

Enable with the logs feature and .enable_logs() on the cache builder. Each tier outcome and operation completion emits a structured tracing event.

Tier events carry cache.name, cache.event, and cache.duration_ns. Operation-complete events carry cache.name, cache.operation, cache.duration_ns, and cache.coalesced.

Use telemetry::attributes constants to filter and match events in a custom tracing_subscriber::Layer:

use cachet::telemetry::attributes;

// Filter by tracing target prefix
let filter = tracing_subscriber::filter::Targets::new()
    .with_target(attributes::TARGET, tracing::Level::DEBUG);

// Match specific events in a Visit impl
if event_value == attributes::EVENT_HIT { /* cache hit */ }

See the telemetry_subscriber example for a complete demonstration.

§Event types

Level	Events
ERROR	`cache.get_error`, `cache.insert_error`, `cache.invalidate_error`, `cache.clear_error`
INFO	`cache.expired`, `cache.refresh_miss`, `cache.inserted`, `cache.insert_rejected`, `cache.invalidated`, `cache.eviction`
DEBUG	`cache.hit`, `cache.miss`, `cache.refresh_hit`, `cache.cleared`

§Event handler callback API

Register a CacheEventHandler via .event_handler(handler) on the cache builder to receive typed CacheTierEvent and CacheOperationEvent callbacks. Events carry a request_id for correlating tier outcomes with their parent operation. Works independently of the logs feature.

See the telemetry_accumulator example for a DashMap-based accumulation pattern.

Modules§

telemetry: Cache telemetry integration.

Structs§

Cache: The main cache type providing user-facing API with optional stampede protection.
CacheBuilder: Builder for constructing a cache with a single tier.
CacheEntry: A cached value with associated metadata.
CacheOperationEvent: Data from a completed top-level cache operation.
CacheTierEvent: Data from a per-tier cache operation.
DynamicCache: A cloneable dynamic cache tier with type erasure.
Error: An error from a cache operation.
FallbackBuilder: Builder for a cache with fallback tiers.
GetRequestservice: Request to get a value from the cache.
InMemoryCachememory: A concurrent in-memory cache tier.
InsertPolicy: Policy that determines when values should be inserted into a cache tier.
InsertRequestservice: Request to insert a value into the cache.
InvalidateRequestservice: Request to invalidate (remove) a value from the cache.
MockCachetest-util: A configurable mock cache for testing.
ServiceAdapterservice: Adapter that converts a Service<CacheOperation> into a CacheTier.
SizeError: An error from a CacheTier::len operation.
TimeToRefresh: Configuration for background cache refresh.
TransformBuilder: Builder that introduces a type-conversion boundary in the cache pipeline.
TransformCodec: A boxed-closure codec for custom bidirectional transforms (values).
TransformEncoder: A boxed-closure encoder for custom one-directional transforms (keys).

Enums§

CacheOptest-util: Recorded cache operation with full context.
CacheOperationservice: A cache operation request.
CacheResponseservice: Response from a cache operation.
DecodeOutcome: The result of a decode operation.

Traits§

CacheEventHandler: Trait for consuming cachet telemetry events.
CacheServiceExtservice: Extension trait providing ergonomic cache methods for any Service<CacheOperation>.
CacheTier: Trait for cache tier implementations.
CacheTierBuilder: A builder that can produce a cache tier.
Codec: A bidirectional codec that converts between types A and B.
Encoder: A one-directional encoder that converts values from type From to type To.

Functions§

infallible: Wraps an infallible closure taking a reference so it can be used where a fallible one is expected.
infallible_owned: Wraps an infallible closure taking an owned value so it can be used where a fallible one is expected.

Type Aliases§

CacheName: Type alias for cache names used in telemetry.
Result: A specialized Result type for cache operations.

Crate cachet

Crate cachet Copy item path

§Why Multi-Tier Caching?

§Why Background Refresh?

§Cache Stampede Protection

§Flexibility

§Why Use This Instead of Moka/Other Caches?

§Major Types

§How Tiers Compose

§Companion Crates

§Cargo Features

§Examples

§Basic In-Memory Cache

§Multi-Tier Cache with Fallback

§Serialization Boundary

§Telemetry

§Tracing events

§Event types

§Event handler callback API

Modules§

Structs§

Enums§

Traits§

Functions§

Type Aliases§

Crate cachet