1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT License.
//! A composable, multi-tier caching library with stampede protection, background
//! refresh, and structured telemetry.
//!
//! # Why Multi-Tier Caching?
//!
//! A single cache is a single point of failure and a capacity ceiling. Multi-tier
//! caching layers fast, small caches in front of slower, larger ones:
//!
//! - **L1 (primary)** - an in-process memory cache: microsecond latency, bounded
//! capacity, evicts under pressure.
//! - **L2 (fallback)** - a remote or larger cache: millisecond latency, much larger
//! capacity, survives process restarts.
//!
//! On a miss in L1, `cachet` transparently queries L2 and optionally *promotes* the
//! value back into L1 so the next request is fast again. The result is lower average
//! latency, reduced load on the backing store, and resilience when either tier is
//! temporarily unavailable.
//!
//! # Why Background Refresh?
//!
//! TTL-based expiration causes a *synchronous* miss every time an entry ages out:
//! the next caller blocks while the value is recomputed. Background refresh
//! (time-to-refresh, TTR) decouples freshness from latency:
//!
//! - While an entry is still within its TTR, all callers receive the cached value
//! immediately (a "refresh hit").
//! - Once the TTR elapses, the *next* caller still receives the stale value, but a
//! background task is spawned to pull a fresh value from the fallback tier.
//! - Subsequent callers continue to hit the cache while the refresh happens, so
//! latency never spikes.
//!
//! Use [`TimeToRefresh`] together with a fallback tier to enable this pattern.
//!
//! # Cache Stampede Protection
//!
//! A **cache stampede** (also called a thundering herd) occurs when many concurrent
//! requests all miss the cache at the same time - for example, after a cold start or
//! after a popular entry expires. Every request independently computes the value,
//! spiking load on the backing store.
//!
//! `cachet` avoids this with *request coalescing* via the [`uniflight`] crate: when
//! stampede protection is enabled, all concurrent requests for the same key are merged
//! so that only one computes the value. The rest wait and share the result, including
//! any error. Enable it with [`CacheBuilder::stampede_protection`].
//!
//! # Flexibility
//!
//! `cachet` is designed to adapt to your infrastructure rather than the other way
//! around:
//!
//! - **Any storage backend** - implement [`CacheTier`] to plug in Redis, Memcached,
//! a database, or any other store.
//! - **Service middleware** - with the `service` feature, any
//! `Service<CacheOperation>` becomes a `CacheTier`, so you can compose retry,
//! timeout, and circuit-breaker middleware around your storage using standard Tower
//! or `layered` patterns.
//! - **Dynamic dispatch** - the builder type-erases the storage tier into a
//! [`DynamicCache<K, V>`], so all builders produce the same `Cache<K, V>`
//! output type regardless of the underlying storage or tier composition.
//! - **Configurable insert policy** - choose whether, and under what conditions,
//! values are inserted into a tier ([`InsertPolicy`]).
//! - **Clock injection** - all time-based logic (TTL, TTR, timestamps) goes through
//! a [`tick::Clock`], making caches fully controllable in tests without sleeping.
//!
//! # Why Use This Instead of Moka/Other Caches?
//!
//! Moka (and similar crates) are excellent single-tier in-process caches. `cachet`
//! builds on top of them and adds:
//!
//! | Feature | Moka | `cachet` |
//! |---|---|---|
//! | In-process memory cache | ✅ | ✅ (via `cachet_memory`) |
//! | Multi-tier / fallback | ❌ | ✅ |
//! | Stampede protection | ❌ | ✅ |
//! | Background refresh | ❌ | ✅ |
//! | Service middleware integration | ❌ | ✅ |
//! | Structured telemetry | ❌ | ✅ |
//! | Pluggable storage backends | ❌ | ✅ |
//! | Clock injection for testing | ❌ | ✅ |
//!
//! If you only need a single in-process cache with no telemetry requirements,
//! `moka` directly may be simpler. If you need any of the above, `cachet` is the
//! right choice.
//!
//! # Major Types
//!
//! | Type | Description |
//! |---|---|
//! | [`Cache`] | The user-facing cache. Wraps any `CacheTier` with `get`, `insert`, `invalidate`, `clear`, `get_or_insert`, `try_get_or_insert`, and `optionally_get_or_insert`. |
//! | [`CacheBuilder`] | Builder for `Cache`. Configure storage, TTL, name, telemetry, fallback, insert policy, stampede protection, and background refresh. |
//! | [`CacheEntry<V>`] | A value together with an optional cached-at timestamp and TTL. Returned by all `get` operations. |
//! | [`CacheTier`] | The core trait for storage backends. Implement this to add your own storage. |
//! | [`InsertPolicy`] | Decides whether a value should be inserted into a tier. |
//! | [`TimeToRefresh`] | Configures background refresh: how stale an entry must be before a background task refreshes it. |
//! | [`Error`] | The error type returned by all fallible cache operations. |
//!
//! # How Tiers Compose
//!
//! Tiers are composed at build time using the builder:
//!
//! ```text
//! Cache::builder::<K, V>(clock)
//! .memory() // L1: fast in-process store
//! .ttl(Duration::from_secs(30)) // entries expire from L1 after 30 s
//! .fallback( // on L1 miss, consult L2
//! Cache::builder::<K, V>(clock)
//! .memory() // L2: a second in-process store (or a remote service)
//! .ttl(Duration::from_secs(300))
//! )
//! .insert_policy(InsertPolicy::always()) // control when values are inserted into L1
//! .time_to_refresh(TimeToRefresh::new(Duration::from_secs(20), spawner)) // refresh L1 in background
//! .build()
//! ```
//!
//! On a `get`:
//! 1. Check L1. If hit and not stale, return immediately.
//! 2. If hit but stale (TTR elapsed), return the stale value *and* spawn a background
//! task to fetch from L2 and repopulate L1.
//! 3. If miss or expired (TTL elapsed), check L2. If found, optionally promote to L1,
//! then return.
//! 4. If both miss, return `Ok(None)`.
//!
//! **Note:** expired entries are not automatically removed from storage. The wrapper
//! uses lazy expiration - it returns `None` but leaves cleanup to the storage
//! backend (e.g. moka built-in eviction).
//!
//! TO-DO add an `ExpirationPolicy` that would make this configurable.
//!
//! Invalidation and clear are sent to **all** tiers concurrently.
//!
//! # Companion Crates
//!
//! `cachet` is the main entry point. The ecosystem is split into focused crates:
//!
//! | Crate | Purpose |
//! |---|---|
//! | [`cachet_tier`] | Core `CacheTier` trait, `CacheEntry`, `Error`, and `MockCache` for testing. |
//! | [`cachet_memory`] | In-process memory cache backed by [moka](https://docs.rs/moka) (`TinyLFU` eviction). |
//! | [`cachet_service`] | Adapters between the `CacheTier` trait and the `layered::Service` / Tower service patterns. |
//!
//! You rarely need to depend on companion crates directly - `cachet` re-exports the
//! most commonly used types from all of them.
//!
//! # Cargo Features
//!
//! | Feature | Default | Description |
//! |---|---|---|
//! | `memory` | ✅ | Enables `InMemoryCache` and the `.memory()` builder method via `cachet_memory`. |
//! | `logs` | ❌ | Enables structured `tracing` log events for every cache operation. Subscribe via [`telemetry::attributes`] constants. |
//! | `service` | ❌ | Enables `ServiceAdapter`, `CacheServiceExt`, and `CacheOperation`/`CacheResponse` types for service middleware integration. |
//! | `serialize` | ❌ | Enables `.serialize()` on builders for automatic postcard serialization of keys and values to `BytesView`. |
//! | `test-util` | ❌ | Enables `MockCache`, frozen-clock utilities, and other test helpers. |
//!
//! # Examples
//!
//! ## Basic In-Memory Cache
//!
//! ```no_run
//! use cachet::{Cache, CacheEntry};
//! use tick::Clock;
//! # async {
//!
//! let clock = Clock::new_tokio();
//! let cache: Cache<String, i32> = Cache::builder(clock).memory().build();
//!
//! cache.insert("key".to_string(), CacheEntry::new(42)).await?;
//! let value = cache.get("key").await?;
//! assert_eq!(*value.unwrap().value(), 42);
//! # Ok::<(), cachet::Error>(())
//! # };
//! ```
//!
//! ## Multi-Tier Cache with Fallback
//!
//! ```no_run
//! use std::time::Duration;
//!
//! use cachet::Cache;
//! use tick::Clock;
//! # async {
//!
//! let clock = Clock::new_tokio();
//! let l2 = Cache::builder::<String, String>(clock.clone()).memory();
//!
//! let cache = Cache::builder::<String, String>(clock)
//! .memory()
//! .ttl(Duration::from_secs(60))
//! .fallback(l2)
//! .build();
//! # };
//! ```
//!
//! ## Serialization Boundary
//!
//! When a fallback tier operates on serialized bytes (e.g., Redis), use `.serialize()`
//! to add a postcard serialization boundary. Keys and values are automatically serialized
//! to [`BytesView`](bytesbuf::BytesView) before reaching the fallback tier, and
//! deserialized on the way back.
//!
//! ```ignore
//! use cachet::{Cache, FallbackPromotionPolicy};
//! use tick::Clock;
//! # async {
//!
//! let clock = Clock::new_tokio();
//! let remote = Cache::builder::<bytesbuf::BytesView, bytesbuf::BytesView>(clock.clone()).memory();
//!
//! let cache = Cache::builder::<String, String>(clock)
//! .memory()
//! .serialize()
//! .fallback(remote)
//! .promotion_policy(FallbackPromotionPolicy::always())
//! .build();
//!
//! // Keys and values are String on the outside, BytesView in the fallback tier.
//! cache.insert("key".to_string(), "value".to_string()).await?;
//! # Ok::<(), cachet::Error>(())
//! # };
//! ```
//!
//! # Telemetry
//!
//! Cachet provides two complementary telemetry channels:
//!
//! ## Tracing events
//!
//! Enable with the `logs` feature and `.enable_logs()` on the cache builder.
//! Each tier outcome and operation completion emits a structured [`tracing`] event.
//!
//! **Tier events** carry `cache.name`, `cache.event`, and `cache.duration_ns`.
//! **Operation-complete events** carry `cache.name`, `cache.operation`,
//! `cache.duration_ns`, and `cache.coalesced`.
//!
//! Use [`telemetry::attributes`] constants to filter and match events in a
//! custom `tracing_subscriber::Layer`:
//!
//! ```ignore
//! use cachet::telemetry::attributes;
//!
//! // Filter by tracing target prefix
//! let filter = tracing_subscriber::filter::Targets::new()
//! .with_target(attributes::TARGET, tracing::Level::DEBUG);
//!
//! // Match specific events in a Visit impl
//! if event_value == attributes::EVENT_HIT { /* cache hit */ }
//! ```
//!
//! See the `telemetry_subscriber` example for a complete demonstration.
//!
//! ### Event types
//!
//! | Level | Events |
//! |-------|--------|
//! | ERROR | `cache.get_error`, `cache.insert_error`, `cache.invalidate_error`, `cache.clear_error` |
//! | INFO | `cache.expired`, `cache.refresh_miss`, `cache.inserted`, `cache.insert_rejected`, `cache.invalidated`, `cache.eviction` |
//! | DEBUG | `cache.hit`, `cache.miss`, `cache.refresh_hit`, `cache.cleared` |
//!
//! ## Event handler callback API
//!
//! Register a [`CacheEventHandler`] via
//! `.event_handler(handler)` on the cache builder to receive typed
//! [`CacheTierEvent`] and
//! [`CacheOperationEvent`] callbacks.
//! Events carry a `request_id` for correlating tier outcomes with their parent
//! operation. Works independently of the `logs` feature.
//!
//! See the `telemetry_accumulator` example for a DashMap-based accumulation pattern.
pub use ;
pub use ;
pub use InMemoryCache;
pub use ;
pub use DynamicCache;
pub use ;
pub use ;
pub use InsertPolicy;
pub use TimeToRefresh;
pub use ;
pub use ;