1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
//! Sequential write policy implementation (Write-Through).
//!
//! This policy writes to L1 first, then writes to L2 sequentially.
//! It's the classic write-through strategy with strong consistency.
use async_trait;
use ;
use Future;
use CompositionWritePolicy;
use crateBackendError;
/// Sequential write policy: Write to L1, then L2 (write-through).
///
/// This is the default and most common strategy. It provides:
/// - Strong consistency (both layers updated before returning)
/// - Atomic updates from caller's perspective
/// - Graceful degradation if L1 succeeds but L2 fails
///
/// # Behavior
/// 1. Call `write_l1(key)`
/// - If fails: Return error immediately (don't write to L2)
/// - If succeeds: Continue to L2
/// 2. Call `write_l2(key)`
/// - If fails: Return error (L1 has data, L2 doesn't - inconsistent state)
/// - If succeeds: Return success
///
/// # Consistency Guarantees
///
/// **Success case (`Ok(())`)**: Both L1 and L2 have been updated successfully.
///
/// **Failure cases (`Err`)**:
/// - **L1 write failed**: Neither layer updated - cache remains consistent
/// - **L2 write failed**: L1 updated, L2 not updated - **inconsistent state**
///
/// ## Inconsistent State Handling
///
/// When L1 succeeds but L2 fails, the cache enters an inconsistent state where:
/// - **L1 contains the new value** - subsequent reads from this client will hit L1
/// - **L2 may contain stale data or no data** - other clients may see stale values
/// - **The error is logged** with tracing::error for monitoring
///
/// ### Mitigation Strategies:
///
/// 1. **Accept inconsistency** - If L1 is much faster and L2 failures are rare,
/// the inconsistency may be acceptable as L1 will mask it for most reads
///
/// 2. **Retry logic** - Implement retry at application level or use a RetryBackend
/// wrapper to retry failed L2 writes
///
/// 3. **Use OptimisticParallelWritePolicy** - Succeeds if either L1 or L2 succeeds,
/// providing better availability at the cost of potential inconsistency
///
/// 4. **Monitor and alert** - Track L2 write failures via metrics and investigate
/// persistent failures that could indicate L2 capacity or connectivity issues
///
/// ### When L2 Failures Are Acceptable:
/// - L2 is a persistent cache for cold starts (L1 mask inconsistency during normal operation)
/// - Cache data is regeneratable from source of truth
/// - Read-heavy workload where L1 hit rate is very high
;