1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
use crate*;
/// Metadata structure for an append-only storage entry.
///
/// This structure stores metadata associated with each entry in the append-only storage.
/// It includes a hash of the key for quick lookups, an offset pointing to the previous
/// entry in the chain, and a checksum for integrity verification.
///
/// ## Entry Storage Layout
///
/// Aligned entry (non-tombstone):
///
/// | Offset Range | Field | Size (Bytes) | Description |
/// |----------------|--------------------|--------------|-----------------------------------|
/// | `P .. P+pad` | Pre-Pad (optional) | `pad` | Zero bytes to align payload start |
/// | `P+pad .. N` | Payload | `N-(P+pad)` | Variable-length data |
/// | `N .. N+8` | Key Hash | `8` | 64-bit XXH3 key hash |
/// | `N+8 .. N+16` | Prev Offset | `8` | Absolute offset of previous tail |
/// | `N+16 .. N+20` | Checksum | `4` | CRC32C of payload |
///
/// Where:
/// - `pad = (A - (prev_tail % A)) & (A - 1)`, `A = PAYLOAD_ALIGNMENT`.
/// - The next entry starts at `N + 20`.
///
/// Tombstone (deletion marker):
///
/// | Offset Range | Field | Size (Bytes) | Description |
/// |---------------|----------|--------------|------------------------|
/// | `T .. T+1` | Payload | `1` | Single byte `0x00` |
/// | `T+1 .. T+21` | Metadata | `20` | Key hash, prev, crc32c |
///
/// Notes:
/// - Using the previous tail in `Prev Offset` lets us insert pre-pad while
/// keeping chain traversal unambiguous.
/// - Readers compute `payload_start = prev_offset + prepad_len(prev_offset)`
/// and use the current metadata position as `payload_end`.
///
/// <img src="https://github.com/jzombie/rust-simd-r-drive/blob/main/assets/storage-layout.png" alt="Storage Layout" />
///
/// ## Notes
/// - The `prev_offset` forms a **backward-linked chain** for each key.
/// - The checksum is **not cryptographically secure** but serves as a quick integrity check.
/// - The first entry for a key has `prev_offset = 0`, indicating no previous version.