dig_blockstore/store.rs
1//! `BlockStore` — RocksDB-backed persistent block and chain state.
2//!
3//! # Architecture
4//!
5//! This module is the primary entry point for all block persistence in the DIG L2
6//! network. It mirrors the storage patterns established by the **Chia blockchain**'s
7//! `BlockStore` in `chia-blockchain/chia/consensus/block_store.py`, adapted for Rust
8//! and RocksDB instead of Python/SQLite:
9//!
10//! | Chia Python pattern | DIG Rust equivalent |
11//! |---------------------|---------------------|
12//! | `full_blocks` SQLite table | [`CF_BLOCKS`](crate::CF_BLOCKS) column family (zstd-compressed bincode) |
13//! | `block_records` SQLite table | In-memory [`BlockRecord`](crate::BlockRecord) cache (never persisted; [`TYP-004`]) |
14//! | `block_cache: LRUCache[bytes32, FullBlock]` | [`ShardedBlockCache`](crate::cache::sharded::ShardedBlockCache) (sharded LRU, [`CAC-001`]) |
15//! | `current_peak` single-row | [`META_TIP`](crate::META_TIP) in [`CF_METADATA`](crate::CF_METADATA) (40-byte [`ChainTip`]) |
16//! | `INSERT OR IGNORE` idempotency | [`put_block`](BlockStore::put_block) existence check → `Ok(false)` |
17//! | `BlockHeightMap` bytearray | [`CF_CANONICAL`](crate::CF_CANONICAL) + `canonical.bin` mmap ([`CAN-001`](../docs/requirements/domains/canonical_chain/specs/CAN-001.md), [`crate::canonical::mmap`](crate::canonical::mmap)) |
18//!
19//! # Column family ownership
20//!
21//! - [`CF_BLOCKS`]: Compressed full block bodies keyed by header hash ([`SER-001`]).
22//! - [`CF_HEADERS`]: Uncompressed bincode headers keyed by header hash ([`SER-002`]).
23//! - [`CF_CANONICAL`]: Dense height→hash index for the canonical chain ([`CAN-001`]).
24//! - [`CF_METADATA`]: Tip, genesis hash, schema version, zstd dictionary ([`TYP-002`]).
25//! - [`CF_ATTESTED`]: [`AttestedBlock`](dig_block::AttestedBlock) rows via [`BlockStore::put_attestation`] ([`BLK-009`](../docs/requirements/domains/block_storage/specs/BLK-009.md)).
26//! - [`CF_CHECKPOINTS`]: Checkpoint storage ([`CKP-*`](../docs/requirements/IMPLEMENTATION_ORDER.md) Phase 9).
27//!
28//! # Three-tier read path
29//!
30//! Every `get_*` method follows a consistent tiered lookup:
31//!
32//! 1. **In-memory cache** — sharded LRU for blocks/headers, `HashMap` for records.
33//! Cache hits return clones with zero RocksDB I/O.
34//! 2. **RocksDB column family** — on miss, raw bytes are fetched, deserialized, and
35//! inserted back into the cache (read-through).
36//! 3. **Absent** — `Ok(None)` when the key does not exist at any tier.
37//!
38//! # Concurrency model
39//!
40//! `BlockStore` uses `&self` for all public methods (no `&mut self`). Interior
41//! mutability is provided by:
42//! - [`parking_lot::RwLock`] for tip and zstd dictionary (read-heavy, rare writes).
43//! - [`parking_lot::Mutex`] for the record cache (short critical sections).
44//! - [`std::sync::atomic::AtomicUsize`] for instrumentation counters (lock-free).
45//! - [`Arc<DB>`] for the RocksDB handle (thread-safe by design).
46//!
47//! # Requirements trace
48//!
49//! - [`STR-004`](../docs/requirements/domains/crate_structure/specs/STR-004.md) — constructors and lifecycle.
50//! - [`BLK-001`](../docs/requirements/domains/block_storage/specs/BLK-001.md) — `put_block` / `put`.
51//! - [`BLK-002`](../docs/requirements/domains/block_storage/specs/BLK-002.md) — `get_block` with block cache.
52//! - [`BLK-003`](../docs/requirements/domains/block_storage/specs/BLK-003.md) — `get_header` with header cache.
53//! - [`BLK-004`](../docs/requirements/domains/block_storage/specs/BLK-004.md) — `get_record` with layered caching.
54//! - [`BLK-005`](../docs/requirements/domains/block_storage/specs/BLK-005.md) — `get_blocks_by_hash` batch retrieval.
55//! - [`BLK-006`](../docs/requirements/domains/block_storage/specs/BLK-006.md) — `stream_blocks_in_range` sequential readahead.
56//! - [`BLK-007`](../docs/requirements/domains/block_storage/specs/BLK-007.md) — async read wrappers (`get_block_async`, …).
57//! - [`BLK-008`](../docs/requirements/domains/block_storage/specs/BLK-008.md) — async write pipeline (`put_pipelined`, batched `WriteBatch`).
58//! - [`BLK-009`](../docs/requirements/domains/block_storage/specs/BLK-009.md) — `put_attestation` / `get_attestation` on [`CF_ATTESTED`].
59//! - [`BLK-010`](../docs/requirements/domains/block_storage/specs/BLK-010.md) — `update_status` on in-memory [`BlockRecord`] only.
60//! - [`BLK-011`](../docs/requirements/domains/block_storage/specs/BLK-011.md) — `has_block` lightweight existence by hash.
61//! - [`BLK-012`](../docs/requirements/domains/block_storage/specs/BLK-012.md) — `stats` aggregate [`StorageStats`](crate::types::StorageStats) snapshot.
62//! - [`BLK-013`](../docs/requirements/domains/block_storage/specs/BLK-013.md) — `flush` / `compact` maintenance on the shared [`rocksdb::DB`].
63//! - [`BLK-014`](../docs/requirements/domains/block_storage/specs/BLK-014.md) — `get_blocks_in_range` and sync `get_block_by_height` over [`CF_CANONICAL`].
64//! - [`BLK-015`](../docs/requirements/domains/block_storage/specs/BLK-015.md) — `get_records_in_range` / `get_record_by_height` (header-derived, no block bodies).
65//! - [`CAN-001`](../docs/requirements/domains/canonical_chain/specs/CAN-001.md) — dual-layer canonical index (`canonical.bin` + [`CF_CANONICAL`]).
66//! - [`CAN-003`](../docs/requirements/domains/canonical_chain/specs/CAN-003.md) — [`set_canonical`](BlockStore::set_canonical) for existing stored blocks.
67//! - [`CAN-004`](../docs/requirements/domains/canonical_chain/specs/CAN-004.md) — [`set_canonical_batch`](BlockStore::set_canonical_batch) (single [`WriteBatch`](rocksdb::WriteBatch) for reorg-scale promotion).
68//! - [`SER-001`](../docs/requirements/domains/serialization/specs/SER-001.md) — bincode + zstd block serialization.
69//! - [`SER-002`](../docs/requirements/domains/serialization/specs/SER-002.md) — bincode-only header serialization.
70//! - [`SER-005`](../docs/requirements/domains/serialization/specs/SER-005.md) — dictionary training and persistence.
71//! - [`CAC-001`](../docs/requirements/domains/caching/specs/CAC-001_sharded_block_cache.md) — sharded block LRU.
72//! - [`CAC-002`](../docs/requirements/domains/caching/specs/CAC-002_sharded_header_cache.md) — sharded header LRU.
73//! - [`CAC-006`](../docs/requirements/domains/caching/specs/CAC-006_cache_warming_on_startup.md) — startup warming.
74//!
75//! **Spec:** `docs/resources/SPEC.md` §15.1 (constructors), §16 (crate boundary).
76
77use std::collections::HashMap;
78use std::path::Path;
79use std::sync::atomic::{AtomicU64, AtomicUsize, Ordering};
80use std::sync::Arc;
81
82use tokio::sync::mpsc;
83
84use chia_protocol::Bytes32;
85use dig_block::{AttestedBlock, BlockStatus, L2Block, L2BlockHeader};
86use parking_lot::{Mutex, RwLock};
87use rocksdb::{Direction, IteratorMode, Options, WriteBatch, DB};
88
89use crate::cache::sharded::{ShardedBlockCache, ShardedHeaderCache, ShardedLruCache};
90use crate::canonical::mmap::CanonicalBin;
91use crate::cf_options;
92use crate::compression::resolve_zstd_dictionary;
93use crate::constants::{
94 CF_ATTESTED, CF_BLOCKS, CF_CANONICAL, CF_CHECKPOINTS, CF_HEADERS, CF_METADATA,
95 META_GENESIS_HASH, META_MIN_HEIGHT, META_TIP,
96};
97use crate::encoding::{hash_key, height_key};
98use crate::error::{
99 BlockStoreError, ERR_ASYNC_JOIN_PREFIX, ERR_INIT_GENESIS_ALREADY_INITIALIZED,
100 ERR_INIT_GENESIS_READ_ONLY, ERR_MUTATION_READ_ONLY, ERR_OPEN_READONLY_PATH_MISSING_PREFIX,
101 ERR_UPDATE_STATUS_RECORD_NOT_CACHED_PREFIX,
102};
103use crate::pipeline::PipelineJob;
104use crate::types::{BlockRecord, ChainTip, ReorgResult, StorageStats};
105use crate::BlockStoreConfig;
106
107pub use crate::pipeline::StreamBlocksInRange;
108
109/// Shared RocksDB + cache state behind [`BlockStore`].
110///
111/// **Why a separate type ([`BLK-007`](../docs/requirements/domains/block_storage/specs/BLK-007.md)):** [`BlockStore`]
112/// wraps this struct in [`Arc`] so [`BlockStore::clone`] is a single refcount increment. Async helpers move a clone
113/// into [`tokio::task::spawn_blocking`] while preserving one logical store (atomics + caches stay shared).
114///
115/// Public API remains on [`BlockStore`] via [`std::ops::Deref`]. The type is `pub` so [`Deref::Target`] is
116/// well-formed; external crates should still depend on [`BlockStore`] methods only ([`BLK-008`](../docs/requirements/domains/block_storage/specs/BLK-008.md) will deepen `Arc` sharing).
117#[doc(hidden)]
118pub struct BlockStoreInner {
119 /// RocksDB handle shared across all operations. Thread-safe via RocksDB's internal locking.
120 /// All six column families ([`TYP-001`]) are created at open time.
121 pub(crate) db: Arc<DB>,
122 /// When `true`, all mutation APIs (`put_block`, `init_genesis`, [`BlockStore::put_attestation`](BlockStore::put_attestation)) return
123 /// [`BlockStoreError::Serialization`] with [`ERR_MUTATION_READ_ONLY`].
124 /// Set by [`BlockStore::open_readonly`]; cannot be toggled after construction.
125 pub(crate) read_only: bool,
126 /// In-memory copy of the chain tip from [`META_TIP`] in [`CF_METADATA`].
127 /// Updated atomically after [`BlockStore::init_genesis`] and future tip-advance APIs.
128 /// Reads use [`RwLock::read`] (very cheap with parking_lot); writes are rare (new blocks).
129 pub(crate) tip: RwLock<Option<ChainTip>>,
130 /// Count of blocks verified present during cache warming at last [`BlockStore::open`].
131 /// Exposed via [`BlockStore::warm_blocks_loaded_count`] for startup diagnostics.
132 pub(crate) warm_blocks_loaded: AtomicUsize,
133 /// Zstd level for [`BlockStore::serialize_block`] / plain fallback ([`SER-001`](../docs/requirements/domains/serialization/specs/SER-001.md) §6).
134 pub(crate) compression_level: i32,
135 /// When true and [`Self::zstd_dict`] is [`Some`], compress with dictionary ([`SER-005`](../docs/requirements/domains/serialization/specs/SER-005.md) precursor).
136 pub(crate) use_compression_dict: bool,
137 /// Cap passed to [`zstd::bulk::Decompressor::decompress`] ([`SER-001`](../docs/requirements/domains/serialization/specs/SER-001.md) implementation notes).
138 pub(crate) max_decompressed_block_bytes: usize,
139 /// Trained dictionary loaded from [`META_ZSTD_DICT`] or [`BlockStoreConfig::zstd_dictionary_override`].
140 ///
141 /// **[`SER-005`](../docs/requirements/domains/serialization/specs/SER-005.md):** [`RwLock`] lets
142 /// [`BlockStore::maybe_train_dictionary`] publish the first trained dictionary **after** the write that crosses
143 /// [`DICT_TRAINING_THRESHOLD`](crate::constants::DICT_TRAINING_THRESHOLD) while keeping [`BlockStore`] on an
144 /// immutable `&self` API surface (matches `put`-style ergonomics slated for [`BLK-001`](../docs/requirements/domains/block_storage/specs/BLK-001.md)).
145 pub(crate) zstd_dict: RwLock<Option<Arc<Vec<u8>>>>,
146 /// [`BlockRecord`] rows derived on write; **never** persisted ([`BLK-001`](../docs/requirements/domains/block_storage/specs/BLK-001.md), [`CAC-003`](../docs/requirements/domains/caching/specs/CAC-003.md) precursor).
147 ///
148 /// **Concurrency:** [`parking_lot::Mutex`] keeps inserts from [`BlockStore::put_block`] / [`BlockStore::init_genesis`] and
149 /// lookups from [`BlockStore::get_record`] safe without `&mut self`.
150 pub(crate) record_cache: Mutex<HashMap<Bytes32, BlockRecord>>,
151 /// Sharded LRU of deserialized [`L2Block`] values ([`BLK-002`](../docs/requirements/domains/block_storage/specs/BLK-002.md)).
152 pub(crate) block_cache: Arc<ShardedBlockCache>,
153 /// Count of RocksDB `get_cf` calls against [`CF_BLOCKS`] issued from [`BlockStore::get_block`] **after** a cache miss.
154 ///
155 /// **Rationale:** Proves AC §2 “no I/O on hit” in `tests/blk_002_tests.rs`; cheap atomic hot path on miss only.
156 /// **Not incremented** by [`BlockStore::get_blocks_by_hash`] (that path uses [`DB::multi_get_cf`](rocksdb::DB::multi_get_cf); see [`BlockStore::cf_blocks_multi_get_batch_count`]).
157 pub(crate) cf_blocks_physical_gets: AtomicUsize,
158 /// Count of [`rocksdb::DB::multi_get_cf`] **batch invocations** from [`BlockStore::get_blocks_by_hash`] when the input
159 /// contains at least one block-cache miss ([`BLK-005`](../docs/requirements/domains/block_storage/specs/BLK-005.md) AC §3).
160 ///
161 /// **Semantics:** Increments by **at most one per `get_blocks_by_hash` call** that performs RocksDB I/O (all misses
162 /// share one `multi_get_cf` round-trip). Stays at zero when every hash hits [`BlockStore::block_cache`] or when `hashes` is empty.
163 pub(crate) cf_blocks_multi_get_batches: AtomicUsize,
164 /// Count of [`DB::get_cf_opt`](rocksdb::DB::get_cf_opt) calls against [`CF_BLOCKS`] from [`StreamBlocksInRange`]
165 /// ([`BLK-006`](../docs/requirements/domains/block_storage/specs/BLK-006.md)) after a block-cache miss.
166 ///
167 /// **Rationale:** Distinct from [`BlockStore::cf_blocks_physical_get_count`] ([`get_block`](BlockStore::get_block)) so tests can
168 /// prove cache hits in a streamed range skip redundant block-blob reads ([`tests/blk_006_tests.rs`]).
169 pub(crate) cf_blocks_stream_physical_gets: AtomicUsize,
170 /// Copy of [`BlockStoreConfig::readahead_size`](crate::BlockStoreConfig::readahead_size) at open time ([`BLK-006`](../docs/requirements/domains/block_storage/specs/BLK-006.md) AC §4).
171 pub(crate) readahead_size: usize,
172 /// Sharded LRU of [`L2BlockHeader`] values ([`BLK-003`](../docs/requirements/domains/block_storage/specs/BLK-003.md)).
173 ///
174 /// **Separate** from [`Self::block_cache`] per BLK-003 implementation notes (tunables: [`BlockStoreConfig::header_cache_capacity`](crate::BlockStoreConfig::header_cache_capacity)).
175 pub(crate) header_cache: Arc<ShardedHeaderCache>,
176 /// Count of RocksDB `get_cf` calls against [`CF_HEADERS`] after **both** [`Self::header_cache`] and
177 /// [`Self::record_cache`] miss — incremented by [`BlockStore::get_header`] and by [`BlockStore::get_record`] ([`BLK-003`](../docs/requirements/domains/block_storage/specs/BLK-003.md), [`BLK-004`](../docs/requirements/domains/block_storage/specs/BLK-004.md)).
178 pub(crate) cf_headers_physical_gets: AtomicUsize,
179 /// Max jobs per RocksDB [`WriteBatch`] flush ([`BlockStoreConfig::write_pipeline_batch_size`](crate::BlockStoreConfig::write_pipeline_batch_size)).
180 pub(crate) pipeline_batch_size: usize,
181 /// Partial-batch flush timer ([`BlockStoreConfig::write_pipeline_flush_ms`](crate::BlockStoreConfig::write_pipeline_flush_ms)).
182 pub(crate) pipeline_flush_ms: u64,
183 /// Bounded channel depth ([`BlockStoreConfig::write_pipeline_channel_capacity`](crate::BlockStoreConfig::write_pipeline_channel_capacity)).
184 pub(crate) pipeline_channel_capacity: usize,
185 /// Count of successful [`DB::write`](rocksdb::DB::write) calls issued **only** by the pipeline worker ([`tests/blk_008_tests.rs`]).
186 pub(crate) pipeline_write_batches: AtomicUsize,
187 /// Dense height→hash mmap sidecar (`canonical.bin`) kept in lockstep with [`CF_CANONICAL`] ([`CAN-001`](../docs/requirements/domains/canonical_chain/specs/CAN-001.md)).
188 ///
189 /// **Reads:** [`parking_lot::RwLock::read`] for [`Self::get_hash_by_height`] (hot path). **Writes:**
190 /// [`RwLock::write`] after every successful RocksDB batch that touches the canonical index (`init_genesis`, [`BlockStore::put_block`], pipeline flush).
191 pub(crate) canonical_bin: RwLock<CanonicalBin>,
192 /// In-memory height→hash cache for hot canonical heights ([`CAC-004`](../docs/requirements/domains/caching/specs/CAC-004_canonical_height_index_cache.md)).
193 ///
194 /// **BTreeMap** provides O(log n) lookup and ordered iteration for range queries.
195 /// Populated from `set_canonical`, `put_block(canonical=true)`, and `get_hash_by_height` read-through.
196 /// Evicted on rollback. Bounded by `canonical_height_cache_capacity` (default 10,000).
197 /// Cached `META_MIN_HEIGHT` for fast access by rollback validation and compaction filter ([`PRN-004`]).
198 /// Loaded from CF_METADATA at startup; updated with `Release` ordering after prune succeeds.
199 /// Shared with the compaction filter ([`PRN-003`]) when `enable_compaction_pruning` is true.
200 /// The filter reads this with `Acquire` ordering; `prune_before_height` writes with `Release`.
201 pub(crate) min_retained_height_cached: Arc<AtomicU64>,
202 pub(crate) canonical_height_cache: RwLock<std::collections::BTreeMap<u64, Bytes32>>,
203 /// Max entries before the lowest-height entry is evicted from [`Self::canonical_height_cache`].
204 pub(crate) canonical_height_cache_capacity: usize,
205 /// Hash→height reverse lookup cache ([`CAC-005`](../docs/requirements/domains/caching/specs/CAC-005_hash_to_height_reverse_cache.md)).
206 ///
207 /// Sharded LRU with `u64` values (block height). Populated on `put_block`, header reads,
208 /// and block reads. Used by `find_common_ancestor` and `set_canonical` to avoid header
209 /// deserialization solely for height extraction.
210 pub(crate) hash_to_height_cache: Arc<ShardedLruCache<u64>>,
211}
212
213/// Primary handle for all block persistence APIs.
214///
215/// # Chia blockchain analogy
216///
217/// This struct corresponds to `BlockStore` in `chia-blockchain/chia/consensus/block_store.py`.
218/// Where Chia uses a single SQLite `full_blocks` table with Python LRU caches, DIG uses
219/// RocksDB column families with Rust sharded LRU caches for higher throughput under
220/// concurrent access. The API surface mirrors Chia's: `add_full_block` → [`put_block`](Self::put_block),
221/// `get_full_block` → [`get_block`](Self::get_block), `get_block_record` → [`get_record`](Self::get_record).
222///
223/// # Ownership
224///
225/// Thin [`Arc`] around [`BlockStoreInner`]: cheap [`Clone`] for [`tokio::task::spawn_blocking`] dispatch ([`BLK-007`](../docs/requirements/domains/block_storage/specs/BLK-007.md)).
226/// Field access on `&BlockStore` transparently reaches [`BlockStoreInner`] via [`std::ops::Deref`].
227///
228/// **Write pipeline ([`BLK-008`](../docs/requirements/domains/block_storage/specs/BLK-008.md)):** [`Self::pipeline_tx`]
229/// holds the lazy [`mpsc::Sender`] **outside** [`BlockStoreInner`]. The worker task also keeps an [`Arc`] to `inner`
230/// for RocksDB; if the sender lived on `inner`, dropping all [`BlockStore`] handles would still leave the sender
231/// alive (circular retention), the channel would never close, and AC §8 “flush on shutdown” would not run.
232///
233/// # Construction
234///
235/// Use [`BlockStore::open`] for read-write access or [`BlockStore::open_readonly`] for read-only
236/// access to an existing database. After construction, call [`BlockStore::init_genesis`] once
237/// to initialize a new chain.
238pub struct BlockStore {
239 pub(crate) inner: Arc<BlockStoreInner>,
240 /// Lazy bounded ingress for [`Self::put_pipelined`] — **not** stored on [`BlockStoreInner`] (see struct docs).
241 pub(crate) pipeline_tx: Arc<tokio::sync::Mutex<Option<mpsc::Sender<PipelineJob>>>>,
242}
243
244impl Clone for BlockStore {
245 fn clone(&self) -> Self {
246 Self {
247 inner: self.inner.clone(),
248 pipeline_tx: self.pipeline_tx.clone(),
249 }
250 }
251}
252
253impl std::ops::Deref for BlockStore {
254 type Target = BlockStoreInner;
255
256 fn deref(&self) -> &Self::Target {
257 &self.inner
258 }
259}
260
261impl BlockStore {
262 /// Open or create a store at `config.path` with all column families ([`STR-004`](../docs/requirements/domains/crate_structure/specs/STR-004.md), [`TYP-008`](../docs/requirements/domains/storage_types/specs/TYP-008.md)).
263 pub fn open(config: BlockStoreConfig) -> Result<Self, BlockStoreError> {
264 let compression_level = config.compression_level;
265 let use_compression_dict = config.use_compression_dict;
266 let max_decompressed_block_bytes = config.max_decompressed_block_bytes;
267 let zstd_dictionary_override = config.zstd_dictionary_override.clone();
268 // ERR-001 has no `Io` variant; surface directory creation failures as [`BlockStoreError::Serialization`]
269 // until the taxonomy adds filesystem errors ([`ERR-001`](../docs/requirements/domains/error_types/specs/ERR-001_blockstoreerror_enum.md)).
270 std::fs::create_dir_all(&config.path).map_err(|e| {
271 BlockStoreError::Serialization(format!(
272 "filesystem error creating database directory {}: {e}",
273 config.path.display()
274 ))
275 })?;
276 let mut opts = Options::default();
277 opts.create_if_missing(true);
278 opts.create_missing_column_families(true);
279 // PRN-003: create shared AtomicU64 for compaction filter BEFORE DB open
280 let prune_threshold = if config.enable_compaction_pruning {
281 Some(Arc::new(AtomicU64::new(0)))
282 } else {
283 None
284 };
285 let cfs = cf_options::column_family_descriptors(&config, prune_threshold.clone());
286 let db = DB::open_cf_descriptors(&opts, &config.path, cfs)?;
287 let db = Arc::new(db);
288 let canonical_bin =
289 RwLock::new(CanonicalBin::open_synced(&db, config.path.as_path(), true)?);
290 let zstd_dict =
291 resolve_zstd_dictionary(&db, use_compression_dict, zstd_dictionary_override)?;
292 let tip = load_tip(&db)?;
293 let warm_cache_on_open = config.warm_cache_on_open;
294 let warm_cache_depth = config.warm_cache_depth;
295 let readahead_size = config.readahead_size;
296 let shards = config.cache_shards.max(1);
297 let block_cache = Arc::new(ShardedBlockCache::new(config.block_cache_capacity, shards));
298 let header_cache = Arc::new(ShardedHeaderCache::new(
299 config.header_cache_capacity,
300 shards,
301 ));
302 let store = Self {
303 inner: Arc::new(BlockStoreInner {
304 db,
305 read_only: false,
306 tip: RwLock::new(tip),
307 warm_blocks_loaded: AtomicUsize::new(0),
308 compression_level,
309 use_compression_dict,
310 max_decompressed_block_bytes,
311 zstd_dict: RwLock::new(zstd_dict),
312 record_cache: Mutex::new(HashMap::new()),
313 block_cache,
314 cf_blocks_physical_gets: AtomicUsize::new(0),
315 cf_blocks_multi_get_batches: AtomicUsize::new(0),
316 cf_blocks_stream_physical_gets: AtomicUsize::new(0),
317 readahead_size,
318 header_cache,
319 cf_headers_physical_gets: AtomicUsize::new(0),
320 pipeline_batch_size: config.write_pipeline_batch_size.max(1),
321 pipeline_flush_ms: config.write_pipeline_flush_ms.max(1),
322 pipeline_channel_capacity: config.write_pipeline_channel_capacity.max(1),
323 pipeline_write_batches: AtomicUsize::new(0),
324 canonical_bin,
325 min_retained_height_cached: prune_threshold
326 .unwrap_or_else(|| Arc::new(AtomicU64::new(0))),
327 canonical_height_cache: RwLock::new(std::collections::BTreeMap::new()),
328 canonical_height_cache_capacity: config.canonical_height_cache_capacity,
329 hash_to_height_cache: Arc::new(ShardedLruCache::new(
330 config.hash_to_height_cache_capacity,
331 shards,
332 )),
333 }),
334 pipeline_tx: Arc::new(tokio::sync::Mutex::new(None)),
335 };
336 // PRN-004: load persisted min_retained_height into AtomicU64
337 if let Ok(Some(h)) = store.read_min_retained_height() {
338 store.min_retained_height_cached.store(h, Ordering::Release);
339 }
340 // CAC-006: warm ALL caches after full construction. get_block_by_height and
341 // get_record_by_height auto-populate block_cache, header_cache, record_cache,
342 // canonical_height_cache (CAC-004), and hash_to_height_cache (CAC-005).
343 if warm_cache_on_open {
344 let warmed = store.warm_caches(warm_cache_depth);
345 store.warm_blocks_loaded.store(warmed, Ordering::Relaxed);
346 }
347 Ok(store)
348 }
349
350 /// Open an existing database read-only; fails if `path` does not exist ([`STR-004`](../docs/requirements/domains/crate_structure/specs/STR-004.md)).
351 pub fn open_readonly(path: impl AsRef<Path>) -> Result<Self, BlockStoreError> {
352 let path = path.as_ref();
353 if !path.exists() {
354 return Err(BlockStoreError::Serialization(format!(
355 "{ERR_OPEN_READONLY_PATH_MISSING_PREFIX}{}",
356 path.display()
357 )));
358 }
359 let opts = Options::default();
360 // CF option structs must match how the DB was created; tests use STR-005 `test_config`, which
361 // mirrors [`BlockStoreConfig::default`] for `enable_blob_db` ([`TYP-003`](../../docs/requirements/domains/storage_types/specs/TYP-003.md)).
362 let readonly_cfg = BlockStoreConfig {
363 path: path.to_path_buf(),
364 ..BlockStoreConfig::default()
365 };
366 let compression_level = readonly_cfg.compression_level;
367 let use_compression_dict = readonly_cfg.use_compression_dict;
368 let max_decompressed_block_bytes = readonly_cfg.max_decompressed_block_bytes;
369 let zstd_dictionary_override = readonly_cfg.zstd_dictionary_override.clone();
370 let cfs = cf_options::column_family_descriptors(&readonly_cfg, None);
371 let db = DB::open_cf_descriptors_read_only(&opts, path, cfs, false)?;
372 let db = Arc::new(db);
373 let canonical_bin = RwLock::new(CanonicalBin::open_synced(&db, path, false)?);
374 let zstd_dict =
375 resolve_zstd_dictionary(&db, use_compression_dict, zstd_dictionary_override)?;
376 let tip = load_tip(&db)?;
377 let readahead_size = readonly_cfg.readahead_size;
378 let shards = readonly_cfg.cache_shards.max(1);
379 let block_cache = Arc::new(ShardedBlockCache::new(
380 readonly_cfg.block_cache_capacity,
381 shards,
382 ));
383 let header_cache = Arc::new(ShardedHeaderCache::new(
384 readonly_cfg.header_cache_capacity,
385 shards,
386 ));
387 let store = Self {
388 inner: Arc::new(BlockStoreInner {
389 db,
390 read_only: true,
391 tip: RwLock::new(tip),
392 warm_blocks_loaded: AtomicUsize::new(0),
393 compression_level,
394 use_compression_dict,
395 max_decompressed_block_bytes,
396 zstd_dict: RwLock::new(zstd_dict),
397 record_cache: Mutex::new(HashMap::new()),
398 block_cache,
399 cf_blocks_physical_gets: AtomicUsize::new(0),
400 cf_blocks_multi_get_batches: AtomicUsize::new(0),
401 cf_blocks_stream_physical_gets: AtomicUsize::new(0),
402 readahead_size,
403 header_cache,
404 cf_headers_physical_gets: AtomicUsize::new(0),
405 pipeline_batch_size: readonly_cfg.write_pipeline_batch_size.max(1),
406 pipeline_flush_ms: readonly_cfg.write_pipeline_flush_ms.max(1),
407 pipeline_channel_capacity: readonly_cfg.write_pipeline_channel_capacity.max(1),
408 pipeline_write_batches: AtomicUsize::new(0),
409 canonical_bin,
410 min_retained_height_cached: Arc::new(AtomicU64::new(0)),
411 canonical_height_cache: RwLock::new(std::collections::BTreeMap::new()),
412 canonical_height_cache_capacity: readonly_cfg.canonical_height_cache_capacity,
413 hash_to_height_cache: Arc::new(ShardedLruCache::new(
414 readonly_cfg.hash_to_height_cache_capacity,
415 shards,
416 )),
417 }),
418 pipeline_tx: Arc::new(tokio::sync::Mutex::new(None)),
419 };
420 // PRN-004: load persisted min_retained_height
421 if let Ok(Some(h)) = store.read_min_retained_height() {
422 store.min_retained_height_cached.store(h, Ordering::Release);
423 }
424 Ok(store)
425 }
426
427 /// Initialize genesis: empty store only; atomic [`WriteBatch`] ([`STR-004`](../docs/requirements/domains/crate_structure/specs/STR-004.md)).
428 pub fn init_genesis(&self, block: &L2Block) -> Result<(), BlockStoreError> {
429 if self.read_only {
430 return Err(BlockStoreError::Serialization(
431 ERR_INIT_GENESIS_READ_ONLY.into(),
432 ));
433 }
434 let meta = self.cf(CF_METADATA)?;
435 if self.db.get_cf(meta, META_TIP.as_bytes())?.is_some()
436 || self
437 .db
438 .get_cf(meta, META_GENESIS_HASH.as_bytes())?
439 .is_some()
440 {
441 return Err(BlockStoreError::Serialization(
442 ERR_INIT_GENESIS_ALREADY_INITIALIZED.into(),
443 ));
444 }
445 let hash = block.hash();
446 if block.height() != 0 {
447 return Err(BlockStoreError::Serialization(format!(
448 "init_genesis: genesis block height must be 0, got {}",
449 block.height()
450 )));
451 }
452 // [`SER-001`](../docs/requirements/domains/serialization/specs/SER-001.md): bincode + zstd (dictionary when configured).
453 let compressed = self.serialize_block(block)?;
454 // [`SER-002`](../docs/requirements/domains/serialization/specs/SER-002.md): headers are bincode-only in `CF_HEADERS`.
455 let header_bytes = Self::serialize_header(&block.header)?;
456 let tip = ChainTip { hash, height: 0 };
457 let mut batch = WriteBatch::default();
458 let cf_b = self.cf(CF_BLOCKS)?;
459 let cf_h = self.cf(CF_HEADERS)?;
460 let cf_c = self.cf(CF_CANONICAL)?;
461 // [`hash_key`] returns `[u8; 32]`; use `.as_slice()` (not `.as_ref()`) so RocksDB keys resolve to
462 // `&[u8]` without ambiguous `AsRef` when the `bitcoin` crate is also in the dependency graph.
463 batch.put_cf(cf_b, hash_key(&hash).as_slice(), &compressed);
464 batch.put_cf(cf_h, hash_key(&hash).as_slice(), &header_bytes);
465 batch.put_cf(cf_c, height_key(0), hash_key(&hash).as_slice());
466 batch.put_cf(meta, META_TIP.as_bytes(), tip.to_bytes().as_slice());
467 batch.put_cf(meta, META_GENESIS_HASH.as_bytes(), hash.as_ref());
468 self.db.write(batch)?;
469 self.canonical_bin.write().extend_write(0, &hash)?;
470 *self.tip.write() = Some(tip);
471 let record = BlockRecord::from_header(&block.header, BlockStatus::Validated);
472 self.record_cache.lock().insert(hash, record);
473 self.block_cache.insert(hash, block.clone());
474 self.header_cache.insert(hash, block.header.clone());
475 self.maybe_train_dictionary()?;
476 Ok(())
477 }
478
479 /// **Diagnostics / tests:** Disable the mmap acceleration layer so height→hash resolution uses [`CF_CANONICAL`]
480 /// only until the next [`BlockStore::open`] ([`CAN-001`](../docs/requirements/domains/canonical_chain/specs/CAN-001.md) test plan: mmap fallback).
481 ///
482 /// **Production:** Do not call — the next process restart re-syncs `canonical.bin` from RocksDB anyway.
483 pub fn disable_canonical_bin_acceleration(&self) {
484 self.canonical_bin.write().disable();
485 }
486
487 /// Current chain tip — hash and height of the highest canonical block.
488 ///
489 /// Returns the in-memory cached copy loaded from [`META_TIP`](crate::META_TIP) at startup
490 /// and updated by [`BlockStore::set_tip`], [`BlockStore::init_genesis`], and future
491 /// `extend_chain` / `rollback_to_height` APIs.
492 ///
493 /// # Performance
494 ///
495 /// This is a hot-path accessor queried on every block ingestion for parent-hash validation.
496 /// The [`parking_lot::RwLock::read`] is lock-free on the uncontended fast path (~2-5ns).
497 /// No RocksDB I/O occurs.
498 ///
499 /// # Chia analogy
500 ///
501 /// Corresponds to `BlockStore.get_peak()` in Chia's `block_store.py`.
502 ///
503 /// **Requirement:** [`CAN-007`](../docs/requirements/domains/canonical_chain/specs/CAN-007.md).
504 pub fn tip(&self) -> Option<ChainTip> {
505 *self.tip.read()
506 }
507
508 /// Convenience accessor for the current canonical chain height.
509 ///
510 /// Returns `tip().map(|t| t.height)` — `None` when the store has no tip (before genesis),
511 /// `Some(0)` after genesis, `Some(n)` after extending to height `n`.
512 ///
513 /// **Requirement:** [`CAN-007`](../docs/requirements/domains/canonical_chain/specs/CAN-007.md) § Accessors.
514 #[must_use]
515 pub fn height(&self) -> Option<u64> {
516 self.tip().map(|t| t.height)
517 }
518
519 /// Persist a new chain tip to [`CF_METADATA`](crate::CF_METADATA) / [`META_TIP`](crate::META_TIP)
520 /// and update the in-memory cache.
521 ///
522 /// # Encoding
523 ///
524 /// The value written is exactly 40 bytes: `hash (32 bytes, raw Bytes32) || height (8 bytes, little-endian u64)`.
525 /// This matches [`ChainTip::to_bytes()`](crate::ChainTip::to_bytes) and the
526 /// [`TYP-006`](../docs/requirements/domains/storage_types/specs/TYP-006.md) wire format.
527 ///
528 /// # Ordering
529 ///
530 /// RocksDB write is performed **before** updating the in-memory `RwLock`. If the write
531 /// fails, the in-memory tip remains unchanged (no stale state visible to concurrent readers).
532 ///
533 /// # Errors
534 ///
535 /// - [`BlockStoreError::Serialization`] with [`ERR_MUTATION_READ_ONLY`](crate::ERR_MUTATION_READ_ONLY)
536 /// when called on a read-only handle.
537 /// - [`BlockStoreError::RocksDb`] on write failure.
538 ///
539 /// # Update points ([`CAN-007`](../docs/requirements/domains/canonical_chain/specs/CAN-007.md))
540 ///
541 /// | Operation | New Tip |
542 /// |-----------|---------|
543 /// | `extend_chain` (CAN-005) | Newly added block |
544 /// | `rollback_to_height` (ROR-001) | Block at target height |
545 /// | `apply_reorg` (ROR-003) | Last block in new chain |
546 /// | `init_genesis` (STR-004) | Genesis block (height 0) |
547 pub fn set_tip(&self, tip: ChainTip) -> Result<(), BlockStoreError> {
548 if self.read_only {
549 return Err(BlockStoreError::Serialization(
550 ERR_MUTATION_READ_ONLY.into(),
551 ));
552 }
553 let cf = self.cf(CF_METADATA)?;
554 // Write the 40-byte encoding to CF_METADATA before updating in-memory state.
555 // On failure, the in-memory tip remains at the old value (no stale state).
556 self.db
557 .put_cf(cf, META_TIP.as_bytes(), tip.to_bytes().as_slice())?;
558 *self.tip.write() = Some(tip);
559 Ok(())
560 }
561
562 /// Blocks successfully verified present while warming on last [`Self::open`] ([`STR-004`](../docs/requirements/domains/crate_structure/specs/STR-004.md) / [`CAC-006`](../docs/requirements/domains/caching/specs/CAC-006_cache_warming_on_startup.md)).
563 pub fn warm_blocks_loaded_count(&self) -> usize {
564 self.warm_blocks_loaded.load(Ordering::Relaxed)
565 }
566
567 /// Serialize a block header for [`CF_HEADERS`] ([`SER-002`](../docs/requirements/domains/serialization/specs/SER-002.md)).
568 ///
569 /// **Write path (normative):** `L2BlockHeader` → [`bincode::serialize`] → raw bytes (no zstd). Headers are
570 /// small and read on every chain walk; skipping compression avoids framing overhead and decode latency
571 /// on the hot path ([`NORMATIVE.md` § SER-002](../docs/requirements/domains/serialization/NORMATIVE.md)).
572 ///
573 /// **Errors:** [`BlockStoreError::Serialization`] — same variant as corrupt block payloads so upper
574 /// layers can treat “bytes unusable” uniformly until ERR-* adds finer codes.
575 ///
576 /// **Write path:** [`Self::put_block`] / [`Self::init_genesis`] insert fresh values so steady-state reads hit RAM.
577 pub fn get_block(&self, hash: &Bytes32) -> Result<Option<L2Block>, BlockStoreError> {
578 if let Some(block) = self.block_cache.get_clone(hash) {
579 return Ok(Some(block));
580 }
581 let cf = self.cf(CF_BLOCKS)?;
582 self.cf_blocks_physical_gets.fetch_add(1, Ordering::Relaxed);
583 let raw_opt = self.db.get_cf(cf, hash_key(hash).as_slice())?;
584 let Some(raw) = raw_opt else {
585 return Ok(None);
586 };
587 let block = self.deserialize_block(&raw)?;
588 self.block_cache.insert(*hash, block.clone());
589 self.header_cache.insert(*hash, block.header.clone());
590 // CAC-005 (API-002): populate hash→height on block read-through
591 self.hash_to_height_cache.insert(*hash, block.height());
592 Ok(Some(block))
593 }
594
595 /// **[`BLK-011`](../docs/requirements/domains/block_storage/specs/BLK-011.md)** — Whether any persisted row exists for `hash` **without** decoding zstd or bincode ([`NORMATIVE.md` § BLK-011](../docs/requirements/domains/block_storage/NORMATIVE.md#blk-011-has-block-has_block)).
596 ///
597 /// **Cache first (AC §2):** [`Self::block_cache`] and [`Self::header_cache`] are consulted via [`ShardedLruCache::contains`](crate::cache::sharded::ShardedLruCache::contains) ([`LruCache::peek`](lru::LruCache::peek) — no LRU promotion).
598 ///
599 /// **RocksDB (AC §1):** If both caches miss, probe [`CF_HEADERS`] then [`CF_BLOCKS`] using [`hash_key`](crate::encoding::hash_key) (same key layout as [`Self::put_block`]). The second probe covers edge cases where only a body row exists; normal [`Self::put_block`] writes both families together ([`BLK-001`](../docs/requirements/domains/block_storage/specs/BLK-001.md)).
600 ///
601 /// **No deserialize / decompress (AC §3):** Uses only [`DB::get_cf`](rocksdb::DB::get_cf) presence checks — returned bytes are discarded without calling [`Self::deserialize_block`] or [`Self::deserialize_header`].
602 ///
603 /// **Instrumentation:** This path does **not** increment [`Self::cf_blocks_physical_get_count`] (that counter remains exclusive to [`Self::get_block`]) so tests can prove cache-fast paths avoid the heavy read API ([`BLK-002`](../docs/requirements/domains/block_storage/specs/BLK-002.md) counter semantics).
604 pub fn has_block(&self, hash: &Bytes32) -> Result<bool, BlockStoreError> {
605 if self.block_cache.contains(hash) || self.header_cache.contains(hash) {
606 return Ok(true);
607 }
608 let key = hash_key(hash);
609 let cf_h = self.cf(CF_HEADERS)?;
610 if self.db.get_cf(cf_h, key.as_slice())?.is_some() {
611 return Ok(true);
612 }
613 let cf_b = self.cf(CF_BLOCKS)?;
614 Ok(self.db.get_cf(cf_b, key.as_slice())?.is_some())
615 }
616
617 /// **[`BLK-012`](../docs/requirements/domains/block_storage/specs/BLK-012.md)** — Aggregate [`StorageStats`](crate::types::StorageStats) for monitoring / diagnostics ([`TYP-007`](../docs/requirements/domains/storage_types/specs/TYP-007.md), [`NORMATIVE` BLK-012](../docs/requirements/domains/block_storage/NORMATIVE.md#blk-012-storage-statistics-stats)).
618 ///
619 /// **Row counts:** Each `*_count` field is the number of keys in the corresponding column family from a linear
620 /// [`DB::iterator_cf`](rocksdb::DB::iterator_cf) scan ([`CF_BLOCKS`], [`CF_HEADERS`], [`CF_CANONICAL`],
621 /// [`CF_CHECKPOINTS`], [`CF_ATTESTED`]). This is **exact** for current store sizes (typical node counts) and
622 /// matches NORMATIVE wording (“reflects the total number of entries”). If full scans become too costly at scale,
623 /// a future revision may offer `rocksdb.estimate-num-keys` behind configuration with documented error bounds.
624 ///
625 /// **Tip / pruning:** [`StorageStats::tip_height`] mirrors [`Self::tip`] (RAM snapshot loaded from [`META_TIP`] on open, updated by [`Self::init_genesis`] today). [`Self::put_block`] does not yet advance the tip ([`CAN-007`](../docs/requirements/domains/canonical_chain/specs/CAN-007.md)); operators should not assume `tip_height == max(block heights)` until chain-tip APIs land. [`StorageStats::min_height`] reads
626 /// [`META_MIN_HEIGHT`] in [`CF_METADATA`] as **8 bytes little-endian** `u64` ([`storage_types/NORMATIVE`](../docs/requirements/domains/storage_types/NORMATIVE.md)); missing key means no prune watermark yet ([`PRN-004`](../docs/requirements/domains/pruning/specs/PRN-004_min_retained_height_tracking.md)).
627 ///
628 /// **Disk estimate:** [`StorageStats::total_size_bytes`] sums per-CF RocksDB property `rocksdb.estimate-live-data-size`
629 /// (live SST + memtable footprint estimate). It is **not** a byte-exact `du` of the directory; callers should treat
630 /// it as an order-of-magnitude health signal; use [`Self::flush`] / [`Self::compact`] before relying on
631 /// filesystem-level durability or space reclamation ([`BLK-013`](../docs/requirements/domains/block_storage/specs/BLK-013.md)).
632 pub fn stats(&self) -> Result<StorageStats, BlockStoreError> {
633 Ok(StorageStats {
634 block_count: self.count_cf_entries(CF_BLOCKS)?,
635 canonical_block_count: self.count_cf_entries(CF_CANONICAL)?,
636 header_count: self.count_cf_entries(CF_HEADERS)?,
637 checkpoint_count: self.count_cf_entries(CF_CHECKPOINTS)?,
638 attested_count: self.count_cf_entries(CF_ATTESTED)?,
639 tip_height: self.tip().map(|t| t.height),
640 min_height: self.read_min_retained_height()?,
641 total_size_bytes: self.sum_cf_live_data_size_estimates()?,
642 })
643 }
644
645 /// Exact key count for `cf_name` — used by [`Self::stats`] ([`BLK-012`](../docs/requirements/domains/block_storage/specs/BLK-012.md)).
646 fn count_cf_entries(&self, cf_name: &'static str) -> Result<u64, BlockStoreError> {
647 let cf = self.cf(cf_name)?;
648 let mut n = 0u64;
649 for entry in self.db.iterator_cf(cf, IteratorMode::Start) {
650 let (_k, _v) = entry?;
651 n += 1;
652 }
653 Ok(n)
654 }
655
656 /// Sum RocksDB `rocksdb.estimate-live-data-size` across all user column families ([`BLK-012`](../docs/requirements/domains/block_storage/specs/BLK-012.md) § Field Population).
657 fn sum_cf_live_data_size_estimates(&self) -> Result<u64, BlockStoreError> {
658 const PROP: &str = "rocksdb.estimate-live-data-size";
659 let mut sum = 0u64;
660 for name in [
661 CF_BLOCKS,
662 CF_HEADERS,
663 CF_CANONICAL,
664 CF_METADATA,
665 CF_ATTESTED,
666 CF_CHECKPOINTS,
667 ] {
668 let cf = self.cf(name)?;
669 if let Some(v) = self.db.property_int_value_cf(cf, PROP)? {
670 sum = sum.saturating_add(v);
671 }
672 }
673 Ok(sum)
674 }
675
676 /// Read persisted minimum retained height, if pruning has written [`META_MIN_HEIGHT`].
677 fn read_min_retained_height(&self) -> Result<Option<u64>, BlockStoreError> {
678 let meta = self.cf(CF_METADATA)?;
679 let Some(bytes) = self.db.get_cf(meta, META_MIN_HEIGHT.as_bytes())? else {
680 return Ok(None);
681 };
682 let arr: [u8; 8] = bytes.as_slice().try_into().map_err(|_| {
683 BlockStoreError::Serialization(format!(
684 "stats: META_MIN_HEIGHT value must be exactly 8 bytes (little-endian u64), got {} bytes",
685 bytes.len()
686 ))
687 })?;
688 Ok(Some(u64::from_le_bytes(arr)))
689 }
690
691 /// **[`BLK-013`](../docs/requirements/domains/block_storage/specs/BLK-013.md)** — Persist buffered engine state
692 /// ([`NORMATIVE` BLK-013](../docs/requirements/domains/block_storage/NORMATIVE.md#blk-013-flush-and-compact)).
693 ///
694 /// **Semantics:** First `rocksdb::DB::flush_wal(true)` so the write-ahead log is **synced** through the OS to
695 /// stable storage, then [`rocksdb::DB::flush`] to flush **all** column-family
696 /// memtables to SST files. Together this matches operators’ “make my recent writes durable” intent while staying
697 /// close to the BLK-013 spec snippet (which only showed `flush()` — WAL sync is required by NORMATIVE item 1’s
698 /// “WAL flush” wording).
699 ///
700 /// **Logical state:** Does not mutate dig-blockstore caches, tip, or row keys — only RocksDB I/O.
701 ///
702 /// **Errors:** Any [`rocksdb::Error`] maps to [`BlockStoreError::RocksDb`] ([`ERR-002`](../docs/requirements/domains/error_types/specs/ERR-002_error_from_conversions.md)).
703 pub fn flush(&self) -> Result<(), BlockStoreError> {
704 self.db.flush_wal(true)?;
705 self.db.flush()?;
706 Ok(())
707 }
708
709 /// **[`BLK-013`](../docs/requirements/domains/block_storage/specs/BLK-013.md)** — Request manual compaction on
710 /// **every** column family in [`crate::constants::ALL_COLUMN_FAMILIES`] ([`TYP-001`](../docs/requirements/domains/storage_types/specs/TYP-001.md)).
711 ///
712 /// **Implementation:** For each family, [`DB::compact_range_cf`](rocksdb::DB::compact_range_cf) with a `None`
713 /// key range compacts the **entire** keyspace (RocksDB schedules background work). The rust-rocksdb binding
714 /// returns `()` from `compact_range_cf` (errors surface asynchronously); callers use this for **space reclamation**
715 /// and read amplification tuning, not as a transactional barrier.
716 ///
717 /// **Logical state:** Compaction does not delete live keys written by [`Self::put_block`] / [`Self::init_genesis`];
718 /// it merges SSTables. Same error mapping as [`Self::flush`] if future APIs gain fallible compaction entry points.
719 pub fn compact(&self) -> Result<(), BlockStoreError> {
720 for &name in crate::constants::ALL_COLUMN_FAMILIES {
721 let cf = self.cf(name)?;
722 self.db.compact_range_cf(cf, None::<&[u8]>, None::<&[u8]>);
723 }
724 Ok(())
725 }
726
727 /// Batch-fetch blocks by hash ([`BLK-005`](../docs/requirements/domains/block_storage/specs/BLK-005.md)).
728 ///
729 /// **Algorithm**
730 /// 1. For each input hash in order, clone from [`Self::block_cache`] when present ([`CAC-001`](../docs/requirements/domains/caching/specs/CAC-001_sharded_block_cache.md)).
731 /// 2. Collect all cache misses; if non-empty, issue **one** [`rocksdb::DB::multi_get_cf`] over [`CF_BLOCKS`]
732 /// (same `(cf, key)` pattern as [`Self::get_block`], [`SER-001`](../docs/requirements/domains/serialization/specs/SER-001.md) payloads).
733 /// 3. For each returned blob: [`Self::deserialize_block`], then insert into [`Self::block_cache`] and [`Self::header_cache`]
734 /// (mirrors single-key read-through in [`Self::get_block`]).
735 ///
736 /// **Ordering:** Output `Vec` index `i` always corresponds to `hashes[i]` (per NORMATIVE BLK-005 §5).
737 ///
738 /// **Missing keys:** `Ok(None)` at that index; RocksDB row absent still consumes one slot in the `multi_get` result vector.
739 ///
740 /// **Empty input:** Returns `Ok(vec![])` without touching RocksDB.
741 ///
742 /// **Chunking:** Very large batches stay single-call for now ([`BLK-005.md`](../docs/requirements/domains/block_storage/specs/BLK-005.md) implementation notes); future work may split to bound peak memory.
743 pub fn get_blocks_by_hash(
744 &self,
745 hashes: &[Bytes32],
746 ) -> Result<Vec<Option<L2Block>>, BlockStoreError> {
747 let mut results: Vec<Option<L2Block>> = vec![None; hashes.len()];
748 let mut miss_indices: Vec<usize> = Vec::new();
749 for (i, hash) in hashes.iter().enumerate() {
750 if let Some(block) = self.block_cache.get_clone(hash) {
751 results[i] = Some(block);
752 } else {
753 miss_indices.push(i);
754 }
755 }
756 if miss_indices.is_empty() {
757 return Ok(results);
758 }
759 let cf = self.cf(CF_BLOCKS)?;
760 self.cf_blocks_multi_get_batches
761 .fetch_add(1, Ordering::Relaxed);
762 let keys: Vec<[u8; 32]> = miss_indices
763 .iter()
764 .map(|&idx| *hash_key(&hashes[idx]))
765 .collect();
766 let db_results = self
767 .db
768 .multi_get_cf(keys.iter().map(|k| (cf, k.as_slice())));
769 for (j, db_result) in db_results.into_iter().enumerate() {
770 let idx = miss_indices[j];
771 let maybe_raw = db_result?;
772 let Some(raw) = maybe_raw else {
773 continue;
774 };
775 let block = self.deserialize_block(&raw)?;
776 self.block_cache.insert(hashes[idx], block.clone());
777 self.header_cache.insert(hashes[idx], block.header.clone());
778 results[idx] = Some(block);
779 }
780 Ok(results)
781 }
782
783 /// Drop a single entry from the in-memory block LRU — **no RocksDB writes** ([`BLK-002`](../docs/requirements/domains/block_storage/specs/BLK-002.md) test plan: simulate eviction).
784 pub fn invalidate_block_cache_entry(&self, hash: &Bytes32) {
785 self.block_cache.remove(hash);
786 }
787
788 /// Look up the canonical block at `height` ([`CAN-006`](../docs/requirements/domains/canonical_chain/specs/CAN-006.md) precursor, [`BLK-014`](../docs/requirements/domains/block_storage/specs/BLK-014.md) building block).
789 ///
790 /// **Algorithm:** [`Self::get_hash_by_height`] (mmap then [`CF_CANONICAL`]) → [`Self::get_block`]
791 /// ([`BLK-002`](../docs/requirements/domains/block_storage/specs/BLK-002.md) decompress + cache).
792 ///
793 /// **Returns:** `Ok(None)` when the height index is absent **or** when the hash is indexed but the body row is
794 /// missing (same as [`Self::get_block`] returning `None`).
795 ///
796 /// **Threading:** Safe on any thread; performs synchronous RocksDB + zstd work — use [`Self::get_block_by_height_async`]
797 /// from async contexts that must not block the runtime ([`BLK-007`](../docs/requirements/domains/block_storage/specs/BLK-007.md)).
798 pub fn get_block_by_height(&self, height: u64) -> Result<Option<L2Block>, BlockStoreError> {
799 let Some(hash) = self.get_hash_by_height(height)? else {
800 return Ok(None);
801 };
802 self.get_block(&hash)
803 }
804
805 /// **[`BLK-014`](../docs/requirements/domains/block_storage/specs/BLK-014.md)** — Collect canonical [`L2Block`]s for
806 /// heights in `[start_height, end_height]` inclusive ([`NORMATIVE` BLK-014](../docs/requirements/domains/block_storage/NORMATIVE.md#blk-014-get-blocks-in-range-get_blocks_in_range)).
807 ///
808 /// **Semantics:** Ascending height order; `start_height > end_height` ⇒ empty `Vec` (not an error); any height with
809 /// no canonical row or no retrievable body is **omitted** (gaps and “beyond tip” behave the same — fewer results).
810 ///
811 /// **vs [`Self::stream_blocks_in_range`] ([`BLK-006`]):** This API eagerly builds a `Vec` with simple point lookups per height.
812 /// [`StreamBlocksInRange`] is better for large scans (single readahead iterator over [`CF_CANONICAL`]).
813 pub fn get_blocks_in_range(
814 &self,
815 start_height: u64,
816 end_height: u64,
817 ) -> Result<Vec<L2Block>, BlockStoreError> {
818 if start_height > end_height {
819 return Ok(Vec::new());
820 }
821 let mut blocks = Vec::with_capacity((end_height - start_height + 1) as usize);
822 for height in start_height..=end_height {
823 if let Some(block) = self.get_block_by_height(height)? {
824 blocks.push(block);
825 }
826 }
827 Ok(blocks)
828 }
829
830 /// Look up the canonical [`BlockRecord`] at `height` ([`BLK-015`](../docs/requirements/domains/block_storage/specs/BLK-015.md), [`BLK-004`](../docs/requirements/domains/block_storage/specs/BLK-004.md)).
831 ///
832 /// **Resolution:** Same [`CF_CANONICAL`] → [`Bytes32`] step as [`Self::get_block_by_height`], then [`Self::get_record`]
833 /// so misses load **bincode headers only** from [`CF_HEADERS`] ([`SER-002`](../docs/requirements/domains/serialization/specs/SER-002.md)) — no zstd frame read from [`CF_BLOCKS`].
834 ///
835 /// **Returns:** `Ok(None)` when the height index is missing or when neither [`CF_HEADERS`] nor caches can supply a header.
836 ///
837 /// **Canonical resolution:** Same [`Self::get_hash_by_height`] dual layer as [`Self::get_block_by_height`] ([`CAN-001`](../docs/requirements/domains/canonical_chain/specs/CAN-001.md)).
838 pub fn get_record_by_height(
839 &self,
840 height: u64,
841 ) -> Result<Option<BlockRecord>, BlockStoreError> {
842 let Some(hash) = self.get_hash_by_height(height)? else {
843 return Ok(None);
844 };
845 self.get_record(&hash)
846 }
847
848 /// Look up the canonical header at `height` ([`CAN-006`](../docs/requirements/domains/canonical_chain/specs/CAN-006.md)).
849 ///
850 /// **Algorithm:** [`Self::get_hash_by_height`] → [`Self::get_header`]
851 /// ([`BLK-003`](../docs/requirements/domains/block_storage/specs/BLK-003.md) cache + bincode).
852 ///
853 /// **Returns:** `Ok(None)` when the height is not canonical or the header row is absent.
854 ///
855 /// **Lighter than `get_block_by_height`:** Headers are uncompressed bincode (~700 bytes)
856 /// versus full block bodies (zstd decompression + larger payload). Use this when only
857 /// header fields are needed (e.g., parent-hash walks, timestamp checks).
858 pub fn get_header_by_height(
859 &self,
860 height: u64,
861 ) -> Result<Option<L2BlockHeader>, BlockStoreError> {
862 let Some(hash) = self.get_hash_by_height(height)? else {
863 return Ok(None);
864 };
865 self.get_header(&hash)
866 }
867
868 /// Collect canonical block hashes for all heights in the given epoch.
869 ///
870 /// **Algorithm:** Uses [`dig_epoch::first_height_in_epoch`] and
871 /// [`dig_epoch::epoch_checkpoint_height`] to derive the inclusive `[start, end]`
872 /// height range, then calls [`Self::get_hash_by_height`] for each height. Stops
873 /// early when a height returns `None` (chain hasn't reached that height yet).
874 ///
875 /// **Returns:** A `Vec<Bytes32>` containing one hash per canonical height in the epoch,
876 /// in ascending height order. May be shorter than `BLOCKS_PER_EPOCH` if the chain is
877 /// still growing into the epoch, or empty if the epoch is entirely beyond the chain tip.
878 ///
879 /// **Requirement:** [`CAN-006`](../docs/requirements/domains/canonical_chain/specs/CAN-006.md) § Epoch Block Hashes.
880 pub fn get_epoch_block_hashes(&self, epoch: u64) -> Result<Vec<Bytes32>, BlockStoreError> {
881 let start = dig_epoch::first_height_in_epoch(epoch);
882 let end = dig_epoch::epoch_checkpoint_height(epoch);
883 let mut hashes = Vec::new();
884 for height in start..=end {
885 if let Some(hash) = self.get_hash_by_height(height)? {
886 hashes.push(hash);
887 } else {
888 break; // chain hasn't reached this height yet
889 }
890 }
891 Ok(hashes)
892 }
893
894 /// **[`BLK-015`](../docs/requirements/domains/block_storage/specs/BLK-015.md)** — Materialize canonical [`BlockRecord`]s for `[start_height, end_height]` inclusive ([`NORMATIVE` BLK-015](../docs/requirements/domains/block_storage/NORMATIVE.md#blk-015-get-records-in-range-get_records_in_range)).
895 ///
896 /// **Semantics:** Matches [`Self::get_blocks_in_range`] ordering and gap rules ([`BLK-014`](../docs/requirements/domains/block_storage/specs/BLK-014.md)), but each row comes from [`Self::get_record_by_height`] so operators avoid zstd decompression on the hot path ([`BLK-015`](../docs/requirements/domains/block_storage/specs/BLK-015.md) § Specification).
897 ///
898 /// **Cache interaction:** [`Self::get_record`] may insert derived rows into [`Self::record_cache`] / [`Self::header_cache`];
899 /// repeated scans therefore become cheaper, mirroring single-hash lookups ([`CAC-003`](../docs/requirements/domains/caching/specs/CAC-003.md) precursor).
900 pub fn get_records_in_range(
901 &self,
902 start_height: u64,
903 end_height: u64,
904 ) -> Result<Vec<BlockRecord>, BlockStoreError> {
905 if start_height > end_height {
906 return Ok(Vec::new());
907 }
908 let mut records = Vec::with_capacity((end_height - start_height + 1) as usize);
909 for height in start_height..=end_height {
910 if let Some(record) = self.get_record_by_height(height)? {
911 records.push(record);
912 }
913 }
914 Ok(records)
915 }
916
917 /// How many times [`Self::get_block`] reached RocksDB [`CF_BLOCKS`] after a cache miss (includes `Ok(None)` probes).
918 ///
919 /// **Tests / ops:** [`tests/blk_002_tests.rs`] asserts hits add zero; misses increment exactly once per call.
920 pub fn cf_blocks_physical_get_count(&self) -> u64 {
921 self.cf_blocks_physical_gets.load(Ordering::Relaxed) as u64
922 }
923
924 /// How many times [`Self::get_blocks_by_hash`] invoked [`rocksdb::DB::multi_get_cf`] because at least one hash missed
925 /// [`Self::block_cache`] ([`BLK-005`](../docs/requirements/domains/block_storage/specs/BLK-005.md); see [`tests/blk_005_tests.rs`]).
926 #[inline]
927 pub fn cf_blocks_multi_get_batch_count(&self) -> u64 {
928 self.cf_blocks_multi_get_batches.load(Ordering::Relaxed) as u64
929 }
930
931 /// RocksDB readahead hint (bytes) copied from [`BlockStoreConfig::readahead_size`](crate::BlockStoreConfig::readahead_size) at open ([`BLK-006`](../docs/requirements/domains/block_storage/specs/BLK-006.md) AC §4).
932 #[must_use]
933 pub fn readahead_size(&self) -> usize {
934 self.readahead_size
935 }
936
937 /// How many times [`StreamBlocksInRange`] issued [`DB::get_cf_opt`](rocksdb::DB::get_cf_opt) against [`CF_BLOCKS`]
938 /// after a block-cache miss while streaming ([`BLK-006`](../docs/requirements/domains/block_storage/specs/BLK-006.md); [`tests/blk_006_tests.rs`]).
939 #[must_use]
940 pub fn cf_blocks_stream_physical_get_count(&self) -> u64 {
941 self.cf_blocks_stream_physical_gets.load(Ordering::Relaxed) as u64
942 }
943
944 /// Retrieve a block header by hash ([`BLK-003`](../docs/requirements/domains/block_storage/specs/BLK-003.md)).
945 ///
946 /// **Order:** [`Self::header_cache`] → on miss, `get_cf` [`CF_HEADERS`] → [`Self::deserialize_header`] (**no zstd**;
947 /// [`SER-002`](../docs/requirements/domains/serialization/specs/SER-002.md)).
948 ///
949 /// **Write path:** [`Self::put_block`] / [`Self::init_genesis`] insert headers in parallel with block bodies.
950 pub fn get_header(&self, hash: &Bytes32) -> Result<Option<L2BlockHeader>, BlockStoreError> {
951 if let Some(header) = self.header_cache.get_clone(hash) {
952 return Ok(Some(header));
953 }
954 let cf = self.cf(CF_HEADERS)?;
955 self.cf_headers_physical_gets
956 .fetch_add(1, Ordering::Relaxed);
957 let raw_opt = self.db.get_cf(cf, hash_key(hash).as_slice())?;
958 let Some(raw) = raw_opt else {
959 return Ok(None);
960 };
961 let header = Self::deserialize_header(&raw)?;
962 self.header_cache.insert(*hash, header.clone());
963 // CAC-005 (API-002): populate hash→height on header read-through
964 self.hash_to_height_cache.insert(*hash, header.height);
965 Ok(Some(header))
966 }
967
968 /// Drop one header from the in-memory LRU ([`BLK-003`](../docs/requirements/domains/block_storage/specs/BLK-003.md) tests / future invalidation).
969 pub fn invalidate_header_cache_entry(&self, hash: &Bytes32) {
970 self.header_cache.remove(hash);
971 }
972
973 /// Count of RocksDB [`CF_HEADERS`] `get_cf` calls from [`Self::get_header`] or [`Self::get_record`]
974 /// when the in-memory header **and** record caches do not already supply the header/record ([`BLK-003`](../docs/requirements/domains/block_storage/specs/BLK-003.md), [`BLK-004`](../docs/requirements/domains/block_storage/specs/BLK-004.md)).
975 ///
976 /// **Note:** [`Self::get_record`] consults [`Self::header_cache`] before touching RocksDB, so a record-cache
977 /// miss with a warm header cache does **not** increment this counter (still satisfies “derive from header”).
978 pub fn cf_headers_physical_get_count(&self) -> u64 {
979 self.cf_headers_physical_gets.load(Ordering::Relaxed) as u64
980 }
981
982 /// **[`BLK-001`](../docs/requirements/domains/block_storage/specs/BLK-001.md)** — Primary name in
983 /// [`IMPLEMENTATION_ORDER.md`](../docs/requirements/IMPLEMENTATION_ORDER.md) Phase 5.
984 ///
985 /// **Pipeline:** zstd payload → [`CF_BLOCKS`], bincode header → [`CF_HEADERS`], optional height index →
986 /// [`CF_CANONICAL`]; [`BlockRecord`] is derived with [`BlockStatus::Validated`] and stored only in
987 /// [`Self::record_cache`] ([`TYP-004`](../docs/requirements/domains/storage_types/specs/TYP-004.md) persistence rule).
988 ///
989 /// **Idempotency:** If the block hash already exists in `CF_BLOCKS`, returns `Ok(false)` and performs no writes
990 /// ([`start.md`](../docs/prompt/start.md) hard requirement §9).
991 ///
992 /// **[`SER-005`](../docs/requirements/domains/serialization/specs/SER-005.md):** A successful insert that makes
993 /// [`Self::block_count`] reach [`DICT_TRAINING_THRESHOLD`] triggers **one-time** dictionary training when
994 /// [`BlockStoreConfig::use_compression_dict`](crate::BlockStoreConfig) is `true`.
995 pub fn put_block(&self, block: &L2Block, canonical: bool) -> Result<bool, BlockStoreError> {
996 if self.read_only {
997 return Err(BlockStoreError::Serialization(
998 ERR_MUTATION_READ_ONLY.into(),
999 ));
1000 }
1001 let hash = block.hash();
1002 let cf_b = self.cf(CF_BLOCKS)?;
1003 if self.db.get_cf(cf_b, hash_key(&hash).as_slice())?.is_some() {
1004 return Ok(false);
1005 }
1006 let compressed = self.serialize_block(block)?;
1007 let header_bytes = Self::serialize_header(&block.header)?;
1008 let mut batch = WriteBatch::default();
1009 let cf_h = self.cf(CF_HEADERS)?;
1010 batch.put_cf(cf_b, hash_key(&hash).as_slice(), &compressed);
1011 batch.put_cf(cf_h, hash_key(&hash).as_slice(), &header_bytes);
1012 if canonical {
1013 let cf_c = self.cf(CF_CANONICAL)?;
1014 batch.put_cf(cf_c, height_key(block.height()), hash_key(&hash).as_slice());
1015 }
1016 self.db.write(batch)?;
1017 if canonical {
1018 self.canonical_bin
1019 .write()
1020 .extend_write(block.height(), &hash)?;
1021 // CAC-004: populate height→hash cache for canonical blocks
1022 self.insert_canonical_height_cache(block.height(), hash);
1023 }
1024 let record = BlockRecord::from_header(&block.header, BlockStatus::Validated);
1025 self.record_cache.lock().insert(hash, record);
1026 self.block_cache.insert(hash, block.clone());
1027 self.header_cache.insert(hash, block.header.clone());
1028 // CAC-005: populate hash→height cache for all blocks (canonical or not)
1029 self.hash_to_height_cache.insert(hash, block.height());
1030 self.maybe_train_dictionary()?;
1031 Ok(true)
1032 }
1033
1034 /// Alias for [`Self::put_block`] — matches the BLK-001 normative snippet name `put` ([`NORMATIVE.md` § BLK-001](../docs/requirements/domains/block_storage/NORMATIVE.md)).
1035 #[inline]
1036 pub fn put(&self, block: &L2Block, canonical: bool) -> Result<bool, BlockStoreError> {
1037 self.put_block(block, canonical)
1038 }
1039
1040 // -----------------------------------------------------------------------
1041 // Rollback & Reorg (ROR domain)
1042 // -----------------------------------------------------------------------
1043
1044 /// Public accessor for the pruning floor: minimum retained block height.
1045 ///
1046 /// Returns `0` when no pruning has occurred (all heights retained).
1047 /// After [`PRN-001`] runs, this reflects the `META_MIN_HEIGHT` value in `CF_METADATA`.
1048 ///
1049 /// **Requirement:** [`ROR-005`](../docs/requirements/domains/rollback_reorg/specs/ROR-005.md).
1050 /// Public accessor for the pruning floor: minimum retained block height.
1051 ///
1052 /// Returns `0` when no pruning has occurred (all heights retained).
1053 /// Uses the cached [`AtomicU64`] for fast lock-free access ([`PRN-004`]).
1054 pub fn min_retained_height(&self) -> Result<u64, BlockStoreError> {
1055 Ok(self.min_retained_height_cached.load(Ordering::Acquire))
1056 }
1057
1058 /// Read-only preview: which canonical blocks would be reverted by a rollback to `target_height`.
1059 ///
1060 /// Returns hashes in **descending** height order (tip first). Returns empty `Vec` when
1061 /// no tip is set or `target_height >= tip.height`. Does NOT modify any state.
1062 ///
1063 /// **Requirement:** [`ROR-006`](../docs/requirements/domains/rollback_reorg/specs/ROR-006.md).
1064 pub fn blocks_to_revert(&self, target_height: u64) -> Result<Vec<Bytes32>, BlockStoreError> {
1065 let Some(current_tip) = self.tip() else {
1066 return Ok(Vec::new());
1067 };
1068 if target_height >= current_tip.height {
1069 return Ok(Vec::new());
1070 }
1071 let mut reverted = Vec::new();
1072 for h in (target_height + 1..=current_tip.height).rev() {
1073 if let Some(hash) = self.get_hash_by_height(h)? {
1074 reverted.push(hash);
1075 }
1076 }
1077 Ok(reverted)
1078 }
1079
1080 /// Revert the canonical chain to `target_height`, removing higher heights from
1081 /// the canonical index and updating the tip.
1082 ///
1083 /// # Validation ([`ROR-005`](../docs/requirements/domains/rollback_reorg/specs/ROR-005.md))
1084 ///
1085 /// Checked in order before any mutation:
1086 /// 1. **NoTip** — no chain tip set.
1087 /// 2. **RollbackAboveTip** — `target_height > tip.height`.
1088 /// 3. **RollbackBelowMin** — `target_height < min_retained_height()`.
1089 ///
1090 /// # Mutation ([`ROR-001`](../docs/requirements/domains/rollback_reorg/specs/ROR-001.md))
1091 ///
1092 /// 1. Collect reverted hashes from tip down to `target_height + 1`.
1093 /// 2. `WriteBatch` deletes on CF_CANONICAL for each reverted height.
1094 /// 3. Truncate `canonical.bin` to `(target_height + 1) * 32`.
1095 /// 4. Update tip to the block at `target_height`.
1096 /// 5. Mark reverted blocks as non-canonical in record_cache.
1097 /// 6. Evict reverted heights from canonical_height_cache.
1098 ///
1099 /// # Returns
1100 ///
1101 /// Hashes of reverted blocks in **descending** height order (tip first).
1102 /// Returns empty Vec for a no-op rollback at the current tip.
1103 /// Block data in CF_BLOCKS is NOT deleted (fork preservation per ROR-004).
1104 pub fn rollback_to_height(&self, target_height: u64) -> Result<Vec<Bytes32>, BlockStoreError> {
1105 if self.read_only {
1106 return Err(BlockStoreError::Serialization(
1107 ERR_MUTATION_READ_ONLY.into(),
1108 ));
1109 }
1110 // ROR-005: boundary validation (in order: NoTip, AboveTip, BelowMin)
1111 let current_tip = self.tip().ok_or(BlockStoreError::NoTip)?;
1112 if target_height > current_tip.height {
1113 return Err(BlockStoreError::RollbackAboveTip {
1114 target: target_height,
1115 tip: current_tip.height,
1116 });
1117 }
1118 let min_height = self.min_retained_height()?;
1119 if target_height < min_height {
1120 return Err(BlockStoreError::RollbackBelowMin {
1121 target: target_height,
1122 min: min_height,
1123 });
1124 }
1125 // No-op: rollback at current tip
1126 if target_height == current_tip.height {
1127 return Ok(Vec::new());
1128 }
1129
1130 // Collect hashes to revert (descending order)
1131 let mut reverted = Vec::new();
1132 for h in (target_height + 1..=current_tip.height).rev() {
1133 if let Some(hash) = self.get_hash_by_height(h)? {
1134 reverted.push(hash);
1135 }
1136 }
1137
1138 // WriteBatch: delete CF_CANONICAL entries for reverted heights
1139 let cf_c = self.cf(CF_CANONICAL)?;
1140 let mut batch = WriteBatch::default();
1141 for h in target_height + 1..=current_tip.height {
1142 batch.delete_cf(cf_c, height_key(h));
1143 }
1144 self.db.write(batch)?;
1145
1146 // Truncate canonical.bin (mmap) to (target_height + 1) * 32
1147 self.canonical_bin
1148 .write()
1149 .truncate_to_height(target_height)?;
1150
1151 // Update tip to block at target_height
1152 if let Some(target_hash) = self.get_hash_by_height(target_height)? {
1153 self.set_tip(ChainTip {
1154 hash: target_hash,
1155 height: target_height,
1156 })?;
1157 }
1158
1159 // Mark reverted blocks as non-canonical in record_cache
1160 {
1161 let mut cache = self.record_cache.lock();
1162 for hash in &reverted {
1163 if let Some(r) = cache.get_mut(hash) {
1164 r.in_canonical_chain = false;
1165 }
1166 }
1167 }
1168
1169 // Evict reverted heights from canonical_height_cache (CAC-004)
1170 {
1171 let mut hcache = self.canonical_height_cache.write();
1172 for h in target_height + 1..=current_tip.height {
1173 hcache.remove(&h);
1174 }
1175 }
1176
1177 Ok(reverted)
1178 }
1179
1180 /// Atomically rollback the canonical chain to `ancestor_height` and re-canonicalize
1181 /// the blocks in `new_chain_hashes`.
1182 ///
1183 /// # Algorithm ([`ROR-003`](../docs/requirements/domains/rollback_reorg/specs/ROR-003.md))
1184 ///
1185 /// 1. **Validate:** NoTip → error. EmptyReorgChain → error. Each hash in `new_chain_hashes`
1186 /// must be in the store (BlockNotInStore if not).
1187 /// 2. **WriteBatch (atomic):**
1188 /// - Delete CF_CANONICAL entries for heights `ancestor_height + 1` .. `current_tip.height`.
1189 /// - Put new canonical entries for each hash in `new_chain_hashes` (height from record).
1190 /// - Write new tip (last hash in `new_chain_hashes`) to META_TIP.
1191 /// 3. **Post-commit:**
1192 /// - Truncate `canonical.bin` to `ancestor_height`, then write new hashes.
1193 /// - Update record_cache: reverted → `in_canonical_chain=false`, applied → `true`.
1194 /// - Update in-memory tip.
1195 /// - Evict/update canonical_height_cache.
1196 ///
1197 /// # Returns
1198 ///
1199 /// [`ReorgResult`] with `reverted` (descending), `applied` (ascending), and `new_tip`.
1200 pub fn apply_reorg(
1201 &self,
1202 ancestor_height: u64,
1203 new_chain_hashes: &[Bytes32],
1204 ) -> Result<ReorgResult, BlockStoreError> {
1205 if self.read_only {
1206 return Err(BlockStoreError::Serialization(
1207 ERR_MUTATION_READ_ONLY.into(),
1208 ));
1209 }
1210 let current_tip = self.tip().ok_or(BlockStoreError::NoTip)?;
1211 if new_chain_hashes.is_empty() {
1212 return Err(BlockStoreError::EmptyReorgChain);
1213 }
1214
1215 // Validate all new chain hashes exist and collect their records
1216 let mut new_records: Vec<(Bytes32, BlockRecord)> =
1217 Vec::with_capacity(new_chain_hashes.len());
1218 for hash in new_chain_hashes {
1219 let record = self
1220 .get_record(hash)?
1221 .ok_or(BlockStoreError::BlockNotInStore(*hash))?;
1222 new_records.push((*hash, record));
1223 }
1224
1225 let cf_c = self.cf(CF_CANONICAL)?;
1226 let cf_meta = self.cf(CF_METADATA)?;
1227 let mut batch = WriteBatch::default();
1228
1229 // Phase 1: Rollback — delete old canonical entries above ancestor_height
1230 let mut reverted = Vec::new();
1231 for h in (ancestor_height + 1..=current_tip.height).rev() {
1232 if let Some(hash) = self.get_hash_by_height(h)? {
1233 reverted.push(hash);
1234 }
1235 batch.delete_cf(cf_c, height_key(h));
1236 }
1237
1238 // Phase 2: Apply new chain
1239 for (hash, record) in &new_records {
1240 batch.put_cf(cf_c, height_key(record.height), hash_key(hash).as_slice());
1241 }
1242
1243 // Phase 3: Write new tip in the same batch
1244 let new_tip_hash = new_chain_hashes
1245 .last()
1246 .copied()
1247 .expect("non-empty checked above");
1248 let new_tip_height = new_records.last().expect("non-empty").1.height;
1249 let new_tip = ChainTip {
1250 hash: new_tip_hash,
1251 height: new_tip_height,
1252 };
1253 batch.put_cf(cf_meta, META_TIP.as_bytes(), new_tip.to_bytes().as_slice());
1254
1255 // Atomic commit
1256 self.db.write(batch)?;
1257
1258 // Post-commit: update mmap
1259 self.canonical_bin
1260 .write()
1261 .truncate_to_height(ancestor_height)?;
1262 for (hash, record) in &new_records {
1263 self.canonical_bin
1264 .write()
1265 .extend_write(record.height, hash)?;
1266 }
1267
1268 // Post-commit: update record cache
1269 {
1270 let mut cache = self.record_cache.lock();
1271 for hash in &reverted {
1272 if let Some(r) = cache.get_mut(hash) {
1273 r.in_canonical_chain = false;
1274 }
1275 }
1276 for (hash, _) in &new_records {
1277 if let Some(r) = cache.get_mut(hash) {
1278 r.in_canonical_chain = true;
1279 }
1280 }
1281 }
1282
1283 // Post-commit: update canonical_height_cache
1284 {
1285 let mut hcache = self.canonical_height_cache.write();
1286 for h in ancestor_height + 1..=current_tip.height {
1287 hcache.remove(&h);
1288 }
1289 for (hash, record) in &new_records {
1290 hcache.insert(record.height, *hash);
1291 }
1292 }
1293
1294 // Post-commit: update in-memory tip
1295 *self.tip.write() = Some(new_tip);
1296
1297 Ok(ReorgResult {
1298 reverted,
1299 applied: new_chain_hashes.to_vec(),
1300 new_tip,
1301 })
1302 }
1303
1304 /// Walk the `parent_hash` chain from `hash` backward, returning the first block
1305 /// that is the canonical block at its height.
1306 ///
1307 /// # Algorithm ([`ROR-002`](../docs/requirements/domains/rollback_reorg/specs/ROR-002.md))
1308 ///
1309 /// For up to `max_depth` steps:
1310 /// 1. Load the [`BlockRecord`] for `current_hash` (cache or CF_HEADERS derive).
1311 /// 2. Check if `get_hash_by_height(record.height) == current_hash` — if so, this
1312 /// block is canonical and is the common ancestor.
1313 /// 3. Otherwise, follow `record.parent_hash` and repeat.
1314 ///
1315 /// # Returns
1316 ///
1317 /// - `Ok(Some((hash, height)))` — the first canonical ancestor found.
1318 /// - `Ok(None)` — hash not in store, parent chain broken, or `max_depth` exceeded.
1319 ///
1320 /// # Use case
1321 ///
1322 /// When a new block arrives whose parent is not the current tip, call this with the
1323 /// new block’s parent hash to find where the fork diverged from the canonical chain.
1324 /// The result feeds into [`apply_reorg`](Self) (ROR-003) as the `ancestor_height`.
1325 ///
1326 /// # Read-only
1327 ///
1328 /// This method does not modify any state. Safe to call concurrently.
1329 pub fn find_common_ancestor(
1330 &self,
1331 hash: &Bytes32,
1332 max_depth: u64,
1333 ) -> Result<Option<(Bytes32, u64)>, BlockStoreError> {
1334 let mut current_hash = *hash;
1335 for _ in 0..max_depth {
1336 // CAC-005 (API-002): try hash_to_height_cache first to avoid header deserialization
1337 // when we only need the height for the canonical check.
1338 let record = match self.get_record(¤t_hash)? {
1339 Some(r) => r,
1340 None => return Ok(None), // block not in store or chain broken
1341 };
1342 let height = record.height;
1343 // Populate hash→height cache on access (read-through for future lookups)
1344 self.hash_to_height_cache.insert(current_hash, height);
1345 // Check if this block is canonical at its height
1346 if let Some(canonical_hash) = self.get_hash_by_height(height)? {
1347 if canonical_hash == current_hash {
1348 return Ok(Some((current_hash, height)));
1349 }
1350 }
1351 // Walk backwards via parent_hash
1352 current_hash = record.parent_hash;
1353 }
1354 Ok(None) // exceeded max_depth
1355 }
1356
1357 /// **[`BLK-009`](../docs/requirements/domains/block_storage/specs/BLK-009.md)** — Persist an [`AttestedBlock`] under the block’s hash key.
1358 ///
1359 /// **Key:** [`hash_key`](crate::encoding::hash_key)(`hash`) — raw 32 bytes in [`CF_ATTESTED`] ([`KEY-001`](../docs/requirements/domains/key_encoding/specs/KEY-001_hash_keys.md)), identical key shape to [`CF_BLOCKS`] / [`CF_HEADERS`].
1360 ///
1361 /// **Value:** [`bincode::serialize`] of `attested` (uncompressed; attestations are small per BLK-009 implementation notes).
1362 ///
1363 /// **Hash vs payload:** Callers normally pass `hash == attested.hash()`; this method does **not** verify that invariant so
1364 /// tests and migration tooling can stage rows independently of body presence in [`CF_BLOCKS`].
1365 ///
1366 /// **Overwrite (AC §4):** A second call with the same `hash` replaces the previous value (`DB::put_cf`).
1367 ///
1368 /// **Read-only:** Returns [`BlockStoreError::Serialization`] with [`ERR_MUTATION_READ_ONLY`] — same contract as [`Self::put_block`].
1369 pub fn put_attestation(
1370 &self,
1371 hash: &Bytes32,
1372 attested: &AttestedBlock,
1373 ) -> Result<(), BlockStoreError> {
1374 if self.read_only {
1375 return Err(BlockStoreError::Serialization(
1376 ERR_MUTATION_READ_ONLY.into(),
1377 ));
1378 }
1379 let bytes = bincode::serialize(attested)
1380 .map_err(|e| BlockStoreError::Serialization(e.to_string()))?;
1381 let cf = self.cf(CF_ATTESTED)?;
1382 self.db.put_cf(cf, hash_key(hash).as_slice(), &bytes)?;
1383 Ok(())
1384 }
1385
1386 /// **[`BLK-009`](../docs/requirements/domains/block_storage/specs/BLK-009.md)** — Read [`AttestedBlock`] bytes from [`CF_ATTESTED`].
1387 ///
1388 /// **Miss:** [`Ok(None)]` when no row exists (AC §3).
1389 ///
1390 /// **No attestation cache (yet):** Each call performs a RocksDB `get_cf` + bincode decode (BLK-009 notes; hot paths may add [`CAC-*`] later).
1391 ///
1392 /// **Corrupt rows:** Malformed bincode surfaces as [`BlockStoreError::Serialization`] so operators can distinguish “missing” vs “bad bytes”.
1393 pub fn get_attestation(
1394 &self,
1395 hash: &Bytes32,
1396 ) -> Result<Option<AttestedBlock>, BlockStoreError> {
1397 let cf = self.cf(CF_ATTESTED)?;
1398 let raw = match self.db.get_cf(cf, hash_key(hash).as_slice())? {
1399 Some(b) => b,
1400 None => return Ok(None),
1401 };
1402 let attested: AttestedBlock = bincode::deserialize(&raw).map_err(|e| {
1403 BlockStoreError::Serialization(format!(
1404 "get_attestation: bincode deserialize failed: {e}"
1405 ))
1406 })?;
1407 Ok(Some(attested))
1408 }
1409
1410 // -----------------------------------------------------------------------
1411 // Checkpoint Storage (CKP domain)
1412 // -----------------------------------------------------------------------
1413
1414 /// Persist a [`StoredCheckpoint`] to [`CF_CHECKPOINTS`] keyed by epoch.
1415 ///
1416 /// **Key:** [`epoch_key`](crate::encoding::epoch_key)(`checkpoint.checkpoint.epoch`) — 8-byte big-endian.
1417 /// **Value:** [`bincode::serialize`] of the full [`StoredCheckpoint`].
1418 /// **Idempotent:** overwrites any existing checkpoint at the same epoch.
1419 ///
1420 /// **Requirement:** [`CKP-001`](../docs/requirements/domains/checkpoint_storage/specs/CKP-001_put_checkpoint.md).
1421 pub fn put_checkpoint(
1422 &self,
1423 checkpoint: &crate::StoredCheckpoint,
1424 ) -> Result<(), BlockStoreError> {
1425 if self.read_only {
1426 return Err(BlockStoreError::Serialization(
1427 ERR_MUTATION_READ_ONLY.into(),
1428 ));
1429 }
1430 let epoch = checkpoint.checkpoint.epoch;
1431 let key = crate::encoding::epoch_key(epoch);
1432 let value = checkpoint
1433 .encode_bincode()
1434 .map_err(|e| BlockStoreError::Serialization(e.to_string()))?;
1435 let cf = self.cf(CF_CHECKPOINTS)?;
1436 self.db.put_cf(cf, key.as_slice(), &value)?;
1437 Ok(())
1438 }
1439
1440 /// Retrieve a [`StoredCheckpoint`] by epoch from [`CF_CHECKPOINTS`].
1441 ///
1442 /// Returns `Ok(None)` if no checkpoint exists for the given epoch.
1443 ///
1444 /// **Requirement:** [`CKP-002`](../docs/requirements/domains/checkpoint_storage/specs/CKP-002_get_checkpoint.md).
1445 pub fn get_checkpoint(
1446 &self,
1447 epoch: u64,
1448 ) -> Result<Option<crate::StoredCheckpoint>, BlockStoreError> {
1449 let cf = self.cf(CF_CHECKPOINTS)?;
1450 let key = crate::encoding::epoch_key(epoch);
1451 let Some(bytes) = self.db.get_cf(cf, key.as_slice())? else {
1452 return Ok(None);
1453 };
1454 let checkpoint = crate::StoredCheckpoint::decode_bincode(&bytes)
1455 .map_err(|e| BlockStoreError::Serialization(e.to_string()))?;
1456 Ok(Some(checkpoint))
1457 }
1458
1459 /// Retrieve the most recent checkpoint (highest epoch) via reverse iterator.
1460 ///
1461 /// Returns `Ok(None)` if no checkpoints are stored.
1462 ///
1463 /// **Requirement:** [`CKP-003`](../docs/requirements/domains/checkpoint_storage/specs/CKP-003_get_latest_checkpoint.md).
1464 pub fn get_latest_checkpoint(
1465 &self,
1466 ) -> Result<Option<crate::StoredCheckpoint>, BlockStoreError> {
1467 let cf = self.cf(CF_CHECKPOINTS)?;
1468 let mut iter = self.db.iterator_cf(cf, IteratorMode::End);
1469 let Some(item) = iter.next() else {
1470 return Ok(None);
1471 };
1472 let (_key, value) = item?;
1473 let checkpoint = crate::StoredCheckpoint::decode_bincode(&value)
1474 .map_err(|e| BlockStoreError::Serialization(e.to_string()))?;
1475 Ok(Some(checkpoint))
1476 }
1477
1478 /// Retrieve all checkpoints within an epoch range `[start_epoch, end_epoch]` inclusive.
1479 ///
1480 /// Returns empty `Vec` if no checkpoints exist in the range. If `start_epoch > end_epoch`,
1481 /// returns empty (no error).
1482 ///
1483 /// **Requirement:** [`CKP-004`](../docs/requirements/domains/checkpoint_storage/specs/CKP-004_get_checkpoints_in_range.md).
1484 pub fn get_checkpoints_in_range(
1485 &self,
1486 start_epoch: u64,
1487 end_epoch: u64,
1488 ) -> Result<Vec<crate::StoredCheckpoint>, BlockStoreError> {
1489 if start_epoch > end_epoch {
1490 return Ok(Vec::new());
1491 }
1492 let cf = self.cf(CF_CHECKPOINTS)?;
1493 let start_key = crate::encoding::epoch_key(start_epoch);
1494 let mode = IteratorMode::From(&start_key, Direction::Forward);
1495 let iter = self.db.iterator_cf(cf, mode);
1496 let mut result = Vec::new();
1497 for item in iter {
1498 let (key_bytes, value) = item?;
1499 if key_bytes.len() != 8 {
1500 continue;
1501 }
1502 let key_arr: [u8; 8] = key_bytes.as_ref().try_into().unwrap_or([0; 8]);
1503 let epoch = crate::encoding::decode_epoch_key(&key_arr);
1504 if epoch > end_epoch {
1505 break;
1506 }
1507 let checkpoint = crate::StoredCheckpoint::decode_bincode(&value)
1508 .map_err(|e| BlockStoreError::Serialization(e.to_string()))?;
1509 result.push(checkpoint);
1510 }
1511 Ok(result)
1512 }
1513
1514 // -----------------------------------------------------------------------
1515 // Pruning (PRN domain)
1516 // -----------------------------------------------------------------------
1517
1518 /// Remove all blocks, headers, attestations, and canonical entries below `height`.
1519 ///
1520 /// # Algorithm ([`PRN-001`](../docs/requirements/domains/pruning/specs/PRN-001_prune_before_height.md))
1521 ///
1522 /// 1. Iterate `CF_CANONICAL` from `min_retained_height` to `height - 1`, collecting hashes.
1523 /// 2. Also scan `CF_HEADERS` for non-canonical blocks below `height` ([`PRN-005`]).
1524 /// 3. Single `WriteBatch` deletes from CF_BLOCKS, CF_HEADERS, CF_ATTESTED, CF_CANONICAL.
1525 /// 4. Update `META_MIN_HEIGHT` in the same batch.
1526 /// 5. Post-commit: evict from all caches, update `AtomicU64`.
1527 ///
1528 /// # Returns
1529 ///
1530 /// Count of blocks pruned (canonical + non-canonical).
1531 pub fn prune_before_height(&self, height: u64) -> Result<usize, BlockStoreError> {
1532 if self.read_only {
1533 return Err(BlockStoreError::Serialization(
1534 ERR_MUTATION_READ_ONLY.into(),
1535 ));
1536 }
1537 let current_min = self.min_retained_height()?;
1538 if height <= current_min {
1539 return Ok(0);
1540 }
1541
1542 let cf_b = self.cf(CF_BLOCKS)?;
1543 let cf_h = self.cf(CF_HEADERS)?;
1544 let cf_a = self.cf(CF_ATTESTED)?;
1545 let cf_c = self.cf(CF_CANONICAL)?;
1546 let cf_meta = self.cf(CF_METADATA)?;
1547
1548 let mut batch = WriteBatch::default();
1549 let mut pruned_hashes: Vec<Bytes32> = Vec::new();
1550
1551 // Phase 1: Canonical blocks below target height
1552 for h in current_min..height {
1553 if let Some(hash) = self.get_hash_by_height(h)? {
1554 batch.delete_cf(cf_b, hash_key(&hash).as_slice());
1555 batch.delete_cf(cf_h, hash_key(&hash).as_slice());
1556 batch.delete_cf(cf_a, hash_key(&hash).as_slice());
1557 batch.delete_cf(cf_c, height_key(h));
1558 pruned_hashes.push(hash);
1559 }
1560 }
1561
1562 // Phase 2 (PRN-005): Non-canonical blocks — scan CF_HEADERS for blocks below height
1563 // that were NOT already collected in the canonical pass.
1564 let canonical_set: std::collections::HashSet<Bytes32> =
1565 pruned_hashes.iter().copied().collect();
1566 let header_iter = self.db.iterator_cf(cf_h, IteratorMode::Start);
1567 for item in header_iter {
1568 let (key_bytes, value_bytes) = item?;
1569 if key_bytes.len() != 32 {
1570 continue;
1571 }
1572 let arr: [u8; 32] = key_bytes.as_ref().try_into().unwrap_or([0; 32]);
1573 let hash = Bytes32::new(arr);
1574 if canonical_set.contains(&hash) {
1575 continue; // already handled in canonical pass
1576 }
1577 // Deserialize header to check height
1578 if let Ok(header) = Self::deserialize_header(&value_bytes) {
1579 if header.height < height {
1580 batch.delete_cf(cf_b, hash_key(&hash).as_slice());
1581 batch.delete_cf(cf_h, hash_key(&hash).as_slice());
1582 batch.delete_cf(cf_a, hash_key(&hash).as_slice());
1583 pruned_hashes.push(hash);
1584 }
1585 }
1586 }
1587
1588 // Update META_MIN_HEIGHT in the same batch
1589 batch.put_cf(cf_meta, META_MIN_HEIGHT.as_bytes(), height.to_le_bytes());
1590
1591 let count = pruned_hashes.len();
1592 self.db.write(batch)?;
1593
1594 // Post-commit: update AtomicU64
1595 self.min_retained_height_cached
1596 .store(height, Ordering::Release);
1597
1598 // Post-commit: evict from all caches
1599 for hash in &pruned_hashes {
1600 self.block_cache.remove(hash);
1601 self.header_cache.remove(hash);
1602 self.record_cache.lock().remove(hash);
1603 self.hash_to_height_cache.remove(hash);
1604 }
1605 // Evict canonical height cache entries below height
1606 {
1607 let mut hcache = self.canonical_height_cache.write();
1608 let to_remove: Vec<u64> = hcache.range(..height).map(|(&h, _)| h).collect();
1609 for h in to_remove {
1610 hcache.remove(&h);
1611 }
1612 }
1613
1614 Ok(count)
1615 }
1616
1617 /// Remove all checkpoints with epoch < `epoch` from CF_CHECKPOINTS.
1618 ///
1619 /// Returns the count of pruned checkpoints.
1620 ///
1621 /// **Requirement:** [`PRN-002`](../docs/requirements/domains/pruning/specs/PRN-002_prune_checkpoints_before_epoch.md).
1622 pub fn prune_checkpoints_before_epoch(&self, epoch: u64) -> Result<usize, BlockStoreError> {
1623 if self.read_only {
1624 return Err(BlockStoreError::Serialization(
1625 ERR_MUTATION_READ_ONLY.into(),
1626 ));
1627 }
1628 if epoch == 0 {
1629 return Ok(0);
1630 }
1631 let cf = self.cf(CF_CHECKPOINTS)?;
1632 let mut batch = WriteBatch::default();
1633 let mut count = 0usize;
1634 let iter = self.db.iterator_cf(cf, IteratorMode::Start);
1635 for item in iter {
1636 let (key_bytes, _value) = item?;
1637 if key_bytes.len() != 8 {
1638 continue;
1639 }
1640 let key_arr: [u8; 8] = key_bytes.as_ref().try_into().unwrap_or([0; 8]);
1641 let e = crate::encoding::decode_epoch_key(&key_arr);
1642 if e >= epoch {
1643 break;
1644 }
1645 batch.delete_cf(cf, key_bytes.as_ref());
1646 count += 1;
1647 }
1648 if count > 0 {
1649 self.db.write(batch)?;
1650 }
1651 Ok(count)
1652 }
1653
1654 /// Look up [`BlockRecord`] by hash ([`BLK-004`](../docs/requirements/domains/block_storage/specs/BLK-004.md)).
1655 ///
1656 /// **Order**
1657 /// 1. [`Self::record_cache`] (Mutex map) — clone on hit; **no** RocksDB I/O.
1658 /// 2. [`Self::header_cache`] — if the header is already deserialized (e.g. after [`Self::put_block`] or
1659 /// [`Self::get_header`]), derive [`BlockRecord::from_header`] with [`BlockStatus::Validated`] and insert into
1660 /// the record cache; **no** RocksDB `get_cf` on [`CF_HEADERS`].
1661 /// 3. Else load raw bytes from [`CF_HEADERS`], increment [`Self::cf_headers_physical_gets`], deserialize via
1662 /// [`Self::deserialize_header`], warm [`Self::header_cache`] + record cache.
1663 ///
1664 /// **Persistence:** [`BlockRecord`] is never written to any column family ([`TYP-004`](../docs/requirements/domains/storage_types/specs/TYP-004.md)); only headers live under [`CF_HEADERS`].
1665 ///
1666 /// **Read-only stores:** Record/header RAM caches start empty; the first lookup may read [`CF_HEADERS`] and populate both caches without mutating on-disk layout beyond normal reads.
1667 pub fn get_record(&self, hash: &Bytes32) -> Result<Option<BlockRecord>, BlockStoreError> {
1668 {
1669 let guard = self.record_cache.lock();
1670 if let Some(r) = guard.get(hash) {
1671 return Ok(Some(r.clone()));
1672 }
1673 }
1674 if let Some(header) = self.header_cache.get_clone(hash) {
1675 let record = BlockRecord::from_header(&header, BlockStatus::Validated);
1676 self.record_cache.lock().insert(*hash, record.clone());
1677 return Ok(Some(record));
1678 }
1679 let cf = self.cf(CF_HEADERS)?;
1680 self.cf_headers_physical_gets
1681 .fetch_add(1, Ordering::Relaxed);
1682 let Some(bytes) = self.db.get_cf(cf, hash_key(hash).as_slice())? else {
1683 return Ok(None);
1684 };
1685 let header = Self::deserialize_header(&bytes)?;
1686 self.header_cache.insert(*hash, header.clone());
1687 let record = BlockRecord::from_header(&header, BlockStatus::Validated);
1688 self.record_cache.lock().insert(*hash, record.clone());
1689 Ok(Some(record))
1690 }
1691
1692 /// Remove one hash from the in-memory [`BlockRecord`] map — **no RocksDB writes** ([`BLK-004`](../docs/requirements/domains/block_storage/specs/BLK-004.md) test plan: simulate record-cache eviction).
1693 pub fn invalidate_record_cache_entry(&self, hash: &Bytes32) {
1694 let mut guard = self.record_cache.lock();
1695 let _ = guard.remove(hash);
1696 }
1697
1698 /// **[`BLK-010`](../docs/requirements/domains/block_storage/specs/BLK-010.md)** — Set [`BlockRecord::status`] for a hash already present in [`Self::record_cache`].
1699 ///
1700 /// **No disk I/O:** [`BlockRecord`] is cache-only ([`TYP-004`](../docs/requirements/domains/storage_types/specs/TYP-004.md)); this method never touches [`rocksdb::WriteBatch`] or [`DB::put_cf`](rocksdb::DB::put_cf).
1701 ///
1702 /// **`in_canonical_chain`:** Recomputed from [`BlockStatus::is_canonical`](dig_block::BlockStatus::is_canonical) so the row stays aligned with [`BlockRecord::from_header`](crate::types::BlockRecord::from_header) ([`TYP-004`](../docs/requirements/domains/storage_types/specs/TYP-004.md) module docs).
1703 ///
1704 /// **Precondition:** The hash must already exist in the record cache (typically after [`Self::put_block`] or a cache-warming [`Self::get_record`]). Otherwise returns [`BlockStoreError::Serialization`] whose message starts with
1705 /// [`ERR_UPDATE_STATUS_RECORD_NOT_CACHED_PREFIX`](crate::error::ERR_UPDATE_STATUS_RECORD_NOT_CACHED_PREFIX)
1706 /// ([`ERR-001`](../docs/requirements/domains/error_types/specs/ERR-001_blockstoreerror_enum.md) caps the public enum at thirteen variants, so this uses the same stable-prefix pattern as read-only mutation guards).
1707 pub fn update_status(
1708 &self,
1709 hash: &Bytes32,
1710 status: BlockStatus,
1711 ) -> Result<(), BlockStoreError> {
1712 let mut guard = self.record_cache.lock();
1713 let record = guard.get_mut(hash).ok_or_else(|| {
1714 BlockStoreError::Serialization(format!(
1715 "{ERR_UPDATE_STATUS_RECORD_NOT_CACHED_PREFIX}{hash}"
1716 ))
1717 })?;
1718 record.status = status;
1719 record.in_canonical_chain = status.is_canonical();
1720 Ok(())
1721 }
1722
1723 /// Async retrieval by hash ([`BLK-007`](../docs/requirements/domains/block_storage/specs/BLK-007.md) AC §1, §4, §5).
1724 ///
1725 /// **Hot path:** [`Self::block_cache`] hits return cloned blocks **before** any `.await`, so the generated
1726 /// future can complete as [`Poll::Ready`] on the first [`poll`](std::future::Future::poll) without scheduling
1727 /// [`tokio::task::spawn_blocking`] (NORMATIVE BLK-007 §1–2).
1728 ///
1729 /// **Cold path:** Delegates to [`Self::get_block`] on the blocking pool so RocksDB + zstd never run on a
1730 /// cooperative tokio worker thread.
1731 pub async fn get_block_async(
1732 &self,
1733 hash: &Bytes32,
1734 ) -> Result<Option<L2Block>, BlockStoreError> {
1735 if let Some(block) = self.block_cache.get_clone(hash) {
1736 return Ok(Some(block));
1737 }
1738 let store = self.clone();
1739 let hash = *hash;
1740 tokio::task::spawn_blocking(move || store.get_block(&hash))
1741 .await
1742 .map_err(Self::map_spawn_join)?
1743 }
1744
1745 /// Async header retrieval ([`BLK-007`](../docs/requirements/domains/block_storage/specs/BLK-007.md) AC §2, §4–5).
1746 pub async fn get_header_async(
1747 &self,
1748 hash: &Bytes32,
1749 ) -> Result<Option<L2BlockHeader>, BlockStoreError> {
1750 if let Some(header) = self.header_cache.get_clone(hash) {
1751 return Ok(Some(header));
1752 }
1753 let store = self.clone();
1754 let hash = *hash;
1755 tokio::task::spawn_blocking(move || store.get_header(&hash))
1756 .await
1757 .map_err(Self::map_spawn_join)?
1758 }
1759
1760 /// Async canonical-height lookup followed by block load ([`BLK-007`](../docs/requirements/domains/block_storage/specs/BLK-007.md) AC §3).
1761 ///
1762 /// **Always `spawn_blocking`:** height→hash uses [`CF_CANONICAL`] I/O; per BLK-007 implementation notes this
1763 /// stays on the blocking pool even when the block body would hit [`Self::block_cache`], avoiding partial
1764 /// “async hits” that still touch RocksDB in the sync prelude.
1765 pub async fn get_block_by_height_async(
1766 &self,
1767 height: u64,
1768 ) -> Result<Option<L2Block>, BlockStoreError> {
1769 let store = self.clone();
1770 tokio::task::spawn_blocking(move || store.get_block_by_height(height))
1771 .await
1772 .map_err(Self::map_spawn_join)?
1773 }
1774
1775 /// Maps a failed [`tokio::task::spawn_blocking`] join handle onto [`BlockStoreError::Serialization`].
1776 ///
1777 /// **Why not a dedicated enum variant:** [`ERR-001`](../docs/requirements/domains/error_types/specs/ERR-001_blockstoreerror_enum.md) caps
1778 /// the error surface at thirteen variants; join failures are rare operational faults surfaced with [`ERR_ASYNC_JOIN_PREFIX`].
1779 #[inline]
1780 fn map_spawn_join(err: tokio::task::JoinError) -> BlockStoreError {
1781 BlockStoreError::Serialization(format!("{ERR_ASYNC_JOIN_PREFIX}{err}"))
1782 }
1783
1784 /// Resolve a column family handle by name, or error if the DB was not opened with it.
1785 ///
1786 /// This is a thin wrapper around [`DB::cf_handle`](rocksdb::DB::cf_handle) that converts
1787 /// the `Option<&ColumnFamily>` to our error type. In practice this should never fail
1788 /// because [`BlockStore::open`] creates all six families via [`cf_options::column_family_descriptors`],
1789 /// but defensive coding prevents silent `None` dereferences if the CF list drifts.
1790 pub(crate) fn cf(&self, name: &'static str) -> Result<&rocksdb::ColumnFamily, BlockStoreError> {
1791 self.db
1792 .cf_handle(name)
1793 .ok_or_else(|| BlockStoreError::Serialization(format!("missing column family {name}")))
1794 }
1795}
1796
1797/// Load the current chain tip from [`CF_METADATA`] / [`META_TIP`].
1798///
1799/// The tip is a 40-byte value encoding `hash (32 bytes) || height (8 bytes LE)`,
1800/// decoded via [`ChainTip::from_bytes`]. Returns `None` for a brand-new database
1801/// that has not yet had [`BlockStore::init_genesis`] called.
1802///
1803/// # Chia analogy
1804///
1805/// Corresponds to reading `current_peak` from the `block_store` metadata in Chia's
1806/// `BlockStore.get_peak()`. The DIG version uses a fixed-width binary encoding
1807/// instead of SQLite row access.
1808///
1809/// # Called by
1810///
1811/// [`BlockStore::open`] and [`BlockStore::open_readonly`] to populate the in-memory
1812/// [`BlockStore::tip`] field at startup.
1813fn load_tip(db: &DB) -> Result<Option<ChainTip>, BlockStoreError> {
1814 let meta = db
1815 .cf_handle(CF_METADATA)
1816 .ok_or_else(|| BlockStoreError::Serialization("missing CF_METADATA".into()))?;
1817 let Some(raw) = db.get_cf(meta, META_TIP.as_bytes())? else {
1818 return Ok(None);
1819 };
1820 ChainTip::from_bytes(&raw).map(Some)
1821}
1822
1823// NOTE: The old `warm_recent_blocks` free function has been replaced by
1824// `BlockStoreInner::warm_caches` (CAC-006) which runs AFTER full construction
1825// and populates ALL caches (block, header, record, height index, hash-to-height)
1826// instead of only counting block existence.