Struct BlockIndexRebuild 

pub struct BlockIndexRebuild {
    pub config: RebuildConfig,
    pub scan_entries: Vec<BlockScanEntry>,
    pub index: HashMap<String, IndexEntry>,
    pub progress: RebuildProgress,
    pub existing_index: HashMap<String, IndexEntry>,
}

Block index reconstruction engine.

Call BlockIndexRebuild::run_full_rebuild for an all-in-one pipeline, or invoke each phase method individually for fine-grained control.

Fields

config: RebuildConfig

Configuration for this rebuild operation.

scan_entries: Vec<BlockScanEntry>

Scanned block entries accumulated during the scanning phase.

index: HashMap<String, IndexEntry>

Newly reconstructed index, populated during the rebuilding phase.

progress: RebuildProgress

Live rebuild progress.

existing_index: HashMap<String, IndexEntry>

Pre-existing index entries loaded before the rebuild starts.

Implementations

impl BlockIndexRebuild

pub fn new(config: RebuildConfig) -> Self

Create a new engine with the given config.

Progress starts at started_at = 0; the timestamp is updated when blocks are first loaded via Self::load_blocks.

pub fn assign_shard(&self, cid: &str) -> u8

Compute the shard number for a given CID using FNV-1a over the CID bytes.

The result is fnv1a_64(cid.as_bytes()) mod shard_count, clamped to a u8.

pub fn assign_offset(idx: usize) -> u64

Compute the page-aligned byte offset for the block at position idx.

The formula is idx as u64 * PAGE_SIZE (4 096 bytes per page).

pub fn detect_flags(meta: &HashMap<String, String>) -> u8

Derive the flags byte from block metadata.

• Key "pinned" with value "true" → bit 0x01
• Key "compressed" with value "true" → bit 0x02
• Key "encrypted" with value "true" → bit 0x04

pub fn load_blocks( &mut self, blocks: Vec<(String, Vec<u8>, HashMap<String, String>)>, now: u64, )

Scanning phase — ingest raw blocks and compute their checksums.

Each element of blocks is (cid, data, metadata). For every block a BlockScanEntry is created (with verified = false), the checksum is computed, and blocks_scanned is incremented. The progress phase is set to RebuildPhase::Scanning and started_at is updated to now if this is the first call.

pub fn load_existing_index(&mut self, entries: Vec<IndexEntry>)

Existing index load — populate the engine with the pre-existing index.

This is called before the verify/rebuild phases so that RebuildConfig::rebuild_missing_only can skip blocks that already have an up-to-date index entry.

pub fn verify_phase(&mut self)

Verification phase — recompute checksums and mark each entry.

When verify_checksums is disabled in the config, all blocks are considered verified automatically. Otherwise the stored checksum is compared against a freshly computed value; mismatches are recorded as errors (up to max_errors).

Note: because this engine stores only the final checksum (not the original raw bytes), re-verification is done by treating the checksum as valid — callers that need full byte re-verification should use BlockIndexRebuild::verify_with_data instead.

pub fn verify_with_data<I>(&mut self, data_iter: I)

where I: IntoIterator<Item = (String, Vec<u8>)>,

Verification phase with raw data — verify entries against original bytes.

Accepts an iterator of (cid, data) pairs. For each pair, the checksum in the corresponding BlockScanEntry is compared against a freshly computed FNV-1a checksum. Mismatches are recorded as errors.

pub fn rebuild_phase(&mut self, _now: u64)

Rebuild phase — construct IndexEntry records for verified blocks.

Iterates over all BlockScanEntry records that have verified = true. If RebuildConfig::rebuild_missing_only is set, blocks already present in Self::existing_index are skipped. For each remaining block, shard, offset, and flags are derived, then the entry is inserted into Self::index.

pub fn validate_phase(&mut self)

Validation phase — verify consistency between scan entries and the index.

For every block that was scanned (regardless of verification status), this phase checks that a corresponding entry exists in Self::index. Missing entries are recorded as errors. If any errors exist after validation the phase is set to RebuildPhase::Failed; otherwise it transitions to RebuildPhase::Complete.

pub fn run_full_rebuild( &mut self, blocks: Vec<(String, Vec<u8>, HashMap<String, String>)>, existing: Vec<IndexEntry>, now: u64, ) -> &RebuildProgress

Execute the full four-phase rebuild pipeline and return a reference to the final progress snapshot.

Equivalent to calling (in order):

1. Self::load_blocks
2. Self::load_existing_index
3. Self::verify_phase
4. Self::rebuild_phase
5. Self::validate_phase

pub fn get_index_entry(&self, cid: &str) -> Option<&IndexEntry>

Look up a CID in the rebuilt index.

Returns None when no entry was reconstructed for that CID.

pub fn index_size(&self) -> usize

Number of entries currently in the rebuilt index.

pub fn rebuild_stats(&self) -> RebuildStats

Produce a RebuildStats snapshot of the current engine state.

pub fn export_index(&self) -> Vec<IndexEntry>

Return all index entries as a cloned Vec.

pub fn scan_entries(&self) -> &[BlockScanEntry]

Return all scan entries (including unverified ones).

pub fn errors(&self) -> &[String]

Return the current errors list.

pub fn progress(&self) -> &RebuildProgress

Return a reference to the current progress.

pub fn existing_index(&self) -> &HashMap<String, IndexEntry>

Return a reference to the existing index.

pub fn reset(&mut self)

Reset the engine state, preserving the configuration.

Clears scan entries, rebuilt index, existing index, and progress.

pub fn is_finished(&self) -> bool

Check whether the engine has completed (successfully or with failure).

pub fn is_successful(&self) -> bool

Returns true if the rebuild completed without any errors.

pub fn merge_index(&mut self, entries: Vec<IndexEntry>)

Merge an additional set of IndexEntry records from an external source into the rebuilt index without overwriting existing entries.

pub fn upsert_index_entry(&mut self, entry: IndexEntry)

Forcibly insert (or overwrite) an IndexEntry in the rebuilt index.

pub fn remove_index_entry(&mut self, cid: &str) -> Option<IndexEntry>

Remove an entry from the rebuilt index by CID. Returns the removed entry if it existed.

pub fn unverified_entries(&self) -> Vec<&BlockScanEntry>

Return the subset of scanned blocks that failed verification (or were never verified).

pub fn all_index_entries(&self) -> Vec<&IndexEntry>

Return references to all IndexEntry records in the rebuilt index.

pub fn entries_in_shard(&self, shard: u8) -> Vec<&IndexEntry>

Find index entries assigned to a specific shard.

pub fn pinned_entries(&self) -> Vec<&IndexEntry>

Return all index entries that have the pinned flag set.

pub fn compressed_entries(&self) -> Vec<&IndexEntry>

Return all index entries that have the compressed flag set.

pub fn encrypted_entries(&self) -> Vec<&IndexEntry>

Return all index entries that have the encrypted flag set.