Crate brk_indexer 

brk_indexer

Full Bitcoin blockchain indexer for fast analytics queries.

What It Enables

Transform raw Bitcoin blockchain data into indexed vectors and key-value stores optimized for analytics. Query any block, transaction, address, or UTXO without scanning the chain.

Key Features

• Multi-phase block processing: Parallel TXID computation, input/output processing, sequential finalization
• Address indexing: Maps addresses to their transaction history and UTXOs per address type
• UTXO tracking: Live outpoint→value lookups, address→unspent outputs
• Reorg handling: Automatic rollback to valid chain state on reorganization
• Collision detection: Validates rapidhash-based prefix lookups against known duplicate TXIDs
• Incremental snapshots: Periodic checkpoints for crash recovery

Core API

let mut indexer = Indexer::forced_import(&outputs_dir)?;

// Index new blocks
let starting_indexes = indexer.index(&blocks, &client, &exit)?;

// Access indexed data
let txindex = indexer.stores.txidprefix_to_txindex.get(&txid_prefix)?;
let blockhash = indexer.vecs.blocks.blockhash.get(height)?;

Data Structures

Vecs (append-only vectors):

• blocks: blockhash, timestamp, difficulty, total_size, weight
• transactions: txid, first_txinindex, first_txoutindex
• inputs: outpoint, txindex
• outputs: value, outputtype, typeindex, txindex
• addresses: Per-type p2pkhbytes, p2shbytes, p2wpkhbytes, etc.

Stores (key-value lookups):

• txidprefix_to_txindex - TXID lookup via 10-byte prefix
• blockhashprefix_to_height - Block lookup via 4-byte prefix
• addresstype_to_addresshash_to_addressindex - Address lookup per type
• addresstype_to_addressindex_and_unspentoutpoint - Live UTXO set per address

Processing Pipeline

1. Block metadata: Store blockhash, difficulty, timestamp
2. Compute TXIDs: Parallel SHA256d across transactions
3. Process inputs: Lookup spent outpoints, resolve address info
4. Process outputs: Extract addresses, assign type indexes
5. Finalize: Sequential store updates, UTXO set mutations
6. Commit: Periodic flush to disk

Performance

Machine	Time	Disk	Peak Disk	Memory	Peak Memory
MBP M3 Pro (36GB, internal SSD)	3h	247 GB	314 GB	5.2 GB	11 GB
Mac Mini M4 (16GB, external SSD)	4.9h	233 GB	303 GB	5.4 GB	11 GB

Full benchmark data: bitcoinresearchkit/benches

Recommended: mimalloc v3

Use mimalloc v3 as the global allocator to reduce memory usage.

Built On

• vecdb for append-only vectors
• brk_cohort for address type handling
• brk_iterator for block iteration
• brk_store for key-value storage
• brk_types for domain types

Macros

parallel_import

Imports multiple items in parallel using thread::scope. Each expression must return Result.

Structs

AddressesVecs

BlockProcessor

Processes a single block, extracting and storing all indexed data.

BlocksVecs

ComputedTx

Indexer

Indexes

Blockchain-level indexes tracking current positions for various entity types. Used by brk_indexer during block processing.

InputsVecs

OutputsVecs

ProcessedOutput

Readers

Readers for vectors that need to be accessed during block processing. These provide consistent snapshots for reading while the main vectors are being modified.

SameBlockOutputInfo

ScriptsVecs

Stores

TransactionsVecs

Vecs

Enums

InputSource

Traits

IndexesExt

Extension trait for Indexes with brk_indexer-specific functionality.

Functions

starting_index