Crate brk_computer

Expand description

§brk_computer

Bitcoin analytics engine that transforms indexed blockchain data into comprehensive metrics

brk_computer is the computational layer of BRK that processes indexed blockchain data to generate analytics across multiple specialized domains. It provides comprehensive Bitcoin metrics with efficient storage and lazy computation for optimal performance.

§What it provides

Comprehensive Analytics: 9 specialized domains covering all aspects of Bitcoin analysis
Lazy Computation: On-demand calculation with dependency tracking and caching
Incremental Updates: Only processes new data since last computation
Memory Efficiency: ~100MB operation footprint via compressed storage and memory mapping
Multi-timeframe Analysis: Daily, weekly, monthly, quarterly, yearly perspectives

§Nine Analytics Domains

The computer processes data through a fixed dependency chain:

indexes - Time-based indexing (date/height mappings, epoch calculations)
constants - Baseline values and reference metrics
blocks - Block analytics (sizes, intervals, transaction counts, weight)
mining - Mining economics (hashrate, difficulty, rewards, epochs)
fetched - External price data integration (optional)
price - OHLC data across multiple timeframes (optional, requires fetched)
transactions - Transaction analysis (fees, sizes, patterns, RBF detection)
market - Price correlations and market metrics (optional, requires price)
stateful - UTXO tracking and accumulated state computations
cointime - Coin age and time-based value analysis

§Key Features

§Computation Strategy

Fixed dependency chain: Ensures data consistency across all domains
Parallel processing: Uses Rayon for performance optimization
State management: Rollback capabilities for error recovery
Incremental updates: Only computes new data since last run

§Analytics Capabilities

Multi-timeframe analysis: Daily, weekly, monthly, quarterly, yearly aggregations
Chain-based metrics: Height, difficulty epoch, halving epoch indexing
Price correlation: Both dollar and satoshi denominated metrics
DCA analysis: Dollar Cost Averaging with configurable periods
Supply analysis: Circulating, realized, unrealized supply metrics
Address cohort tracking: Analysis across different Bitcoin address types
UTXO cohort analysis: Realized/unrealized gains tracking
Coin time analysis: Understanding Bitcoin velocity and dormancy

§Storage Optimization

Compressed vectors: Efficient disk storage with lazy computation
Memory mapping: Minimal RAM usage during operation
Version management: Automatic invalidation on schema changes
Dependency tracking: Smart recomputation based on data changes

§Usage

§Basic Setup (No Price Data)

use brk_computer::Computer;
use brk_indexer::Indexer;
use vecdb::Exit;

// Setup without external price data
let indexer = Indexer::forced_import("./brk_data")?;
let mut computer = Computer::forced_import("./brk_data", &indexer, None)?;

// Setup exit handler
let exit = Exit::new();
exit.set_ctrlc_handler();

// Compute all analytics
let starting_indexes = indexer.get_starting_indexes();
computer.compute(&indexer, starting_indexes, &exit)?;

§Advanced Setup (With Price Data)

use brk_fetcher::Fetcher;

// Setup with external price data for market analytics
let fetcher = Some(Fetcher::import(true, None)?);
let mut computer = Computer::forced_import("./brk_data", &indexer, fetcher)?;

// Compute all analytics including price/market domains
computer.compute(&indexer, starting_indexes, &exit)?;

§Accessing Computed Data

// Access all computed vectors
let all_vecs = computer.vecs(); // Returns Vec<&dyn AnyCollectableVec>

// Access specific domain data
let block_metrics = &computer.blocks;
let mining_data = &computer.mining;
let transaction_stats = &computer.transactions;

// Access price data (if available)
if let Some(price_data) = &computer.price {
    // Use OHLC data
}

§Incremental Updates

// Continuous computation loop
loop {
    // Get latest indexes from indexer
    let current_indexes = indexer.get_current_indexes();
    
    // Compute only new data
    computer.compute(&indexer, current_indexes, &exit)?;
    
    // Check for exit signal
    if exit.is_signaled() {
        break;
    }
    
    // Wait before next update
    sleep(Duration::from_secs(60));
}

§Core Computer Structure

pub struct Computer {
    pub indexes: indexes::Vecs,           // Time indexing
    pub constants: constants::Vecs,       // Baseline values
    pub blocks: blocks::Vecs,            // Block analytics
    pub mining: mining::Vecs,            // Mining economics
    pub market: market::Vecs,            // Market metrics (optional)
    pub price: Option<price::Vecs>,      // OHLC price data (optional)
    pub transactions: transactions::Vecs, // Transaction analysis
    pub stateful: stateful::Vecs,        // UTXO tracking
    pub fetched: Option<fetched::Vecs>,  // External data (optional)
    pub cointime: cointime::Vecs,        // Coin age analysis
}

§Performance Characteristics

Benchmarked on MacBook Pro M3 Pro:

Initial computation: ~6-7 hours for complete Bitcoin blockchain
Storage efficiency: All computed datasets total ~40GB
Incremental updates: 3-5 seconds per new block
Memory footprint: Peak ~7-8GB during computation, ~100MB during operation
Dependencies: Price data domains optional (fetched, price, market)

§Domain-Specific Analytics

§Block Analytics

Block sizes, weights, transaction counts
Block intervals and mining statistics
Fee analysis per block

§Mining Economics

Hashrate estimation and difficulty tracking
Mining reward analysis
Epoch-based calculations

§Transaction Analysis

Fee rate distributions
RBF (Replace-By-Fee) detection
Output type analysis
Transaction size patterns

§Market Metrics (Optional)

Price correlations with on-chain metrics
Market cap calculations
DCA analysis across timeframes

§Stateful Analysis

UTXO set tracking
Address cohort analysis
Realized/unrealized gains
Supply distribution metrics

§Requirements

Indexed data: Requires completed brk_indexer output
Storage space: Additional ~40GB for computed datasets
Memory: 8GB+ RAM recommended for initial computation
CPU: Multi-core recommended for parallel processing
Price data: Optional external price feeds for market analytics

§Dependencies

brk_indexer - Source of indexed blockchain data
brk_fetcher - External price data (optional)
vecdb - Vector database with lazy computation
rayon - Parallel processing framework
brk_structs - Bitcoin-aware type system

This README was generated by Claude Code

Structs§

Computer
PriceToAmount