bitcoin_block_parser

Module blocks

Source
Expand description

The BlockParser trait allows you to implement a custom parser or use one of the predefined ones.

For example, imagine you want to take the biggest tx using Transaction::total_size from every block and sum their output bitcoin::Amount.

You can use the DefaultParser to simply iterate over the Block:

use bitcoin_block_parser::*;

let mut amount = bitcoin::Amount::ZERO;
for block in DefaultParser.parse_dir("/path/to/blocks").unwrap() {
    let txs = block.unwrap().txdata;
    let max = txs.into_iter().max_by_key(|tx| tx.total_size());
    for output in max.unwrap().output {
        amount += output.value;
    }
}
println!("Sum of txids: {}", amount);

If you only want to run on a subset of blocks use HeaderParser. If you need to process blocks in-order use InOrderParser.

use bitcoin_block_parser::*;

let headers = HeaderParser::parse("/path/to/blocks").unwrap();
// Skip the first 200,000 blocks
for block in InOrderParser.parse(&headers[200_000..]) {
  // Do whatever you need with the blocks
}

If you wish to increase performance you may need to use BlockParser::parse_map. This example uses ~2x less memory and less time since it reduces the data size and runs on multiple threads. The more compute and memory your algorithm uses, the more you may benefit from this. The mapping will occur after BlockParser::batch, keeping the ordering.

use bitcoin::*;
use bitcoin_block_parser::*;
use anyhow::Result;
use crossbeam_channel::Receiver;

let headers = HeaderParser::parse("/path/to/blocks").unwrap();
let receiver: Receiver<Result<Amount>> = DefaultParser.parse_map(&headers, |block| {
    // Code in this closure runs in parallel
    let mut amount = bitcoin::Amount::ZERO;
    let max = block.txdata.into_iter().max_by_key(|tx| tx.total_size());
    for output in max.unwrap().output {
        amount += output.value;
    }
    amount
});

let mut total_amount = bitcoin::Amount::ZERO;
for amount in receiver {
  total_amount += amount.unwrap();
}
println!("Sum of txids: {}", total_amount);

You can implement your own BlockParser which contains shared state using an Arc. Updating any locked stated should take place in BlockParser::batch to reduce the contention on the lock.

use std::sync::*;
use bitcoin::*;
use bitcoin_block_parser::*;

// Parser with shared state, must implement Clone for parallelism
#[derive(Clone, Default)]
struct AmountParser(Arc<Mutex<Amount>>);

// Custom implementation of a parser
impl BlockParser<Amount> for AmountParser {
    // Runs in parallel on each block
    fn extract(&self, block: bitcoin::Block) -> Vec<Amount> {
        let max = block.txdata.iter().max_by_key(|tx| tx.total_size());
        vec![max.unwrap().output.iter().map(|out| out.value).sum()]
    }

    // Runs on batches of items from the extract function
    fn batch(&self, items: Vec<Amount>) -> Vec<Amount> {
        // We should access our Mutex here to reduce contention on the lock
        let mut sum = self.0.lock().unwrap();
        for item in items {
            *sum += item;
        }
        vec![]
    }
}

let parser = AmountParser::default();
for _ in parser.parse_dir("/path/to/blocks").unwrap() {}
println!("Sum of txids: {:?}", parser.0);

Structs§

  • DefaultParser
    Parser that returns Block for users that don’t implement a custom BlockParser
  • InOrderParser
    Parse all the blocks represented by the headers, ensuring the blocks are returned in the same order the ParsedHeader were passed in.
  • Options
    Options to tune the performance of the parser, generally you can stick to the defaults unless you run into memory issues.

Traits§