Crate brk_parser

Expand description

§BRK Parser

Size

A very fast and simple Rust library which reads raw block files (blkXXXXX.dat) from Bitcoin Core node and creates an iterator over all the requested blocks in sequential order (0, 1, 2, …).

The element returned by the iterator is a tuple which includes the:

Height: Height
Block: Block (from bitcoin-rust)
Block’s Hash: BlockHash (also from bitcoin-rust)

Tested with Bitcoin Core v25.0..=v28.1

§Requirements

Even though it reads blkXXXXX.dat files, it needs bitcoind (with no particular parameters) to run with the RPC server to filter out forks.

Peak memory should be around 500MB.

XOR-ed blocks are supported.

§Disclaimer

A state of the local chain is saved in {bitcoindir}/blocks/blk_index_to_blk_recap.json to allow for faster starts (see benchmark below) but doesn’t yet support locking. Thus, it is highly recommended to run one instance of brk_parser at a time.

§Benchmark

	brk_parser	bitcoin-explorer (deprecated)	blocks_iterator*
Runs with `bitcoind`	Yes ✅	No ❌	Yes ✅
Runs without `bitcoind`	No ❌	Yes ✅	Yes ✅
`0..=855_000`	4mn 10s	4mn 45s	> 2h
`800_000..=855_000`	0mn 52s (4mn 10s if first run)	0mn 55s	> 2h

* blocks_iterator is with the default config (and thus with skip_prevout = false which does a lot more than just iterate over blocks) so it isn’t an apples to apples comparaison and the numbers are misleading. You should expect much closer times. Will update the benchmark with skip_prevout = true as soon as possible.

Benchmarked on a Macbook Pro M3 Pro

§Example

use bitcoincore_rpc::{Auth, Client};
use brk_core::{Height, default_bitcoin_path};
use brk_parser::Parser;

fn main() {
    let i = std::time::Instant::now();

    let bitcoin_dir = default_bitcoin_path();

    let rpc = Box::leak(Box::new(
        Client::new(
            "http://localhost:8332",
            Auth::CookieFile(bitcoin_dir.join(".cookie")),
        )
        .unwrap(),
    ));

    let start = None;
    let end = None;

    let parser = Parser::new(bitcoin_dir.join("blocks"), rpc);

    parser
        .parse(start, end)
        .iter()
        .for_each(|(height, _block, hash)| {
            println!("{height}: {hash}");
        });

    println!(
        "{}",
        parser
            .get(Height::new(0))
            .txdata
            .first()
            .unwrap()
            .output
            .first()
            .unwrap()
            .script_pubkey
    );

    println!(
        "{}",
        parser
            .get(Height::new(840_000))
            .txdata
            .first()
            .unwrap()
            .output
            .first()
            .unwrap()
            .value
    );

    dbg!(i.elapsed());
}