bitcoin-explorer

Downloads

bitcoin_explorer is an efficient library for decoding transaction information from bitcoin blockchain.

Support bitcoin MainNet, might support other networks in the future.

Features

1. Block & Script Decoding

Query blocks based on block heights.
Decode Bitcoin Core native levelDB info.
Decode addresses from output scripts (supporting all standard scripts, including Multi-Sig scripts).
Find input addresses using on-disk UTXO cache (iter_connected_block()).

2. Concurrency + Iterator + Sequential Output

Fast concurrent reading but producing sequential output.
Work-stealing with registry to maximize efficiency.
Native Iterator interface (support for in syntax).
Robust exception handling in multi-threading.
Data racing and deadlock free data model.

3. Small Memory Footprint (< 4 GB RAM)

Compress 256-bit TxID to u128 keys in UTXO.
Use a fast on-disk UTXO storage (RocksDB).
Configurable in-memory UTXO cache (non-default feature).

4. Generic User Interface

Use static generic to offer different output formats for various APIs.

5. Thorough Testing

Thoroughly tested for correctness using rust integration tests.

6. Build for Rust + Python (Multi-OS PyPI wheels)

Built and published PyPI wheels for python 3.6-3.10 across Windows x86/x64, MacOS x86_64/arm64, and Linux x86_64.
Rust library on crates.io called bitcoin-explorer.

Documentation

See Rust Documentation

Examples

Get total number of blocks and transactions available on disk

use bitcoin_explorer::{BitcoinDB, FConnectedBlock, SConnectedBlock};
use std::path::Path;

fn main() {

    let path = Path::new("/Users/me/bitcoin");
    let db = BitcoinDB::new(path, false).unwrap();

    let block_count = db.get_block_count();

    let total_number_of_transactions = (0..block_count)
        .map(|i| db.get_header(i).unwrap().n_tx)
        .sum::<u32>();

}

Get a block (i.e., see doc for what is full/simple format (`FBlock`/`SBlock`) )

use bitcoin_explorer::{BitcoinDB, FBlock, SBlock, Block};
use std::path::Path;

fn main() {
    let path = Path::new("/Users/me/bitcoin");

    // launch without reading txindex
    let db = BitcoinDB::new(path, false).unwrap();

    // get block of height 600000 (in different formats)
    let block: Block = db.get_block(600000).unwrap();
    let block: FBlock = db.get_block(600000).unwrap();
    let block: SBlock = db.get_block(600000).unwrap();
}

Get a transaction (in different formats)

Note: this requires building tx index with --txindex=1 flag using Bitcoin Core.

use bitcoin_explorer::{BitcoinDB, Transaction, FTransaction, STransaction, Txid, FromHex};
use std::path::Path;

fn main() {
    let path = Path::new("/Users/me/bitcoin");

    // !!must launch with txindex=true!!
    let db = BitcoinDB::new(path, true).unwrap();

    // get transaction
    // e3bf3d07d4b0375638d5f1db5255fe07ba2c4cb067cd81b84ee974b6585fb468
    let txid_str = "e3bf3d07d4b0375638d5f1db5255fe07ba2c4cb067cd81b84ee974b6585fb468";
    let txid = Txid::from_hex(txid_str).unwrap();

    // get transactions in different formats
    let tx: Transaction = db.get_transaction(&txid).unwrap();
    let tx: FTransaction = db.get_transaction(&txid).unwrap();
    let tx: STransaction = db.get_transaction(&txid).unwrap();
}

Iterate through all blocks (in different formats)

use bitcoin_explorer::{BitcoinDB, Block, SBlock, FBlock};
use std::path::Path;

fn main() {
    let path = Path::new("/Users/me/bitcoin");

    // launch without reading txindex
    let db = BitcoinDB::new(path, false).unwrap();

    // iterate over block from 0 to 1000
    for block in db.iter_block::<Block>(0, 1000) {
        for tx in block.txdata {
            println!("do something for this transaction");
        }
    }

    // iterate over block from 1000 to end
    for block in db.iter_block::<FBlock>(1000, db.get_block_count()) {
        for tx in block.txdata {
            println!("do something for this transaction");
        }
    }

    // iterate over block from 0 to end
    for block in db.iter_block::<SBlock>(0, db.get_block_count()) {
        for tx in block.txdata {
            println!("do something for this transaction");
        }
    }
}

Iterate through all blocks with Input Addresses Found (`ConnectedBlock`)

use bitcoin_explorer::{BitcoinDB, FConnectedBlock, SConnectedBlock};
use std::path::Path;

fn main() {

    let path = Path::new("/Users/me/bitcoin");

    // launch without reading txindex
    let db = BitcoinDB::new(path, false).unwrap();
    let end = db.get_block_count();

    // iterate over all blocks found (simple connected format)
    for block in db.iter_connected_block::<SConnectedBlock>(end) {
        for tx in block.txdata {
            println!("do something for this transaction");
        }
    }
}

Hardware Requirements

Memory Requirement

Memory requirement: 8 GB physical RAM.

Disk Requirement

SSD for better performance.

Benchmarking

OS: x86_64 Windows 10
CPU: Intel i7-9700 @ 3.00GHZ (4-core, 8-threads)
Memory: 16 GB 2667 Mhz
Disk: WDC SN730 512GB (SSD)

Iteration Through All Blocks (0 - 700000)

db.iter_block::<SBlock>(0, 700000)

Time: about 10 minutes
Peak Memory: <= 500 MB

Iteration Through All Blocks (0 - 700000) With Input Addresses

db.iter_connected_block::<SConnectedBlock>(700000)

Using default configuration

Compile with default features (Cargo.toml):

bitcoin-explorer = "^2.1"

Time: about 2.5 hours
Peak Memory: 4 GB

Using non-default configuration (large RAM for good performance)

Compile with non-default features (Cargo.toml):

bitcoin-explorer = { version = "^2.1", default-features = false }

Time: about 30 minutes
Peak Memory: 32 GB

Notes

Compatibility

This package deals with the binary file of another software Bitcoin Core. It might not be compatible with older Bitcoin Core versions.

Non-Default Feature (In-Memory-UTXO cache)

If you have more than 32 GB memory, you might try default-features = false for faster performance on db.iter_connected_block()

bitcoin-explorer = { version = "^2.1", default-features = false }

bitcoin-explorer 1.2.8