Bitcoin slices

ZERO allocations parse library for Bitcoin data structures.

Data is accessed by providing visitor structs for the data the user is interested in.

Data structures are read-only and parsed data must be in memory, no streaming API.

Tradeoffs

Check the CONS before using this library, use rust-bitcoin if they are too restrictive for your case.

Pros

• Deserialization is amazingly fast, since no allocation is made during parsing.
• Serialization is incredibly fast, since a slice of the serialized data is kept in the structure.
• hashing a little faster because slice are ready without the need of re-serializing data.
• No mandatory dependency.
• No standard.
• Calculate txid and block hash via optional dep bitcoin_hashes or sha2.
• Visitor pattern to visit just what you are interested in.

Cons

• Full data must be in memory, there is no streaming (Read/Write) API.
• Data structure are read-only, cannot be modified.
• Visitor pattern requires user-built data structure for visiting.

Test

cargo test

Bench

RUSTFLAGS='--cfg=bench' cargo +nightly bench --all-features

test bsl::block::bench::block_deserialize            ... bench:     344,768 ns/iter (+/- 683)
test bsl::block::bench::block_deserialize_bitcoin    ... bench:   1,565,341 ns/iter (+/- 174,542)
test bsl::block::bench::block_sum_outputs            ... bench:     347,763 ns/iter (+/- 2,027)
test bsl::block::bench::block_sum_outputs_bitcoin    ... bench:   1,469,288 ns/iter (+/- 143,139)
test bsl::block::bench::hash_block_txs               ... bench:   4,443,346 ns/iter (+/- 88,908)
test bsl::block::bench::hash_block_txs_bitcoin       ... bench:   6,303,477 ns/iter (+/- 45,197)
test bsl::block::bench::hash_block_txs_sha2          ... bench:   1,012,167 ns/iter (+/- 1,492)
test bsl::block_header::bench::block_hash            ... bench:         734 ns/iter (+/- 4)
test bsl::block_header::bench::block_hash_bitcoin    ... bench:         800 ns/iter (+/- 6)
test bsl::transaction::bench::tx_deserialize         ... bench:          58 ns/iter (+/- 1)
test bsl::transaction::bench::tx_deserialize_bitcoin ... bench:         139 ns/iter (+/- 2)
test bsl::transaction::bench::txid                   ... bench:       1,223 ns/iter (+/- 1)
test bsl::transaction::bench::txid_bitcoin           ... bench:       1,358 ns/iter (+/- 8)
test bsl::transaction::bench::txid_sha2              ... bench:         196 ns/iter (+/- 0)

• benches ending with _bitcoin use rust-bitcoin
• benches ending with _sha2 use sha2 lib instead of bitcoin_hashes

Fuzz

Use cargo fuzz Run fuzzing with transaction as target.

cargo +nightly fuzz run transaction

Other target available in fuzz/fuzz_targets

Miniminze corpus:

cargo +nightly fuzz cmin transaction

Previous work and credits

• Bitiodine use similar visitor pattern (parser credited to Mathias Svensson)
• Some previous work on the idea to parse while reducing allocations in this PR
• Matt Corallo mentioned something like this in a comment in that PR

TODO

• create rotating buffer that consume and produce keeping a linear memory (rotate back when it can't append), this would overcome a bit the absence of streaming API
• implements network types
• add github actions CI