Expand description
A fast and flexible Merkle Tree library for Rust, providing efficient construction of Merkle Trees, verification of Merkle Proofs for single and multiple elements, and generic support for any hashable data type.
§What is a Merkle Tree?
A Merkle Tree is a tree data structure. where it contains a set of properties such as:
- Each leaf node contains the hash of a data block
- Each non-leaf node contains the hash of its child nodes
- The root hash represents a cryptographic fingerprint of all the data in the tree
This data structure enables efficient and secure verification that a data element is part of a larger dataset without needing to download the entire dataset.
§Use Cases
- Blockchain & Cryptocurrencies: Bitcoin and other cryptocurrencies use Merkle Trees to efficiently verify transactions
- Distributed Systems: Verify data integrity across distributed networks
- File Systems: Git uses Merkle Trees to track changes and verify repository integrity
- Database Verification: Ensure data hasn’t been tampered with
- Peer-to-Peer Networks: Verify chunks of data in distributed file sharing
§Example
import torch
from mrkle import MrkleTree
def namespaced_state_dict(model: torch.nn.Module) -> dict[str, torch.Tensor]:
"""
Returns a state_dict with the model name prefixed to every key.
"""
sd = model.state_dict()
return {f"{model.__class__.__name__.lower()}.{k}": v.detach().cpu().numpy() for k, v in sd.items()}
class ToyModel(torch.nn.Module):
def __init__(self, in_feature: int, out_feature: int):
super().__init__()
self.ln = torch.nn.Linear(in_feature, out_feature)
self.output = torch.nn.Linear(out_feature, 1)
def forward(self, x: torch.Tensor):
x = self.ln(x)
logits = self.output(torch.tanh(x))
return logits, torch.sigmoid(x)
# Create model + state dict
model = ToyModel(10, 10)
state_dict = namespaced_state_dict(model)
# Construct Merkle tree over model parameters
tree = MrkleTree.from_dict(state_dict, name="sha256", fmt="flatten")
# Root hash identifies the entire model uniquely
print(tree.root())Construct a basic binary Merkle Tree by chunking data into byte slices:
use mrkle::MrkleTree;
use sha2::Sha256;
// Input data (could also be read from a file)
let data = b"The quick brown fox jumps over the lazy dog. \
This is some extra data to make it larger. \
Merkle trees are cool!";
// Split into fixed-size chunks
let chunk_size = 16;
let chunks: Vec<&[u8]> = data.chunks(chunk_size).collect();
// Build the Merkle tree from chunks
let tree = MrkleTree::<&[u8], Sha256>::from(chunks);
// Get the Merkle root
let root = tree.root();Re-exports§
pub use crate::hasher::GenericArray;pub use crate::hasher::Hasher;pub use crate::hasher::MrkleHasher;pub use crate::tree::IndexIter;pub use crate::tree::IndexType;pub use crate::tree::Iter;pub use crate::tree::MutNode;pub use crate::tree::Node;pub use crate::tree::NodeIndex;pub use crate::tree::Tree;pub use crate::tree::TreeView;
Modules§
- error
- Error types for the Merkle tree crate.
- hasher
- Cryptographic hash utilities and traits used in Merkle trees.
- tree
- Core tree structures and nodes for the Merkle tree implementation.
Structs§
- HexDisplay
- A helper wrapper for displaying an
entryas a hexadecimal string. - Mrkle
Default Builder - A builder for constructing Merkle trees with default configuration.
- Mrkle
Node - A generic immutable node in a Merkle Tree.
- Mrkle
Proof - A cryptographic proof verifying that a set of leaves belong to a specific
MrkleTreeroot. - Mrkle
Tree - A cryptographic hash tree data structure.
- Proof
Level - A single level in a
ProofPath. - Proof
Path - A complete path from a leaf to the root within a
MrkleTree. - entry
- Object that holds the bytes hashed
MrkleNodeused for reference of the array.
Enums§
- Payload
- Represents the contents of a node in a Merkle tree.