Expand description
Chunking and DAG (Directed Acyclic Graph) support for large file handling
This module provides functionality for:
- Splitting large files into content-addressed blocks
- Creating Merkle DAG structures
- Reassembling files from chunks
Structs§
- Chunked
File - Result of chunking a file
- Chunker
- Chunker for splitting large data into blocks
- Chunking
Config - Configuration for chunking operations
- Chunking
Config Builder - Builder for creating custom chunking configurations
- DagBuilder
- Builder for creating Merkle DAG structures incrementally
- DagLink
- A link to another block in the DAG
- DagNode
- A node in the Merkle DAG structure
- Deduplication
Stats - Statistics for chunk deduplication
Enums§
- Chunking
Strategy - Chunking strategy for splitting data
Constants§
- DEFAULT_
CHUNK_ SIZE - Default chunk size (256 KiB) - a good balance between overhead and deduplication
- MAX_
CHUNK_ SIZE - Maximum chunk size (1 MiB, leaving room for metadata in a 2 MiB block)
- MAX_
LINKS_ PER_ NODE - Maximum number of links in a single DAG node This prevents any single node from being too large
- MIN_
CHUNK_ SIZE - Minimum chunk size (1 KiB)