Crate xet_data 

Data processing pipeline for chunking, deduplication, and file reconstruction, used in the Hugging Face Xet storage tools.

Provides content-defined chunking via gear hashing, deduplication against metadata shards, and file reconstruction from deduplicated chunk references.

Re-exports

pub use error::DataError;

pub use error::Result;

Modules

deduplication

error

file_reconstruction

processing

progress_tracking