[−][src]Crate gearhash
The GEAR hashing function is a fast, rolling hash function that is well suited for content defined chunking. In particular, it is used as a building block for the FastCDC algorithm.
The implementation provided in this crate consists of both a simple, scalar variant, as well as optimized versions for the SSE4.2 and AVX2 instruction sets.
Example
fn find_all_chunks(buf: &[u8], mask: u64) -> Vec<&[u8]> { // set up initial state let mut chunks = vec![]; let mut offset = 0; // create new hasher let mut hasher = gearhash::Hasher::default(); // loop through all matches, and push the corresponding chunks while let Some(boundary) = hasher.next_match(&buf[offset..], mask) { chunks.push(&buf[offset..offset + boundary]); offset += boundary; } // push final chunk chunks.push(&buf[offset..]); chunks }
Structs
Hasher | Gear hash state. Processes bytes to find chunk boundaries. |
Statics
DEFAULT_TABLE | Default hash table, using random (but static) integers. |
Type Definitions
Table | Gear hash table. |