Struct Scanner 

pub struct Scanner { /* private fields */ }

An engine that hashes data, looking for good split points. When it finds a split point, it emits the length of the block and its hash (SHA-256, Blake2, or Blake3).

You can configure the window size, preferred block size, and min/max block sizes (all in powers of 2).

Implementations

impl Scanner

pub fn new( hash_type: HashType, min_bits: u8, pref_bits: u8, max_bits: u8, window_bits: u8, ) -> Scanner

Create a new scanner with:

• hash_type: which hashing function to use
• min_bits: minimum block size
• pref_bits: preferred block size
• max_bits: maximum block size
• window_bits: size of the rolling buffer, or how far back to include in the rolling hash

All sizes are in powers of 2, so Scanner::new(18, 20, 22, 10) will use a window size of 1K, a preferred block size of 1M, and a min/max of 256K/4M.

pub fn feed(&mut self, data: &[u8]) -> (usize, Option<Block>)

Process a chunk of data and determine if we’ve hit a block boundary. Returns the number of bytes consumed, and optionally block metadata. If it returns a block, it may not have finished processing this chunk of data, so if consumed < data.len(), then you should call this method again, with &data[consumed..].

pub fn finish(&mut self) -> Option<Block>

Terminate the scanner. Return a block if there was one in flight. It may be smaller than the minimum size.

pub fn reader_iter<'a, R: Read>( &'a mut self, reader: &'a mut R, buffer: &'a mut [u8], digest: &'a mut Hashing, ) -> ScanIterator<'a, R>

Scan a reader (Read), using a given temporary buffer, and computing a SHA-256 hash as we go.

pub fn scan_reader( &mut self, reader: &mut impl Read, buffer: &mut [u8], hash_type: HashType, ) -> Result<FileBlocks>

Exhaustively scan a Read and return a vector of blocks. The scanner is reset at the end so that it can be immediately reused on another reader.