hash_roll

Trait ChunkIncr

Source 
pub trait ChunkIncr {
    // Required method
    fn push(&mut self, data: &[u8]) -> Option<usize>;

    // Provided methods
    fn iter_slices<'a>(self, data: &'a [u8]) -> IterSlices<'a, Self> ⓘ
       where Self: Sized { ... }
    fn iter_slices_strict<'a>(
        self,
        data: &'a [u8],
    ) -> IterSlicesStrict<'a, Self> ⓘ
       where Self: Sized { ... }
}
Expand description

Accept incrimental input and provide indexes of split points

Compared to Chunk, ChunkIncr allows avoiding having to buffer all input data in memory, and avoids the need to use a single buffer for storing the input data (even if all data is in memory).

Data fed into a given ChunkIncr instance is considered to be part of the same data “source”. This affects chunking algorithms that maintain some state between chunks (like ZstdRsyncable does). If you have multiple “sources”, one should obtain new instances of ChunkIncr for each of them (typically via ToChunkIncr).

Note that for some splitting/chunking algorithms, the incrimental api will be less efficient compared to the non-incrimental API. In particular, algorithms like [Rsyncable] that require the use of previously examined data to shift their “window” (resulting in needing a circular buffer which all inputed data passes through) will perform more poorly using ChunkIncr compared with non-incrimental interfaces

Required Methods§

Source

fn push(&mut self, data: &[u8]) -> Option<usize>

The data “contained” within a implimentor of this trait is the history of all data slices passed to feed.

In other words, all previous data (or no previous data) may be used in determining the point to split.

Returns None if the data has no split point. Otherwise, returns an index in the most recently passed data.

Note that returning the index in the current slice makes most “look-ahead” splitting impossible (as it is permissible to pass 1 byte at a time).

Provided Methods§

Source

fn iter_slices<'a>(self, data: &'a [u8]) -> IterSlices<'a, Self> ⓘ
where Self: Sized,

Given a ChunkIncr and a single slice, return a list of slices chunked by the chunker.

Will always return enough slices to form the entire content of data, even if the trailing part of data is not a chunk (ie: does not end on a chunk boundary)

Source

fn iter_slices_strict<'a>(self, data: &'a [u8]) -> IterSlicesStrict<'a, Self> ⓘ
where Self: Sized,

Given a ChunkIncr and a single slice, return a list of slices chunked by the chunker. Does not return the remainder (if any) in the iteration. Use [IterSlices::take_rem()] or IterSlices::into_parts() to get the remainder.

Note that this is a non-incrimental interface. Calling this on an already fed chunker or using this multiple times on the same chunker may provide unexpected results

Implementors§