[−][src]Trait hash_roll::ChunkIncr

pub trait ChunkIncr {
    fn push(&mut self, data: &[u8]) -> Option<usize>;

    fn iter_slices<'a>(self, data: &'a [u8]) -> IterSlices<'a, Self>ⓘ
Notable traits for IterSlices<'a, C>
impl<'a, C: ChunkIncr> Iterator for IterSlices<'a, C>    type Item = &'a [u8];

    where
        Self: Sized,
    { ... }

    fn iter_slices_strict<'a>(
        self, 
        data: &'a [u8]
    ) -> IterSlicesStrict<'a, Self>ⓘ
Notable traits for IterSlicesStrict<'a, C>
impl<'a, C: ChunkIncr> Iterator for IterSlicesStrict<'a, C>    type Item = &'a [u8];

    where
        Self: Sized,
    { ... }
}

Accept incrimental input and provide indexes of split points

Compared to Chunk, ChunkIncr allows avoiding having to buffer all input data in memory, and avoids the need to use a single buffer for storing the input data (even if all data is in memory).

Data fed into a given ChunkIncr instance is considered to be part of the same data "source". This affects chunking algorithms that maintain some state between chunks (like ZstdRsyncable does). If you have multiple "sources", one should obtain new instances of ChunkIncr for each of them (typically via ToChunkIncr).

Note that for some splitting/chunking algorithms, the incrimental api will be less efficient compared to the non-incrimental API. In particular, algorithms like [Rsyncable] that require the use of previously examined data to shift their "window" (resulting in needing a circular buffer which all inputed data passes through) will perform more poorly using ChunkIncr compared with non-incrimental interfaces

Required methods

fn push(&mut self, data: &[u8]) -> Option<usize>

The data "contained" within a implimentor of this trait is the history of all data slices passed to feed.

In other words, all previous data (or no previous data) may be used in determining the point to split.

Returns None if the data has no split point. Otherwise, returns an index in the most recently passed data.

Note that returning the index in the current slice makes most "look-ahead" splitting impossible (as it is permissible to pass 1 byte at a time).

Loading content...

Provided methods

fn iter_slices<'a>(self, data: &'a [u8]) -> IterSlices<'a, Self>ⓘ

Notable traits for IterSlices<'a, C>

impl<'a, C: ChunkIncr> Iterator for IterSlices<'a, C> type Item = &'a [u8];
 where
    Self: Sized, 

Given a ChunkIncr and a single slice, return a list of slices chunked by the chunker.

Will always return enough slices to form the entire content of data, even if the trailing part of data is not a chunk (ie: does not end on a chunk boundary)

fn iter_slices_strict<'a>(self, data: &'a [u8]) -> IterSlicesStrict<'a, Self>ⓘ

Notable traits for IterSlicesStrict<'a, C>

impl<'a, C: ChunkIncr> Iterator for IterSlicesStrict<'a, C> type Item = &'a [u8];
 where
    Self: Sized, 

Given a ChunkIncr and a single slice, return a list of slices chunked by the chunker. Does not return the remainder (if any) in the iteration. Use [IterSlices::take_rem()] or IterSlices::into_parts() to get the remainder.

Note that this is a non-incrimental interface. Calling this on an already fed chunker or using this multiple times on the same chunker may provide unexpected results

Loading content...

Implementors

impl ChunkIncr for RollSumIncr[src]

impl ChunkIncr for GzipRsyncableIncr[src]

impl ChunkIncr for MiiIncr[src]

impl ChunkIncr for PigzRsyncableIncr[src]

impl ChunkIncr for RamIncr[src]

impl ChunkIncr for ZpaqIncr[src]

impl ChunkIncr for ZstdIncr[src]

impl<'a> ChunkIncr for FastCdcIncr<'a>[src]

impl<'a> ChunkIncr for GearIncr32<'a>[src]

impl<H: BuzHashHash> ChunkIncr for BuzHashIncr<H>[src]

fn push(&mut self, data: &[u8]) -> Option<usize>[src]

Return the index in data immeidately following the hash matching.

Note that you can call this multiple times to examine "subsequent" data slices, but the index returned will always refer to the current data slice.

Loading content...