[−][src]Trait asuran_chunker::Chunker

pub trait Chunker: Clone {
    type Chunks: Iterator<Item = Result<Vec<u8>, ChunkerError>> + 'static;
    fn chunk_boxed(&self, read: Box<dyn Read + Send + 'static>) -> Self::Chunks;

    fn chunk<R: Read + Send + 'static>(&self, read: R) -> Self::Chunks { ... }

    fn chunk_slice<R: AsRef<[u8]> + Send + 'static>(
        &self, 
        slice: R
    ) -> Self::Chunks { ... }
}

Describes something that can slice objects in a defined, repeatable manner

Chunkers must meet the following properties: 1.) Data must be split into one or more chunks 2.) Data must be identical to original after a simple reconstruction by concatenation 3.) The same data and settings must produce the same slices every time 4.) Chunkers (that have a max size) should not produce any chunks larger than their max_size 5.) Chunkers (that have a min size) should produce, at most, 1 slice smaller than its min_size, and should only do as such when there is not enough data left to produce a min size chunk

For the time being given the lack of existential types, Chunkers use Box<dyn Read + 'static>.

If/when existential types get stabilized in a way that helps, this will be switched to an existential type, to drop the dynamic dispatch.

Chunkers should, ideally, contain only a small number of settings for the chunking algorithm, and should there for be cloneable with minimal overhead. Ideally, they should implement copy, but that is not supplied as a bound to increase the flexibility in implementation

The Send bound on the Read is likely temporary, it is currently required to make the streams feature work properly.

Associated Types

type Chunks: Iterator<Item = Result<Vec<u8>, ChunkerError>> + 'static

The return type of the functions in this trait is an iterator over the chunks of their input.

The returned iterator must be owned, hence the 'static bound.

Loading content...

Required methods

fn chunk_boxed(&self, read: Box<dyn Read + Send + 'static>) -> Self::Chunks

Core function, takes a boxed owned Read and produces an iterator of Vec over it

Loading content...

Provided methods

fn chunk<R: Read + Send + 'static>(&self, read: R) -> Self::Chunks

Convenience function that boxes a bare Read for you, and passes it to chunk_boxed

This will be the primary source of interaction wth the API for most use cases

fn chunk_slice<R: AsRef<[u8]> + Send + 'static>(&self, slice: R) -> Self::Chunks

Convenience function that boxes an AsRef<u8> wrapped in a cursor and passes it to chunk_boxed. Implementations are encouraged to overwrite when sensible.

This method is provided to ensure API compatibility when implementations are using memory mapped io or the like. When chunkers can sensibly override this, they are encouraged to, as it would otherwise result in a performance overhead for consumers using memmaped IO.

Loading content...

Implementors

impl Chunker for BuzHash[src]

type Chunks = BuzHashChunker

impl Chunker for FastCDC[src]

type Chunks = FastCDCChunker

impl Chunker for StaticSize[src]

type Chunks = StaticSizeChunker

Loading content...