Module deduplication 

Modules

constants

test_utils

Structs

Chunk

Chunker

Chunk Generator given an input stream. Do not use directly. Use chunk_target_default.

DataAggregator

DeduplicationMetrics

FileDeduper

RawXorbData

This struct is the data needed to cut a

Traits

DeduplicationDataInterface

The interface needed for the deduplication routines to run. To use the deduplication code, define a struct that implements these methods. This struct must be given by value to the FileDeduper struct on creation.

Functions

find_partitions

Find valid partition points in a file where we can chunk in parallel. Returns the start points of each partition (i.e. file offset 0 is always the first entry, and file_size is never in the result). Note that reader position is modified and not restored.