Struct StreamDedupIndex 

pub struct StreamDedupIndex { /* private fields */ }

Index of stream fingerprints for near-duplicate detection.

Files are added by name via ingest; duplicates are retrieved via find_duplicates.

Implementations

impl StreamDedupIndex

pub fn new(config: StreamChunkerConfig) -> Self

Create a new, empty index with the given chunker config.

pub fn ingest<R: Read>( &mut self, name: &str, reader: R, ) -> Result<StreamFingerprint>

Ingest a stream and store its fingerprint under name.

Returns the computed StreamFingerprint.

Errors

Propagates io::Error from reader.

pub fn len(&self) -> usize

Number of indexed entries.

pub fn is_empty(&self) -> bool

Return true if the index is empty.

pub fn jaccard_similarity( &self, a: &StreamFingerprint, b: &StreamFingerprint, ) -> f64

Compute the Jaccard similarity between two fingerprints.

pub fn find_duplicates(&self, threshold: f64) -> Vec<(String, String, f64)>

Find all pairs of indexed entries whose Jaccard similarity exceeds threshold.

Returns a list of (name_a, name_b, similarity) tuples sorted by descending similarity.

pub fn get(&self, name: &str) -> Option<&StreamFingerprint>

Retrieve the fingerprint stored under name, if any.

Trait Implementations

impl Debug for StreamDedupIndex

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.