Struct DuplicateDetector 

pub struct DuplicateDetector { /* private fields */ }

Detects potential duplicates in a dataset.

Implementations

impl DuplicateDetector

pub fn new(similarity_threshold: f64, comparison_fields: Vec<String>) -> Self

Creates a new duplicate detector.

pub fn string_similarity(&self, a: &str, b: &str) -> f64

Calculates similarity between two strings (Jaccard similarity).

pub fn are_duplicates<T: Duplicatable>(&self, a: &T, b: &T) -> bool

Checks if two records are potential duplicates.

pub fn find_duplicates<T: Duplicatable>( &self, records: &[T], ) -> Vec<(usize, usize, f64)>

Finds all duplicate pairs in a collection.