Struct LshIndex 

pub struct LshIndex { /* private fields */ }

LSH index for finding candidate similar document pairs.

Documents are inserted into buckets based on their band hashes. Documents in the same bucket are candidate pairs for similarity checking.

Implementations

impl LshIndex

pub fn new(config: &Config) -> Result<Self>

Create a new LSH index from configuration.

Errors

Returns an error if the configuration has incompatible parameters.

pub fn clear(&mut self)

Clear all indexed documents and derived cluster state, keeping the band and row structure and the similarity threshold.

This is the infallible way to empty an index for reuse. Unlike rebuilding via Self::new (which is fallible and, when a caller swallowed its error, could leave a stale populated index while surrounding state was reset), clear() cannot fail and cannot leave the index half-reset.

pub fn insert(&mut self, signature: MinHashSignature) -> Result<Vec<usize>>

Insert a signature into the LSH index.

Returns a list of candidate similar document indices that collided in at least one bucket.

Errors

Returns Error::InvalidConfig when doc_id exceeds the configured maximum (prevents unbounded allocation from adversarial identifiers).

pub fn query(&self, signature: &MinHashSignature) -> Vec<usize>

Query for candidate similar documents.

Returns document indices that collided with the given signature in at least one LSH bucket.

pub fn verify_similarity(&self, doc_a: usize, doc_b: usize) -> Option<f64>

Verify similarity between two documents using their signatures.

pub fn find_clusters(&mut self) -> &[DuplicateCluster]

Find all duplicate clusters in the index.

This performs pairwise verification of all candidate pairs and groups documents into clusters.

pub fn get_cluster_for_doc(&self, doc_id: usize) -> Option<&DuplicateCluster>

Get the cluster for a document index.

pub fn is_duplicate(&self, doc_id: usize) -> bool

Check if a document is a duplicate (belongs to any cluster).

pub fn get_unique_indices(&self) -> Vec<usize>

Get all unique documents (first in each cluster + non-duplicate documents).

pub const fn doc_count(&self) -> usize

Get the total number of documents indexed.

pub fn cluster_count(&self) -> usize

Get the number of duplicate clusters.

pub fn duplicate_count(&self) -> usize

Get the number of duplicate documents (documents in clusters, excluding representatives).

pub fn stats(&self) -> LshStats

Get statistics about the LSH index.

pub fn memory_usage(&self) -> usize

Estimate memory usage in bytes.