Module simhash 

SimHash — locality-sensitive hash for deduplication of large documents.

Generates a 64-bit fingerprint where similar documents have similar fingerprints. Similarity is measured by the Hamming distance between fingerprints.

Performance

O(n) time where n = number of features. O(1) memory (single u64).

Structs

SimHashFingerprint

A 64-bit SimHash fingerprint.

Functions

compare

One-shot: hash both strings and compare.

compare_default

Default one-shot with tetragram shingles.

fingerprint

Generate a SimHash fingerprint for a text.

fingerprint_default

Default fingerprint with tetragram shingles.

hamming_distance

Hamming distance between two SimHash fingerprints.

similarity

Normalized similarity from two SimHash fingerprints.