Module algo 

The Algorithm Arsenal

Categorized similarity algorithms, each returning a normalized f64 score where 1.0 = identical and 0.0 = completely dissimilar.

Short Strings & Typos

Algorithm	Best for	Time
levenshtein	Edit distance (typos)	O(n·m)
jaro_winkler	Name matching	O(n·m)
hamming	Equal-length strings	O(n)

Sets & Documents

Algorithm	Best for	Time
jaccard	N-gram set similarity	O(n+m)
minhash	Large-document fingerprints	O(k·n)
simhash	Deduplication fingerprints	O(n)

Statistical Meaning

Algorithm	Best for	Time
bm25	Search relevance ranking	O(n·m)
tfidf	Term-weighted cosine similarity	O(n·m)

Each function in this module takes pre-processed inputs (strings) and returns an f64 in [0.0, 1.0].

Modules

bm25

BM25 (Best Matching 25) — a probabilistic retrieval function for ranking documents against a query. An improvement over TF-IDF for contextual similarity without the overhead of an LLM.

hamming

Hamming Distance — counts positions where corresponding symbols differ for strings of equal length. Normalized to [0.0, 1.0].

jaccard

Jaccard Similarity — measures similarity between two sets of words or n-grams. Normalized to [0.0, 1.0].

jaro_winkler

Jaro-Winkler Distance — exceptionally good for name matching as it heavily weights matching prefixes. Normalized to [0.0, 1.0].

levenshtein

Levenshtein Distance — minimum single-character edits to change one string into another. Normalized to [0.0, 1.0] where 1.0 = identical.

minhash

MinHash — probabilistic data structure for estimating Jaccard similarity between large documents. Uses k hash functions to create compact fingerprints.

simhash

SimHash — locality-sensitive hash for deduplication of large documents.

tfidf

TF-IDF + Cosine — counts word frequencies, weights them by rarity, and calculates the cosine angle between document vectors.