Skip to main content

winnow

Function winnow 

Source
pub fn winnow(hashes: &[u64], w: usize) -> Vec<(u64, usize)>
Expand description

Winnowing: select fingerprints from k-gram hashes.

Over every window of w consecutive hashes the minimum is selected (rightmost on ties). Inputs shorter than one window select their global minimum, so short segments are still fingerprinted.

Returns deduplicated (hash, gram index) pairs in ascending index order. The selection guarantees that any shared token run of at least w + k - 1 tokens produces at least one shared fingerprint.