Module shingle 

Shingling (k-gram generation) for text documents.

Converts documents into sets of overlapping subsequences (shingles/k-grams) which are then hashed for MinHash computation.

Structs

HashedShingleIterator

Iterator that yields hashed shingles directly.

ShingleIterator

Iterator over shingles (k-grams) of a byte sequence.

WordShingleIterator

Create word-level shingles (n-grams of words).

Functions

get_hashed_shingles

Compute all hashed shingles for a document.

get_shingles

Compute all shingles for a document.

normalize_text

Normalize text by lowercasing and removing excess whitespace.