Skip to main content

text_to_sparse

Function text_to_sparse 

Source
pub fn text_to_sparse(text: &str) -> SparseVector
Expand description

Tokenize text into a term-frequency SparseVector: dimension ids are token ids (term_id) and values are within-text term counts. The ingest side of the BM25 path (ADR-0046).