Crate kevy_text 

kevy-text — dictionary-free full-text core: script-aware tokenization (Latin words + CJK bigrams), per-shard inverted segments maintained synchronously with writes, BM25 ranking with shard-local statistics.

Modules

cold

The frozen half of a text index: the codec a cold bucket segment’s posting payloads use, the freeze that produces them, and the scorer that reads them back — all pure (no I/O; the segment file itself is the engine’s concern).

Structs

CorpusStats

Corpus statistics supplied from outside a segment, for scoring one shard’s documents against the whole corpus rather than its own slice.

Distinct

Collapse the page so only the best document per value of a stored field appears.

Facet

Count the values of a stored field over the whole match set.

FacetedMatches

One faceted query’s answer: the page, and a count per value for each requested field.

Filter

A non-scoring predicate over a document’s stored values.

KevyTokenizer

The default dictionary-free tokenizer.

QueryOpts

Everything a MATCH query carries beyond its text and result limit.

SegmentShape

What an index declares, in the terms a segment is built from.

Sort

An order to select the top hits by, other than the score.

TextMatch

One ranked hit.

TextSegment

One shard’s inverted segment.

TextStats

Sizing counters (memory formula + IDX.LIST).

Constants

BM25_B

Length normalization strength.

BM25_K1

Term-frequency saturation.

Traits

Tokenizer

Pluggable tokenizer (tokenize is the only shipped impl).

Functions

parse_clauses

Split a query into bare terms, quoted phrases, and word* prefixes. A "…" group of two or more tokens is a phrase (a shorter group joins the bare terms — a one-word “phrase” is just that word); an unquoted word ending in * is a prefix. An unterminated quote is lenient: the remainder is read as plain text rather than rejected.

sorted_order

The order a page sorted by a stored value is in: a document WITH a value outranks one without (in both directions — missing is not a value), then by that value’s order key, then by row key so ties are stable.

tokenize

Tokenize per the default rules (see module doc).

tokenize_spans

Tokens with the byte span each came from in the ORIGINAL text: (token, start, end) where start..end indexes text (a CJK bigram spans both its characters, a lowercased word its source range). This is what highlighting re-analyses a winning document’s field with to point <em> spans at the source.

Type Aliases

Bucket

One value bucket: the identity a cross-shard merge sums by, a spelling of it that occurs in the corpus, and how many documents matched with it.

Clauses

Parsed query clauses: bare terms, phrases (each a token sequence) and prefix stems.