Crate tokengrams

Modules

mmap_slice

Structs

InMemoryIndex

An in-memory index exposes suffix table functionality over text corpora small enough to fit in memory.

MemmapIndex

A memmap index exposes suffix table functionality over text corpora too large to fit in memory.

SuffixTable

A suffix table is a sequence of lexicographically sorted suffixes. The table supports n-gram statistics computation and language modeling over text corpora.