Module vibrato::tokenizer

source · [−]
Expand description

Viterbi-based tokenizer.

Modules

worker
Provider of a routine for tokenization.

Structs

Tokenizer
Tokenizer.