List of all items

Structs

• dict::Dict
• freq::FreqMap
• segmenter::Tokenizer
• segmenter::TokenizerBuilder
• token::Token

Enums

• error::KhamError
• token::TokenKind

Functions

• dict::builtin_dict
• normalizer::normalize
• pre_tokenizer::classify_char
• pre_tokenizer::is_emoji
• pre_tokenizer::pre_tokenize
• tcc::tcc_boundaries
• tcc::tcc_iter

Statics

• dict::BUILTIN_WORDS