Struct FreqMap 

pub struct FreqMap(/* private fields */);

A word→frequency lookup table backed by the Thai National Corpus (TNC).

Frequencies are raw TNC occurrence counts. Words absent from the table return 0, which is a safe default (the DP scorer simply ignores them).

The built-in table is loaded with FreqMap::builtin. Custom tables can be constructed from any tab-separated word\tcount text via FreqMap::from_tsv.

Implementations

impl FreqMap

pub fn from_tsv(data: &str) -> Self

Parse a tab-separated word\tcount text (one entry per line).

pub fn builtin() -> Self

Load the built-in TNC frequency table.

Parses the embedded tnc_freq.txt (106k entries) into a BTreeMap. This is a pay-once startup cost; the returned FreqMap should be reused across segmentation calls (e.g. stored in Tokenizer).

pub fn get(&self, word: &str) -> u32

Look up a word’s frequency; returns 0 if not found.

pub fn max_freq(&self) -> u32

Return the maximum frequency value in the table, or 0 if the table is empty.

Used by KeyExtractor to compute the IDF numerator for TF-IDF scoring.

Examples

use kham_core::freq::FreqMap;

let m = FreqMap::from_tsv("กิน\t100\nข้าว\t500\nที่\t9999\n");
assert_eq!(m.max_freq(), 9999);

let empty = FreqMap::from_tsv("");
assert_eq!(empty.max_freq(), 0);