Skip to main content

Module ranking

Module ranking 

Source

Structs§

ApproxTokenCounter
v0: Approximate GPT-style tokenization tokens(content) := ceil(len(content) / 4)
TermFrequencyScorer
v0: Simple Term Frequency Scorer

Traits§

Scorer
TokenCounter