Skip to main content

Module lexicon

Module lexicon 

Source
Expand description

Lexicon-based importance scoring — LexiconScorer trait, ConfigLexiconScorer, DefaultEnglishScorer, CompositeLexiconScorer, LexiconAppender, and LexiconProposal. Lexicon-based importance scoring.

Provides LexiconScorer — a trait for domain-specific importance weighting injected at crate::ContextForge construction time via the builder API — and ConfigLexiconScorer, the default TOML-driven implementation.

The scorer runs inside crate::engine::ContextEngine::assemble after BM25 + recency decay, before the token-budget cut. Importance weighting must happen at this point to influence which entries survive the budget cut, not re-rank survivors after.

§Two-layer design

The builder always pre-seeds a DefaultEnglishScorer that recognizes plain-English importance signals (“confirmed”, “never mind”, etc.). A persona scorer (ConfigLexiconScorer loaded from a TOML file) is optional and stacks on top via CompositeLexiconScorer. Both layers are additive — the engine applies the -1.0 floor clamp after fusion.

Structs§

CompositeLexiconScorer
Applies multiple LexiconScorers in sequence and sums their boosts.
ConfigLexiconScorer
LexiconScorer backed by a TOML config file.
DefaultEnglishScorer
Always-on baseline scorer for plain-English importance signals.
LexiconAppender
Atomically appends or removes entries from a TOML lexicon file.
LexiconConfig
TOML schema for ConfigLexiconScorer.
LexiconPatterns
A list of case-insensitive substring patterns.
LexiconProposal
A candidate term proposed for addition to the lexicon.

Traits§

LexiconScorer
Domain-specific importance scorer, injected at construction time.

Functions§

bootstrap_prompt
Generate a calibration prompt for bootstrapping a persona lexicon via an LLM.