Skip to main content

Module token

Module token 

Source
Expand description

Token system — QATBE, SCS, PDS, budget management (PRD SS17-18, SS27).

Re-exports§

pub use budget::compute_budget;
pub use budget::suggest_pds_tier;
pub use budget::AdaptiveBudget;
pub use budget::BudgetConfig;
pub use budget::BudgetTier;
pub use counter::count_tokens;
pub use counter::estimate_tokens_fast;
pub use counter::TokenBudget;

Modules§

budget
Adaptive Token Budget (ATB) — dynamic token allocation based on query complexity.
counter
Token counter and budget tracking.
pds
Progressive Detail Streaming (PDS) — PRD §27.
qatbe
Query-Aware Token-Budgeted Extraction (QATBE).
scs
Semantic Content Segmentation (SCS).