Token budget management for LLM conversations.
This module provides token budget management to prevent context window overflow while maintaining conversation continuity. It implements a hybrid approach combining rolling summary with recent message windowing.
Key Components
- [
types]: Core types likeTokenBudget,BudgetStrategy,PreparedContext - [
counter]: Token counting via BPE tokenizer (tiktoken) or heuristic estimation - [
segmenter]: Message segmentation preserving tool-call atomicity - [
limits]: Model context window limits registry - [
preparation]: Context preparation with budget enforcement - [
summarizer]: Conversation summarization for context preservation