Expand description
Token counting and message trimming utilities.
Provides TokenCounter trait for pluggable token counting,
CharTokenCounter for test approximation, and trim_messages
for context window management.
Based on Group 16.6 and Group 19 of the pre-plan.
Structs§
- Char
Token Counter - Approximate token counter using character count / 4.
- Trim
Options - Options for
trim_messages.
Enums§
- Message
Role - Which message roles are acceptable at window boundaries.
- Trim
Strategy - Strategy for which messages to keep when trimming.
Traits§
- Token
Counter - How messages should be counted for trimming.
Functions§
- trim_
messages - Trim a message list to fit within a token budget.