Module token 

Structs

HeuristicTokenCounter

Default token counter: ~4 chars per token (heuristic for English text).

Traits

TokenCounter

Pluggable token counting strategy.

Functions

content_tokens

Estimate tokens for a Content slice.

estimate_tokens

Rough token estimate: ~4 chars per token for English text. See TokenCounter for pluggable alternatives.

message_tokens

Estimate tokens for a single message.

resolve_counter

Returns the provided counter, or HeuristicTokenCounter if None.

total_tokens

Estimate total tokens for a message list.