Expand description
Token usage out of provider responses, as they stream past.
UsageParser takes a response body in pieces of any size, as a JSON
document or as text/event-stream, and keeps the latest usage it saw.
It reads the shapes of OpenAI’s Chat Completions (usage.prompt_tokens
/ completion_tokens, in the final chunk of a stream), OpenAI’s
Responses API (usage.input_tokens / output_tokens, nested under
response in its response.completed event) and Anthropic’s Messages
API (message_start → message.usage, message_delta → usage, whose
output_tokens is cumulative). Anthropic’s
cache_creation_input_tokens and cache_read_input_tokens count as
input: they are billed, so they count toward budgets.
Loops only, every buffer bounded: provider responses are untrusted input.
Structs§
- Usage
- Tokens a response reported.
- Usage
Parser - Usage from a response body fed in pieces. See the module docs.