Expand description
Request-level deduplication — THE critical accounting step (CLAUDE.md §8.1).
WHY: Claude Code (and agents like it) write one JSONL line per content block,
and every line repeats the request’s message.usage. A single API request
therefore appears as 2–10 lines sharing one requestId (observed on real
data: 83 lines -> 26 requests; 642 -> 262). Naively summing per-line usage
multiplies token counts by that factor.
RULE: group assistant events by requestId and take the field-wise MAX of
each usage counter (see crate::model::Usage::merge_max). Lines of one
request either repeat identical numbers or grow monotonically while
streaming, so MAX recovers the final per-request value in both cases without
double counting. Events with no requestId are never merged with each other.
This pass also performs thinking ATTRIBUTION (§8.2). Claude Code’s
output_tokens ALREADY includes extended-thinking tokens (verified on real
data — thinking-only requests report output far larger than their visible
text), so there is no undercount to “fix”. Instead we measure the visible
thinking chars per request (the basis for an estimated thinking-token SHARE of
output) and flag requests whose thinking was encrypted/redacted, where that
share is unmeasurable (absence ≠ zero, §8.5).
Structs§
- Dedup
Stats - Proof-of-work counters from the dedup pass.
- Usage
Record - One deduplicated API request with its finalized usage.
Functions§
- dedup_
session - Collapse a session’s assistant events into per-request usage records.