Module stats 

F-046: savings ledger, stats aggregation, and JSON/CSV export (roadmap.md F-046, interfaces.md §7.1 “Stats and Analytics JSON”).

StatsSummary/LedgerRecord mirror interfaces.md’s documented JSON shapes exactly. aggregate() is the one pure aggregation path shared by the CLI’s stats/gain/session subcommands (each just calls it and then tweaks framing fields like scope/window).

Ledger storage format: tokenfold.toml’s [analytics].ledger_db path is documented as ending in .db, but this pass stores newline-delimited JSON (JSONL) inside it rather than embedding a sqlite dependency — there is no sqlite crate anywhere in this workspace’s dependency graph. The .db extension is only what the config schema names the path; the format inside is plain JSONL. Upgrade path is a real embedded database (sqlite or similar) if this ever needs concurrent-writer safety; a single local CLI process appending its own ledger doesn’t need that yet.

Fields with no honest data source yet are zero-filled rather than invented, each with a comment at its computation site: cache (no cache subsystem exists anywhere in this codebase), retrieval.hits/misses/expired (only store-time marker counts are tracked, not later retrieval-attempt outcomes), latency (no per-request timing is threaded through CompressionReport/LedgerRecord yet — every TransformReport.elapsed_micros in this codebase is already always None), and untrusted_filter_count (the F-047 filter registry doesn’t exist yet).

Structs

CacheStats

LatencyStats

LedgerGcOutcome

LedgerRecord

Matches interfaces.md §7.1’s redacted recent_requests[] item shape verbatim. This is also the exact shape persisted (one per line, as JSON) by LedgerStore — there is no separate “storage” representation, so no raw prompt/response/command-arg/path/header bytes can ever end up on disk through this type.

LedgerStore

F-046’s optional local ledger: appends/reads/garbage-collects redacted LedgerRecord metadata at a JSONL file path (see the module doc for the .db-named-but-JSONL decision).

RetrievalStats

StatsSummary

Matches interfaces.md §7.1’s full StatsSummary JSON shape verbatim.

Enums

SavingsProvenance

Whether a savings/tokens figure came from an exact estimator ("measured"), the byte heuristic ("heuristic"), or is a derived extrapolation like estimated_lost_tokens ("estimated") rather than a directly observed value. There is no per-provider pricing data anywhere in this codebase, so this labels token-count provenance, not dollar cost.

Constants

SCHEMA_VERSION

Functions

aggregate

The one shared aggregation path: pure, total-order-independent, takes only ledger-shaped records. scope/window/project are generic defaults here — callers (the CLI’s stats/gain/session entry points) overwrite those framing fields on the returned StatsSummary to suit each command’s emphasis; the underlying counts never change.

filter_since

Keeps only records whose timestamp is within window_secs of now (both in Unix seconds). Records with an unparsable timestamp are dropped — unlike LedgerStore::gc, which fails safe by keeping what it can’t confidently date, a --since view is explicitly asking for “recent”, so an undatable record can’t honestly satisfy that.

format_unix_timestamp

Formats Unix seconds as an RFC3339 UTC timestamp (YYYY-MM-DDTHH:MM:SSZ) without pulling in a date/time crate: this is the one place in the codebase that needs calendar math, so a compact civil-from-days conversion (Howard Hinnant’s well-known civil_from_days algorithm, see http://howardhinnant.github.io/date_algorithms.html) is a fair ponytail trade against adding chrono/time as a dependency for a couple of call sites.

generate_request_id

A short synthetic ID in the documented tc-XXXXXXXX shape (8 lowercase hex chars), derived from the current time and process ID. Good enough for a human-scannable, practically-unique local identifier; not a security-sensitive value, so DefaultHasher (not a cryptographic hash) is an appropriate, dependency-free choice.

now_unix

parse_duration_secs

Parses a duration shorthand like "30d", "24h", "90m", "120s", or a bare integer (seconds) — used by tokenfold gain --since.

record_from_report

Builds the LedgerRecord for one CompressionReport, used both by the CLI’s post-run ledger-recording hook and by ad-hoc tokenfold stats <report-glob> aggregation (turning a standalone report file into the same shape). surface is derived from the report itself ("wrap" when CommandReport is present, "cli" otherwise) rather than passed in, so both callers classify it identically.

savings_provenance

Labels a directly-observed token count by estimator exactness (see EstimatorInfo.is_exact). Derived/extrapolated figures (e.g. estimated_lost_tokens) are always SavingsProvenance::Estimated by construction, independent of this function.

to_csv

Renders summary as two CSV sections separated by a blank line: a one-row summary table, then the recent_requests table. A manual writer (no new dependency) is enough for this shape — every field is a plain number or a short known-alphabet string, so the only escaping that can ever matter is on project/project_hash/bypass_reason, handled by [csv_field].