trusty-mpm 1.8.2

//! Per-session token-savings ledger (#6958).
//!
//! Why: trusty-mpm spends real effort not sending tokens — folding instruction
//! sources into one compiled prompt, diverting a large file read to a cheap
//! worker, compressing gate output before an agent reads it. None of that was
//! visible anywhere, so the owner's question ("how much are we saving?") had no
//! answer at all. This module is that answer's storage: one append-only log
//! every producer writes a row to, and one read-time fold the statusline (and,
//! later, the console) renders.
//!
//! What: `~/.trusty-mpm/usage/savings.jsonl` — one JSON object per line,
//! `{ts, session_id, technique, tokens_saved, tokens_before, cost_saved_usd,
//! basis}`. [`append_row`] is the single writer, [`fold_session`] and
//! [`fold_all`] the single readers. `technique` is an open string, so a new
//! producer needs no schema change here.
//!
//! Two properties the rest of the feature depends on:
//!
//! - **Append-only, one row per `O_APPEND` write.** No cross-process lock, no
//!   rollup file that can drift from the log. The total is derived at read
//!   time, matching the owner's 2026-07-29 ruling for the Costs surface.
//! - **A bad row cannot move the total.** A line that does not parse, a row
//!   with `tokens_saved <= 0`, and a row with a non-positive or non-finite
//!   `cost_saved_usd` are each skipped with a `warn!` and contribute nothing.
//!   A producer bug that undercounts its baseline therefore shows as a missing
//!   contribution, never as a negative or inflated displayed figure.
//!
//! **Who writes this ledger (#7636).** Six files share the `savings*` name
//! prefix with no map between them — the #7658 engineer read ~1100 lines
//! before a 30-line fix, and the original brief for that fix described a
//! repair arm ([`crate::core::savings_repair`]) that already existed. This is
//! that map: producer, the `technique` it writes, how the row reaches the
//! ledger, and what stops it duplicating.
//!
//! | Producer | Technique | Append path | Dedup guard |
//! |---|---|---|---|
//! | `savings` (this module) | none — shared writer/reader, not a producer | [`append_row`] and [`append_row_once`] are the ledger's only two writers | `append_row`: none, every call appends. `append_row_once`: [`row_presence`] on `(session_id, technique, basis)`, held under an exclusive file lock |
//! | [`crate::core::savings_instructions`] | `instruction-compression` | `record_instruction_compression_in_with` calls [`append_row_once`] directly | inherited from `append_row_once` |
//! | [`crate::core::savings_sidecar`] | `instruction-compression` (staged) | stages a [`SavingsRow`] to `pending-savings/<key>.json` when no session id exists yet; a later hook's `emit_staged_row` claims it via an atomic rename and calls [`append_row_once`] | `append_row_once`'s guard, plus the claim rename — of two racing hooks, only the one whose rename succeeds appends |
//! | [`crate::core::savings_repair`] | none — not a producer | never appends; rewrites the ledger atomically (temp file + rename) from the kept rows' original bytes | N/A — quarantines already-written rows to a sidecar, never adds new ones |
//! | [`crate::core::savings_divert`] | `divert` | `record_divert` calls [`append_row`] directly | none — a second diversion of the same file is a second real saving, so suppressing it would delete data |
//! | [`crate::core::savings_compress`] | `compress` | `record_compress` calls [`append_row`] directly | none, same reasoning as `savings_divert` |
//!
//! Only the two `instruction-compression` producers route through
//! `append_row_once`; `divert` and `compress` keep the unguarded [`append_row`]
//! on purpose (see its own doc for why).
//!
//! **Who READS which rows (owner ruling 2026-09-14, #7867).** The `💸`
//! statusline segment folds [`PER_CALL_TECHNIQUES`] only — `compress` and
//! `divert`, the per-tool-call measurements — through
//! [`fold_session_per_call`] and, since #8063, [`session_row_percents`]
//! ([`fold_sessions_per_call`] feeds the cross-session mean instead).
//! `instruction-compression` rows stay on the ledger and stay readable, but
//! feed no part of that segment: they are one launch-time comparison of the
//! compiled prompt against its sources, on a different clock from a figure that
//! is supposed to move when a tool call is compressed. `tm doctor`'s
//! `instruction_fold` row is where that measurement surfaces instead.
//!
//! **[`SavingsTotal::percent_saved`] denominator (owner ruling 2026-09-08,
//! #7179; re-scoped by the 2026-09-15 ruling, #8063).** The method takes its
//! denominator as an `Option<u64>` argument because this module cannot reach
//! one of the two: "session actual tokens" is a cumulative counter
//! `crate::commands::statusline::compaction` (the `tm` binary) folds across
//! every auto-compaction reset of the `statusLine` hook's
//! `total_input_tokens`, keyed by session id in
//! `~/.trusty-mpm/statusline/<session_id>.json`, since that raw figure resets
//! on every compaction and would otherwise make the percent jump
//! non-monotonically for reasons unrelated to anything the harness saved.
//!
//! - `Some(actual)` prices a **session share**, `saved / (actual + saved)` —
//!   what `tm commit-trailers` reports for a whole session.
//! - `None` prices the ledger's own `tokens_saved / tokens_before` ratio, and
//!   that is what the `💸` segment renders since #8063, once PER ROW: the badge
//!   shows the latest row's own reduction beside the mean of this session's
//!   rows, because a session share rounded an 18.9 % per-call reduction to a
//!   1 % badge and so read as "the harness saves nothing".
//!
//! `#[serde(default)]` on `tokens_before` so a pre-#7179 row folds with
//! `tokens_before = 0` and is excluded from that denominator, rather than
//! failing to parse.
//!
//! Everything here fails soft: a missing, unreadable, or truncated ledger folds
//! to zero rather than erroring, because the consumer is a status bar on
//! Claude Code's hot render path.
//!
//! Test: the inline suite in `savings_tests.rs` — `append_then_fold_round_trips`,
//! `fold_skips_a_malformed_line_and_keeps_the_valid_total`,
//! `a_negative_row_cannot_raise_the_total`, `fold_ignores_other_sessions`,
//! `fold_of_a_missing_ledger_is_zero`, `percent_saved_rounds_to_nearest_whole_number`.

use std::path::{Path, PathBuf};

use serde::{Deserialize, Serialize};

/// Characters per token used by every estimate in this feature.
///
/// Why: the owner's directive is explicit that the figure "doesn't have to be
/// exact", and a byte-delta producer has no tokenizer available at its call
/// site. Four characters per token is the conventional English-prose
/// approximation Anthropic's own guidance uses; stating it once here is what
/// keeps two producers from picking two different divisors and reporting
/// incomparable numbers.
/// What: the divisor applied to a byte delta to reach a token delta.
/// Test: `instruction_compression_tokens_use_the_shared_divisor`.
pub const BYTES_PER_TOKEN: f64 = 4.0;

/// `technique` value written by the instruction/language-compression producer.
///
/// Test: `instruction_compression_row_carries_the_named_technique`.
pub const TECHNIQUE_INSTRUCTION_COMPRESSION: &str = "instruction-compression";

/// `technique` value written by the bulk-read diversion producer (#6959).
///
/// Test: `divert_row_carries_the_named_technique`.
pub const TECHNIQUE_DIVERT: &str = "divert";

/// `technique` value written by the tool-output compression producer.
///
/// Why: `tm compress` shrinks a gate's output before an agent reads it, which
/// avoids sending tokens exactly the way the other two producers do. Until it
/// wrote a row, the `💸` segment reported everything the harness saves EXCEPT
/// bash and tool output.
/// Test: `a_compress_row_carries_the_named_technique`.
pub const TECHNIQUE_COMPRESS: &str = "compress";

/// The techniques the `💸` statusline segment folds (owner ruling 2026-09-14,
/// #7867).
///
/// Why: the segment reports what rtk- and shunt-style tool-output compression
/// saves — a measurement one tool call makes about its own output, from the
/// bytes that call did not send, recorded after every call.
/// [`TECHNIQUE_INSTRUCTION_COMPRESSION`] is a different measurement on a
/// different clock: one comparison per session launch, of the compiled prompt
/// against the corpus it was folded from. Summing the two into one percent made
/// a per-call figure move for a reason no tool call caused, and made a launch
/// whose prompt did not shrink read as a session that compressed nothing.
/// What: `compress` and `divert` — the two producers that append per call, with
/// no once-per-session dedup (see the module header's producer table).
/// Test: `the_per_call_techniques_are_compress_and_divert`.
pub const PER_CALL_TECHNIQUES: [&str; 2] = [TECHNIQUE_COMPRESS, TECHNIQUE_DIVERT];

/// Whether `technique` is one the `💸` segment folds (#7867).
///
/// Test: `the_per_call_techniques_are_compress_and_divert`.
pub fn is_per_call_technique(technique: &str) -> bool {
    PER_CALL_TECHNIQUES.contains(&technique)
}

/// The environment variable Claude Code exports carrying the session's own id.
///
/// Why (#7209): every row on this ledger is folded back by
/// [`fold_session`] under the `session_id` Claude Code sends the statusline on
/// stdin, which is this variable's value. A producer that keys a row by any
/// other id writes a row the segment can never match, and the `💸` segment then
/// omits itself instead of rendering a percent.
/// What: `CLAUDE_CODE_SESSION_ID`.
/// Test: `claude_code_session_id_names_the_harness_variable`.
pub const CLAUDE_CODE_SESSION_ID_ENV: &str = "CLAUDE_CODE_SESSION_ID";

/// The Claude Code session id, when the harness exported one.
///
/// Why: this is the single read of [`CLAUDE_CODE_SESSION_ID_ENV`] both savings
/// producers route through — the divert producer in the `tm` binary and the
/// instruction-compression producer in this crate — so the two cannot drift
/// into keying their rows differently.
/// What: the variable's value, or `None` when it is absent or blank.
/// Test: `claude_code_session_id_names_the_harness_variable`, and the
/// producers' own suites.
pub fn claude_code_session_id() -> Option<String> {
    std::env::var(CLAUDE_CODE_SESSION_ID_ENV)
        .ok()
        .filter(|value| !value.trim().is_empty())
}

/// One producer's claim that a technique avoided sending some tokens.
///
/// Why: estimates and measurements share one row shape, which is why `basis` is
/// free text rather than a typed formula — a divert row's basis is three token
/// counts, an instruction-compression row's is two byte counts, and a future
/// producer's will be something else again. Making `basis` structured would
/// force a schema change per producer, which is exactly what an open
/// `technique` string is here to avoid.
/// What: `tokens_saved` is signed so a producer bug that writes a negative is
/// *representable* and therefore rejectable at fold time (see the module
/// header) rather than deserialising into a huge unsigned value.
/// Test: `a_negative_row_cannot_raise_the_total`, `fold_ignores_other_sessions`.
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
pub struct SavingsRow {
    /// RFC 3339 UTC timestamp of the moment the producer wrote the row.
    pub ts: String,
    /// The session the saving is attributed to; the fold filters on this.
    pub session_id: String,
    /// Open technique name — `instruction-compression`, `divert`, `compress`, …
    pub technique: String,
    /// Estimated tokens not sent. Rows at or below zero are skipped.
    pub tokens_saved: i64,
    /// Estimated pre-saving token count for this row — `tokens_saved` plus the
    /// tokens actually sent instead (#7179).
    ///
    /// Why: [`SavingsTotal::percent_saved`]'s primary denominator is the
    /// session's actual-token counter (see the module header), but that value
    /// is only available once a `statusLine` tick has landed for this session.
    /// This field folds into the *fallback* denominator used when it has not —
    /// the ledger's own before-figure, folded across every row alongside
    /// `tokens_saved`.
    /// What: `u64` (a plain count, never negative by construction).
    /// `#[serde(default)]` so a row written before this field existed folds as
    /// `0` — excluded from the fallback denominator rather than failing to
    /// parse. Producers that cannot state a before-figure simply omit it.
    /// Test: `percent_saved_rounds_to_nearest_whole_number`,
    /// `a_row_written_before_tokens_before_existed_still_folds`.
    #[serde(default)]
    pub tokens_before: u64,
    /// Estimated USD not spent. Rows at or below zero are skipped.
    pub cost_saved_usd: f64,
    /// Free text stating how the two figures above were arrived at.
    pub basis: String,
    /// Which source named the model the row was priced at (#6972).
    ///
    /// Why: every producer prices its token delta at a model's published input
    /// rate, and before #6972 a row priced from the config chain's Sonnet
    /// default was indistinguishable from one priced at the model the session
    /// was really running. That is how a whole session of Opus diversions
    /// under-reported by five times without a single wrong-looking row. Naming
    /// the source makes a wrong price diagnosable off the ledger line.
    /// What: one of
    /// [`crate::core::session_model::MODEL_SOURCE_ENV`],
    /// `MODEL_SOURCE_STATUSLINE`, `MODEL_SOURCE_CONFIG_FALLBACK`, or
    /// `MODEL_SOURCE_LAUNCH_CONFIG`. `#[serde(default)]` so rows written before
    /// #6972 still fold, reading back as `""`.
    /// Test: `divert_row_names_the_model_source`,
    /// `instruction_compression_row_names_its_model_source`.
    #[serde(default)]
    pub model_source: String,
}

/// The folded total of every accepted row in one read.
///
/// Why: the statusline needs `tokens_saved` and `tokens_before` together to
/// compute a percent, and `cost_saved_usd` for `tm`'s own dollar-denominated
/// reporting commands (the ledger stays the one source both surfaces read).
/// `rows` is what lets a caller distinguish "the ledger held nothing" from
/// "every row was rejected", which read very differently in a bug report.
/// What: sums of the accepted rows only; skipped rows contribute nothing.
/// Test: `fold_skips_a_malformed_line_and_keeps_the_valid_total`.
#[derive(Debug, Clone, Copy, Default, PartialEq)]
pub struct SavingsTotal {
    /// Sum of `tokens_saved` across accepted rows.
    pub tokens_saved: u64,
    /// Sum of `tokens_before` across accepted rows — the percent denominator.
    pub tokens_before: u64,
    /// Sum of `cost_saved_usd` across accepted rows.
    pub cost_saved_usd: f64,
    /// How many rows were accepted.
    pub rows: usize,
}

impl SavingsTotal {
    /// Whether the fold found nothing to display.
    ///
    /// Why: the statusline segment is omitted entirely on this condition — a
    /// rendered `$0.00` is indistinguishable from "no savings" and states a
    /// measurement that was never made.
    /// What: true when no row was accepted, or when the accepted rows sum to
    /// zero on both axes.
    /// Test: `zero_fold_is_zero`, `savings_segment_is_absent_on_a_zero_fold`.
    pub fn is_zero(&self) -> bool {
        self.rows == 0 || (self.tokens_saved == 0 && self.cost_saved_usd <= 0.0)
    }

    /// Whole-number percent of tokens the harness avoided sending, as a share
    /// of the session (owner ruling 2026-09-08, #7179).
    ///
    /// Why: a caller renders one percentage — how much of what this session
    /// actually spent plus what it avoided did the harness avoid. (#8063: the
    /// `💸` segment now calls this once per ROW with `None`; `tm
    /// commit-trailers` is what still asks for the session share.)
    /// `session_actual_tokens` is the caller-supplied cumulative token
    /// count from `crate::commands::statusline::compaction` (see the module
    /// header for why that counter, rather than a raw `total_input_tokens`
    /// read, is what survives an auto-compaction). This method has no
    /// filesystem access of its own — it is a pure function of the fold plus
    /// whatever the caller already read — which is what keeps it unit-testable
    /// against hand-built totals with no I/O.
    /// What: primary path, `session_actual_tokens = Some(actual)`:
    /// `round(100 * tokens_saved / (actual + tokens_saved))`. Fallback path,
    /// `None` — no compaction tick has landed yet for this session:
    /// `round(100 * tokens_saved / tokens_before)`, the pre-#7179 formula.
    /// This fallback is never distinguished in the rendered string — both
    /// paths produce the identical `💸<N>%` shape; only this doc comment
    /// records which formula ran. Either path clamps to `[1, 100]` whenever
    /// the fold accepted at least one row and the denominator is nonzero — the
    /// lower clamp mirrors the pre-#7179 dollar segment's "never render a
    /// false zero" rule: a sub-0.5% ratio would otherwise round down to a
    /// literal `0%`, which reads as "we saved nothing" and states a
    /// measurement that was never made. The upper clamp covers a mixed
    /// old/new ledger on the fallback path, where rows written before
    /// `tokens_before` existed fold their share of that denominator as `0`
    /// (see the module header) and can otherwise push the raw ratio above 100.
    /// `None` when there is nothing to divide: [`Self::is_zero`], or (on the
    /// fallback path only) a `tokens_before` sum of `0` — every accepted row
    /// predates #7179 and no compaction tick has landed either.
    /// Test: `percent_saved_uses_the_session_actual_denominator_when_given`,
    /// `percent_saved_falls_back_to_tokens_before_when_actual_is_unknown`,
    /// `percent_saved_rounds_to_nearest_whole_number`,
    /// `percent_saved_is_none_on_a_zero_fold`,
    /// `percent_saved_clamps_a_legacy_mixed_fold`,
    /// `percent_saved_never_rounds_down_to_zero`.
    pub fn percent_saved(&self, session_actual_tokens: Option<u64>) -> Option<u32> {
        if self.is_zero() {
            return None;
        }
        let ratio = match session_actual_tokens {
            Some(actual) => {
                let denominator = actual + self.tokens_saved;
                if denominator == 0 {
                    return None;
                }
                self.tokens_saved as f64 / denominator as f64
            }
            None => {
                if self.tokens_before == 0 {
                    return None;
                }
                self.tokens_saved as f64 / self.tokens_before as f64
            }
        };
        let pct = (ratio * 100.0).round();
        // A true zero is already excluded by the checks above, so any ratio
        // reaching here is a real (if tiny) measurement — round it up to the
        // smallest displayable percent rather than down to a false zero.
        let pct = if pct < 1.0 { 1.0 } else { pct };
        Some(pct.clamp(1.0, 100.0) as u32)
    }
}

/// The savings ledger under an explicit framework root.
///
/// Why: the ledger sits in `<root>/usage/` beside where the daemon usage
/// accounting of #6873 will put `usage.redb`, so the two halves of "what did
/// this session cost and what did it avoid" live in one directory. Taking the
/// root as an argument is what lets a test — and the operator's
/// `--root`/`TRUSTY_MPM_ROOT` override — redirect it without touching `$HOME`.
/// What: `<root>/usage/savings.jsonl`.
/// Test: `savings_log_in_nests_under_usage`.
pub fn savings_log_in(root: &Path) -> PathBuf {
    root.join("usage").join("savings.jsonl")
}

/// The savings ledger under the default framework root (`~/.trusty-mpm`).
///
/// Why: producers run inside the library, where the `--root`/`TRUSTY_MPM_ROOT`
/// override the `tm` binary resolves is not in scope; they use the same
/// home-relative root every other `FrameworkPaths` consumer does.
/// What: [`savings_log_in`] against `FrameworkPaths::default().root`.
/// Test: `default_savings_log_is_under_the_framework_root`.
pub fn default_savings_log() -> PathBuf {
    savings_log_in(&crate::core::paths::FrameworkPaths::default().root)
}

/// Append one row to the ledger, creating the `usage/` directory if absent.
///
/// Why: one writer means the on-disk shape cannot drift between producers, and
/// an `O_APPEND` write of a single line is atomic enough for this file's size
/// that no cross-process lock is needed — two producers racing interleave rows,
/// never bytes within a row. That second half only holds because the row and
/// its newline leave here as ONE write (#7579); see [`write_row_line`].
/// What: serialises `row` to one line of JSON and appends it with a trailing
/// newline. Returns the IO error unchanged; every producer treats a failure as
/// non-fatal, since a missing savings row must never cost a session its launch.
/// Test: `append_then_fold_round_trips`, `append_creates_the_usage_directory`.
pub fn append_row(ledger: &Path, row: &SavingsRow) -> std::io::Result<()> {
    crate::core::home_write_fence::check(ledger); // #8545
    if let Some(parent) = ledger.parent() {
        std::fs::create_dir_all(parent)?;
    }
    let mut file = std::fs::OpenOptions::new()
        .create(true)
        .append(true)
        .open(ledger)?;
    write_row_line(&mut file, row)
}

/// Whether the ledger already carries a given measurement — or could not be
/// asked (#7658).
///
/// Why: a two-valued answer forces an unreadable ledger to be reported as one
/// of "present" or "absent", and both are wrong in a way that costs something.
/// Reporting it absent is the FAIL-OPEN: every producer then appends, and an
/// unreadable ledger becomes an unbounded one — which is the failure this whole
/// change exists to stop. Reporting it present would silently discard a genuine
/// measurement. The third value lets the caller do neither: skip the write and
/// say so in the log.
/// What: `Unreadable` covers every read error EXCEPT `NotFound`, because a
/// ledger that does not exist yet is the ordinary state on a machine no
/// producer has run on, not a fault.
/// Test: `row_presence_distinguishes_a_different_basis`,
/// `append_row_once_skips_an_unreadable_ledger`,
/// `row_presence_of_a_missing_ledger_is_absent`.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum RowPresence {
    /// The ledger holds a row with this `(session_id, technique, basis)`.
    Present,
    /// The ledger was read and holds no such row.
    Absent,
    /// The ledger could not be read, so the question was not answered.
    Unreadable,
}

/// Does the ledger already carry this exact measurement?
///
/// Why (#7658): [`has_row`] answers "does this session have a row of this
/// technique", which is the right question for the re-derivation guard and the
/// wrong one for an append — a session whose prompt genuinely changed mid-run
/// owes a second, DIFFERENT row. Keying on `basis` as well is what separates
/// "the same measurement again" from "a new measurement", so the guard
/// suppresses only the former.
/// What: scans every parseable row, whatever its values. Deliberately NOT
/// [`for_each_accepted_row`]: a row the fold rejects is still on the file, and
/// treating it as absent would let a producer re-append it on every invocation
/// forever.
/// Test: `row_presence_distinguishes_a_different_basis`,
/// `row_presence_finds_a_row_the_fold_rejects`,
/// `append_row_once_skips_an_unreadable_ledger`.
pub fn row_presence(ledger: &Path, session_id: &str, technique: &str, basis: &str) -> RowPresence {
    let text = match std::fs::read_to_string(ledger) {
        Ok(text) => text,
        Err(source) if source.kind() == std::io::ErrorKind::NotFound => return RowPresence::Absent,
        Err(source) => {
            tracing::warn!(
                ledger = %ledger.display(),
                %source,
                "could not read the savings ledger to check for an existing row"
            );
            return RowPresence::Unreadable;
        }
    };
    let found = text
        .lines()
        .filter(|line| !line.trim().is_empty())
        .filter_map(|line| serde_json::from_str::<SavingsRow>(line).ok())
        .any(|row| {
            row.session_id == session_id && row.technique == technique && row.basis == basis
        });
    if found {
        RowPresence::Present
    } else {
        RowPresence::Absent
    }
}

/// What [`append_row_once`] did.
///
/// Test: `append_row_once_writes_one_row_for_n_attempts`.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum AppendOnce {
    /// The row was not on the ledger and is now.
    Appended,
    /// An identical measurement was already recorded; nothing was written.
    AlreadyPresent,
    /// The ledger could not be read, so nothing was written (#7658).
    LedgerUnreadable,
}

/// Append `row` unless the ledger already carries the same measurement.
///
/// Why (#7658): THE choke point. The ledger grew 312 byte-identical
/// `instruction-compression` rows for one live session because both producers
/// that write this technique — the compose-time append in
/// [`crate::core::savings_instructions`] and the sweep's append in
/// [`crate::core::savings_sidecar`] — called [`append_row`] without ever
/// consulting the file. Guarding each producer separately would leave the next
/// producer free to re-open the hole, so the check lives HERE, below every one
/// of them: a caller that reaches the ledger at all reaches it through this.
/// What: [`row_presence`] on `(session_id, technique, basis)`, then
/// [`append_row`] only for [`RowPresence::Absent`] — the pair held under
/// `trusty_common::file_lock::with_exclusive_lock` on the ledger, because a
/// read-then-append that is not atomic is the SAME duplicate class bounded by
/// the number of racers: two hooks in separate processes both observe `Absent`
/// and both append. That lock is the workspace's one advisory-locking primitive
/// (#5344) and it serialises threads and processes alike, so no second
/// implementation is introduced here. An unreadable ledger skips the write and
/// reports it — never treated as "no row exists", which would make an
/// unreadable ledger append on every invocation.
///
/// Only the instruction-compression producers route through this. `divert` and
/// `compress` keep [`append_row`]: a second diversion of the same file is a
/// second real saving, so suppressing it would delete data, and neither pays the
/// lock.
///
/// NOT reentrant, inheriting `with_exclusive_lock`'s contract — never call this
/// from inside another lock on the same ledger.
/// Test: `append_row_once_writes_one_row_for_n_attempts`,
/// `racing_threads_append_exactly_one_row`,
/// `append_row_once_skips_an_unreadable_ledger`,
/// `append_row_once_still_records_a_different_basis`.
pub fn append_row_once(ledger: &Path, row: &SavingsRow) -> std::io::Result<AppendOnce> {
    crate::core::home_write_fence::check(ledger); // #8545
    // #7658: the ledger's parent must exist before the lock sidecar can be
    // created beside it. `append_row` creates it too, but that is inside the
    // critical section and too late for the lock file itself.
    if let Some(parent) = ledger.parent() {
        std::fs::create_dir_all(parent)?;
    }
    trusty_common::file_lock::with_exclusive_lock(ledger, || append_row_once_locked(ledger, row))?
}

/// [`append_row_once`]'s critical section, with the lock already held.
///
/// Why: separating it keeps the lock acquisition one line and makes the
/// read-then-append pair readable as the single indivisible step it has to be.
/// What: see [`append_row_once`].
/// Test: `append_row_once_writes_one_row_for_n_attempts`,
/// `racing_threads_append_exactly_one_row`.
fn append_row_once_locked(ledger: &Path, row: &SavingsRow) -> std::io::Result<AppendOnce> {
    match row_presence(ledger, &row.session_id, &row.technique, &row.basis) {
        RowPresence::Present => {
            tracing::debug!(
                ledger = %ledger.display(),
                session_id = %row.session_id,
                technique = %row.technique,
                "the ledger already carries this measurement; appending nothing"
            );
            Ok(AppendOnce::AlreadyPresent)
        }
        RowPresence::Unreadable => {
            tracing::warn!(
                ledger = %ledger.display(),
                session_id = %row.session_id,
                technique = %row.technique,
                "the savings ledger could not be read, so whether this row is already \
                 recorded is unknown; skipping the append rather than risking a duplicate"
            );
            Ok(AppendOnce::LedgerUnreadable)
        }
        RowPresence::Absent => {
            append_row(ledger, row)?;
            Ok(AppendOnce::Appended)
        }
    }
}

/// Write one row and its terminator to `sink` in a single `write_all`.
///
/// Why (#7579): `writeln!` expands to `write_fmt`, which hands the formatter's
/// pieces to the sink one at a time — the row's bytes, then the newline, as two
/// separate `O_APPEND` writes. Two producers racing between those two writes
/// interleave BYTES, not rows: the operator's ledger carries ten physical lines
/// holding two complete rows with no separator, each followed by the blank line
/// the two stranded newlines left. Building the whole line in memory first is
/// what makes the module header's "never bytes within a row" claim true, since
/// a single `write_all` to an `O_APPEND` descriptor is not split.
/// What: serialises `row`, appends `\n` to the same buffer, and issues one
/// `write_all`. A serialisation failure becomes an `InvalidData` IO error so
/// callers keep one error type.
/// Test: `a_row_and_its_newline_leave_in_one_write`,
/// `append_then_fold_round_trips`.
pub fn write_row_line(sink: &mut impl std::io::Write, row: &SavingsRow) -> std::io::Result<()> {
    let mut line = serde_json::to_string(row)
        .map_err(|source| std::io::Error::new(std::io::ErrorKind::InvalidData, source))?;
    line.push('\n');
    sink.write_all(line.as_bytes())
}

/// Fold every accepted row belonging to `session_id`.
///
/// Why: the statusline renders one session's figure, and a machine's ledger
/// carries every session's rows. Filtering at read time — rather than keeping a
/// file per session — is what keeps the writer a bare append.
/// What: [`fold`] with a session filter. A missing or unreadable ledger folds
/// to zero.
/// Test: `fold_ignores_other_sessions`, `fold_of_a_missing_ledger_is_zero`.
pub fn fold_session(ledger: &Path, session_id: &str) -> SavingsTotal {
    fold(ledger, Some(session_id))
}

/// Fold `session_id`'s per-call tool-output compression rows only (#7867).
///
/// Why: the `💸` segment's figure is per-call tool-output compression, which
/// [`fold_session`] cannot answer — it sums every technique, so an
/// instruction-fold row written once at launch moved a figure that is supposed
/// to track what tool calls saved. Filtering here, rather than dropping rows
/// from the ledger, keeps the instruction-fold measurement recorded and
/// readable by its own surfaces.
/// What: [`fold_session`]'s walk, accumulating only rows whose technique passes
/// [`is_per_call_technique`]. A missing ledger folds to zero; an unreadable one
/// warns (see [`for_each_accepted_row`]) and folds to zero.
/// Test: `fold_session_per_call_ignores_an_instruction_compression_row`,
/// `fold_session_per_call_sums_compress_and_divert`.
pub fn fold_session_per_call(ledger: &Path, session_id: &str) -> SavingsTotal {
    let mut total = SavingsTotal::default();
    for_each_accepted_row(ledger, Some(session_id), |row| {
        if is_per_call_technique(&row.technique) {
            accumulate(&mut total, row);
        }
    });
    total
}

/// Does the ledger already hold an accepted `technique` row for `session_id`?
///
/// Why (#7411): the hook re-derives an instruction-compression row from the
/// compiled prompt when nothing was staged for the session, and a Claude
/// session raises `SessionStart` more than once — a resume and a compact each
/// raise it again. Without a "this session already has one" read, every one of
/// those would append a second copy of the same measurement and double the
/// `💸` figure. [`fold_session`] cannot answer it: it sums across every
/// technique, so a session that had only a `divert` row would read as already
/// covered.
/// What: scans the accepted rows for `session_id` and reports whether any
/// carries `technique`. A missing or unreadable ledger reads as `false`.
/// Test: `has_row_sees_only_the_named_technique`.
pub fn has_row(ledger: &Path, session_id: &str, technique: &str) -> bool {
    let mut found = false;
    for_each_accepted_row(ledger, Some(session_id), |row| {
        found |= row.technique == technique;
    });
    found
}

/// Fold every accepted row, whatever session wrote it.
///
/// Why: a machine-wide figure is what a future `tm usage`/console surface wants,
/// and deriving it from the same reader is what stops the two surfaces
/// disagreeing about a total.
/// What: [`fold`] with no filter.
/// Test: `fold_all_sums_every_session`.
pub fn fold_all(ledger: &Path) -> SavingsTotal {
    fold(ledger, None)
}

/// Fold every accepted row, grouped by the session that wrote it (#7074).
///
/// Why: the `💸` segment's average is the mean of each session's OWN percentage
/// — percentages do not sum into a lifetime total but do average cleanly (owner
/// ruling 2026-09-09, option A). That needs one total per session, which no
/// existing reader produced. Deriving it from the same row-acceptance walk as
/// [`fold`] is what keeps the average and the per-session figure from
/// disagreeing about which rows count.
/// What: [`for_each_accepted_row`] with no filter, accumulating into one
/// [`SavingsTotal`] per `session_id`. A missing or unreadable ledger yields an
/// empty map — never a map of zeros.
/// Test: `fold_sessions_groups_by_session`,
/// `fold_sessions_of_a_missing_ledger_is_empty`.
pub fn fold_sessions(ledger: &Path) -> std::collections::BTreeMap<String, SavingsTotal> {
    let mut by_session: std::collections::BTreeMap<String, SavingsTotal> =
        std::collections::BTreeMap::new();
    for_each_accepted_row(ledger, None, |row| {
        accumulate(by_session.entry(row.session_id.clone()).or_default(), row);
    });
    by_session
}

/// [`fold_sessions`] over the per-call techniques only (#7867).
///
/// Why: the average beside the session's own figure must be the mean of the
/// same measurement, or the two halves of one segment would report different
/// things — an instruction-fold row in another session would raise the average
/// a per-call figure is compared against.
/// What: [`fold_sessions`]'s walk with [`is_per_call_technique`] applied. A
/// session whose every row is filtered out contributes no entry at all, rather
/// than an entry of zeros that would drag the mean down.
/// Test: `fold_sessions_per_call_drops_a_session_with_only_instruction_rows`.
pub fn fold_sessions_per_call(ledger: &Path) -> std::collections::BTreeMap<String, SavingsTotal> {
    let mut by_session: std::collections::BTreeMap<String, SavingsTotal> =
        std::collections::BTreeMap::new();
    for_each_accepted_row(ledger, None, |row| {
        if is_per_call_technique(&row.technique) {
            accumulate(by_session.entry(row.session_id.clone()).or_default(), row);
        }
    });
    by_session
}

/// The per-call fold of the whole ledger, with the last row's timestamp
/// (#7867).
///
/// Why: `tm doctor`'s `tool_output_compression` check reports whether anything
/// has been compressed yet AND when, and an unreadable ledger must reach it as
/// an error rather than as the same zero an empty ledger produces — a silent
/// zero there would state that nothing was compressed when nothing was read.
/// [`fold_all`] cannot carry either fact.
/// What: reads the ledger once; `Ok` with a zero total and no timestamp when
/// the file does not exist, `Err` carrying the IO error's text when it exists
/// and cannot be read, otherwise the [`is_per_call_technique`] fold plus the
/// `ts` of the last accepted per-call row in file order.
/// Test: `try_fold_per_call_reports_the_last_timestamp`,
/// `try_fold_per_call_errors_on_an_unreadable_ledger`,
/// `try_fold_per_call_of_a_missing_ledger_is_an_empty_ok`.
pub fn try_fold_per_call(ledger: &Path) -> Result<PerCallFold, String> {
    let text = match read_ledger_text(ledger) {
        Ok(Some(text)) => text,
        Ok(None) => return Ok(PerCallFold::default()),
        Err(source) => return Err(source.to_string()),
    };
    let mut fold = PerCallFold::default();
    visit_accepted_rows(ledger, &text, None, &mut |row: &SavingsRow| {
        if is_per_call_technique(&row.technique) {
            accumulate(&mut fold.total, row);
            fold.last_ts = Some(row.ts.clone());
        }
    });
    Ok(fold)
}

/// What [`try_fold_per_call`] read back.
///
/// Why: the doctor check states both "how much" and "how recently", and a
/// rows-only total cannot answer the second.
/// What: the folded total, and the `ts` of the last accepted per-call row —
/// `None` when the fold accepted none.
/// Test: `try_fold_per_call_reports_the_last_timestamp`.
#[derive(Debug, Clone, Default, PartialEq)]
pub struct PerCallFold {
    /// The per-call rows' folded total.
    pub total: SavingsTotal,
    /// The last accepted per-call row's RFC 3339 timestamp.
    pub last_ts: Option<String>,
}

/// Add one accepted row into `total`.
///
/// Why: four folds share these four sums; a fifth that added them by hand could
/// count a field differently.
/// Test: `append_then_fold_round_trips`.
fn accumulate(total: &mut SavingsTotal, row: &SavingsRow) {
    total.tokens_saved += row.tokens_saved as u64;
    total.tokens_before += row.tokens_before;
    total.cost_saved_usd += row.cost_saved_usd;
    total.rows += 1;
}

/// The mean of every session's own savings percentage (#7074).
///
/// Why: the owner ruled the statusline shows a per-session AVERAGE beside the
/// current session's figure (2026-09-09, option A). Averaging the PERCENTAGES —
/// rather than folding all rows into one ratio — is what makes a short session
/// that saved 60 % count as much as a long one that saved 10 %, which is the
/// question "how much are we saving per session" actually asks.
/// What: computes [`SavingsTotal::percent_saved`] for each session in
/// `by_session`, passing that session's own actual-token count from
/// `session_actual_tokens` (the caller supplies the lookup, so this function
/// does no I/O and is testable against a fixture map), and returns the
/// arithmetic mean rounded to a whole percent. Sessions whose percent is `None`
/// — a zero fold, or no denominator on either path — contribute nothing, so a
/// ledger holding only such sessions yields `None` rather than a false `0%`.
/// Test: `average_percent_is_the_mean_of_each_sessions_percent`,
/// `average_of_one_session_is_that_sessions_percent`,
/// `average_of_sessions_with_no_denominator_is_none`,
/// `average_skips_corrupt_rows_and_never_reports_a_false_zero`.
// #8063: the `💸` segment no longer folds this — its right-hand figure is now
// the mean of THIS session's rows ([`session_row_percents`]). This cross-session
// mean stays the ledger's answer to "how much does the harness save in general",
// which the per-session reporting surfaces read.
pub fn average_percent_saved(
    by_session: &std::collections::BTreeMap<String, SavingsTotal>,
    session_actual_tokens: impl Fn(&str) -> Option<u64>,
) -> Option<u32> {
    let percents: Vec<u32> = by_session
        .iter()
        .filter_map(|(session_id, total)| total.percent_saved(session_actual_tokens(session_id)))
        .collect();
    mean_percent(&percents)
}

/// Each accepted per-call row's OWN percent for `session_id`, in ledger order
/// (#8063).
///
/// Why: owner ruling 2026-09-15 — the `💸` segment shows the latest row's
/// figure beside the mean of this session's rows. A whole-session share turned
/// an 18.9 % per-call reduction into a 1 % badge, so the segment read as "the
/// harness saves nothing" while the ledger said otherwise (#8063). Neither
/// figure exists in a session-level fold: both need one percent PER ROW, which
/// no other reader here produces.
/// What: [`for_each_accepted_row`] with a session filter — the same acceptance
/// rules every other fold applies — keeping the [`is_per_call_technique`] rows
/// (#7867) and pricing each one alone through
/// [`SavingsTotal::percent_saved`] with no session denominator, so the figure
/// is the row's own `tokens_saved / tokens_before` reduction. A row that
/// predates `tokens_before` has no denominator and contributes nothing rather
/// than a fabricated percent. Order is the ledger's append order, which is also
/// timestamp order (every producer stamps with [`now_ts`]), so the last element
/// is the newest row. A missing or unreadable ledger yields an empty vector.
/// Test: `session_row_percents_are_one_per_row_in_ledger_order`,
/// `session_row_percents_skip_a_row_with_no_before_figure`,
/// `session_row_percents_ignore_other_sessions_and_instruction_rows`.
pub fn session_row_percents(ledger: &Path, session_id: &str) -> Vec<u32> {
    let mut percents = Vec::new();
    for_each_accepted_row(ledger, Some(session_id), |row| {
        if !is_per_call_technique(&row.technique) {
            return;
        }
        let mut one_row = SavingsTotal::default();
        accumulate(&mut one_row, row);
        if let Some(percent) = one_row.percent_saved(None) {
            percents.push(percent);
        }
    });
    percents
}

/// The arithmetic mean of whole percents, rounded and clamped to `[1, 100]`.
///
/// Why: two averages share this rule — the cross-session one (#7074) and the
/// per-row one the `💸` segment renders (#8063) — and a second hand-rolled mean
/// could round or clamp differently, so the same rows would read as two
/// different numbers on two surfaces.
/// What: `None` for an empty slice, never a `0%`: an average is only ever shown
/// beside a figure that already exists, so a false zero there would claim a
/// measurement that was not made.
/// Test: `mean_percent_of_nothing_is_none`,
/// `average_percent_is_the_mean_of_each_sessions_percent`,
/// `savings_segment_renders_the_average_beside_the_latest_row`.
pub fn mean_percent(percents: &[u32]) -> Option<u32> {
    if percents.is_empty() {
        return None;
    }
    let sum: u64 = percents.iter().map(|p| u64::from(*p)).sum();
    let mean = sum as f64 / percents.len() as f64;
    Some(mean.round().clamp(1.0, 100.0) as u32)
}

/// The one fold both readers share.
///
/// Why: the skip rules — unparseable line, non-positive tokens, non-positive or
/// non-finite cost — must be identical for every consumer, or a per-session
/// figure and a machine-wide figure computed from the same file could disagree
/// about which rows count.
/// What: [`for_each_accepted_row`] with the caller's filter, summed.
/// Test: `fold_skips_a_malformed_line_and_keeps_the_valid_total`,
/// `a_negative_row_cannot_raise_the_total`,
/// `fold_skips_a_row_whose_cost_is_not_a_number`.
fn fold(ledger: &Path, session_id: Option<&str>) -> SavingsTotal {
    let mut total = SavingsTotal::default();
    for_each_accepted_row(ledger, session_id, |row| accumulate(&mut total, row));
    total
}

/// Walk the ledger once, handing `visit` every row that passes the skip rules.
///
/// Why (#7074): [`fold`] and [`fold_sessions`] must accept and reject exactly
/// the same rows, or the current session's figure and the average beside it
/// could be computed from different subsets of one file. One walk, one set of
/// rules, two accumulators.
/// What: reads the whole file (it is one short line per saving event), parses
/// each non-blank line, applies the optional session filter, and calls `visit`
/// for what survives. Each rejection emits one `warn!` naming the reason. A
/// missing or unreadable ledger visits nothing.
/// Test: `fold_skips_a_malformed_line_and_keeps_the_valid_total`,
/// `a_negative_row_cannot_raise_the_total`,
/// `fold_skips_a_row_whose_cost_is_not_a_number`,
/// `fold_sessions_groups_by_session`.
fn for_each_accepted_row(
    ledger: &Path,
    session_id: Option<&str>,
    mut visit: impl FnMut(&SavingsRow),
) {
    match read_ledger_text(ledger) {
        Ok(Some(text)) => visit_accepted_rows(ledger, &text, session_id, &mut visit),
        // Absent is the ordinary state before any producer has run; it is not a
        // fault and must not be logged as one.
        Ok(None) => {}
        // #7867: an unreadable ledger is NOT an empty one. The fold still
        // yields zero — the statusline is on a hot render path and has nothing
        // better to show — but the reason reaches the log rather than being
        // swallowed into a figure that claims nothing was saved.
        Err(source) => tracing::warn!(
            ledger = %ledger.display(),
            %source,
            "the savings ledger exists but could not be read; folding it as zero"
        ),
    }
}

/// Read the ledger, distinguishing "not there yet" from "cannot be read".
///
/// Why (#7867, Fail-Open Check): every caller treats an absent ledger as an
/// ordinary zero, and no caller may treat a permission or IO failure as the
/// same thing.
/// What: `Ok(None)` for `NotFound`, `Ok(Some(text))` for a readable file, `Err`
/// otherwise.
/// Test: `try_fold_per_call_errors_on_an_unreadable_ledger`,
/// `try_fold_per_call_of_a_missing_ledger_is_an_empty_ok`.
fn read_ledger_text(ledger: &Path) -> std::io::Result<Option<String>> {
    match std::fs::read_to_string(ledger) {
        Ok(text) => Ok(Some(text)),
        Err(source) if source.kind() == std::io::ErrorKind::NotFound => Ok(None),
        Err(source) => Err(source),
    }
}

/// Apply the skip rules to already-read ledger text.
///
/// Why (#7867): [`try_fold_per_call`] needs the read and the walk separated so
/// it can report a read failure, and both must still accept exactly the rows
/// [`fold`] accepts.
/// What: parses each non-blank line, applies the optional session filter and the
/// skip rules, and calls `visit` for what survives. Each rejection emits one
/// `warn!` naming the reason.
/// Test: `fold_skips_a_malformed_line_and_keeps_the_valid_total`,
/// `a_negative_row_cannot_raise_the_total`.
fn visit_accepted_rows(
    ledger: &Path,
    text: &str,
    session_id: Option<&str>,
    visit: &mut impl FnMut(&SavingsRow),
) {
    for (index, line) in text.lines().enumerate() {
        if line.trim().is_empty() {
            continue;
        }
        let row: SavingsRow = match serde_json::from_str(line) {
            Ok(row) => row,
            Err(source) => {
                tracing::warn!(
                    ledger = %ledger.display(),
                    line = index + 1,
                    %source,
                    "skipping a malformed savings row"
                );
                continue;
            }
        };
        if session_id.is_some_and(|wanted| row.session_id != wanted) {
            continue;
        }
        if row.tokens_saved <= 0 {
            tracing::warn!(
                ledger = %ledger.display(),
                line = index + 1,
                technique = %row.technique,
                tokens_saved = row.tokens_saved,
                "skipping a savings row with non-positive tokens_saved"
            );
            continue;
        }
        if !row.cost_saved_usd.is_finite() || row.cost_saved_usd <= 0.0 {
            tracing::warn!(
                ledger = %ledger.display(),
                line = index + 1,
                technique = %row.technique,
                cost_saved_usd = row.cost_saved_usd,
                "skipping a savings row with non-positive cost_saved_usd"
            );
            continue;
        }
        visit(&row);
    }
}

/// The current instant as an RFC 3339 UTC timestamp, for a row's `ts`.
///
/// Why: every producer stamps its row the same way, so the ledger sorts
/// chronologically by plain string comparison.
/// What: `chrono::Utc::now()` in RFC 3339 form with second precision.
/// Test: `now_ts_is_rfc3339`.
pub fn now_ts() -> String {
    chrono::Utc::now()
        .to_rfc3339_opts(chrono::SecondsFormat::Secs, true)
        .to_string()
}

#[cfg(test)]
#[path = "savings_tests.rs"]
mod tests;

#[cfg(test)]
#[path = "savings_average_tests.rs"]
mod average_tests;