Skip to main content

mecha_core/
step.rs

1//! What a finished step actually did — the deterministic half of step
2//! appraisal (`docs/GOAL-SYSTEM-DESIGN.md` §5.5).
3//!
4//! **A step is marked done by the agent, and nothing checked it.** The
5//! symmetry with the tier above is the whole argument: a *board task* is
6//! closed by the owner (`TASK-AGENT-DESIGN.md` D6), so a person is the check;
7//! a *todo step* is closed by the model, so there is no person and the check
8//! has to be structural. D5's rule — state is derived from the record, never
9//! self-reported — reaches one tier further down than it was written for.
10//!
11//! **Pure, and unit-tested rather than trialled**, for `compact.rs`'s reason:
12//! getting it wrong is silent. A finding that fires on honest work is a line
13//! the model learns to skip, which is how a check that protects nothing
14//! survives; a finding that never fires is indistinguishable from a plan that
15//! always lands.
16//!
17//! Two readings only, and the omissions are deliberate. §5.5's table lists
18//! five signals; the two here — *no calls at all* and *the last call did not
19//! succeed* — are **facts about the span**. The other three (a span far longer
20//! than its siblings, a verify-shaped call that passed, the same target read
21//! repeatedly) are *comparisons*, and each needs either a threshold nobody has
22//! measured here or a guess about what a tool call meant. A threshold that
23//! cries wolf is doctor's named failure, and the escalation to a model that
24//! would settle the ambiguous cases is rung 7's, not this one's. The
25//! same-target reading is boredom's (§9.1) and belongs one mechanism over.
26//!
27//! What this module does **not** do is act. The finding is rendered onto the
28//! `todo` result and the plan action — accept, revise the step, revise the
29//! plan, escalate — is the model's, because the plan is the model's. The
30//! harness has no way to author a decomposition and no business having one.
31
32use crate::agent::ToolCallTrace;
33use anyhow::{Context, Result};
34use serde::Deserialize;
35
36/// How one executed call ended, as far as the run's own record knows.
37///
38/// **`Refused` is not `Failed`, and the split is load-bearing.** A denied
39/// trace carries `is_error: true` *as well as* `denied: true`, so any counter
40/// spelled `is_error` alone reports the approver doing its job as the step
41/// going wrong — the same miscount the eval rig names on
42/// `ended_on_failed_call`, which is why the fold below spells it
43/// `unknown || (is_error && !denied)`. A step whose calls were refused was
44/// *blocked*; a step whose calls failed did not land. Telling the model the
45/// first is the second would send it to fix code that is working.
46#[derive(Debug, Clone, Copy, PartialEq, Eq)]
47pub enum Outcome {
48    Ok,
49    Failed,
50    Refused,
51}
52
53impl Outcome {
54    fn of(call: &ToolCallTrace) -> Self {
55        if call.denied {
56            // Includes a call withheld by policy: nothing ran, and nobody
57            // claimed it did.
58            Outcome::Refused
59        } else if call.unknown || call.is_error {
60            // A named tool that does not exist is the model's mistake rather
61            // than the environment's refusal, and it counts with the failures
62            // for the same reason the eval rig counts it there.
63            Outcome::Failed
64        } else {
65            // A staged call is a success: the draft was written, and the send
66            // is waiting on a person rather than having gone wrong.
67            Outcome::Ok
68        }
69    }
70}
71
72/// The run's work so far, as of one tool call.
73///
74/// Cumulative, and only ever read as a **difference** between two points — the
75/// turn a step went `in_progress` and the turn it was marked done. That is why
76/// it is a handful of integers rather than a list: the span is arithmetic, and
77/// keeping the calls themselves would make this a second copy of the trace.
78///
79/// Stamped on [`ToolCtx`](crate::tool::ToolCtx) by the loop, which does not
80/// know which tool cares — the `taint` and `call_id` precedent.
81#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]
82pub struct Work {
83    /// Every attempt, refusals included. A step that tried three times and was
84    /// refused three times is blocked, not empty, and counting only what ran
85    /// would report it as having done nothing.
86    pub calls: u32,
87    pub failed: u32,
88    pub refused: u32,
89    /// Successful calls that look like they verified something — a `shell`
90    /// call whose command matches a small test-runner-shaped keyword list.
91    /// See [`looks_like_verification`]. Folded here rather than kept as a
92    /// raw trace for the same reason `calls`/`failed`/`refused` are: the span
93    /// is arithmetic, and a tool asking "was there a check in this span"
94    /// needs a count, not the calls themselves.
95    pub verify_like: u32,
96    /// Every *successful* `shell` call, matched or not — the denominator
97    /// `verify_like` needs to mean anything, so it has to count the same
98    /// population `verify_like` draws from (`Outcome::Ok` only). A refused
99    /// or failed `shell` call never ran, and counting it here would reopen
100    /// exactly the false positive this field exists to close: on a
101    /// read-only run (`shell` denied) or a surface where `shell` is not
102    /// registered at all, that is precisely the shape a step's one attempt
103    /// takes. `looks_like_verification` can only recognise a check shaped as
104    /// `shell`, so `verify_like == 0` is ambiguous on its own: it is true
105    /// both when a step's checks used some other tool (an MCP test runner,
106    /// `cargo check`-via-a-non-`shell` wrapper) and on any surface where
107    /// `shell` is not even registered (a mail-only trigger, a
108    /// `tools:`-narrowed skill, a read-only run) — cases where nothing could
109    /// have set the counter regardless of what actually happened. See
110    /// [`escalation_candidate`]'s `UnverifiedClaim` branch, which reads this
111    /// alongside `verify_like` for exactly that reason.
112    pub shell_calls: u32,
113    /// How the most recent attempt ended. `None` before the run makes one.
114    pub last: Option<Outcome>,
115    /// Calls approved in *this* turn whose results are not back yet — the
116    /// siblings of the call reading this.
117    ///
118    /// mecha executes a turn's calls concurrently, so a model that does the
119    /// work and ticks the box in one batch has that work invisible to the fold
120    /// below. Without this the commonest efficient shape in the corpus would
121    /// report as the null step, which is the false positive that would teach
122    /// people to ignore the reading.
123    pub in_flight: u32,
124    /// Calls settled *this turn* without becoming approved work, whose
125    /// target step is unknown to the harness. Named for the commonest case
126    /// (the approver, a hook, the interlock) but not only that: an unknown
127    /// or withheld tool name and a failed staging attempt settle the same
128    /// way, without ever being denied by anyone.
129    ///
130    /// Any of these is settled the instant it happens — unlike `in_flight`
131    /// it is already in the trace — but the batch it happened in is exactly
132    /// the shape `in_flight` exists for: a model ticking a step and
133    /// reaching for the next one's tool in the same turn. `trace.push` for
134    /// one of these runs ahead of the calls it approved, so `Work::of` folds
135    /// it in as the raw trace's last entry regardless of which call it sat
136    /// beside — blaming *this* step for an outcome that belongs to the
137    /// next one. Carried alongside `in_flight` for the same reason: a batch
138    /// holding either supports no finding at all.
139    pub denied: u32,
140    /// Which run these counters belong to.
141    ///
142    /// **The trace is per run and a conversation is many runs.** In chat and
143    /// the TUI one submission is one run, so the counters restart at zero
144    /// while the plan carries on — and a step started before the user last
145    /// spoke would difference against a larger number, saturate to zero and
146    /// report as the null step. That is the loudest reading this module has,
147    /// firing on the commonest shape there is, which is how a check gets
148    /// switched off. So a mark from another run is *unmeasurable* rather than
149    /// empty, and [`Work::since`] says so by returning nothing.
150    ///
151    /// Only inequality is ever read, which is what makes a process-local
152    /// counter enough: two runs in one process must differ, and nothing
153    /// compares this across processes or across a restart.
154    pub run: u64,
155}
156
157/// A fresh run identity. Monotonic within the process, meaningless outside it.
158pub fn next_run() -> u64 {
159    static RUNS: std::sync::atomic::AtomicU64 = std::sync::atomic::AtomicU64::new(1);
160    RUNS.fetch_add(1, std::sync::atomic::Ordering::Relaxed)
161}
162
163impl Work {
164    /// Fold the run's trace so far.
165    ///
166    /// Called once per turn, not once per call: the numbers are the same for
167    /// every call in a batch, and the walk is over every call the run has
168    /// made.
169    pub fn of(trace: &[ToolCallTrace]) -> Self {
170        let mut work = Work::default();
171        for call in trace {
172            let outcome = Outcome::of(call);
173            work.calls += 1;
174            match outcome {
175                Outcome::Failed => work.failed += 1,
176                Outcome::Refused => work.refused += 1,
177                Outcome::Ok => {
178                    if call.name == "shell" {
179                        work.shell_calls += 1;
180                    }
181                    if looks_like_verification(call) {
182                        work.verify_like += 1;
183                    }
184                }
185            }
186            work.last = Some(outcome);
187        }
188        work
189    }
190
191    pub fn with_in_flight(mut self, n: u32) -> Self {
192        self.in_flight = n;
193        self
194    }
195
196    pub fn with_denied(mut self, n: u32) -> Self {
197        self.denied = n;
198        self
199    }
200
201    pub fn in_run(mut self, run: u64) -> Self {
202        self.run = run;
203        self
204    }
205
206    /// Work done since `start`, less `bookkeeping` calls the caller knows were
207    /// its own.
208    ///
209    /// The adjustment exists because the only caller is a tool that appears in
210    /// its own count. A model that revises its plan three times mid-step would
211    /// otherwise show three calls of "work" for a step where nothing happened
212    /// — the null step masked by the bookkeeping that announced it. The
213    /// argument for putting the subtraction here rather than in the loop is
214    /// the loop's own invariant: it stamps run state without learning which
215    /// tool reads it, so "the plan tool is not work" is a judgement only the
216    /// plan tool can make, and this is where the arithmetic it needs lives.
217    ///
218    /// **`last` is the caller's to supply, and `self.last` is the wrong
219    /// answer.** `self.last` is the raw trace's most recent entry, which is
220    /// this same bookkeeping tool's own call whenever one lands last — a
221    /// successful revision masks an earlier failure (`EndedOnFailure` never
222    /// fires), and a *rejected* one, which never reaches this method's caller
223    /// at all, reads as the step's own failure (`EndedOnFailure` fires on
224    /// work that landed). `bookkeeping` is a count and cannot say which
225    /// position it occupied, so only a caller tracking its own calls as they
226    /// happen — [`crate::tool::todo::Tracked`] does, incrementally — can name
227    /// the outcome that actually belongs to the span.
228    ///
229    /// `None` when `start` was taken in another run — see [`Work::run`]. An
230    /// unmeasurable span supports no finding, which is doctor's dash one
231    /// mechanism over: could-not-look and nothing-happened are opposite
232    /// answers.
233    pub fn since(&self, start: Work, bookkeeping: u32, last: Option<Outcome>) -> Option<Span> {
234        if start.run != self.run {
235            return None;
236        }
237        Some(Span {
238            calls: self
239                .calls
240                .saturating_sub(start.calls)
241                .saturating_sub(bookkeeping),
242            failed: self.failed.saturating_sub(start.failed),
243            refused: self.refused.saturating_sub(start.refused),
244            verify_like: self.verify_like.saturating_sub(start.verify_like),
245            shell_calls: self.shell_calls.saturating_sub(start.shell_calls),
246            last,
247            in_flight: self.in_flight,
248            denied: self.denied,
249        })
250    }
251}
252
253/// What happened between a step starting and the model calling it done.
254#[derive(Debug, Clone, Copy, PartialEq, Eq)]
255pub struct Span {
256    pub calls: u32,
257    pub failed: u32,
258    pub refused: u32,
259    /// See [`Work::verify_like`]. Read by [`escalation_candidate`], never by
260    /// [`appraise`] — a verify-shaped call is evidence for the escalation to
261    /// weigh, not a fact the deterministic reading changes on.
262    pub verify_like: u32,
263    /// See [`Work::shell_calls`].
264    pub shell_calls: u32,
265    /// The run's most recent finished attempt — which is the *span's* most
266    /// recent one whenever the span holds any, since calls happen in order.
267    /// Meaningless when `calls` is zero, and [`appraise`] reads it only after
268    /// establishing that it is not.
269    pub last: Option<Outcome>,
270    /// Siblings still running. Any of them may be the work, or the recovery,
271    /// so a span holding one supports no finding at all.
272    pub in_flight: u32,
273    /// Siblings denied this turn. See [`Work::denied`] — the denial cannot be
274    /// attributed to this step over any other in the same batch, so a span
275    /// holding one supports no finding either.
276    pub denied: u32,
277}
278
279/// What the span says about the step.
280#[derive(Debug, Clone, Copy, PartialEq, Eq)]
281pub enum Finding {
282    /// The common path, and it says nothing. Silence is the point: this rides
283    /// on a tool result in an append-only transcript, so a line per completed
284    /// step is bulk carried for the rest of the run in exchange for confirming
285    /// what the model already believes.
286    Landed,
287    /// Nothing was attempted — the step-level null run, which `WORK_FLOOR`
288    /// exists to catch one tier up.
289    Null,
290    /// The last thing tried failed, and nothing after it succeeded.
291    EndedOnFailure,
292    /// The last thing tried was refused. The step was blocked, not done.
293    EndedOnRefusal,
294}
295
296/// The deterministic reading. No model, no threshold, no tuned constant.
297pub fn appraise(span: Span) -> Finding {
298    // Unknown beats every other reading. A sibling still in flight can be the
299    // work the span looks empty without, or the recovery after the failure
300    // that ended it — so the honest answer is no finding rather than the
301    // finding the visible half would support. This is the same direction the
302    // taint snapshot takes on uncovered runs: an absence is not evidence.
303    // A sibling denied this turn gets the same treatment: the refusal is
304    // settled, but which step it belongs to is not.
305    if span.in_flight > 0 || span.denied > 0 {
306        return Finding::Landed;
307    }
308    if span.calls == 0 {
309        return Finding::Null;
310    }
311    // Only the *last* attempt counts, which is the eval rig's rule for
312    // `ended_on_failed_call` one tier down: a failure among successes is
313    // recovery, and recovery is the model working.
314    match span.last {
315        Some(Outcome::Failed) => Finding::EndedOnFailure,
316        Some(Outcome::Refused) => Finding::EndedOnRefusal,
317        _ => Finding::Landed,
318    }
319}
320
321impl Finding {
322    /// One line for the `todo` result, or nothing at all.
323    ///
324    /// **Wording is load-bearing**, on `EMPTY_TURN_NUDGE`'s and `ask_user`'s
325    /// evidence: a vague nudge makes a model restart work it had done, and
326    /// "use your best judgment" measurably makes it invent. So each line
327    /// states the fact first and offers exactly one continuation — and the
328    /// null line offers the model the *reading* rather than the verdict,
329    /// because a step that was genuinely a decision made no calls and is not
330    /// wrong. Naming that case is what keeps the line from being one the model
331    /// learns to skip.
332    ///
333    /// No tool is named in any of them. `ask_user` is registered only by a
334    /// front-end that owns a human, so an unattended run told to ask would
335    /// spend a turn on a call that can only fail — `compact`'s own description
336    /// declines to name `todo` for the same reason.
337    pub fn line(self, step: &str, again: bool) -> Option<String> {
338        let step = ellipsize(step, 60);
339        let body = match self {
340            Finding::Landed => return None,
341            Finding::Null => format!(
342                "step \"{step}\" was marked done with no tool calls behind it. \
343                 If it was a decision rather than work, that is what it should say; \
344                 if it was work, it has not been done yet."
345            ),
346            Finding::EndedOnFailure => format!(
347                "step \"{step}\" was marked done with its last call still failing, \
348                 and nothing after it succeeded. Check that it landed before moving on."
349            ),
350            Finding::EndedOnRefusal => format!(
351                "step \"{step}\" was marked done with its last call refused. \
352                 It was blocked rather than finished — the plan should say which."
353            ),
354        };
355        Some(if again {
356            // §5.5's bound: one revision per step. A second identical reading
357            // means the revision did not work, and a third attempt is how
358            // "revise the step" becomes the local minimum the drive above it
359            // exists to escape — arriving through the door meant to prevent
360            // it.
361            format!(
362                "{body} This is the second time this step has come back that way; \
363                 rather than trying it again, say what you need to get past it."
364            )
365        } else {
366            body
367        })
368    }
369}
370
371/// Keep a long step from being most of the line it appears in.
372///
373/// On a char boundary, because a step's content is whatever the model typed
374/// and slicing a multi-byte character panics — in the one code path that runs
375/// on every plan revision of every run.
376fn ellipsize(s: &str, max: usize) -> String {
377    if s.chars().count() <= max {
378        return s.to_string();
379    }
380    let head: String = s.chars().take(max - 1).collect();
381    format!("{}…", head.trim_end())
382}
383
384// ─── The model half: escalation (§5.5, rung 7) ──────────────────────────────
385//
386// Two of §5.5's five signals are *comparisons* rather than facts about one
387// span, and each needs either a threshold nobody has measured or a guess
388// about what a call meant — which is why the deterministic reading above
389// declines both. What follows is the escalation itself: a cheap deterministic
390// pre-filter decides *whether* to ask (never the answer), and one quarantined
391// model call settles the ambiguous case.
392//
393// **Live, not offline.** Unlike the appraiser (§5.1), which reviews a
394// finished session from outside it, a step's plan action has to reach the
395// *same* run before it wastes more turns on a bad decomposition — so this
396// has no CLI surface of its own. `agent.rs`'s loop calls `escalate` directly,
397// the same way it already calls the compaction summariser, and folds the
398// verdict into the turn the way `boredom.rs`'s notices already do.
399//
400// **What this may see, and what it may never say back.** The step's own
401// text (and its siblings') is this same model's own prior plan output —
402// already fully trusted in-context every turn, not a new place for
403// third-party text to reach a decision, which is what made the appraiser's
404// evidence numbers-only. But the model's free-text `reasoning` here never
405// re-enters the conversation: a model's paraphrase of step text it just read
406// is `frontdoor`'s "a paraphrase of an injection is the injection rearranged"
407// risk, arriving through the one channel that *does* reach context.
408// `templated_nudge` is fully templated by which trigger fired; the model
409// only ever decides the binary accept/revise_plan.
410
411/// Does this successful call look like it checked something, rather than
412/// merely done something? A coarse keyword match on a `shell` command —
413/// argued, not measured, on this module's own convention for its constants.
414/// Only `shell` is matched: a project's own test runner, wired up as an MCP
415/// tool, has no name this module could know in advance, and guessing at one
416/// would be the same mistake as guessing at a threshold.
417fn looks_like_verification(call: &ToolCallTrace) -> bool {
418    const KEYWORDS: &[&str] = &[
419        "cargo test",
420        "pytest",
421        "npm test",
422        "npm run test",
423        "yarn test",
424        "pnpm test",
425        "make test",
426        "go test",
427        "rspec",
428        "jest",
429    ];
430    if call.name != "shell" {
431        return false;
432    }
433    let Some(command) = call.input.get("command").and_then(|v| v.as_str()) else {
434        return false;
435    };
436    let command = command.to_ascii_lowercase();
437    KEYWORDS.iter().any(|k| command.contains(k))
438}
439
440/// A step's own words that read as a checkable claim. Argued, not measured,
441/// same convention as [`looks_like_verification`]'s keyword list.
442///
443/// This is only half the trigger — [`escalation_candidate`] also requires
444/// `span.shell_calls > 0`, because the evidence side
445/// ([`looks_like_verification`]) can only recognise a check shaped as
446/// `shell`. A step verified through some other tool, or a run where `shell`
447/// is not registered at all, is meaningless to compare against a keyword
448/// list that only ever looks at `shell` commands.
449fn reads_as_a_verification_claim(step: &str) -> bool {
450    // Single words, matched on a word boundary — `"test"` as a plain
451    // substring also matches `"latest"`, `"attest"`, `"contest"`, of which
452    // `"latest"` is the one that actually turns up in plans ("pull the
453    // latest changes"). Multi-word phrases below stay substring matches:
454    // they cannot collide with an unrelated word the same way.
455    const WORDS: &[&str] = &["test", "verify", "confirm", "ensure"];
456    const PHRASES: &[&str] = &["check that", "make sure"];
457    let step = step.to_ascii_lowercase();
458    let tokens: std::collections::HashSet<&str> = step
459        .split(|c: char| !c.is_alphanumeric())
460        .filter(|w| !w.is_empty())
461        .collect();
462    WORDS.iter().any(|k| tokens.contains(k)) || PHRASES.iter().any(|p| step.contains(p))
463}
464
465/// Which comparison flagged a landed step for a second opinion.
466#[derive(Debug, Clone, Copy, PartialEq, Eq)]
467pub enum EscalationReason {
468    /// This step's span took far more calls than the plan's other completed
469    /// steps — maybe the decomposition was wrong, maybe the work was just
470    /// harder. The model, not a threshold, tells the two apart.
471    SpanOutlier,
472    /// The step's own words read as a checkable claim, but nothing in its
473    /// span looks like a check. The eval rig's "grade the artifact, never
474    /// the claim", one tier down.
475    UnverifiedClaim,
476}
477
478/// What the quarantined escalation call is handed. See the module note above
479/// on why the step's own text is safe to include here in a way an
480/// appraiser's evidence (§5.1) could not be.
481#[derive(Debug, Clone, PartialEq)]
482pub struct StepEscalation {
483    pub reason: EscalationReason,
484    pub step: String,
485    /// A sample of other completed steps in the same plan, most recent
486    /// first — never the whole history, which `Tracked` bounds but does not
487    /// make small. Empty for [`EscalationReason::UnverifiedClaim`], which
488    /// needs no comparison.
489    pub siblings: Vec<String>,
490    pub calls: u32,
491    /// The mean call count of the completed steps this was compared
492    /// against. Only set for [`EscalationReason::SpanOutlier`].
493    pub sibling_mean_calls: Option<f32>,
494    /// How many completed steps `sibling_mean_calls` is a mean *over* —
495    /// `completed.len()` at the time of the comparison, not `siblings.len()`.
496    /// The two diverge once the plan has more completed steps than
497    /// `ESCALATION_SIBLING_SAMPLE`: the mean is still over all of them, but
498    /// `siblings` is a truncated sample for the model to read, and stating
499    /// the sample's length beside the full mean would describe a mean over
500    /// 5 steps that was actually taken over 20.
501    pub sibling_count: usize,
502}
503
504/// A span is a clear enough outlier to be worth a second opinion at this
505/// ratio against the mean of the plan's other completed steps...
506const SPAN_OUTLIER_RATIO: f32 = 3.0;
507/// ...and at least this many calls outright, so a plan of tiny steps does
508/// not escalate on a difference of one or two calls that means nothing.
509const SPAN_OUTLIER_FLOOR: u32 = 6;
510/// Fewer completed steps than this and there is no "the plan's other steps"
511/// to compare against yet.
512const SPAN_OUTLIER_MIN_SIBLINGS: usize = 2;
513/// How many prior steps' text ride along as context — enough to judge "does
514/// this decomposition look right", not the whole plan's history.
515const ESCALATION_SIBLING_SAMPLE: usize = 5;
516
517/// The escalation's own pre-filter: cheap, deterministic, and it only ever
518/// decides *whether to ask*, never the answer.
519///
520/// **The caller must already know `appraise(span) == Finding::Landed`.** A
521/// step with its own deterministic finding needs no second opinion, and this
522/// function takes that as given rather than re-deriving it, because deriving
523/// it needs the same `span` this function already has — asking the caller to
524/// check first is one comparison, not two.
525pub fn escalation_candidate(
526    span: Span,
527    step: &str,
528    completed: &[(String, u32)],
529) -> Option<StepEscalation> {
530    if completed.len() >= SPAN_OUTLIER_MIN_SIBLINGS {
531        let mean = completed.iter().map(|(_, n)| *n as f32).sum::<f32>() / completed.len() as f32;
532        if span.calls as f32 >= mean * SPAN_OUTLIER_RATIO && span.calls >= SPAN_OUTLIER_FLOOR {
533            return Some(StepEscalation {
534                reason: EscalationReason::SpanOutlier,
535                step: step.to_string(),
536                siblings: completed
537                    .iter()
538                    .rev()
539                    .take(ESCALATION_SIBLING_SAMPLE)
540                    .map(|(s, _)| s.clone())
541                    .collect(),
542                calls: span.calls,
543                sibling_mean_calls: Some(mean),
544                sibling_count: completed.len(),
545            });
546        }
547    }
548    // `span.shell_calls > 0` first: `looks_like_verification` can only
549    // recognise a check shaped as `shell`, so `verify_like == 0` alone is
550    // ambiguous between "a shell call ran and none of them looked like a
551    // check" (the real case this trigger is for) and "no shell call could
552    // have set the counter at all" — a step verified through an MCP test
553    // runner, or a run where `shell` is not even registered (a mail-only
554    // trigger, a `tools:`-narrowed skill, a read-only run). Reading the
555    // second case as a positive would make every claim-shaped step on such
556    // a run escalate, unconditionally, straight to
557    // `MAX_STEP_ESCALATIONS_PER_RUN` — the same "absence of evidence is not
558    // evidence of absence" rule `appraise` already takes on `in_flight`/
559    // `denied`, one door over.
560    if reads_as_a_verification_claim(step) && span.shell_calls > 0 && span.verify_like == 0 {
561        return Some(StepEscalation {
562            reason: EscalationReason::UnverifiedClaim,
563            step: step.to_string(),
564            siblings: Vec::new(),
565            calls: span.calls,
566            sibling_mean_calls: None,
567            sibling_count: 0,
568        });
569    }
570    None
571}
572
573/// What the escalation decided: nothing further, or a plan-revision nudge is
574/// worth surfacing.
575#[derive(Debug, Clone, Copy, PartialEq, Eq)]
576pub enum StepVerdict {
577    Accept,
578    RevisePlan,
579}
580
581/// How much of a step's own text — the model's own prior plan output, but
582/// nothing in `todo`'s schema bounds its length — reaches the quarantined
583/// call. Longer than [`ellipsize`]'s 60-char display cap on
584/// [`templated_nudge`]'s step name: this text is what the call judges from,
585/// not a label, so it gets room to be useful while still being bounded.
586const ESCALATION_PROMPT_TEXT_CHARS: usize = 400;
587
588/// The prompt the quarantined pass runs. Reasoning first, the typed field
589/// last — the front door's and the appraiser's own finding: constrained
590/// output degrades reasoning when the answer precedes the thinking.
591pub fn escalation_prompt(escalation: &StepEscalation) -> String {
592    let step_text = ellipsize(&escalation.step, ESCALATION_PROMPT_TEXT_CHARS);
593    let question = match escalation.reason {
594        EscalationReason::SpanOutlier => format!(
595            "This step just finished after {} tool calls. The plan's other completed \
596             steps averaged {:.1} calls each ({} of them). Does the size of this step \
597             suggest the plan's decomposition should be revised for the steps still \
598             ahead, or was this step just harder than the others with nothing wrong \
599             in how the plan divided the work?",
600            escalation.calls,
601            escalation.sibling_mean_calls.unwrap_or(0.0),
602            escalation.sibling_count,
603        ),
604        EscalationReason::UnverifiedClaim => format!(
605            "This step was marked done. Its own wording reads as claiming something \
606             was tested, verified, or confirmed, but none of its {} tool call(s) \
607             looked like a check — grade the calls, not the claim. Does this look \
608             like the step actually verified what it says, or like an unverified \
609             claim the plan should revisit?",
610            escalation.calls,
611        ),
612    };
613    let siblings = if escalation.siblings.is_empty() {
614        String::new()
615    } else {
616        format!(
617            "\n\nThe {} most recent of them, for context:\n{}",
618            escalation.siblings.len(),
619            escalation
620                .siblings
621                .iter()
622                .map(|s| format!("- {}", ellipsize(s, ESCALATION_PROMPT_TEXT_CHARS)))
623                .collect::<Vec<_>>()
624                .join("\n")
625        )
626    };
627    format!(
628        "You are reviewing one step of your own plan from outside the run that made \
629         it — you have no tools and cannot act, only judge.\n\n\
630         The step: \"{}\"\n\
631         {question}{siblings}\n\n\
632         Return exactly this JSON and nothing else:\n\
633         {{\n  \"reasoning\": \"one or two sentences\",\n  \
634         \"verdict\": \"accept | revise_plan\"\n}}\n\n\
635         `accept` is the common, correct answer when the work looks sound; \
636         `revise_plan` only when there is a real reason to reconsider the \
637         decomposition.",
638        step_text,
639    )
640}
641
642/// Parse what the escalation returned.
643///
644/// The bracket-matching leniency is `frontdoor::parse_extraction`'s: models
645/// wrap JSON in prose and code fences however firmly they are asked not to.
646/// `reasoning`, if present, is logged at `debug` and never returned — see the
647/// module note on why it must not reach [`templated_nudge`].
648pub fn parse_step_verdict(text: &str) -> Result<StepVerdict> {
649    let start = text
650        .find('{')
651        .context("the escalation returned no JSON object")?;
652    let end = text
653        .rfind('}')
654        .context("the escalation returned no JSON object")?;
655    if end <= start {
656        anyhow::bail!("the escalation returned no JSON object");
657    }
658
659    #[derive(Deserialize)]
660    struct Wire {
661        #[serde(default)]
662        reasoning: Option<String>,
663        verdict: String,
664    }
665    let wire: Wire = serde_json::from_str(&text[start..=end]).with_context(|| {
666        let cut = crate::text::char_boundary_at_or_before(text, end.min(start + 400) + 1);
667        format!("parsing the escalation's verdict: {}", &text[start..cut])
668    })?;
669
670    if let Some(reasoning) = &wire.reasoning {
671        tracing::debug!(%reasoning, "step escalation reasoning (never shown to the model)");
672    }
673    match wire.verdict.as_str() {
674        "accept" => Ok(StepVerdict::Accept),
675        "revise_plan" => Ok(StepVerdict::RevisePlan),
676        other => anyhow::bail!("the escalation returned an unrecognised verdict `{other}`"),
677    }
678}
679
680// **The quarantined call itself is `agent.rs`'s to make, not this module's.**
681// This is the one place rung 7's two halves genuinely differ: the appraiser
682// (§5.1) is offline, so a bare `&dyn Provider` and a plain retry loop are the
683// whole story. This escalation runs *inside* a live, cancellable run, so it
684// has to go through `Agent::complete` the same way `compact`'s summariser and
685// `compact_validate`'s check already do — that is what wires it into the
686// run's own cancellation token and folds its spend into `RunStats`, neither
687// of which a bare provider call can reach. `escalation_prompt` and
688// `parse_step_verdict` above are what stay pure and testable here; the retry
689// loop that drives them lives beside `compact` in `agent.rs`.
690
691/// Marks a folded nudge as the harness's own words, on `boredom::NOTICE_STEM`'s
692/// exact precedent: `agent::is_harness_voice` is a closed list the learning
693/// miner filters every tool-result message's text through before deciding
694/// whether it is a user's `Steer`/`Followup` intervention. Without an entry
695/// here, `templated_nudge`'s output — folded into the very message
696/// `escalation_candidate` also carries tool results in — would be mined as if
697/// a person had typed it, `escalation.step` and all, and could ride into a
698/// future prompt as a `Clean`-origin learned rule derived from nobody's words.
699pub const STEP_ESCALATION_STEM: &str = "A second opinion on your plan:";
700
701/// The nudge folded into the run when the escalation says `revise_plan`.
702///
703/// Fully templated — the model's own free-text reasoning never reaches this
704/// output, on `frontdoor`'s rule one door over: a paraphrase of text the
705/// model just read is the same risk as the text itself, arriving through
706/// the one channel that re-enters context. Wording follows `Finding::line`'s
707/// own discipline: state the fact, offer one continuation.
708pub fn templated_nudge(escalation: &StepEscalation) -> String {
709    let step = ellipsize(&escalation.step, 60);
710    let body = match escalation.reason {
711        EscalationReason::SpanOutlier => format!(
712            "step \"{step}\" took {} tool call(s) against the plan's other completed \
713             steps' average of {:.1} — worth checking whether the remaining steps in \
714             the plan need to be broken down differently, or re-scoped.",
715            escalation.calls,
716            escalation.sibling_mean_calls.unwrap_or(0.0),
717        ),
718        EscalationReason::UnverifiedClaim => format!(
719            "step \"{step}\" reads as claiming something was tested or verified, but \
720             nothing in its tool calls looked like a check — worth confirming it \
721             actually landed before moving on."
722        ),
723    };
724    format!("{STEP_ESCALATION_STEM} {body}")
725}
726
727#[cfg(test)]
728mod tests {
729    use super::*;
730    use serde_json::json;
731
732    fn call(is_error: bool, denied: bool, unknown: bool) -> ToolCallTrace {
733        ToolCallTrace {
734            name: "shell".into(),
735            input: json!({}),
736            is_error,
737            denied,
738            unknown,
739            staged: false,
740        }
741    }
742
743    fn ok() -> ToolCallTrace {
744        call(false, false, false)
745    }
746    fn failed() -> ToolCallTrace {
747        call(true, false, false)
748    }
749    fn denied() -> ToolCallTrace {
750        // As the loop writes it: a denial is an error *and* a denial.
751        call(true, true, false)
752    }
753
754    #[test]
755    fn a_denial_is_never_counted_as_a_failure() {
756        let work = Work::of(&[ok(), denied()]);
757        assert_eq!(work.calls, 2);
758        assert_eq!(
759            work.failed, 0,
760            "the approver doing its job is not a failure"
761        );
762        assert_eq!(work.refused, 1);
763        assert_eq!(work.last, Some(Outcome::Refused));
764
765        // And the two produce different findings, which is the whole reason
766        // the split exists: one says fix your work, the other says you were
767        // blocked.
768        let refused = appraise(work.since(Work::default(), 0, work.last).unwrap());
769        assert_eq!(refused, Finding::EndedOnRefusal);
770        let broke = appraise(
771            Work::of(&[ok(), failed()])
772                .since(Work::default(), 0, Some(Outcome::Failed))
773                .unwrap(),
774        );
775        assert_eq!(broke, Finding::EndedOnFailure);
776    }
777
778    #[test]
779    fn an_unknown_tool_counts_with_the_failures() {
780        let work = Work::of(&[call(true, false, true)]);
781        assert_eq!(work.failed, 1);
782        assert_eq!(work.refused, 0);
783    }
784
785    #[test]
786    fn a_step_with_nothing_behind_it_is_the_null_step() {
787        let start = Work::of(&[ok(), ok()]);
788        // Two turns later, and not one call in between.
789        let now = start;
790        assert_eq!(
791            appraise(now.since(start, 0, now.last).unwrap()),
792            Finding::Null
793        );
794    }
795
796    #[test]
797    fn plan_bookkeeping_does_not_count_as_work() {
798        let start = Work::default();
799        // Three calls since the step started — all of them the plan tool
800        // rewriting the list. Nothing was done.
801        let now = Work::of(&[ok(), ok(), ok()]);
802        assert_eq!(
803            appraise(now.since(start, 3, now.last).unwrap()),
804            Finding::Null,
805            "a step whose whole span is plan revision did nothing"
806        );
807        // One real call among them and it is no longer null.
808        assert_eq!(
809            appraise(now.since(start, 2, now.last).unwrap()),
810            Finding::Landed
811        );
812    }
813
814    #[test]
815    fn a_bookkeeping_call_landing_last_does_not_mask_a_real_failure() {
816        // build fails, then the plan tool revises the list (successfully) —
817        // `now.last` is the revision's `Ok`, but the caller (`Tracked`) knows
818        // the failure is the outcome that actually belongs to the span.
819        let start = Work::default();
820        let now = Work::of(&[failed(), ok()]);
821        let span = now.since(start, 1, Some(Outcome::Failed)).unwrap();
822        assert_eq!(span.calls, 1, "the bookkeeping call is excluded from work");
823        assert_eq!(
824            appraise(span),
825            Finding::EndedOnFailure,
826            "the caller's `last` overrides the raw trace tail"
827        );
828    }
829
830    #[test]
831    fn a_bookkeeping_call_landing_last_does_not_manufacture_a_failure() {
832        // A rejected plan write is itself a failed call, but it is still the
833        // plan tool touching its own state rather than work on the step —
834        // the caller excludes it from both the count and `last`.
835        let start = Work::default();
836        let now = Work::of(&[ok(), failed()]);
837        let span = now.since(start, 1, Some(Outcome::Ok)).unwrap();
838        assert_eq!(span.calls, 1);
839        assert_eq!(
840            appraise(span),
841            Finding::Landed,
842            "a rejected bookkeeping call must not read as the step's own failure"
843        );
844    }
845
846    #[test]
847    fn a_denied_sibling_supports_no_finding_either() {
848        // The batched shape `in_flight` exists for, except the sibling is
849        // denied rather than still running: settled, but not attributable to
850        // this step.
851        let span = Work::of(&[ok()])
852            .with_denied(1)
853            .since(Work::default(), 0, Some(Outcome::Ok))
854            .unwrap();
855        assert_eq!(appraise(span), Finding::Landed);
856
857        // Same when the visible half would otherwise report a refusal.
858        let span = Work::of(&[ok(), denied()])
859            .with_denied(1)
860            .since(Work::default(), 0, Some(Outcome::Refused))
861            .unwrap();
862        assert_eq!(
863            appraise(span),
864            Finding::Landed,
865            "the denial in the same batch is not necessarily this step's"
866        );
867    }
868
869    #[test]
870    fn a_failure_recovered_from_is_the_model_working() {
871        let start = Work::default();
872        let now = Work::of(&[failed(), ok()]);
873        let span = now.since(start, 0, now.last).unwrap();
874        assert_eq!(span.failed, 1, "the failure is still counted");
875        assert_eq!(
876            appraise(span),
877            Finding::Landed,
878            "only the last attempt decides; recovery is not a finding"
879        );
880    }
881
882    #[test]
883    fn a_sibling_still_running_supports_no_finding() {
884        // The work and the tick in one batch: nothing has landed in the trace
885        // yet, so the visible half says null and the honest answer is nothing.
886        let empty = Work::default().with_in_flight(1);
887        assert_eq!(
888            appraise(empty.since(Work::default(), 0, empty.last).unwrap()),
889            Finding::Landed
890        );
891
892        // Same for a failure a sibling may still be recovering from.
893        let failing = Work::of(&[failed()]).with_in_flight(1);
894        assert_eq!(
895            appraise(failing.since(Work::default(), 0, failing.last).unwrap()),
896            Finding::Landed
897        );
898    }
899
900    #[test]
901    fn a_failure_before_the_step_started_is_not_this_step_s() {
902        let start = Work::of(&[failed()]);
903        let now = Work::of(&[failed(), ok(), ok()]);
904        let span = now.since(start, 0, now.last).unwrap();
905        assert_eq!(span.failed, 0);
906        assert_eq!(appraise(span), Finding::Landed);
907    }
908
909    /// The chat shape: a step started before the user last spoke. The
910    /// counters restarted with the run, so the span is unmeasurable — and
911    /// saying so is the whole point, because the arithmetic alone would
912    /// saturate to zero and announce the null step on ordinary work.
913    #[test]
914    fn a_mark_from_another_run_is_unmeasurable_rather_than_empty() {
915        let first = Work::of(&[ok(), ok(), ok()]).in_run(1);
916        let second = Work::of(&[ok()]).in_run(2);
917        assert_eq!(second.since(first, 0, second.last), None);
918
919        // And within one run it measures as usual.
920        assert!(Work::of(&[ok(), ok(), ok(), ok()])
921            .in_run(1)
922            .since(first, 0, Some(Outcome::Ok))
923            .is_some());
924    }
925
926    #[test]
927    fn the_common_path_says_nothing() {
928        assert_eq!(Finding::Landed.line("read the config", false), None);
929    }
930
931    #[test]
932    fn a_second_identical_reading_stops_asking_for_a_revision() {
933        let first = Finding::Null.line("fix the port", false).unwrap();
934        let second = Finding::Null.line("fix the port", true).unwrap();
935        assert!(first.contains("fix the port") && !first.contains("second time"));
936        assert!(second.contains("second time"));
937        // Neither names a tool: an unattended run has no `ask_user` to reach
938        // for, and pointing at an absent tool spends a turn on a call that can
939        // only fail.
940        for line in [&first, &second] {
941            assert!(!line.contains("ask_user") && !line.contains('`'));
942        }
943    }
944
945    #[test]
946    fn a_long_step_is_cut_on_a_char_boundary() {
947        let step = "é".repeat(200);
948        let line = Finding::Null.line(&step, false).unwrap();
949        assert!(line.contains('…'));
950    }
951
952    // --- the model half: escalation ---
953
954    fn shell(command: &str) -> ToolCallTrace {
955        ToolCallTrace {
956            name: "shell".into(),
957            input: json!({"command": command}),
958            is_error: false,
959            denied: false,
960            unknown: false,
961            staged: false,
962        }
963    }
964
965    /// Every existing caller of this helper models a span made of `shell`
966    /// calls (some verify-shaped, some not) — the ordinary case the
967    /// `UnverifiedClaim` trigger is for — so `shell_calls` defaults to
968    /// `calls` here. `a_claim_with_no_shell_call_at_all_does_not_escalate`
969    /// builds its own `Span` literal for the case this helper does not
970    /// model.
971    fn span(calls: u32, verify_like: u32) -> Span {
972        Span {
973            calls,
974            failed: 0,
975            refused: 0,
976            verify_like,
977            shell_calls: calls,
978            last: Some(Outcome::Ok),
979            in_flight: 0,
980            denied: 0,
981        }
982    }
983
984    #[test]
985    fn a_shell_call_matching_a_test_runner_looks_like_verification() {
986        for command in [
987            "cargo test --workspace",
988            "pytest tests/",
989            "npm test",
990            "make test",
991            "CARGO TEST -p mecha-core",
992        ] {
993            assert!(
994                looks_like_verification(&shell(command)),
995                "{command:?} should have matched"
996            );
997        }
998    }
999
1000    #[test]
1001    fn an_ordinary_shell_call_does_not_look_like_verification() {
1002        assert!(!looks_like_verification(&shell("cargo build --release")));
1003        assert!(!looks_like_verification(&call(false, false, false)));
1004    }
1005
1006    #[test]
1007    fn work_folds_verify_like_only_for_successful_calls() {
1008        // A failed test invocation did not confirm anything.
1009        let mut failing_test = shell("cargo test");
1010        failing_test.is_error = true;
1011        let work = Work::of(&[shell("cargo test"), failing_test]);
1012        assert_eq!(work.verify_like, 1);
1013    }
1014
1015    /// The review finding: a refused or failed `shell` call never ran, so
1016    /// counting it would let a single denied attempt (a read-only run, or
1017    /// any surface `shell` isn't registered on) satisfy `shell_calls > 0`
1018    /// and reopen the exact false positive that check exists to close.
1019    #[test]
1020    fn work_folds_shell_calls_only_for_successful_calls() {
1021        // `call(is_error, denied, unknown)` already names itself `"shell"`.
1022        let refused = call(false, true, false);
1023        let failed = call(true, false, false);
1024        let work = Work::of(&[refused, failed]);
1025        assert_eq!(
1026            work.shell_calls, 0,
1027            "neither call actually ran, so shell_calls must stay at zero"
1028        );
1029        let work = Work::of(&[ok()]);
1030        assert_eq!(work.shell_calls, 1);
1031    }
1032
1033    #[test]
1034    fn a_span_far_longer_than_its_siblings_is_a_span_outlier_candidate() {
1035        let completed = vec![
1036            ("read the config".to_string(), 2),
1037            ("write the file".to_string(), 3),
1038        ];
1039        let escalation = escalation_candidate(span(20, 0), "do the big thing", &completed)
1040            .expect("20 calls against a mean of 2.5 should escalate");
1041        assert_eq!(escalation.reason, EscalationReason::SpanOutlier);
1042        assert_eq!(escalation.calls, 20);
1043        assert_eq!(escalation.sibling_mean_calls, Some(2.5));
1044        assert_eq!(escalation.siblings.len(), 2);
1045    }
1046
1047    #[test]
1048    fn a_tiny_plan_never_fires_the_span_outlier_trigger() {
1049        // Only one prior completed step: nothing to compare against yet.
1050        let completed = vec![("read the config".to_string(), 2)];
1051        assert!(escalation_candidate(span(20, 0), "do the big thing", &completed).is_none());
1052    }
1053
1054    #[test]
1055    fn a_step_within_the_floor_never_fires_even_against_a_tiny_mean() {
1056        // 3x a mean of 1 is 3, which is under the absolute floor.
1057        let completed = vec![("a".to_string(), 1), ("b".to_string(), 1)];
1058        assert!(escalation_candidate(span(3, 0), "a small step", &completed).is_none());
1059    }
1060
1061    #[test]
1062    fn a_step_only_moderately_bigger_than_its_siblings_does_not_escalate() {
1063        let completed = vec![("a".to_string(), 5), ("b".to_string(), 5)];
1064        // 2x the mean, not 3x.
1065        assert!(escalation_candidate(span(10, 0), "a somewhat bigger step", &completed).is_none());
1066    }
1067
1068    #[test]
1069    fn a_step_that_claims_verification_with_none_in_its_span_escalates() {
1070        let escalation = escalation_candidate(span(3, 0), "test that the API responds", &[])
1071            .expect("a verification claim with no verify-shaped call should escalate");
1072        assert_eq!(escalation.reason, EscalationReason::UnverifiedClaim);
1073        assert!(escalation.siblings.is_empty());
1074        assert_eq!(escalation.sibling_mean_calls, None);
1075    }
1076
1077    #[test]
1078    fn a_step_that_claims_verification_and_has_it_does_not_escalate() {
1079        assert!(escalation_candidate(span(3, 1), "test that the API responds", &[]).is_none());
1080    }
1081
1082    #[test]
1083    fn an_ordinary_step_with_no_claim_and_no_outlier_never_escalates() {
1084        let completed = vec![("a".to_string(), 4), ("b".to_string(), 5)];
1085        assert!(escalation_candidate(span(4, 0), "write the docs", &completed).is_none());
1086    }
1087
1088    /// The bug the review found: `"test"` as a plain substring also matches
1089    /// `"latest"`, so an ordinary step about pulling the latest changes read
1090    /// as a verification claim with nothing to back it.
1091    #[test]
1092    fn a_word_containing_test_as_a_substring_is_not_a_verification_claim() {
1093        for step in [
1094            "pull the latest changes",
1095            "read the latest config",
1096            "copy the latest bundle",
1097        ] {
1098            assert!(
1099                escalation_candidate(span(3, 0), step, &[]).is_none(),
1100                "{step:?} must not read as a verification claim"
1101            );
1102        }
1103        // The word-boundary match must still catch the real thing.
1104        assert!(escalation_candidate(span(3, 0), "test that the API responds", &[]).is_some());
1105    }
1106
1107    /// The review finding: `verify_like == 0` is ambiguous between "a shell
1108    /// call ran and didn't look like a check" and "nothing could have set
1109    /// the counter" — a step verified through some other tool, or a run
1110    /// where `shell` is not even registered. Only the first should escalate.
1111    #[test]
1112    fn a_claim_with_no_shell_call_at_all_does_not_escalate() {
1113        let no_shell_calls = Span {
1114            calls: 1,
1115            failed: 0,
1116            refused: 0,
1117            verify_like: 0,
1118            shell_calls: 0,
1119            last: Some(Outcome::Ok),
1120            in_flight: 0,
1121            denied: 0,
1122        };
1123        assert!(
1124            escalation_candidate(no_shell_calls, "test that the API responds", &[]).is_none(),
1125            "no shell call ran in this span, so absence of a match proves nothing"
1126        );
1127    }
1128
1129    #[test]
1130    fn only_the_most_recent_siblings_ride_along() {
1131        let completed: Vec<(String, u32)> = (0..20).map(|i| (format!("step {i}"), 2)).collect();
1132        let escalation = escalation_candidate(span(30, 0), "a big step", &completed).unwrap();
1133        assert_eq!(escalation.siblings.len(), ESCALATION_SIBLING_SAMPLE);
1134        // Most recent first.
1135        assert_eq!(escalation.siblings[0], "step 19");
1136        // The review finding: the mean is over all 20, and the prompt must
1137        // say so — not the length of the truncated sample listed below it.
1138        assert_eq!(escalation.sibling_count, 20);
1139        let prompt = escalation_prompt(&escalation);
1140        assert!(prompt.contains("(20 of them)"));
1141        assert!(prompt.contains(&format!(
1142            "The {ESCALATION_SIBLING_SAMPLE} most recent of them"
1143        )));
1144    }
1145
1146    /// Unlike `templated_nudge`, which ellipsizes before ever using step
1147    /// text, `escalation_prompt` used to embed the step and every sibling
1148    /// verbatim — and nothing in the `todo` tool's schema bounds a step's
1149    /// length. A single very long step (or sibling) would have gone into
1150    /// the quarantined call whole.
1151    #[test]
1152    fn a_very_long_step_or_sibling_is_bounded_in_the_prompt() {
1153        let long_step = "x".repeat(5_000);
1154        let long_sibling = "y".repeat(5_000);
1155        let escalation = StepEscalation {
1156            reason: EscalationReason::SpanOutlier,
1157            step: long_step.clone(),
1158            siblings: vec![long_sibling.clone()],
1159            calls: 20,
1160            sibling_mean_calls: Some(2.5),
1161            sibling_count: 1,
1162        };
1163        let prompt = escalation_prompt(&escalation);
1164        assert!(
1165            !prompt.contains(&long_step),
1166            "the full 5,000-char step must not reach the prompt whole"
1167        );
1168        assert!(!prompt.contains(&long_sibling));
1169        assert!(prompt.len() < long_step.len() + long_sibling.len());
1170    }
1171
1172    fn span_outlier_escalation() -> StepEscalation {
1173        StepEscalation {
1174            reason: EscalationReason::SpanOutlier,
1175            step: "do the big thing".into(),
1176            siblings: vec!["read the config".into()],
1177            calls: 20,
1178            sibling_mean_calls: Some(2.5),
1179            sibling_count: 1,
1180        }
1181    }
1182
1183    #[test]
1184    fn the_prompt_asks_for_reasoning_before_the_typed_field() {
1185        let prompt = escalation_prompt(&span_outlier_escalation());
1186        assert!(prompt.find("\"reasoning\"").unwrap() < prompt.find("\"verdict\"").unwrap());
1187        assert!(prompt.contains("do the big thing"));
1188        assert!(prompt.contains("read the config"));
1189    }
1190
1191    #[test]
1192    fn parsing_an_accept_verdict() {
1193        let v = parse_step_verdict(r#"{"reasoning": "looks fine", "verdict": "accept"}"#).unwrap();
1194        assert_eq!(v, StepVerdict::Accept);
1195    }
1196
1197    #[test]
1198    fn parsing_a_revise_plan_verdict_wrapped_in_prose() {
1199        let text = "Here you go:\n```json\n{\"reasoning\": \"too broad\", \"verdict\": \"revise_plan\"}\n```\n";
1200        assert_eq!(parse_step_verdict(text).unwrap(), StepVerdict::RevisePlan);
1201    }
1202
1203    #[test]
1204    fn an_unrecognised_verdict_is_refused() {
1205        assert!(parse_step_verdict(r#"{"reasoning": "x", "verdict": "maybe"}"#).is_err());
1206    }
1207
1208    #[test]
1209    fn an_unparseable_reply_is_an_error() {
1210        assert!(parse_step_verdict("I could not do that.").is_err());
1211    }
1212
1213    /// The property the whole design turns on: whatever the model wrote as
1214    /// `reasoning` must never appear in the nudge shown back to it.
1215    #[test]
1216    fn the_nudge_never_contains_the_models_own_reasoning() {
1217        let escalation = span_outlier_escalation();
1218        let nudge = templated_nudge(&escalation);
1219        assert!(nudge.contains("do the big thing"));
1220        assert!(!nudge.contains("looks fine"));
1221        assert!(!nudge.contains("too broad"));
1222        // Fully templated: two calls with the same escalation produce the
1223        // same nudge regardless of what any model said.
1224        assert_eq!(nudge, templated_nudge(&escalation));
1225    }
1226
1227    #[test]
1228    fn the_unverified_claim_nudge_names_no_siblings() {
1229        let escalation = StepEscalation {
1230            reason: EscalationReason::UnverifiedClaim,
1231            step: "test that the API responds".into(),
1232            siblings: Vec::new(),
1233            calls: 3,
1234            sibling_mean_calls: None,
1235            sibling_count: 0,
1236        };
1237        let nudge = templated_nudge(&escalation);
1238        assert!(nudge.contains("test that the API responds"));
1239    }
1240
1241    /// The review finding: a nudge not registered in `agent::is_harness_voice`
1242    /// gets mined by the learning miner as if a person had typed it —
1243    /// `escalation.step` included — exactly the bug `boredom::NOTICE_STEM`
1244    /// and `mailbox::DELIVERY_STEM` were each added to fix for their own
1245    /// voice. Both `EscalationReason` variants must be recognised.
1246    #[test]
1247    fn the_nudge_is_recognised_as_the_harness_own_voice() {
1248        assert!(crate::agent::is_harness_voice(&templated_nudge(
1249            &span_outlier_escalation()
1250        )));
1251        let unverified = StepEscalation {
1252            reason: EscalationReason::UnverifiedClaim,
1253            step: "test that the API responds".into(),
1254            siblings: Vec::new(),
1255            calls: 3,
1256            sibling_mean_calls: None,
1257            sibling_count: 0,
1258        };
1259        assert!(crate::agent::is_harness_voice(&templated_nudge(
1260            &unverified
1261        )));
1262    }
1263
1264    // The retry loop that used to live here moved to `agent.rs`'s
1265    // `Agent::escalate_step`, which needs `self.complete` for cancellation
1266    // and usage accounting — see the module note above `parse_step_verdict`.
1267    // Its tests live beside `agent.rs`'s own `ScriptedProvider`.
1268}