Skip to main content

magi/
triage.rs

1//! Held-task triage: walking `held` tasks so a hold left by an accident does
2//! not sit unread forever next to one a human placed on purpose.
3//!
4//! [`crate::queue::HoldSource`] already distinguishes "the daemon or
5//! conductor held this during its own recovery" ([`HoldSource::Machine`],
6//! documented as recoverable) from "an operator held this on purpose"
7//! ([`HoldSource::Manual`]). What was missing was anything that actually acts
8//! on that distinction: nothing walked the held list and asked whether a
9//! machine hold's cause was still true, and a record written before
10//! `hold_source` existed (schema < 3, `None`) was silently protected forever
11//! by [`Task::operator_held`]'s conservative default - never wrong, but also
12//! never looked at again by anything.
13//!
14//! [`run_once`] is that walk. For every `held` task it finds:
15//!
16//! - [`HoldSource::Machine`]: if [`machine_cause_resolved`] can tell the
17//!   cause is gone, the task goes straight back to `queued` - the same effect
18//!   as `magi task release`, just automatic. When it cannot tell, a
19//!   [`Question`] is filed once and the task stays held until answered.
20//! - `None` (a legacy record, or a hold nobody explained): always a question,
21//!   exactly once - the whole point being that "protected forever" must not
22//!   mean "never shown to anyone" either.
23//! - [`HoldSource::Manual`]: never touched automatically. Only once the hold
24//!   has sat untouched past [`MANUAL_STALE_AFTER`] does it earn a question of
25//!   its own, asking whether it is still wanted.
26//!
27//! Every question this module files carries [`NODE`] and uses the task id as
28//! [`Question::run`] - the same convention `crate::conduct`'s own questions
29//! use for a task rather than a run (see `conduct::apply_one`'s own comment
30//! on why the dedupe check there also filters on `node`, not `run` alone: an
31//! ordinary graph question's `run` is a real run id, and a coincidental
32//! equality with some task's id must not be read as "about this task").
33//! [`latest_triage_question`] follows the identical rule.
34//!
35//! # Why answers apply here rather than through `crate::queue::Task::block`
36//!
37//! `crate::conduct` blocks a task on its own question
38//! (`Task::block(vec![question_id], …)`), and `crate::daemon::resolve_blockers`
39//! unblocks it - unconditionally, back to `queued` - the moment that question
40//! is answered, whatever the answer actually said. That is correct for
41//! `conduct`: the *content* of the answer is meant for whoever reads
42//! `Task::answers` next, not for the resolver.
43//!
44//! A triage question's answer is different: "not yet" and "discard it" do two
45//! entirely different, non-resuming things, and only "resume it" may put the
46//! task back in line. Reusing the generic blocked/unblock path would resume
47//! every answer alike, so this module never calls [`Task::block`] and never
48//! leaves a triaged task anything but `held` while its question is open.
49//! [`interpret_answer`] reads [`Question::resolution`] itself and
50//! [`run_once`] acts on it directly: [`Task::release`] for an actual "resume
51//! it" choice, [`Queue::remove`] for "discard it" (捨ててよい really means
52//! "you may throw this away", not "leave it sitting held" - the English
53//! wording must say the same thing, not "leave it held"), and
54//! [`Task::hold_manual`] for anything else - which both keeps the task held
55//! and reclassifies it as a hold an operator has now actually seen, one
56//! `crate::conduct` and a later triage pass leave alone.
57//!
58//! The choice is read by its **position** in [`Question::choices`]
59//! ([`Wording::choices3`]/[`Wording::choices2`] always put "resume" first and
60//! "discard" third), never by comparing the answer text against [`Wording`]'s
61//! own strings picked from whatever config is in force *now* - the language a
62//! question was filed in and the language a later `run_once` call happens to
63//! read back are not guaranteed to be the same call's [`Config`], and a text
64//! comparison would silently misread a real "resume" answer as "keep held"
65//! the moment they disagree.
66//!
67//! # Idempotency, without a new field
68//!
69//! [`run_once`] runs on every daemon idle tick (see `crate::daemon::poll`)
70//! and on every `magi task triage`, so applying the *same* answered question
71//! twice has to be harmless - and, once a `HoldSource::Manual`/`None`
72//! question has been answered "not yet", finding a *fresh* one for the same
73//! task later (once it goes stale again) has to still be possible. Neither
74//! [`crate::queue::Task`] nor [`crate::ask::Question`] has a field for "this
75//! answer was already applied", so [`already_applied`] reads the same
76//! [`Question::short`] id back out of [`Task::hold_reason`] that
77//! [`keep_held_note`] appended to it - the same trick [`Question::abandon`]
78//! already uses to fold a fact into a text field that has no dedicated one.
79//! Appended, not written wholesale: the reason the hold happened in the first
80//! place is still worth reading in `magi task show` after an operator says
81//! "not yet". A "resume" or "discard" answer needs no marker at all: the task
82//! either leaves `held` entirely or stops existing, and either way it is
83//! never looked at by this module again.
84
85use std::path::{Path, PathBuf};
86use std::time::Duration;
87
88use jiff::Timestamp;
89
90use crate::ask::{Question, QuestionStatus, Questions};
91use crate::config::Config;
92use crate::disk;
93use crate::queue::{HoldSource, Queue, Task, TaskStatus};
94
95/// Node recorded on every question this module files - `crate::conduct::NODE`
96/// for the same idea applied to a `crate::conduct` decision instead.
97pub const NODE: &str = "triage";
98
99/// Seat name on a filed question. Not a real agent seat - there is no model
100/// call anywhere in this module - but every [`Question`] needs one, and every
101/// other deterministic filer (`crate::land`'s merge approval) names itself
102/// the same way.
103const SEAT: &str = "triage";
104
105/// How long a [`HoldSource::Manual`] hold sits untouched before triage asks
106/// whether it is still wanted.
107///
108/// A judgement call, not a `magi.toml` setting - the same reasoning
109/// `ask::REPLY_QUIET_WINDOW` documents for itself: there is no operator
110/// preference for "how long is too long to ignore my own hold" that a
111/// per-repository config could be *right* about. Seven days is long enough
112/// that an ordinary multi-day hold (waiting on a dependency, waiting on the
113/// operator's own schedule) never gets nagged, and short enough that a hold
114/// nobody has looked at in a week surfaces again rather than aging into the
115/// kind of silent backlog this feature exists to prevent.
116const MANUAL_STALE_AFTER: Duration = Duration::from_secs(7 * 24 * 60 * 60);
117
118/// Localised strings for a filed question, the same idea as `crate::land`'s
119/// own `Words`/`words()` - only the languages magi can actually check are
120/// translated, and anything else falls back to English.
121struct Wording {
122    lang: &'static str,
123    resume: &'static str,
124    wait: &'static str,
125    discard: &'static str,
126    resume_now: &'static str,
127    keep_held: &'static str,
128}
129
130const EN: Wording = Wording {
131    lang: "en",
132    resume: "resume it",
133    wait: "not yet",
134    discard: "discard it",
135    resume_now: "resume it",
136    keep_held: "keep it held",
137};
138
139const JA: Wording = Wording {
140    lang: "ja",
141    resume: "再開してよい",
142    wait: "まだ待って",
143    discard: "捨ててよい",
144    resume_now: "再開する",
145    keep_held: "まだ止めておく",
146};
147
148/// Pick the wording. Codes and names both, the same acceptance
149/// `crate::land::words` gives `[graph] language`.
150fn wording(language: &str) -> &'static Wording {
151    let l = language.trim();
152    if l.eq_ignore_ascii_case("ja")
153        || l.eq_ignore_ascii_case("jp")
154        || l.eq_ignore_ascii_case("japanese")
155        || l.eq_ignore_ascii_case("日本語")
156    {
157        &JA
158    } else {
159        &EN
160    }
161}
162
163impl Wording {
164    fn choices3(&self) -> Vec<String> {
165        vec![
166            self.resume.to_owned(),
167            self.wait.to_owned(),
168            self.discard.to_owned(),
169        ]
170    }
171
172    fn choices2(&self) -> Vec<String> {
173        vec![self.resume_now.to_owned(), self.keep_held.to_owned()]
174    }
175
176    fn source_label(&self, source: Option<HoldSource>) -> &'static str {
177        match (self.lang, source) {
178            ("ja", Some(HoldSource::Machine)) => "machine(機械による自動保留)",
179            ("ja", Some(HoldSource::Manual)) => "manual(操作者による手動保留)",
180            ("ja", None) => "unknown(schema 3 未満の旧レコード、または理由未記録)",
181            (_, Some(HoldSource::Machine)) => "machine (automatic recovery hold)",
182            (_, Some(HoldSource::Manual)) => "manual (an operator held this)",
183            (_, None) => "unknown (pre-schema-3 record, or never recorded)",
184        }
185    }
186
187    /// The body under the summary: everything an operator needs to judge this
188    /// without opening a terminal - id, title, hold reason, hold source.
189    ///
190    /// Falls back to [`Task::last_error`] when [`Task::hold_reason`] is empty:
191    /// the most common `HoldSource::Machine` hold of all - `Task::fail` once
192    /// attempts run out - only ever sets `last_error`, never `hold_reason`, so
193    /// reading `hold_reason` alone would leave the question blank for exactly
194    /// the case requirement 4 exists for.
195    fn detail(&self, task: &Task, why: &str) -> String {
196        let none = if self.lang == "ja" {
197            "(記録なし)"
198        } else {
199            "(none recorded)"
200        };
201        let reason = task
202            .hold_reason
203            .as_deref()
204            .or(task.last_error.as_deref())
205            .unwrap_or(none);
206        format!(
207            "task: {} ({})\ntitle: {}\nhold source: {}\nhold reason: {reason}\n\n{why}",
208            task.id,
209            task.short(),
210            task.title,
211            self.source_label(task.hold_source),
212        )
213    }
214
215    fn summary_machine_unknown(&self, task: &Task) -> String {
216        if self.lang == "ja" {
217            format!("保留タスク {} の再開可否を判断してください", task.short())
218        } else {
219            format!("decide whether to resume held task {}", task.short())
220        }
221    }
222
223    fn why_machine(&self) -> &'static str {
224        if self.lang == "ja" {
225            "機械的な保留(machine hold)ですが、原因がすでに解消しているかを自動では判断できませんでした。"
226        } else {
227            "This is a machine hold, but whether its cause has resolved could not be \
228             checked automatically."
229        }
230    }
231
232    fn summary_legacy(&self, task: &Task) -> String {
233        if self.lang == "ja" {
234            format!(
235                "hold_source が不明な保留タスク {} を確認してください",
236                task.short()
237            )
238        } else {
239            format!(
240                "held task {} has no recorded hold source - please take a look",
241                task.short()
242            )
243        }
244    }
245
246    fn why_legacy(&self) -> &'static str {
247        if self.lang == "ja" {
248            "hold_source が記録されていません。schema 3 より前のレコードか、理由が記録されなかった \
249             holdです。人が意図して止めたのか、クラッシュや強制再起動で宙に浮いただけなのか、\
250             このデータからは区別できません。"
251        } else {
252            "No hold_source was recorded - either a pre-schema-3 record, or a hold whose \
253             reason was never written down. Whether this was a deliberate hold or the \
254             leftover of a crash cannot be told from the data alone."
255        }
256    }
257
258    fn summary_manual_stale(&self, task: &Task, days: i64) -> String {
259        if self.lang == "ja" {
260            format!(
261                "{days}日間 保留されたままの手動保留タスク {} を確認してください",
262                task.short()
263            )
264        } else {
265            format!(
266                "held task {} has been on a manual hold for {days} day(s)",
267                task.short()
268            )
269        }
270    }
271
272    fn why_manual(&self) -> &'static str {
273        if self.lang == "ja" {
274            "操作者が明示的に止めた保留ですが、長期間そのままになっています。まだ止めておくか、\
275             再開するか教えてください。"
276        } else {
277            "An operator held this on purpose, but it has sat untouched for a while. Say \
278             whether to keep holding it or resume it."
279        }
280    }
281}
282
283/// Which of the three situations this module recognises a held task is in.
284#[derive(Debug, Clone, Copy, PartialEq, Eq)]
285enum Bucket {
286    /// `HoldSource::Machine`, cause not verifiably resolved.
287    MachineUnknown,
288    /// `hold_source` is `None`.
289    Legacy,
290    /// `HoldSource::Manual`, held past [`MANUAL_STALE_AFTER`].
291    ManualStale,
292}
293
294/// What one [`run_once`] pass did, task ids in each list.
295#[derive(Debug, Clone, Default)]
296pub struct Report {
297    /// A `HoldSource::Machine` hold whose cause was found resolved, put back
298    /// in line automatically.
299    pub resumed: Vec<String>,
300    /// A fresh question was filed this pass.
301    pub asked: Vec<String>,
302    /// An operator's answer to an earlier triage question was applied.
303    pub answered: Vec<String>,
304}
305
306impl Report {
307    /// Is there nothing to report? Callers use this to skip logging an empty
308    /// pass rather than repeating "triaged 0 held task(s)" on every idle tick.
309    pub fn is_empty(&self) -> bool {
310        self.resumed.is_empty() && self.asked.is_empty() && self.answered.is_empty()
311    }
312}
313
314/// The repository a task's config and disk check should read from. Mirrors
315/// `crate::conduct::repo_for`'s own fallback, duplicated rather than shared
316/// because that one is private to its module and the two are one `if` each.
317fn repo_for(task: &Task) -> PathBuf {
318    if task.repo.as_os_str().is_empty() {
319        PathBuf::from(".")
320    } else {
321        task.repo.clone()
322    }
323}
324
325/// Recognise a `HoldSource::Machine` hold caused by the free-space gate
326/// (`crate::disk::gate`'s message, or `daemon::disk_gate`'s "could not
327/// measure" fallback) from `hold_reason` text alone.
328///
329/// There is no field recording *why* a machine hold happened - `hold_machine`
330/// takes only a reason string - so this is the one signal available, and disk
331/// pressure is the one cause this module can safely re-measure without
332/// touching git, a run, or an agent CLI. Keep the two prefixes here in sync
333/// with `crate::disk::gate`'s formatted string and `daemon::disk_gate`'s own
334/// message if either changes; nothing else ties them together.
335fn is_disk_hold(task: &Task) -> bool {
336    task.hold_reason.as_deref().is_some_and(|r| {
337        r.starts_with("not enough free space to start a run:")
338            || r.starts_with("could not measure free space on ")
339    })
340}
341
342/// Has a `HoldSource::Machine` hold's cause resolved? `Some(true)` means yes -
343/// safe to requeue. `Some(false)` means the same cause was checked and is
344/// still in force. `None` means this hold's cause is not one this module
345/// knows how to re-check at all, and a human has to look.
346fn machine_cause_resolved(task: &Task, cfg: &Config) -> Option<bool> {
347    if !is_disk_hold(task) {
348        return None;
349    }
350    let min = cfg.disk.min_free_bytes;
351    if min == 0 {
352        // The operator turned the gate off since this hold was placed - its
353        // one possible cause is gone by construction, no measurement needed.
354        return Some(true);
355    }
356    let free = disk::free_bytes(&repo_for(task)).ok()?;
357    Some(disk::gate(free, min).is_none())
358}
359
360/// Is a `HoldSource::Manual` hold old enough to earn a "still wanted?"
361/// question? Same comparison `crate::clean::due` uses for a run's fold grace,
362/// against [`MANUAL_STALE_AFTER`] instead of a configured one.
363fn manual_is_stale(task: &Task, now: Timestamp) -> bool {
364    now.as_second() - task.updated_at.as_second() > MANUAL_STALE_AFTER.as_secs() as i64
365}
366
367/// The marker [`apply_answer`] writes into [`Task::hold_reason`] and
368/// [`already_applied`] reads back - see this module's own doc on why.
369fn marker_for(q: &Question) -> String {
370    format!("[triage:{}]", q.short())
371}
372
373/// Has `q`'s answer already been applied to `task`? See this module's doc.
374fn already_applied(task: &Task, q: &Question) -> bool {
375    let marker = marker_for(q);
376    task.hold_reason
377        .as_deref()
378        .is_some_and(|r| r.contains(marker.as_str()))
379}
380
381/// The most recent question this module filed for `task_id`, any status -
382/// open (still waiting), answered (may need applying), or abandoned (settled
383/// with nothing decided). Filters on both `node` and `run`, never `run`
384/// alone - see this module's doc on why a bare `run` match is not safe.
385fn latest_triage_question(questions: &Questions, task_id: &str) -> Option<Question> {
386    questions
387        .list()
388        .into_iter()
389        .filter(|q| q.node == NODE && q.run == task_id)
390        .max_by(|a, b| a.id.cmp(&b.id))
391}
392
393/// What an answered triage question's choice means, independent of which
394/// language it was filed in.
395#[derive(Debug, Clone, Copy, PartialEq, Eq)]
396enum AnswerAction {
397    /// The first choice, always "resume it" / "再開してよい" / "再開する" -
398    /// see [`Wording::choices3`] and [`Wording::choices2`], whose first entry
399    /// is always the resume choice.
400    Resume,
401    /// The third choice, present only in [`Wording::choices3`]: "discard it"
402    /// / "捨ててよい".
403    Discard,
404    /// Anything else: the second choice ("not yet" / "keep it held"), or an
405    /// answer that does not match any offered choice at all (should not
406    /// happen for a multiple-choice question, but the safe default is to
407    /// keep holding rather than guess at "resume").
408    KeepHeld,
409}
410
411/// Read `q`'s answer as an [`AnswerAction`].
412///
413/// Matched by **position in `q.choices`**, never by comparing the answer text
414/// against [`Wording`]'s own strings: [`Wording`] is picked from the task's
415/// *current* repository config, which can differ from whatever language was
416/// in force when the question was filed (a `--config` override on one `magi
417/// task triage` call and not the next, or the repository's config edited in
418/// between). Comparing text would then silently misread an actual "resume"
419/// answer as "keep held" - the choices themselves are fixed at filing time,
420/// in [`file_question`], and never change afterwards, so their position is
421/// the one thing that stays true regardless of which language reads them
422/// back.
423fn interpret_answer(q: &Question) -> AnswerAction {
424    let resolution = q.resolution().unwrap_or_default();
425    match q.choices.iter().position(|c| *c == resolution) {
426        Some(0) => AnswerAction::Resume,
427        Some(2) => AnswerAction::Discard,
428        _ => AnswerAction::KeepHeld,
429    }
430}
431
432/// The note [`already_applied`] looks for, appended to (never replacing)
433/// whatever [`Task::hold_reason`] already said - the original cause is still
434/// worth reading in `magi task show` after the operator answers "not yet",
435/// and [`Task::hold_manual`] would otherwise overwrite it outright.
436fn keep_held_note(task: &Task, q: &Question, resolution: &str) -> String {
437    let marker = format!("{} operator: {resolution}", marker_for(q));
438    match task.hold_reason.as_deref() {
439        Some(existing) if !existing.is_empty() => format!("{existing}\n{marker}"),
440        _ => marker,
441    }
442}
443
444/// File a fresh triage question for `task` and return it. The caller is
445/// responsible for having already established there is no open one - see
446/// [`latest_triage_question`] - so this never checks again.
447fn file_question(
448    questions: &Questions,
449    task: &Task,
450    bucket: Bucket,
451    w: &Wording,
452    now: Timestamp,
453) -> Option<Question> {
454    let (summary, why, choices) = match bucket {
455        Bucket::MachineUnknown => (
456            w.summary_machine_unknown(task),
457            w.why_machine(),
458            w.choices3(),
459        ),
460        Bucket::Legacy => (w.summary_legacy(task), w.why_legacy(), w.choices3()),
461        Bucket::ManualStale => {
462            let days = (now.as_second() - task.updated_at.as_second()) / (24 * 60 * 60);
463            (
464                w.summary_manual_stale(task, days),
465                w.why_manual(),
466                w.choices2(),
467            )
468        }
469    };
470    let mut q = Question::new(
471        task.id.clone(),
472        NODE.to_owned(),
473        SEAT.to_owned(),
474        summary,
475        w.detail(task, why),
476        choices,
477    );
478    questions.put(&mut q).ok()?;
479    Some(q)
480}
481
482/// Run one deterministic triage pass over every `held` task in `queue`. No
483/// model call anywhere in this function - see this module's own doc for what
484/// each `HoldSource` gets instead.
485///
486/// `config_override` is threaded straight to [`Config::discover`], the same
487/// role `daemon::Opts::config` plays for `daemon::prepare` - an explicit
488/// `--config` from the caller, or `None` to let each task's own repository
489/// pick its layers.
490///
491/// Safe to call on every daemon idle tick and from `magi task triage` alike:
492/// a task already answered and applied is left alone (see
493/// [`already_applied`]), and a task with an open question is left alone too,
494/// so repeated calls with nothing new to say do nothing.
495pub fn run_once(
496    queue: &Queue,
497    questions: &Questions,
498    config_override: Option<&Path>,
499    now: Timestamp,
500) -> Report {
501    let mut report = Report::default();
502    for listed in queue.list() {
503        if listed.status != TaskStatus::Held {
504            continue;
505        }
506        let Ok(_claim) = queue.claim(&listed.id) else {
507            continue;
508        };
509        let Ok(mut task) = queue.get(&listed.id) else {
510            continue;
511        };
512        // Re-read under the claim: a release or a re-hold landed by a human
513        // between the listing above and the claim just taken must not be
514        // clobbered by a decision based on the stale copy.
515        if task.status != TaskStatus::Held {
516            continue;
517        }
518
519        let cfg = Config::discover(&repo_for(&task), config_override)
520            .ok()
521            .map(|(c, _)| c);
522        let w = wording(cfg.as_ref().map_or("en", |c| c.graph.language.as_str()));
523
524        if let Some(q) = latest_triage_question(questions, &task.id) {
525            if q.status.open() {
526                // Already asked, still waiting - nothing to do this pass.
527                continue;
528            }
529            if q.status == QuestionStatus::Answered && !already_applied(&task, &q) {
530                match interpret_answer(&q) {
531                    AnswerAction::Resume => {
532                        task.release();
533                        if queue.put(&mut task).is_ok() {
534                            report.answered.push(task.id.clone());
535                        }
536                    }
537                    AnswerAction::Discard => {
538                        if queue.remove(&task.id, false).is_ok() {
539                            report.answered.push(task.id.clone());
540                        }
541                    }
542                    AnswerAction::KeepHeld => {
543                        let resolution = q.resolution().unwrap_or_default();
544                        let note = keep_held_note(&task, &q, &resolution);
545                        task.hold_manual(Some(note));
546                        if queue.put(&mut task).is_ok() {
547                            report.answered.push(task.id.clone());
548                        }
549                    }
550                }
551                continue;
552            }
553            // Abandoned, or an already-applied answer: fall through to the
554            // ordinary per-source handling below, which is how a stale
555            // `HoldSource::Manual` re-ask - or a fresh machine/legacy
556            // question, once a prior one settled the task back into a hold -
557            // gets filed.
558        }
559
560        match task.hold_source {
561            Some(HoldSource::Machine) => {
562                if cfg.as_ref().and_then(|c| machine_cause_resolved(&task, c)) == Some(true) {
563                    task.release();
564                    if queue.put(&mut task).is_ok() {
565                        report.resumed.push(task.id.clone());
566                    }
567                } else if file_question(questions, &task, Bucket::MachineUnknown, w, now).is_some()
568                {
569                    report.asked.push(task.id.clone());
570                }
571            }
572            None => {
573                if file_question(questions, &task, Bucket::Legacy, w, now).is_some() {
574                    report.asked.push(task.id.clone());
575                }
576            }
577            Some(HoldSource::Manual) => {
578                if manual_is_stale(&task, now)
579                    && file_question(questions, &task, Bucket::ManualStale, w, now).is_some()
580                {
581                    report.asked.push(task.id.clone());
582                }
583            }
584        }
585    }
586    report
587}
588
589/// The open triage question about `task_id`, if any - what `magi task show`
590/// prints so a held task's card names the question waiting on it, not only
591/// its hold reason. `None` once it is answered or abandoned: nothing is
592/// waiting on it anymore.
593pub fn open_question_for(questions: &Questions, task_id: &str) -> Option<Question> {
594    latest_triage_question(questions, task_id).filter(|q| q.status.open())
595}
596
597/// Does this module still have unfinished business with `task`?
598///
599/// True while its latest triage question is still open (waiting on an
600/// answer), and true for a beat longer than [`open_question_for`] alone
601/// would say: once answered, the question sits [`QuestionStatus::Answered`]
602/// but unread until the next [`run_once`] pass actually applies it (see
603/// [`already_applied`]), and [`run_once`] only ever runs on a fully idle
604/// daemon tick - far less often than `crate::conduct` polls. A caller that
605/// only checked "is a question open" would walk straight through that gap
606/// the moment the operator answers, moving the task out of `held` before
607/// [`run_once`] gets a turn - orphaning the very answer it was about to
608/// apply, the same failure mode this function exists to keep `crate::conduct`
609/// out of. `crate::conduct::apply_one` is exactly that caller.
610pub fn pending_for(questions: &Questions, task: &Task) -> bool {
611    match latest_triage_question(questions, &task.id) {
612        Some(q) if q.status.open() => true,
613        Some(q) if q.status == QuestionStatus::Answered => !already_applied(task, &q),
614        _ => false,
615    }
616}
617
618/// Every task id with an open triage question right now - what `magi task
619/// list` uses to mark a held task that is already waiting on an operator
620/// decision, rather than have it read identically to one nobody has looked
621/// at yet.
622pub fn open_task_ids(questions: &Questions) -> std::collections::BTreeSet<String> {
623    questions
624        .list()
625        .into_iter()
626        .filter(|q| q.node == NODE && q.status.open())
627        .map(|q| q.run)
628        .collect()
629}
630
631#[cfg(test)]
632mod tests {
633    use super::*;
634    use crate::ask::Answer;
635    use crate::queue::Source;
636    use jiff::SignedDuration;
637
638    fn store() -> (tempfile::TempDir, Queue, Questions) {
639        let dir = tempfile::tempdir().unwrap();
640        let q = Queue::at(dir.path().join("queue"));
641        let s = Questions::at(dir.path().join("questions"));
642        (dir, q, s)
643    }
644
645    fn task(title: &str, repo: PathBuf) -> Task {
646        Task::new(title.to_owned(), format!("do {title}"), repo, Source::Human)
647    }
648
649    /// `[disk] min_free_bytes = 0` written next to a fictional repo, so
650    /// `machine_cause_resolved` never has to ask the real disk anything - the
651    /// same fixture pattern `daemon`'s own idle-loop tests use.
652    fn gate_disabled_config(dir: &std::path::Path) -> PathBuf {
653        let config = dir.join("magi.toml");
654        std::fs::write(&config, "[disk]\nmin_free_bytes = 0\n").unwrap();
655        config
656    }
657
658    #[test]
659    fn a_resolved_machine_hold_is_requeued_automatically() {
660        let (dir, q, questions) = store();
661        let config = gate_disabled_config(dir.path());
662        let mut t = task("disk pressure", dir.path().join("repo"));
663        t.hold_machine(Some(
664            "not enough free space to start a run: 10 bytes free, 100 required by \
665             `[disk] min_free_bytes`"
666                .to_owned(),
667        ));
668        q.put(&mut t).unwrap();
669
670        let report = run_once(&q, &questions, Some(&config), Timestamp::now());
671        assert_eq!(report.resumed, [t.id.clone()]);
672        assert!(report.asked.is_empty());
673
674        let back = q.get(&t.id).unwrap();
675        assert_eq!(back.status, TaskStatus::Queued);
676        assert!(back.hold_source.is_none());
677        assert!(questions.list().is_empty(), "nothing needed asking");
678    }
679
680    #[test]
681    fn a_machine_hold_with_no_recognised_cause_gets_one_question_not_two() {
682        let (dir, q, questions) = store();
683        let mut t = task("gate went red", dir.path().join("repo"));
684        t.hold_machine(Some("gate red".to_owned()));
685        q.put(&mut t).unwrap();
686
687        let first = run_once(&q, &questions, None, Timestamp::now());
688        assert_eq!(first.asked, [t.id.clone()]);
689        assert!(first.resumed.is_empty());
690
691        let open: Vec<_> = questions
692            .list()
693            .into_iter()
694            .filter(|q| q.status.open())
695            .collect();
696        assert_eq!(open.len(), 1);
697        assert_eq!(open[0].run, t.id);
698        assert_eq!(open[0].node, NODE);
699        assert_eq!(open[0].choices.len(), 3);
700
701        // A second pass with nothing new must not file a second question.
702        let second = run_once(&q, &questions, None, Timestamp::now());
703        assert!(second.asked.is_empty());
704        assert_eq!(
705            questions
706                .list()
707                .into_iter()
708                .filter(|q| q.status.open())
709                .count(),
710            1
711        );
712    }
713
714    #[test]
715    fn a_legacy_hold_with_no_recorded_source_gets_exactly_one_question() {
716        let (dir, q, questions) = store();
717        let mut t = task("schema 1 record", dir.path().join("repo"));
718        t.status = TaskStatus::Held;
719        assert!(t.hold_source.is_none(), "the case this test is about");
720        q.put(&mut t).unwrap();
721
722        let first = run_once(&q, &questions, None, Timestamp::now());
723        assert_eq!(first.asked, [t.id.clone()]);
724
725        let second = run_once(&q, &questions, None, Timestamp::now());
726        assert!(
727            second.asked.is_empty(),
728            "the same legacy hold must not be asked about twice"
729        );
730        assert_eq!(
731            questions
732                .list()
733                .into_iter()
734                .filter(|q| q.status.open())
735                .count(),
736            1
737        );
738    }
739
740    #[test]
741    fn a_manual_hold_is_never_auto_resumed() {
742        let (dir, q, questions) = store();
743        let mut t = task("operator stopped this", dir.path().join("repo"));
744        t.hold_manual(Some("waiting on a decision".to_owned()));
745        q.put(&mut t).unwrap();
746
747        let report = run_once(&q, &questions, None, Timestamp::now());
748        assert!(report.resumed.is_empty());
749        // Fresh, not stale yet - no question either.
750        assert!(report.asked.is_empty());
751
752        let back = q.get(&t.id).unwrap();
753        assert_eq!(back.status, TaskStatus::Held);
754        assert_eq!(back.hold_source, Some(HoldSource::Manual));
755        assert!(questions.list().is_empty());
756    }
757
758    #[test]
759    fn a_stale_manual_hold_earns_a_two_choice_question() {
760        let (dir, q, questions) = store();
761        let mut t = task("been sitting a while", dir.path().join("repo"));
762        t.hold_manual(Some("waiting on a decision".to_owned()));
763        q.put(&mut t).unwrap();
764        // Back-date the hold past MANUAL_STALE_AFTER without waiting a week.
765        let mut back = q.get(&t.id).unwrap();
766        back.updated_at = Timestamp::now() - SignedDuration::new(8 * 24 * 60 * 60, 0);
767        std::fs::write(
768            q.path_of(&back.id),
769            serde_json::to_string_pretty(&back).unwrap(),
770        )
771        .unwrap();
772
773        let report = run_once(&q, &questions, None, Timestamp::now());
774        assert_eq!(report.asked, [t.id.clone()]);
775        let open: Vec<_> = questions
776            .list()
777            .into_iter()
778            .filter(|q| q.status.open())
779            .collect();
780        assert_eq!(open.len(), 1);
781        assert_eq!(open[0].choices.len(), 2);
782    }
783
784    #[test]
785    fn answering_resume_releases_the_task() {
786        let (dir, q, questions) = store();
787        let mut t = task("gate went red", dir.path().join("repo"));
788        t.hold_machine(Some("gate red".to_owned()));
789        q.put(&mut t).unwrap();
790        run_once(&q, &questions, None, Timestamp::now());
791
792        let mut asked = questions
793            .list()
794            .into_iter()
795            .find(|q| q.run == t.id)
796            .unwrap();
797        asked.answer(Answer::Choice(EN.resume.to_owned())).unwrap();
798        questions.put(&mut asked).unwrap();
799
800        let report = run_once(&q, &questions, None, Timestamp::now());
801        assert_eq!(report.answered, [t.id.clone()]);
802        let back = q.get(&t.id).unwrap();
803        assert_eq!(back.status, TaskStatus::Queued);
804        assert!(back.hold_source.is_none());
805    }
806
807    #[test]
808    fn answering_not_yet_keeps_it_held_as_a_manual_hold_and_does_not_reapply() {
809        let (dir, q, questions) = store();
810        let mut t = task("gate went red", dir.path().join("repo"));
811        t.hold_machine(Some("gate red".to_owned()));
812        q.put(&mut t).unwrap();
813        run_once(&q, &questions, None, Timestamp::now());
814
815        let mut asked = questions
816            .list()
817            .into_iter()
818            .find(|q| q.run == t.id)
819            .unwrap();
820        asked.answer(Answer::Choice(EN.wait.to_owned())).unwrap();
821        questions.put(&mut asked).unwrap();
822
823        let report = run_once(&q, &questions, None, Timestamp::now());
824        assert_eq!(report.answered, [t.id.clone()]);
825        let back = q.get(&t.id).unwrap();
826        assert_eq!(back.status, TaskStatus::Held);
827        assert_eq!(back.hold_source, Some(HoldSource::Manual));
828        assert!(
829            back.hold_reason
830                .as_deref()
831                .is_some_and(|r| r.contains("gate red")),
832            "the original cause must survive a \"not yet\" answer, not just the \
833             triage marker: {:?}",
834            back.hold_reason
835        );
836
837        // A third pass, same instant: the manual hold is fresh (just
838        // touched), so nothing more happens - in particular the already
839        // answered question is not re-applied.
840        let third = run_once(&q, &questions, None, Timestamp::now());
841        assert!(third.answered.is_empty());
842        assert!(third.asked.is_empty());
843    }
844
845    #[test]
846    fn answering_discard_removes_the_task_entirely() {
847        let (dir, q, questions) = store();
848        let mut t = task("gate went red", dir.path().join("repo"));
849        t.hold_machine(Some("gate red".to_owned()));
850        q.put(&mut t).unwrap();
851        run_once(&q, &questions, None, Timestamp::now());
852
853        let mut asked = questions
854            .list()
855            .into_iter()
856            .find(|q| q.run == t.id)
857            .unwrap();
858        asked.answer(Answer::Choice(EN.discard.to_owned())).unwrap();
859        questions.put(&mut asked).unwrap();
860
861        let report = run_once(&q, &questions, None, Timestamp::now());
862        assert_eq!(report.answered, [t.id.clone()]);
863        assert!(
864            q.get(&t.id).is_err(),
865            "\"discard it\" (捨ててよい) must actually discard the task, not \
866             just leave it sitting held forever"
867        );
868    }
869
870    #[test]
871    fn an_answer_is_read_by_its_position_in_choices_not_by_the_callers_current_language() {
872        // Filed while the repo's config reads Japanese...
873        let (dir, q, questions) = store();
874        let ja_config = dir.path().join("ja.toml");
875        std::fs::write(&ja_config, "[graph]\nlanguage = \"ja\"\n").unwrap();
876        let mut t = task("gate went red", dir.path().join("repo"));
877        t.hold_machine(Some("gate red".to_owned()));
878        q.put(&mut t).unwrap();
879        run_once(&q, &questions, Some(&ja_config), Timestamp::now());
880
881        let mut asked = questions
882            .list()
883            .into_iter()
884            .find(|q| q.run == t.id)
885            .unwrap();
886        assert_eq!(asked.choices[0], JA.resume, "filed in Japanese");
887        asked.answer(Answer::Choice(JA.resume.to_owned())).unwrap();
888        questions.put(&mut asked).unwrap();
889
890        // ...but applied against an English config (a later `--config`, or an
891        // edited repository config). The Japanese "再開してよい" answer must
892        // still be read as a resume, not silently misread as "keep held"
893        // because it fails a text comparison against the English wording.
894        let en_config = dir.path().join("en.toml");
895        std::fs::write(&en_config, "[graph]\nlanguage = \"en\"\n").unwrap();
896        let report = run_once(&q, &questions, Some(&en_config), Timestamp::now());
897        assert_eq!(report.answered, [t.id.clone()]);
898        let back = q.get(&t.id).unwrap();
899        assert_eq!(
900            back.status,
901            TaskStatus::Queued,
902            "a resume answer must resume the task regardless of which \
903             language it is read back in"
904        );
905    }
906
907    #[test]
908    fn a_question_falls_back_to_last_error_when_hold_reason_was_never_set() {
909        // `Task::fail` - the ordinary "out of attempts" machine hold - only
910        // ever sets `last_error`, never `hold_reason`. The question detail
911        // must still name a cause rather than reading "(none recorded)".
912        let (dir, q, questions) = store();
913        let mut t = task("kept failing the gate", dir.path().join("repo"));
914        t.start("run-1".to_owned());
915        t.fail("gate red three times running", 1);
916        assert_eq!(t.status, TaskStatus::Held);
917        assert!(t.hold_reason.is_none(), "the case this test is about");
918        q.put(&mut t).unwrap();
919
920        run_once(&q, &questions, None, Timestamp::now());
921        let asked = questions
922            .list()
923            .into_iter()
924            .find(|q| q.run == t.id)
925            .unwrap();
926        assert!(
927            asked.detail.contains("gate red three times running"),
928            "the question must surface `last_error` when there is no \
929             `hold_reason` to show instead: {}",
930            asked.detail
931        );
932    }
933
934    #[test]
935    fn open_question_for_and_open_task_ids_reflect_only_what_is_still_waiting() {
936        let (dir, q, questions) = store();
937        let mut t = task("gate went red", dir.path().join("repo"));
938        t.hold_machine(Some("gate red".to_owned()));
939        q.put(&mut t).unwrap();
940
941        assert!(open_question_for(&questions, &t.id).is_none());
942        assert!(!open_task_ids(&questions).contains(&t.id));
943
944        run_once(&q, &questions, None, Timestamp::now());
945        assert!(open_question_for(&questions, &t.id).is_some());
946        assert!(open_task_ids(&questions).contains(&t.id));
947
948        let mut asked = questions
949            .list()
950            .into_iter()
951            .find(|q| q.run == t.id)
952            .unwrap();
953        asked.answer(Answer::Choice(EN.resume.to_owned())).unwrap();
954        questions.put(&mut asked).unwrap();
955
956        assert!(
957            open_question_for(&questions, &t.id).is_none(),
958            "an answered question is no longer open"
959        );
960        assert!(!open_task_ids(&questions).contains(&t.id));
961    }
962
963    #[test]
964    fn pending_for_stays_true_between_an_answer_and_the_next_run_once_pass() {
965        // `open_question_for` alone goes `None` the instant the operator
966        // answers, well before `run_once` - idle-tick only - gets a turn to
967        // actually apply that answer (see `already_applied`). `pending_for`
968        // exists so a caller polling far more often than `run_once` does -
969        // `crate::conduct::apply_one` - does not walk through that gap.
970        let (dir, q, questions) = store();
971        let mut t = task("gate went red", dir.path().join("repo"));
972        t.hold_machine(Some("gate red".to_owned()));
973        q.put(&mut t).unwrap();
974
975        assert!(!pending_for(&questions, &t));
976
977        run_once(&q, &questions, None, Timestamp::now());
978        let held = q.get(&t.id).unwrap();
979        assert!(pending_for(&questions, &held), "still waiting on an answer");
980
981        let mut asked = questions
982            .list()
983            .into_iter()
984            .find(|q| q.run == t.id)
985            .unwrap();
986        asked.answer(Answer::Choice(EN.wait.to_owned())).unwrap();
987        questions.put(&mut asked).unwrap();
988        assert!(!asked.status.open());
989
990        // The race window: answered, but `run_once` has not run again yet.
991        let still_held = q.get(&t.id).unwrap();
992        assert!(
993            pending_for(&questions, &still_held),
994            "answered but not yet applied is still pending"
995        );
996
997        run_once(&q, &questions, None, Timestamp::now());
998        let after = q.get(&t.id).unwrap();
999        assert!(
1000            !pending_for(&questions, &after),
1001            "the answer is applied now, nothing left pending"
1002        );
1003    }
1004}