Skip to main content

runner_manager_platform/
service.rs

1// owner: d3-service-installers
2
3//! Registering `daemon run` with the operating system, so that a home host
4//! resumes work by itself after a reboot with nobody logged in.
5//!
6//! Journey 5 is the whole of the requirement: *"the machine reboots with nobody
7//! logged in; the boot-start service starts the agent, which reads the user
8//! access token from the machine-scoped secret store"*. Everything in this file
9//! exists to make that sentence true on Windows, macOS, and Linux, and to make
10//! `service status` say so — or say precisely why not.
11//!
12//! # The shape: render, then apply
13//!
14//! A service manager is the one dependency a `cargo test` run cannot have. So
15//! this module is split in two, and the split is the reason most of it is
16//! testable on a developer's laptop whatever OS that laptop runs:
17//!
18//! * **Rendering** is pure. [`ServiceDefinition`] turns an [`InstallPlan`] into
19//!   the exact text the platform consumes — a systemd unit, a launchd property
20//!   list, a Task Scheduler XML document, or a canonical descriptor of the
21//!   Windows service parameters. No `cfg`, no privileges, no I/O. A Windows
22//!   developer renders and asserts the systemd unit; a Linux CI leg renders and
23//!   asserts the launchd plist.
24//! * **Applying** is behind [`ServiceControl`], one trait with a backend per
25//!   platform and a public in-memory double ([`RecordingControls`]) that `f3`
26//!   and this crate's own tests drive without touching the host.
27//!
28//! [`ServiceOperations`] is the layer above both, and it is where the logic
29//! that is *not* platform-specific lives: refusing an install while the
30//! single-instance lock is held, recording the resolved absolute binary path,
31//! detecting a stale one, switching start mode without reinstalling, and
32//! uninstalling without deleting a byte of configuration, secrets, or cache.
33//!
34//! # Two domains, not two products
35//!
36//! `--start-at boot` and `--start-at login` are the same daemon registered in
37//! two different *domains*:
38//!
39//! | | boot | login |
40//! |---|---|---|
41//! | Windows | a service in the Service Control Manager, `LocalSystem` | a Task Scheduler task with a logon trigger, running at `LeastPrivilege` |
42//! | macOS | a LaunchDaemon in `/Library/LaunchDaemons` | a LaunchAgent in `~/Library/LaunchAgents` |
43//! | Linux | a system unit in `/etc/systemd/system` | a user unit in `~/.config/systemd/user` |
44//!
45//! The Windows row is the one that is not symmetric, and it is worth saying why
46//! rather than leaving a reader to wonder. **Windows services cannot start at
47//! logon.** Service trigger-start covers domain join, an IP address becoming
48//! available, a device arriving, a firewall port event and a group-policy
49//! change; there is no logon trigger, and there is no user-session service
50//! type this product could use instead. Task Scheduler is the mechanism Windows
51//! actually provides for "run this when the operator signs in", so that is what
52//! `--start-at login` uses there. It is registered, inspected, and removed
53//! through the same [`ServiceControl`] trait as everything else, so nothing
54//! above this module has to know.
55//!
56//! # What the account can reach, and why it is not more
57//!
58//! `05-infrastructure.md` requires *"a least-privilege account that can read the
59//! machine-scoped secret store and write its configured cache and runtime
60//! directories"*. Those two clauses pull in opposite directions on every
61//! platform, and the resolution is recorded per platform in
62//! `docs/service-account.md` and checked by [`review_least_privilege`], which
63//! reads the rendered definition back and reports anything it grants beyond the
64//! requirement.
65//!
66//! The Windows resolution is the one that surprises people. `d2` protects the
67//! machine-scoped store with `D:P(A;;FA;;;SY)(A;;FA;;;BA)(A;;FA;;;OW)` and
68//! documents that the DACL *is* the access control, because a machine-scope
69//! DPAPI blob is decryptable by any process on the host. `NT AUTHORITY\
70//! LocalService` and `NT AUTHORITY\NetworkService` — the two accounts a
71//! "least privilege service" reflex reaches for — are named by none of those
72//! three ACEs and therefore **cannot read the token at all**. Widening the DACL
73//! to reach them would grant every service on the host read access to the one
74//! credential this product holds, which is strictly worse than running as
75//! `LocalSystem`. `secrets.rs` is not this task's file and is not widened; the
76//! service runs as `LocalSystem`, and [`review_least_privilege`] records the
77//! account together with the reason it is the minimum that satisfies the
78//! requirement rather than pretending it is small.
79//!
80//! # What is deliberately not claimed
81//!
82//! **A reboot is not something a test suite can have.** Every assertion here is
83//! about configuration a boot-time start depends on — the start type, the
84//! account, the recorded absolute path, the restart policy the service manager
85//! reports back — and about a store that is readable outside any login session,
86//! which `d2` proves separately. That the machine actually comes back up and
87//! the agent actually resumes is human gate 3 in `06-migration-rollout.md`, and
88//! nothing in this file is evidence for it.
89
90use std::collections::BTreeMap;
91use std::ffi::OsString;
92use std::fmt;
93use std::path::{Path, PathBuf};
94use std::time::Duration;
95
96use chrono::{DateTime, Utc};
97use runner_manager_domain::model::StartMode;
98use runner_manager_domain::path::LocalAbsolutePath;
99use serde::{Deserialize, Serialize};
100
101use crate::lock::{HostLock, LockError, LockKind};
102use crate::paths::AppPaths;
103#[cfg(windows)]
104use crate::runner_root_access::RootAdmission;
105use crate::runner_root_access::{
106    Reversal, RootAccessChange, RootAccessError, RootAccessReport, RootAccessSummary,
107};
108
109// ---------------------------------------------------------------------------
110// Names
111// ---------------------------------------------------------------------------
112
113/// The product's own service name, on every platform that wants a short one.
114pub const SERVICE_NAME: &str = "runner-manager";
115/// Hidden CLI marker carried only by Windows boot-service registrations.
116///
117/// Application-data arguments are shared by every service manager and cannot
118/// distinguish SCM from Task Scheduler. This marker is the durable routing
119/// contract between the Windows installer and the shipping binary.
120pub const WINDOWS_SCM_HOST_ARGUMENT: &str = "--windows-service-host";
121
122/// What an operator sees in `services.msc`, `launchctl list`, or
123/// `systemctl status`.
124pub const DISPLAY_NAME: &str = "GitHub Actions Runner Manager";
125
126/// One line of explanation, for the same three places.
127pub const DESCRIPTION: &str =
128    "Starts ephemeral GitHub Actions self-hosted runners on this machine on demand.";
129
130/// The arguments the installed command line carries.
131///
132/// `05-infrastructure.md`: *"`service install` registers `daemon run` for the
133/// current host"*. Spelled once, here, so the installer and `f3`'s command
134/// surface cannot disagree about what was registered.
135pub const DAEMON_ARGUMENTS: [&str; 2] = ["daemon", "run"];
136
137/// The file `install` writes and `status` reads, inside `config/`.
138///
139/// `config/` and not `state/`: `05-infrastructure.md` gives `config/` to
140/// *"non-secret TOML"*, and this is exactly that — a record of what was
141/// registered, holding no credential. It is also the one file `uninstall`
142/// removes, which is what keeps "uninstall deletes no configuration" honest:
143/// the record is not configuration, it is the registration's own footprint.
144pub const RECORD_FILE: &str = "service.toml";
145
146/// The file the daemon touches after every successful GitHub call, inside
147/// `state/`.
148///
149/// See [`record_github_contact`] for the contract; `service status` reads it
150/// and Journey 5 step 4 requires it.
151pub const CONTACT_FILE: &str = "github-contact.toml";
152
153/// Present while GitHub is rejecting the daemon's credential, inside `state/`.
154///
155/// See [`record_github_credential_rejected`] for the contract.
156pub const CREDENTIAL_REJECTION_FILE: &str = "github-credential-rejected.toml";
157
158/// Present while the daemon cannot read its own stored credential, inside
159/// `state/`.
160///
161/// See [`record_credential_unreadable`] for the contract.
162pub const CREDENTIAL_UNREADABLE_FILE: &str = "credential-unreadable.toml";
163
164/// The `service status` subject a [`CREDENTIAL_UNREADABLE_FILE`] is reported
165/// under, which the TUI matches to offer the same command.
166pub const CREDENTIAL_UNREADABLE_SUBJECT: &str = "stored credential";
167
168/// The `service status` subject blocked runner launches are reported under.
169pub const LAUNCHES_BLOCKED_SUBJECT: &str = "runner launches";
170
171/// The [`StatusProblem::subject`] of a daemon that has stopped making progress.
172pub const DAEMON_STALLED_SUBJECT: &str = "daemon progress";
173
174/// The [`StatusProblem::subject`] of a daemon that refuses to start runners
175/// because a required host check fails from its own session.
176pub const HOST_UNFIT_SUBJECT: &str = "host fitness";
177
178/// The agent's last runner-root refusal, for `service status` to report.
179///
180/// See [`record_runner_root_refusal`] for the contract.
181pub const ROOT_REFUSAL_FILE: &str = "runner-root-refusal.toml";
182
183/// The rotating diagnostic log the **daemon** writes, inside `logs/`.
184///
185/// The appender adds its own date suffix, so this is the stem rather than a
186/// file that exists. `service status` reports the directory and the stem,
187/// because that is what an operator needs in order to find today's file and
188/// yesterday's.
189///
190/// It is [`crate::logging::SERVICE_LOG_STEM`] and not the operator's, because a
191/// boot-mode daemon runs under a different account and writes a different file.
192/// [`crate::logging::LogRole`] says why the two were separated.
193pub const LOG_FILE_STEM: &str = crate::logging::SERVICE_LOG_STEM;
194
195// ---------------------------------------------------------------------------
196// Identity
197// ---------------------------------------------------------------------------
198
199/// What the operating system calls this registration.
200///
201/// A type rather than three constants, for one reason that is not tidiness: the
202/// privileged installer tests register a **real** service on a **real** machine,
203/// and they must not be able to collide with — or remove — an operator's
204/// installation. [`ServiceIdentity::fixture`] produces a name that is
205/// unmistakably a test artefact, and every backend takes its name from here, so
206/// there is no path by which a test reaches the product's own registration.
207#[derive(Debug, Clone, PartialEq, Eq)]
208pub struct ServiceIdentity {
209    name: String,
210    display_name: String,
211    description: String,
212}
213
214impl ServiceIdentity {
215    /// The registration an operator installs.
216    #[must_use]
217    pub fn product() -> Self {
218        Self {
219            name: SERVICE_NAME.to_string(),
220            display_name: DISPLAY_NAME.to_string(),
221            description: DESCRIPTION.to_string(),
222        }
223    }
224
225    /// A disposable registration for a privileged test, named so that it cannot
226    /// be mistaken for — or collide with — [`ServiceIdentity::product`].
227    ///
228    /// `tag` distinguishes concurrent runs; it is reduced to ASCII alphanumerics
229    /// and `-` so that it is a legal service name, launchd label, systemd unit
230    /// name, and Task Scheduler task name at once.
231    #[must_use]
232    pub fn fixture(tag: &str) -> Self {
233        let tag: String = tag
234            .chars()
235            .map(|c| {
236                if c.is_ascii_alphanumeric() {
237                    c.to_ascii_lowercase()
238                } else {
239                    '-'
240                }
241            })
242            .collect();
243        let name = format!("{SERVICE_NAME}-selftest-{tag}");
244        Self {
245            display_name: format!("{DISPLAY_NAME} (self-test fixture {tag})"),
246            description: "Disposable fixture created by runner-manager's own installer tests. \
247                          Safe to remove."
248                .to_string(),
249            name,
250        }
251    }
252
253    /// Whether this identity is a disposable test fixture.
254    ///
255    /// The privileged tests assert this before they delete anything, which is
256    /// the mechanical half of *"never touch a service you did not create"*.
257    #[must_use]
258    pub fn is_fixture(&self) -> bool {
259        self.name.starts_with(&format!("{SERVICE_NAME}-selftest-"))
260    }
261
262    /// The Windows service name, the systemd unit stem, and the Task Scheduler
263    /// task name.
264    #[must_use]
265    pub fn name(&self) -> &str {
266        &self.name
267    }
268
269    /// The human-readable name.
270    #[must_use]
271    pub fn display_name(&self) -> &str {
272        &self.display_name
273    }
274
275    /// One line of explanation.
276    #[must_use]
277    pub fn description(&self) -> &str {
278        &self.description
279    }
280
281    /// The launchd label — reverse-domain, as launchd expects.
282    ///
283    /// Built from the same three product segments [`crate::paths`] resolves the
284    /// application-data directories with, so a label and a directory cannot
285    /// drift apart.
286    #[must_use]
287    pub fn launchd_label(&self) -> String {
288        format!(
289            "{}.{}.{}",
290            crate::paths::QUALIFIER,
291            crate::paths::ORGANIZATION,
292            self.name
293        )
294    }
295
296    /// The systemd unit file name.
297    #[must_use]
298    pub fn systemd_unit(&self) -> String {
299        format!("{}.service", self.name)
300    }
301
302    /// The service manager's own command that restarts this registration.
303    ///
304    /// The product has no `service stop` or `service start`, and
305    /// `service install` is no restart either: it refuses while the running
306    /// daemon holds the agent lock. So a remedy that restarts the daemon names
307    /// the service manager's command. A forced restart does not wait for jobs;
308    /// it is for a daemon that has stopped making progress anyway.
309    #[must_use]
310    pub fn restart_command(&self, mode: StartMode) -> String {
311        let name = &self.name;
312        if cfg!(windows) {
313            match mode {
314                StartMode::Boot => {
315                    format!("Restart-Service -Name {name}")
316                }
317                StartMode::Login => format!("schtasks /End /TN {name}; schtasks /Run /TN {name}"),
318            }
319        } else if cfg!(target_os = "macos") {
320            let label = self.launchd_label();
321            match mode {
322                StartMode::Boot => format!("sudo launchctl kickstart -k system/{label}"),
323                StartMode::Login => format!("launchctl kickstart -k gui/$(id -u)/{label}"),
324            }
325        } else {
326            let unit = self.systemd_unit();
327            match mode {
328                StartMode::Boot => format!("sudo systemctl restart {unit}"),
329                StartMode::Login => format!("systemctl --user restart {unit}"),
330            }
331        }
332    }
333}
334
335impl fmt::Display for ServiceIdentity {
336    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
337        f.write_str(&self.name)
338    }
339}
340
341// ---------------------------------------------------------------------------
342// Restart policy
343// ---------------------------------------------------------------------------
344
345/// The restart-on-failure policy, and the bound on how fast it may retry.
346///
347/// `05-infrastructure.md` item 3: *"set a restart-on-failure policy with bounded
348/// delay"*. Both halves matter and they are different requirements. A service
349/// that does not come back after a crash defeats Journey 5; a service that comes
350/// back instantly, forever, turns one bad configuration into a fork bomb against
351/// GitHub's rate limit. So the delay has a floor as well as a ceiling, and the
352/// floor is what `does not restart-loop faster than that bound` is measured
353/// against.
354#[derive(Debug, Clone, Copy, PartialEq, Eq)]
355pub struct RestartPolicy {
356    delay: Duration,
357    reset_after: Duration,
358}
359
360impl RestartPolicy {
361    /// The shortest delay this will accept.
362    ///
363    /// One second is not a tuning preference. Below it, launchd's own throttle
364    /// and systemd's start-limit logic both take over and the configured number
365    /// stops being the number in force, so a policy under this floor would be a
366    /// value `service status` reported and the platform ignored.
367    pub const MIN_DELAY: Duration = Duration::from_secs(1);
368
369    /// The longest delay this will accept. Past five minutes an operator
370    /// watching a restart would reasonably conclude the service is simply gone.
371    pub const MAX_DELAY: Duration = Duration::from_secs(300);
372
373    /// What `service install` uses when nothing is said.
374    pub const DEFAULT_DELAY: Duration = Duration::from_secs(15);
375
376    /// How long the service must stay up before its failure count is forgotten.
377    pub const DEFAULT_RESET_AFTER: Duration = Duration::from_secs(600);
378
379    /// # Errors
380    ///
381    /// [`ServiceError::RestartDelay`] when `delay` is outside
382    /// [`RestartPolicy::MIN_DELAY`]..=[`RestartPolicy::MAX_DELAY`], or when
383    /// `reset_after` is not longer than `delay` — a reset window shorter than
384    /// the delay can never elapse between two restarts, so the failure count
385    /// would reset on every attempt and no start-limit could ever trip.
386    pub fn new(delay: Duration, reset_after: Duration) -> Result<Self, ServiceError> {
387        if delay < Self::MIN_DELAY || delay > Self::MAX_DELAY {
388            return Err(ServiceError::RestartDelay {
389                requested_secs: delay.as_secs(),
390                min_secs: Self::MIN_DELAY.as_secs(),
391                max_secs: Self::MAX_DELAY.as_secs(),
392            });
393        }
394        if reset_after <= delay {
395            return Err(ServiceError::RestartResetWindow {
396                reset_secs: reset_after.as_secs(),
397                delay_secs: delay.as_secs(),
398            });
399        }
400        Ok(Self { delay, reset_after })
401    }
402
403    /// How long the service manager waits before restarting a failed service.
404    #[must_use]
405    pub const fn delay(&self) -> Duration {
406        self.delay
407    }
408
409    /// How long the service must run before its failure count resets.
410    #[must_use]
411    pub const fn reset_after(&self) -> Duration {
412        self.reset_after
413    }
414
415    /// The delay a given manager can actually express.
416    ///
417    /// Three of the four take seconds and enforce exactly what they are given.
418    /// **Windows Task Scheduler does not**: `RestartOnFailure/Interval` is
419    /// expressed in whole minutes with a one-minute floor, and it *rejects* the
420    /// registration outright rather than rounding — `PT15S` comes back as
421    /// "The task XML contains a value which is incorrectly formatted or out of
422    /// range", which is how this was found.
423    ///
424    /// So the interval is rounded **up**, never down, and never below one
425    /// minute. The direction is the point: the requirement is that the service
426    /// *"does not restart-loop faster than that bound"*, and a delay longer
427    /// than the configured one still satisfies it, while a shorter one would
428    /// not. `service status` reports the difference as a note so that an
429    /// operator reading `15s` in the record and `60s` from the manager is told
430    /// why rather than left to wonder.
431    #[must_use]
432    pub const fn effective_delay(&self, kind: DefinitionKind) -> Duration {
433        match kind {
434            DefinitionKind::WindowsScheduledTask => {
435                let seconds = self.delay.as_secs();
436                let minutes = seconds.div_ceil(60);
437                Duration::from_secs(if minutes == 0 { 60 } else { minutes * 60 })
438            }
439            _ => self.delay,
440        }
441    }
442}
443
444impl Default for RestartPolicy {
445    fn default() -> Self {
446        Self {
447            delay: Self::DEFAULT_DELAY,
448            reset_after: Self::DEFAULT_RESET_AFTER,
449        }
450    }
451}
452
453impl fmt::Display for RestartPolicy {
454    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
455        write!(
456            f,
457            "restart after {}s, failure count resets after {}s",
458            self.delay.as_secs(),
459            self.reset_after.as_secs()
460        )
461    }
462}
463
464// ---------------------------------------------------------------------------
465// The account
466// ---------------------------------------------------------------------------
467
468/// The account a registration runs under.
469///
470/// Not an operator choice. It is a function of the start mode and the platform,
471/// because the start mode decides which secret store the daemon must read and
472/// the store's own access control decides which accounts can read it. See
473/// `docs/service-account.md`, and [`review_least_privilege`] for the check.
474#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
475#[serde(rename_all = "snake_case")]
476pub enum ServiceAccount {
477    /// `NT AUTHORITY\SYSTEM`. The only stock Windows account named by the
478    /// machine-scoped store's DACL.
479    LocalSystem,
480    /// `root`. What a LaunchDaemon and a systemd system unit run as, and what
481    /// the macOS System Keychain's root-only master key requires.
482    Root,
483    /// The operator's own account, for a login-mode registration.
484    InvokingUser,
485}
486
487impl ServiceAccount {
488    /// The account a given kind of definition obliges.
489    ///
490    /// Keyed on the definition rather than on `cfg!(windows)`, and the
491    /// difference is not academic: a Windows developer rendering the launchd
492    /// property list would otherwise write `UserName = NT AUTHORITY\SYSTEM`
493    /// into it, which is not an account macOS has. A definition's account is a
494    /// property of the definition, and the whole point of rendering being pure
495    /// is that any host can render any platform's and get the right answer.
496    #[must_use]
497    pub const fn for_definition(kind: DefinitionKind, mode: StartMode) -> Self {
498        match (kind, mode) {
499            (DefinitionKind::WindowsService, _) => Self::LocalSystem,
500            (DefinitionKind::WindowsScheduledTask, _) | (_, StartMode::Login) => Self::InvokingUser,
501            (_, StartMode::Boot) => Self::Root,
502        }
503    }
504
505    /// The account the given start mode obliges on the platform this binary was
506    /// built for.
507    #[must_use]
508    pub const fn for_start_mode(mode: StartMode) -> Self {
509        Self::for_definition(host_definition_kind(mode), mode)
510    }
511
512    /// How the platform spells it.
513    #[must_use]
514    pub const fn as_str(&self) -> &'static str {
515        match self {
516            Self::LocalSystem => "NT AUTHORITY\\SYSTEM",
517            Self::Root => "root",
518            Self::InvokingUser => "the invoking user",
519        }
520    }
521
522    /// Why this is the *minimum* account that can do the job, not merely the
523    /// convenient one.
524    ///
525    /// Printed by `service status` and by [`review_least_privilege`], because a
526    /// privileged account with no stated reason is indistinguishable from a
527    /// privileged account nobody thought about.
528    #[must_use]
529    pub const fn justification(&self) -> &'static str {
530        match self {
531            Self::LocalSystem => {
532                "the machine-scoped store's DACL names SY, BA and OW only; LocalService and \
533                 NetworkService cannot read it, and widening the DACL to reach them would grant \
534                 every service on this host read access to the one credential this product holds"
535            }
536            Self::Root => {
537                "a boot-time registration runs outside every login session: on macOS the System \
538                 Keychain is unlocked by /var/db/SystemKey, which is root-only, and on Linux the \
539                 machine-scoped store is a 0600 file under /var/lib that only root can open \
540                 before a session exists"
541            }
542            Self::InvokingUser => {
543                "a login-mode registration reads the user-scoped store, which is deliberately \
544                 readable by exactly one account and needs no elevation at all"
545            }
546        }
547    }
548
549    /// Whether registering under this account needs administrative rights.
550    #[must_use]
551    pub const fn needs_elevation(&self) -> bool {
552        matches!(self, Self::LocalSystem | Self::Root)
553    }
554}
555
556impl fmt::Display for ServiceAccount {
557    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
558        f.write_str(self.as_str())
559    }
560}
561
562// ---------------------------------------------------------------------------
563// Errors
564// ---------------------------------------------------------------------------
565
566/// Something went wrong installing, inspecting, or removing the registration.
567///
568/// Every variant is formatted straight into an operator-facing message, so each
569/// one says what to do next rather than only what failed. No variant carries a
570/// secret: this module never reads the token, only the *location* `d2` publishes
571/// for exactly this purpose.
572#[derive(Debug, thiserror::Error)]
573pub enum ServiceError {
574    /// The single-instance lock is held, so a second agent must not be
575    /// registered.
576    ///
577    /// `05-infrastructure.md` item 1. The message is `d1`'s, verbatim, because
578    /// `d1` already worked out what to tell an operator who lost that race.
579    #[error("cannot install the service while an agent is already running on this host: {source}")]
580    LockHeld {
581        /// What `d1` said, including who holds it.
582        #[source]
583        source: Box<LockError>,
584    },
585
586    /// The lock could not be inspected at all, which is not the same as it being
587    /// free.
588    #[error(
589        "cannot tell whether an agent is already running on this host, so the service was not \
590         installed: {source}. Fix the reported problem with the state directory and try again."
591    )]
592    LockUnreadable {
593        /// The underlying failure.
594        #[source]
595        source: Box<LockError>,
596    },
597
598    /// The runner root this registration would run jobs under could not be
599    /// created, inspected, or given the access the registration's account needs.
600    ///
601    /// Reported *before* anything is registered. A registration whose workspaces
602    /// would land in a directory ordinary local users can write is a
603    /// registration that should not exist, and `04-security-recovery.md` asks
604    /// for exactly that refusal rather than a warning.
605    #[error(
606        "this registration would run jobs under the default runner root, and that root could \
607         not be prepared, so nothing was registered: {source}"
608    )]
609    RunnerRoot {
610        /// What [`crate::runner_root_access`] reported, including the remedy.
611        #[source]
612        source: Box<RootAccessError>,
613    },
614
615    /// The path of the running binary could not be resolved.
616    #[error(
617        "cannot resolve the absolute path of this executable, so there is nothing to register: \
618         {detail}. Run the installer from the installed binary rather than through a shell \
619         function or a wrapper that replaces argv[0]."
620    )]
621    BinaryPath {
622        /// What the platform reported.
623        detail: String,
624    },
625
626    /// The binary named for the registration is not there.
627    #[error(
628        "{} is not a file, so registering it would create a service that cannot start. Install \
629         the product first, then run `service install` from the installed binary.",
630        path.display()
631    )]
632    BinaryMissing {
633        /// The path that was to be registered.
634        path: PathBuf,
635    },
636
637    /// The requested restart delay is outside the bound.
638    #[error(
639        "a restart delay of {requested_secs}s is outside the supported range \
640         {min_secs}s-{max_secs}s. Below the floor the platform's own throttle overrides the \
641         value, so `service status` would report a delay that is not the one in force."
642    )]
643    RestartDelay {
644        /// What was asked for.
645        requested_secs: u64,
646        /// The floor.
647        min_secs: u64,
648        /// The ceiling.
649        max_secs: u64,
650    },
651
652    /// The failure-count reset window is not longer than the restart delay.
653    #[error(
654        "a failure-count reset window of {reset_secs}s is not longer than the {delay_secs}s \
655         restart delay, so the count would reset between every pair of restarts and no \
656         start limit could ever apply."
657    )]
658    RestartResetWindow {
659        /// What was asked for.
660        reset_secs: u64,
661        /// The delay it has to exceed.
662        delay_secs: u64,
663    },
664
665    /// A registration is already there, in the **other** start mode.
666    ///
667    /// Only the other one. A registration in the mode being asked for is
668    /// replaced rather than refused — see [`Operations::install`], and the
669    /// outage that taught us why.
670    #[error(
671        "{name} is already registered to start at {existing}, and this asks for {requested}. \
672         Changing the start mode moves the registration between two different service \
673         managers, so it is not something an install does by itself: run `service uninstall` \
674         first, or change the start mode in the terminal UI (`runner-manager tui`), which \
675         switches it in place and keeps the registration running throughout."
676    )]
677    AlreadyInstalled {
678        /// Which registration.
679        name: String,
680        /// The mode it currently carries.
681        existing: StartMode,
682        /// The mode this install asked for.
683        requested: StartMode,
684    },
685
686    /// No registration is there.
687    #[error("{name} is not registered on this host, so there is nothing to {operation}.")]
688    NotInstalled {
689        /// Which registration.
690        name: String,
691        /// What was being attempted.
692        operation: &'static str,
693    },
694
695    /// The record `install` wrote could not be read or written.
696    #[error("cannot {operation} the service record {}: {detail}", path.display())]
697    Record {
698        /// Read, write, or remove.
699        operation: &'static str,
700        /// The record file.
701        path: PathBuf,
702        /// The underlying failure.
703        detail: String,
704    },
705
706    /// The record is there but is not one this version wrote.
707    #[error(
708        "the service record {} was not written by this product, or was written by a version \
709         this one cannot read: {detail}. Run `service uninstall` and `service install` again; \
710         neither touches configuration, secrets, or the cache.",
711        path.display()
712    )]
713    RecordUnreadable {
714        /// The record file.
715        path: PathBuf,
716        /// What was wrong with it.
717        detail: String,
718    },
719
720    /// The record is there and **this account** may not read it.
721    ///
722    /// Its own variant rather than a [`ServiceError::Record`] with a
723    /// `Permission denied` in the detail, because it is the one record failure
724    /// that is not a fault in the record: `service status` reports it and
725    /// carries on, where every other read failure ends the command. See
726    /// [`Operations::status`].
727    ///
728    /// A version before this one wrote the record at `0600` through a temporary
729    /// file, so a boot-mode `sudo service install` left it owned by `root` and
730    /// unreadable by the operator whose profile it is in. `service status` then
731    /// failed on their own host, with a remedy naming the command that had just
732    /// failed.
733    #[error(
734        "the service record {} is there and this account may not read it: {detail}. It was \
735         written by whichever account installed the service -- on a boot-mode host, `sudo \
736         service install`. Running `service install` again rewrites it readable; until then \
737         only what the service manager itself reports is available.",
738        path.display()
739    )]
740    RecordNotPermitted {
741        /// The record file.
742        path: PathBuf,
743        /// What the operating system said.
744        detail: String,
745    },
746
747    /// The application-data directories could not be resolved or created.
748    #[error("cannot prepare this host's application-data directories: {source}")]
749    Paths {
750        /// The underlying failure.
751        #[source]
752        source: Box<crate::paths::PathsError>,
753    },
754
755    /// The platform's service manager refused, or could not be reached.
756    #[error("cannot {operation} {name} through {manager}: {detail}")]
757    Control {
758        /// What was being attempted.
759        operation: &'static str,
760        /// Which registration.
761        name: String,
762        /// Which service manager.
763        manager: &'static str,
764        /// What it said.
765        detail: String,
766    },
767
768    /// An operation failed and its compensating action failed too.
769    #[error(
770        "cannot {operation} {name}: {cause}. The attempted rollback also failed: {rollback}. \
771         Inspect `service status` before retrying."
772    )]
773    Rollback {
774        /// The transaction that could not be completed.
775        operation: &'static str,
776        /// Which registration was involved.
777        name: String,
778        /// The original failure.
779        cause: String,
780        /// The failure while restoring the previous state.
781        rollback: String,
782    },
783
784    /// The machine's registration is another installation's (see
785    /// [`ForeignService`]), and changing it from this account would act on it
786    /// with this account's directories.
787    #[error(
788        "this machine's {name} service was installed from {installed_by}, not from this \
789         account's data root, so this account does not {operation} it: doing so would \
790         re-register or remove that installation's service with this account's directories. \
791         Do it {act_as}. If that account no longer exists, remove the registration with the \
792         service manager itself first."
793    )]
794    ForeignInstallation {
795        /// Which registration.
796        name: String,
797        /// What was refused.
798        operation: &'static str,
799        /// [`ForeignService::installed_by`].
800        installed_by: String,
801        /// [`ForeignService::act_as`].
802        act_as: String,
803    },
804
805    /// `host forget-local` found this account's configuration in use, and
806    /// moved nothing.
807    #[error("this account's local configuration is in use, so nothing was archived: {detail}.")]
808    LocalConfigurationInUse {
809        /// What uses it.
810        detail: String,
811    },
812
813    /// The operation needs administrative rights it does not have.
814    #[error("{operation} {name} needs administrative rights: {detail}. {remedy}")]
815    NeedsElevation {
816        /// What was being attempted.
817        operation: &'static str,
818        /// Which registration.
819        name: String,
820        /// What the platform said.
821        detail: String,
822        /// How to get them, in this platform's own terms.
823        remedy: &'static str,
824    },
825}
826
827// ---------------------------------------------------------------------------
828// Directories
829// ---------------------------------------------------------------------------
830
831/// The four directories the registration was installed against.
832///
833/// Recorded rather than re-derived, because `05-infrastructure.md` item 2 speaks
834/// of the *configured* cache and runtime directories: the account the service
835/// runs under and the account that ran `service install` do not always resolve
836/// the same platform-standard locations, and a claim about what the service can
837/// write is only checkable against a specific set of paths.
838#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
839pub struct ServiceDirectories {
840    /// Non-secret TOML and the SQLite database.
841    pub config: PathBuf,
842    /// The agent lock, the attempt journal, and the runner package cache.
843    pub state: PathBuf,
844    /// Per-attempt disposable runner workspaces.
845    pub runtime: PathBuf,
846    /// Rotating redacted diagnostics.
847    pub logs: PathBuf,
848}
849
850impl ServiceDirectories {
851    /// Snapshots the four directories a resolved [`AppPaths`] names.
852    #[must_use]
853    pub fn of(paths: &AppPaths) -> Self {
854        Self {
855            config: paths.config_dir().to_path_buf(),
856            state: paths.state_dir().to_path_buf(),
857            runtime: paths.runtime_dir().to_path_buf(),
858            logs: paths.logs_dir().to_path_buf(),
859        }
860    }
861
862    /// The four, in the order `05-infrastructure.md` lists them.
863    #[must_use]
864    pub fn all(&self) -> [&Path; 4] {
865        [
866            self.config.as_path(),
867            self.state.as_path(),
868            self.runtime.as_path(),
869            self.logs.as_path(),
870        ]
871    }
872
873    /// The rotating diagnostic log's stem, which is what `service status`
874    /// reports and what `05-infrastructure.md` item 4 requires be preserved.
875    #[must_use]
876    pub fn log_file(&self) -> PathBuf {
877        self.logs.join(LOG_FILE_STEM)
878    }
879
880    /// The four as the hidden `daemon run` options a registration passes them
881    /// in, [`SERVICE_DIRECTORY_OPTIONS`] order.
882    #[must_use]
883    pub fn arguments(&self) -> [OsString; 8] {
884        let [config, state, runtime, logs] = SERVICE_DIRECTORY_OPTIONS.map(OsString::from);
885        [
886            config,
887            self.config.clone().into_os_string(),
888            state,
889            self.state.clone().into_os_string(),
890            runtime,
891            self.runtime.clone().into_os_string(),
892            logs,
893            self.logs.clone().into_os_string(),
894        ]
895    }
896
897    /// Reads the four back out of a registration's command line.
898    ///
899    /// `None` unless all four options are there: a registration made before
900    /// they existed starts the daemon against whatever its account's
901    /// platform-standard directories are, and which those are cannot be told
902    /// from here.
903    #[must_use]
904    pub fn from_command_line(command_line: &str) -> Option<Self> {
905        let arguments = split_command_line(command_line);
906        let value = |option: &str| {
907            let at = arguments.iter().position(|argument| argument == option)?;
908            arguments.get(at + 1).map(PathBuf::from)
909        };
910        let [config, state, runtime, logs] = SERVICE_DIRECTORY_OPTIONS;
911        Some(Self {
912            config: value(config)?,
913            state: value(state)?,
914            runtime: value(runtime)?,
915            logs: value(logs)?,
916        })
917    }
918
919    /// The deepest directory all four sit under: what an operator would call
920    /// the installation's data root. The configuration directory when they
921    /// share nothing.
922    #[must_use]
923    pub fn root(&self) -> PathBuf {
924        let mut shared: Vec<std::path::Component<'_>> = self.config.components().collect();
925        for other in [&self.state, &self.runtime, &self.logs] {
926            let common = shared
927                .iter()
928                .zip(other.components())
929                .take_while(|(left, right)| **left == *right)
930                .count();
931            shared.truncate(common);
932        }
933        if shared.is_empty() {
934            self.config.clone()
935        } else {
936            shared.iter().collect()
937        }
938    }
939
940    /// The four as an [`AppPaths`], to read the records a daemon running
941    /// against them writes.
942    #[must_use]
943    pub fn paths(&self) -> AppPaths {
944        AppPaths::from_directories(&self.config, &self.state, &self.runtime, &self.logs)
945    }
946}
947
948/// The hidden `daemon run` options that carry the installing account's four
949/// application-data directories across to the service's account, in
950/// [`ServiceDirectories`] order. Written by `service install`, and read back
951/// by [`ServiceDirectories::from_command_line`] to tell whose data root a
952/// registration runs against.
953pub const SERVICE_DIRECTORY_OPTIONS: [&str; 4] = [
954    "--service-config-dir",
955    "--service-state-dir",
956    "--service-runtime-dir",
957    "--service-logs-dir",
958];
959
960// ---------------------------------------------------------------------------
961// The install plan
962// ---------------------------------------------------------------------------
963
964/// What `service install` was asked for, before anything has been resolved.
965///
966/// Separate from [`InstallPlan`] because a request is the operator's words and
967/// a plan is what the host actually supports: the plan carries a *resolved
968/// absolute* binary path, the account the start mode obliges, and the four
969/// directories the registration is installed against, none of which the caller
970/// supplies.
971#[derive(Debug, Clone, PartialEq, Eq)]
972pub struct InstallRequest {
973    start_mode: StartMode,
974    binary: Option<PathBuf>,
975    source_binary: Option<PathBuf>,
976    arguments: Vec<OsString>,
977    restart: RestartPolicy,
978    on_demand: bool,
979}
980
981impl InstallRequest {
982    /// Register the binary that is running this call.
983    #[must_use]
984    pub fn new(start_mode: StartMode) -> Self {
985        Self {
986            start_mode,
987            binary: None,
988            source_binary: None,
989            arguments: DAEMON_ARGUMENTS.iter().map(OsString::from).collect(),
990            restart: RestartPolicy::default(),
991            on_demand: false,
992        }
993    }
994
995    /// Registers the service but does not ask the manager to start it by
996    /// itself.
997    ///
998    /// Production never uses this: a boot-mode registration that does not start
999    /// at boot is the failure `service status` reports. **The privileged
1000    /// installer tests use it for every fixture they create**, so that a
1001    /// registration which somehow escaped its cleanup guard cannot start with
1002    /// the owner's machine on the next reboot. A leaked service is bad; a
1003    /// leaked service that runs is worse, and the difference costs one flag.
1004    #[must_use]
1005    pub const fn started_on_demand(mut self) -> Self {
1006        self.on_demand = true;
1007        self
1008    }
1009
1010    /// Register a named binary instead of the running one.
1011    ///
1012    /// The privileged installer tests use this to register a fixture service
1013    /// host. Production does not: `05-infrastructure.md` item 6 is specifically
1014    /// about the path of *the running binary*, and letting an operator name a
1015    /// different one would defeat the stale-path detection it asks for.
1016    #[must_use]
1017    pub fn for_binary(mut self, binary: impl Into<PathBuf>) -> Self {
1018        self.binary = Some(binary.into());
1019        self
1020    }
1021
1022    /// Where the registered binary was copied from.
1023    ///
1024    /// # Why a registration has a source at all
1025    ///
1026    /// A service registered directly against a package manager's own file can
1027    /// never be upgraded on Windows, because the running service holds that
1028    /// file open and `npm i -g` cannot replace it. What it does instead is
1029    /// worse than failing: it rewrites the package metadata, reports success,
1030    /// and leaves the old executable in place — so the operator is told the new
1031    /// version is installed while the old one keeps running. That was observed,
1032    /// twice in a row, before this existed.
1033    ///
1034    /// So the service runs a copy the product owns, and this records where that
1035    /// copy came from. The source is what a package manager updates, what
1036    /// [`inspect_binary`] watches for item 6's stale-path case, and what the
1037    /// daemon compares its own version against to know an upgrade is waiting.
1038    #[must_use]
1039    pub fn copied_from(mut self, source: impl Into<PathBuf>) -> Self {
1040        self.source_binary = Some(source.into());
1041        self
1042    }
1043
1044    /// Replace the registered arguments. Defaults to [`DAEMON_ARGUMENTS`].
1045    #[must_use]
1046    pub fn with_arguments<I, S>(mut self, arguments: I) -> Self
1047    where
1048        I: IntoIterator<Item = S>,
1049        S: Into<OsString>,
1050    {
1051        self.arguments = arguments.into_iter().map(Into::into).collect();
1052        self
1053    }
1054
1055    /// Replace the restart-on-failure policy. Defaults to
1056    /// [`RestartPolicy::default`].
1057    #[must_use]
1058    pub const fn with_restart(mut self, restart: RestartPolicy) -> Self {
1059        self.restart = restart;
1060        self
1061    }
1062
1063    /// The start mode asked for.
1064    #[must_use]
1065    pub const fn start_mode(&self) -> StartMode {
1066        self.start_mode
1067    }
1068}
1069
1070/// Everything a backend needs, resolved and validated.
1071///
1072/// Constructing one is where `05-infrastructure.md` item 6 is actually
1073/// satisfied: the binary path is made absolute and is confirmed to be a file
1074/// *before* anything is registered, so a registration that would fail at boot
1075/// is refused at install time instead.
1076#[derive(Debug, Clone, PartialEq, Eq)]
1077pub struct InstallPlan {
1078    identity: ServiceIdentity,
1079    start_mode: StartMode,
1080    binary: PathBuf,
1081    source_binary: Option<PathBuf>,
1082    arguments: Vec<OsString>,
1083    account: ServiceAccount,
1084    restart: RestartPolicy,
1085    directories: ServiceDirectories,
1086    secret_guard: Option<PathBuf>,
1087    on_demand: bool,
1088}
1089
1090impl InstallPlan {
1091    /// Resolves a request against this host.
1092    ///
1093    /// # Errors
1094    ///
1095    /// [`ServiceError::BinaryPath`] when the running executable cannot be
1096    /// located, and [`ServiceError::BinaryMissing`] when the path that would be
1097    /// registered is not a file.
1098    pub fn resolve(
1099        identity: ServiceIdentity,
1100        request: &InstallRequest,
1101        directories: ServiceDirectories,
1102    ) -> Result<Self, ServiceError> {
1103        let binary = match &request.binary {
1104            Some(named) => absolute(named)?,
1105            None => running_executable()?,
1106        };
1107        if !binary.is_file() {
1108            return Err(ServiceError::BinaryMissing { path: binary });
1109        }
1110        // `d2` publishes the guard file the store lives in so the systemd
1111        // sandbox can admit only its parent for atomic token rotation. A store
1112        // that cannot be resolved is not fatal here; it merely cannot receive
1113        // that narrow writable-path exception.
1114        let secret_guard = crate::secrets::PlatformSecretStore::for_start_mode(request.start_mode)
1115            .ok()
1116            .map(|store| store.guard());
1117        Ok(Self {
1118            identity,
1119            start_mode: request.start_mode,
1120            binary,
1121            source_binary: request.source_binary.clone(),
1122            arguments: request.arguments.clone(),
1123            account: ServiceAccount::for_start_mode(request.start_mode),
1124            restart: request.restart,
1125            directories,
1126            secret_guard,
1127            on_demand: request.on_demand,
1128        })
1129    }
1130
1131    /// Builds a plan from values a caller already holds, without touching the
1132    /// filesystem.
1133    ///
1134    /// This is what makes the renderers testable on a host that has none of the
1135    /// paths involved: a Windows developer can render the systemd unit for
1136    /// `/opt/runner-manager/bin/runner-manager` and assert every line of it.
1137    #[must_use]
1138    pub fn unchecked(
1139        identity: ServiceIdentity,
1140        start_mode: StartMode,
1141        binary: impl Into<PathBuf>,
1142        directories: ServiceDirectories,
1143    ) -> Self {
1144        Self {
1145            identity,
1146            start_mode,
1147            binary: binary.into(),
1148            source_binary: None,
1149            arguments: DAEMON_ARGUMENTS.iter().map(OsString::from).collect(),
1150            account: ServiceAccount::for_start_mode(start_mode),
1151            restart: RestartPolicy::default(),
1152            directories,
1153            secret_guard: None,
1154            on_demand: false,
1155        }
1156    }
1157
1158    /// Registers without arming the manager's automatic start. See
1159    /// [`InstallRequest::started_on_demand`].
1160    #[must_use]
1161    pub const fn started_on_demand(mut self) -> Self {
1162        self.on_demand = true;
1163        self
1164    }
1165
1166    /// Whether the manager was asked not to start this by itself.
1167    #[must_use]
1168    pub const fn is_on_demand(&self) -> bool {
1169        self.on_demand
1170    }
1171
1172    /// Names the file the secret store lives in, so systemd can permit atomic
1173    /// replacement of it under `ProtectSystem=strict`.
1174    ///
1175    /// [`InstallPlan::resolve`] fills this from `d2`; a test names it directly
1176    /// so that a Windows or macOS CI leg can assert the Linux unit's writable
1177    /// path for a store that platform does not have.
1178    #[must_use]
1179    pub fn with_secret_guard(mut self, guard: impl Into<PathBuf>) -> Self {
1180        self.secret_guard = Some(guard.into());
1181        self
1182    }
1183
1184    /// The file the machine-scoped secret store lives in, when one resolved.
1185    #[must_use]
1186    pub fn secret_guard(&self) -> Option<&Path> {
1187        self.secret_guard.as_deref()
1188    }
1189
1190    /// Overrides the restart policy on an already-built plan.
1191    #[must_use]
1192    pub const fn with_restart(mut self, restart: RestartPolicy) -> Self {
1193        self.restart = restart;
1194        self
1195    }
1196
1197    /// Overrides the registered arguments on an already-built plan.
1198    #[must_use]
1199    pub fn with_arguments<I, S>(mut self, arguments: I) -> Self
1200    where
1201        I: IntoIterator<Item = S>,
1202        S: Into<OsString>,
1203    {
1204        self.arguments = arguments.into_iter().map(Into::into).collect();
1205        self
1206    }
1207
1208    /// What the operating system calls this registration.
1209    #[must_use]
1210    pub const fn identity(&self) -> &ServiceIdentity {
1211        &self.identity
1212    }
1213
1214    /// Boot or login.
1215    #[must_use]
1216    pub const fn start_mode(&self) -> StartMode {
1217        self.start_mode
1218    }
1219
1220    /// The resolved absolute path of the binary to register.
1221    #[must_use]
1222    pub fn binary(&self) -> &Path {
1223        &self.binary
1224    }
1225
1226    /// Where [`Self::binary`] was copied from, when it is a copy.
1227    ///
1228    /// `None` is the legacy layout: the registration names the file a package
1229    /// manager owns, and cannot be upgraded while it runs. See
1230    /// [`InstallRequest::copied_from`].
1231    #[must_use]
1232    pub fn source_binary(&self) -> Option<&Path> {
1233        self.source_binary.as_deref()
1234    }
1235
1236    /// The arguments it is registered with.
1237    #[must_use]
1238    pub fn arguments(&self) -> &[OsString] {
1239        &self.arguments
1240    }
1241
1242    /// The account it runs under.
1243    #[must_use]
1244    pub const fn account(&self) -> &ServiceAccount {
1245        &self.account
1246    }
1247
1248    /// The restart-on-failure policy.
1249    #[must_use]
1250    pub const fn restart(&self) -> RestartPolicy {
1251        self.restart
1252    }
1253
1254    /// The four directories it was installed against.
1255    #[must_use]
1256    pub const fn directories(&self) -> &ServiceDirectories {
1257        &self.directories
1258    }
1259
1260    /// The command line as one string, quoted the way each platform expects.
1261    #[must_use]
1262    pub fn command_line(&self) -> String {
1263        let mut out = quote_argument(&self.binary.to_string_lossy());
1264        for argument in &self.arguments {
1265            out.push(' ');
1266            out.push_str(&quote_argument(&argument.to_string_lossy()));
1267        }
1268        out
1269    }
1270}
1271
1272/// Undoes a runner-root change, and says what could not be undone.
1273///
1274/// `None` when the rollback was complete — either because there was nothing to
1275/// undo, or because the directory this operation created was removed again and
1276/// the descriptor it replaced was written back. `Some` is the "report any
1277/// non-reversible existing directory state explicitly" half of the requirement,
1278/// and the caller folds it into the failure it was already reporting.
1279fn retained_runner_root(change: &RootAccessChange) -> Option<String> {
1280    let reversal = change.revert();
1281    matches!(reversal, Reversal::Retained { .. }).then(|| reversal.to_string())
1282}
1283
1284/// The error a failed step returns once both of its rollbacks have been
1285/// attempted.
1286///
1287/// `retained` is [`retained_runner_root`]'s answer and `rollback` is what
1288/// undoing the *registration* reported — whatever it reports on success, which
1289/// a failure has no use for. Both arrive already evaluated, because each may be
1290/// attempted exactly once and the order is the caller's to choose.
1291///
1292/// A complete rollback leaves `cause` exactly as it was: the operator's problem
1293/// is what failed, not the tidying up afterwards. An incomplete one is promoted
1294/// to [`ServiceError::Rollback`], which is the variant that exists to say "this
1295/// failed *and* something is left behind" — and it names everything that did.
1296fn rolled_back<T>(
1297    retained: Option<String>,
1298    rollback: Result<T, ServiceError>,
1299    operation: &'static str,
1300    identity: &ServiceIdentity,
1301    cause: ServiceError,
1302) -> ServiceError {
1303    let left_behind = match (rollback.err(), retained) {
1304        (Some(rollback), Some(retained)) => Some(format!("{rollback}; {retained}")),
1305        (Some(rollback), None) => Some(rollback.to_string()),
1306        (None, retained) => retained,
1307    };
1308    match left_behind {
1309        Some(rollback) => ServiceError::Rollback {
1310            operation,
1311            name: identity.name().to_string(),
1312            cause: cause.to_string(),
1313            rollback,
1314        },
1315        None => cause,
1316    }
1317}
1318
1319/// [`rolled_back`] for a failure with nothing registered yet, where the runner
1320/// root is the only thing there is to undo.
1321fn undo_runner_root(
1322    change: &RootAccessChange,
1323    operation: &'static str,
1324    identity: &ServiceIdentity,
1325    cause: ServiceError,
1326) -> ServiceError {
1327    rolled_back(
1328        retained_runner_root(change),
1329        Ok(()),
1330        operation,
1331        identity,
1332        cause,
1333    )
1334}
1335
1336/// Makes a path absolute without resolving symlinks.
1337///
1338/// Symbolic links are deliberately left alone. A Homebrew installation puts a
1339/// stable link at `/opt/homebrew/bin/runner-manager` and moves the file it
1340/// points at on every upgrade; recording the link's target would make an
1341/// ordinary `brew upgrade` look exactly like the npm failure this module exists
1342/// to detect, and recording the link records the thing the operator installed.
1343///
1344/// Linux is the one platform where this is not fully in the caller's hands:
1345/// `std::env::current_exe` reads `/proc/self/exe`, which the kernel has already
1346/// resolved. There the recorded path is the real file rather than the shim —
1347/// which is still correct for the npm case, because a real file under a Node
1348/// prefix disappears with the prefix.
1349fn absolute(path: &Path) -> Result<PathBuf, ServiceError> {
1350    std::path::absolute(path).map_err(|error| ServiceError::BinaryPath {
1351        detail: format!("{} could not be made absolute: {error}", path.display()),
1352    })
1353}
1354
1355/// The absolute path of the executable running this call.
1356fn running_executable() -> Result<PathBuf, ServiceError> {
1357    let raw = std::env::current_exe().map_err(|error| ServiceError::BinaryPath {
1358        detail: error.to_string(),
1359    })?;
1360    absolute(&raw)
1361}
1362
1363/// Quotes one command-line argument when it needs it.
1364///
1365/// The rule is Windows', because Windows is the platform where a command line is
1366/// a single string the callee re-splits, and because `windows-service` applies
1367/// exactly this rule when it builds `lpBinaryPathName`. The Unix backends embed
1368/// arguments in a plist array and in a systemd `ExecStart=`, both of which
1369/// accept the same quoting, so one rule serves all three rather than three
1370/// nearly-identical ones.
1371pub(crate) fn quote_argument(argument: &str) -> String {
1372    if !argument.is_empty() && !argument.contains([' ', '"', '\t', '\n']) {
1373        return argument.to_string();
1374    }
1375    let mut out = String::with_capacity(argument.len() + 2);
1376    out.push('"');
1377    let mut backslashes = 0usize;
1378    for c in argument.chars() {
1379        match c {
1380            '\\' => {
1381                backslashes += 1;
1382                out.push('\\');
1383            }
1384            '"' => {
1385                // Every backslash immediately before a quote must be doubled,
1386                // and the quote itself escaped.
1387                for _ in 0..=backslashes {
1388                    out.push('\\');
1389                }
1390                out.push('"');
1391                backslashes = 0;
1392            }
1393            other => {
1394                backslashes = 0;
1395                out.push(other);
1396            }
1397        }
1398    }
1399    // Trailing backslashes would otherwise escape the closing quote.
1400    for _ in 0..backslashes {
1401        out.push('\\');
1402    }
1403    out.push('"');
1404    out
1405}
1406
1407/// Reads the executable back out of a command line quoted by
1408/// [`quote_argument`].
1409///
1410/// `service status` needs this because the Windows Service Control Manager
1411/// stores the binary and its arguments as one string and hands the whole thing
1412/// back from `QueryServiceConfigW`. Comparing a record against a registration
1413/// means splitting that string the same way Windows itself does.
1414///
1415/// Returns `None` for an empty or whitespace-only command line, which is the
1416/// only input with no first argument to find.
1417#[must_use]
1418pub fn executable_from_command_line(command_line: &str) -> Option<PathBuf> {
1419    split_command_line(command_line)
1420        .into_iter()
1421        .next()
1422        .filter(|executable| !executable.is_empty())
1423        .map(PathBuf::from)
1424}
1425
1426/// Splits a command line quoted by [`quote_argument`] back into its
1427/// arguments, by the rule Windows itself applies: whitespace separates
1428/// arguments outside quotes, `2n` backslashes before a quote are `n`
1429/// backslashes and a quote that opens or closes, `2n+1` are `n` backslashes
1430/// and a literal quote, and every other backslash is literal.
1431#[must_use]
1432pub fn split_command_line(command_line: &str) -> Vec<String> {
1433    let mut arguments = Vec::new();
1434    let mut current = String::new();
1435    let mut started = false;
1436    let mut quoted = false;
1437    let mut backslashes = 0usize;
1438    for c in command_line.chars() {
1439        match c {
1440            '\\' => {
1441                backslashes += 1;
1442                started = true;
1443            }
1444            '"' => {
1445                current.extend(std::iter::repeat_n('\\', backslashes / 2));
1446                if backslashes.is_multiple_of(2) {
1447                    quoted = !quoted;
1448                } else {
1449                    current.push('"');
1450                }
1451                backslashes = 0;
1452                started = true;
1453            }
1454            ' ' | '\t' if !quoted => {
1455                current.extend(std::iter::repeat_n('\\', backslashes));
1456                backslashes = 0;
1457                if started {
1458                    arguments.push(std::mem::take(&mut current));
1459                    started = false;
1460                }
1461            }
1462            other => {
1463                current.extend(std::iter::repeat_n('\\', backslashes));
1464                backslashes = 0;
1465                current.push(other);
1466                started = true;
1467            }
1468        }
1469    }
1470    current.extend(std::iter::repeat_n('\\', backslashes));
1471    if started {
1472        arguments.push(current);
1473    }
1474    arguments
1475}
1476
1477// ---------------------------------------------------------------------------
1478// Rendered definitions
1479// ---------------------------------------------------------------------------
1480
1481/// Which of the four things a platform reads.
1482#[derive(Debug, Clone, Copy, PartialEq, Eq)]
1483pub enum DefinitionKind {
1484    /// The parameters `CreateServiceW` is called with, rendered as a canonical
1485    /// descriptor so that they can be reviewed and asserted like the other
1486    /// three. The Service Control Manager has no file.
1487    WindowsService,
1488    /// A Task Scheduler XML document, for `--start-at login` on Windows.
1489    WindowsScheduledTask,
1490    /// A launchd property list — a LaunchDaemon at boot, a LaunchAgent at login.
1491    LaunchdPlist,
1492    /// A systemd unit — a system unit at boot, a user unit at login.
1493    SystemdUnit,
1494}
1495
1496impl DefinitionKind {
1497    /// What to call the thing that reads it, in an operator-facing message.
1498    #[must_use]
1499    pub const fn manager(self) -> &'static str {
1500        match self {
1501            Self::WindowsService => "the Windows Service Control Manager",
1502            Self::WindowsScheduledTask => "Windows Task Scheduler",
1503            Self::LaunchdPlist => "launchd",
1504            Self::SystemdUnit => "systemd",
1505        }
1506    }
1507}
1508
1509impl fmt::Display for DefinitionKind {
1510    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
1511        f.write_str(self.manager())
1512    }
1513}
1514
1515/// One platform's definition of the registration, as text.
1516///
1517/// Producing this is pure: no `cfg`, no privileges, no filesystem. That is what
1518/// lets every leg of the CI matrix assert every platform's definition, instead
1519/// of each leg asserting only its own and the other two being reviewed by
1520/// reading.
1521#[derive(Debug, Clone, PartialEq, Eq)]
1522pub struct ServiceDefinition {
1523    kind: DefinitionKind,
1524    text: String,
1525    install_path: Option<PathBuf>,
1526}
1527
1528impl ServiceDefinition {
1529    /// Which platform this is for.
1530    #[must_use]
1531    pub const fn kind(&self) -> DefinitionKind {
1532        self.kind
1533    }
1534
1535    /// The definition itself.
1536    #[must_use]
1537    pub fn text(&self) -> &str {
1538        &self.text
1539    }
1540
1541    /// Where the platform expects the file, when the platform reads a file.
1542    ///
1543    /// `None` for [`DefinitionKind::WindowsService`], whose "definition" is a
1544    /// set of arguments to `CreateServiceW`, and for a Task Scheduler document,
1545    /// which is handed to `schtasks` from a temporary file and is not stored
1546    /// where it was written.
1547    #[must_use]
1548    pub fn install_path(&self) -> Option<&Path> {
1549        self.install_path.as_deref()
1550    }
1551
1552    /// The definition this host's own service manager would be given.
1553    ///
1554    /// The value-level twin of [`host_definition_kind`]. Rendering needs no
1555    /// privileges and no service manager, so anything that wants to *see* what
1556    /// an install would register — `f3`'s `service install --dry-run`, a
1557    /// support bundle, [`RecordingControls`] — can have it without registering
1558    /// anything.
1559    ///
1560    /// # Errors
1561    ///
1562    /// [`ServiceError::Control`] when a Windows logon-triggered registration is
1563    /// asked for and the session reports no account to register it for.
1564    pub fn for_host(plan: &InstallPlan) -> Result<Self, ServiceError> {
1565        Ok(match host_definition_kind(plan.start_mode()) {
1566            DefinitionKind::WindowsService => Self::windows_service(plan),
1567            DefinitionKind::WindowsScheduledTask => {
1568                Self::windows_scheduled_task(plan, &TaskPrincipal::current()?)
1569            }
1570            DefinitionKind::LaunchdPlist => Self::launchd(plan, host_home().as_deref()),
1571            DefinitionKind::SystemdUnit => Self::systemd(plan, host_home().as_deref()),
1572        })
1573    }
1574
1575    /// The Windows service parameters, as a canonical descriptor.
1576    #[must_use]
1577    pub fn windows_service(plan: &InstallPlan) -> Self {
1578        Self {
1579            kind: DefinitionKind::WindowsService,
1580            text: windows_service_descriptor(plan),
1581            install_path: None,
1582        }
1583    }
1584
1585    /// A Task Scheduler document for `--start-at login`.
1586    #[must_use]
1587    pub fn windows_scheduled_task(plan: &InstallPlan, principal: &TaskPrincipal) -> Self {
1588        Self {
1589            kind: DefinitionKind::WindowsScheduledTask,
1590            text: windows_scheduled_task_xml(plan, principal),
1591            install_path: None,
1592        }
1593    }
1594
1595    /// A launchd property list.
1596    ///
1597    /// `home` is only consulted for `--start-at login`, where the file belongs
1598    /// in the operator's own `~/Library/LaunchAgents`.
1599    #[must_use]
1600    pub fn launchd(plan: &InstallPlan, home: Option<&Path>) -> Self {
1601        let file = format!("{}.plist", plan.identity().launchd_label());
1602        let install_path = match plan.start_mode() {
1603            StartMode::Boot => Some(PathBuf::from(LAUNCH_DAEMONS_DIR).join(file)),
1604            StartMode::Login => home.map(|home| home.join(LAUNCH_AGENTS_SUBDIR).join(file)),
1605        };
1606        Self {
1607            kind: DefinitionKind::LaunchdPlist,
1608            text: launchd_plist(plan),
1609            install_path,
1610        }
1611    }
1612
1613    /// Wraps text this module did not render.
1614    ///
1615    /// Two callers need it and both matter. A test builds a deliberately
1616    /// widened definition and watches [`review_least_privilege`] reject it —
1617    /// without which the review would be a function nobody had ever seen fail.
1618    /// And `service status` can review the definition **actually on disk**,
1619    /// which on Linux and macOS an operator is free to edit after installation.
1620    #[must_use]
1621    pub fn from_text(kind: DefinitionKind, text: impl Into<String>) -> Self {
1622        Self {
1623            kind,
1624            text: text.into(),
1625            install_path: None,
1626        }
1627    }
1628
1629    /// A systemd unit.
1630    ///
1631    /// `home` is only consulted for `--start-at login`, where the unit belongs
1632    /// in the operator's own `~/.config/systemd/user`.
1633    #[must_use]
1634    pub fn systemd(plan: &InstallPlan, home: Option<&Path>) -> Self {
1635        let file = plan.identity().systemd_unit();
1636        let install_path = match plan.start_mode() {
1637            StartMode::Boot => Some(PathBuf::from(SYSTEMD_SYSTEM_DIR).join(file)),
1638            StartMode::Login => home.map(|home| home.join(SYSTEMD_USER_SUBDIR).join(file)),
1639        };
1640        Self {
1641            kind: DefinitionKind::SystemdUnit,
1642            text: systemd_unit(plan),
1643            install_path,
1644        }
1645    }
1646}
1647
1648/// Where a LaunchDaemon lives.
1649pub const LAUNCH_DAEMONS_DIR: &str = "/Library/LaunchDaemons";
1650/// Where a LaunchAgent lives, under the operator's home directory.
1651pub const LAUNCH_AGENTS_SUBDIR: &str = "Library/LaunchAgents";
1652/// Where a systemd system unit lives.
1653pub const SYSTEMD_SYSTEM_DIR: &str = "/etc/systemd/system";
1654/// Where a systemd user unit lives, under the operator's home directory.
1655pub const SYSTEMD_USER_SUBDIR: &str = ".config/systemd/user";
1656
1657/// The product's own page, cited by every definition so an operator who finds
1658/// one on a host can find out what put it there.
1659const DOCUMENTATION: &str = "https://github.com/IvanMurzak/GitHub-Runner-Scaler-UI";
1660
1661/// How many consecutive failures the platform tolerates before it stops
1662/// retrying.
1663///
1664/// The companion to [`RestartPolicy::reset_after`]: the delay bounds how *fast*
1665/// a restart may come, and this bounds how *many* come before the host gives
1666/// up and leaves the failure visible instead of hiding it behind an endless
1667/// retry.
1668pub const START_LIMIT_BURST: u32 = 5;
1669
1670// -- systemd -----------------------------------------------------------------
1671
1672/// Renders the systemd unit for this plan.
1673///
1674/// # Why the sandbox is this tight, and what it costs
1675///
1676/// `05-infrastructure.md` item 2 ends *"and write its configured cache and
1677/// runtime directories — and no more"*. `ProtectSystem=strict` plus an explicit
1678/// `ReadWritePaths` is what "and no more" means on Linux: everything outside the
1679/// recorded application-data directories and the credential store directory
1680/// is read-only to this unit. The latter must be writable because refresh-token
1681/// rotation atomically replaces the credential file.
1682///
1683/// **The runner inherits it.** The agent spawns the GitHub Actions runner as a
1684/// child, so a workflow running on this host also runs inside this sandbox: it
1685/// cannot write outside those directories and its private `/tmp`, and
1686/// `NoNewPrivileges=yes` means it cannot `sudo`. `07-security.md` assumes a
1687/// hostile workflow may run here, so that is the intended direction — but it is
1688/// a real behavioural limit and `docs/service-account.md` states it where an
1689/// operator will find it.
1690#[must_use]
1691pub fn systemd_unit(plan: &InstallPlan) -> String {
1692    let identity = plan.identity();
1693    let restart = plan.restart();
1694    let directories = plan.directories();
1695
1696    let mut out = String::new();
1697    out.push_str("[Unit]\n");
1698    out.push_str(&format!("Description={}\n", identity.display_name()));
1699    out.push_str(&format!("Documentation={DOCUMENTATION}\n"));
1700    out.push_str("After=network-online.target\n");
1701    out.push_str("Wants=network-online.target\n");
1702    // In `[Unit]` rather than `[Service]`: systemd moved these in v229 and only
1703    // accepts them here without a deprecation warning.
1704    out.push_str(&format!(
1705        "StartLimitIntervalSec={}\n",
1706        restart.reset_after().as_secs()
1707    ));
1708    out.push_str(&format!("StartLimitBurst={START_LIMIT_BURST}\n"));
1709
1710    out.push_str("\n[Service]\n");
1711    out.push_str("Type=simple\n");
1712    // A runner is a child of the daemon. In particular, systemd must not
1713    // signal it merely because a cooperative daemon handover replaces the
1714    // manager process; startup recovery supervises any survivor instead.
1715    out.push_str("KillMode=process\n");
1716    out.push_str(&format!("ExecStart={}\n", plan.command_line()));
1717    out.push_str(&format!(
1718        "WorkingDirectory={}\n",
1719        directories.state.display()
1720    ));
1721    out.push_str(&format!("SyslogIdentifier={identity}\n"));
1722    out.push_str("Restart=on-failure\n");
1723    out.push_str(&format!("RestartSec={}\n", restart.delay().as_secs()));
1724
1725    out.push_str("\n# Least privilege. See docs/service-account.md.\n");
1726    for directive in SYSTEMD_HARDENING {
1727        out.push_str(directive);
1728        out.push('\n');
1729    }
1730    let mut writable = directories.all().to_vec();
1731    if let Some(secret_directory) = plan.secret_guard().and_then(Path::parent) {
1732        writable.push(secret_directory);
1733    }
1734    out.push_str(&format!(
1735        "ReadWritePaths={}\n",
1736        writable
1737            .iter()
1738            .map(|path| quote_argument(&path.to_string_lossy()))
1739            .collect::<Vec<_>>()
1740            .join(" ")
1741    ));
1742
1743    out.push_str("\n[Install]\n");
1744    out.push_str(match plan.start_mode() {
1745        StartMode::Boot => "WantedBy=multi-user.target\n",
1746        StartMode::Login => "WantedBy=default.target\n",
1747    });
1748    out
1749}
1750
1751/// The hardening directives every unit carries, in the order they are rendered.
1752///
1753/// A constant rather than a sequence of `push_str` calls so that
1754/// [`review_least_privilege`] can be written against the same list the renderer
1755/// emits: a directive dropped from the unit is a directive the review then
1756/// reports as missing, rather than one that silently stops being checked.
1757pub const SYSTEMD_HARDENING: [&str; 13] = [
1758    "NoNewPrivileges=yes",
1759    "CapabilityBoundingSet=",
1760    "AmbientCapabilities=",
1761    "PrivateTmp=yes",
1762    "PrivateDevices=yes",
1763    "ProtectSystem=strict",
1764    "ProtectKernelTunables=yes",
1765    "ProtectKernelModules=yes",
1766    "ProtectControlGroups=yes",
1767    "RestrictNamespaces=yes",
1768    "RestrictRealtime=yes",
1769    "RestrictSUIDSGID=yes",
1770    "RestrictAddressFamilies=AF_INET AF_INET6 AF_UNIX",
1771];
1772
1773// -- launchd -----------------------------------------------------------------
1774
1775/// The `ProcessType` every rendered plist carries.
1776///
1777/// # Why not `Background`
1778///
1779/// launchd applies a job's `ProcessType` to the whole process tree it starts,
1780/// not only to the daemon. Under `Background` every runner the daemon launched
1781/// ran at base priority 4, on efficiency cores with throttled I/O, so each job
1782/// took 4 to 20 times as long as on a classic runner.
1783///
1784/// A child cannot leave it. This is the process type launchd chose at spawn
1785/// (`spawn type = background (5)` in `launchctl print`), not the
1786/// `PRIO_DARWIN_BG` flag `setpriority` controls: measured on macOS 27, a child
1787/// of a `Background` agent reads `getpriority(PRIO_DARWIN_PROCESS) == 0`, and
1788/// `setpriority(PRIO_DARWIN_PROCESS, 0, 0)` returns success and leaves it at
1789/// 4, before `exec` and after it alike. 0.4.31 tried exactly that.
1790///
1791/// The measured alternatives, as the base priority of a runner the agent
1792/// starts: `Background` 4, `Adaptive` 4, `Standard` (the default when the key
1793/// is absent) 20, `Interactive` 31 (the same as a classic runner, and the value
1794/// GitHub's own runner service template uses). The daemon itself mostly waits,
1795/// so it costs nothing to run it at that priority too.
1796pub const LAUNCHD_PROCESS_TYPE: &str = "Interactive";
1797
1798/// Renders the launchd property list for this plan.
1799///
1800/// `KeepAlive` is a dictionary rather than `<true/>` on purpose: the
1801/// requirement is *restart on failure*, and a bare `KeepAlive` also restarts a
1802/// job that exited cleanly, which would turn a deliberate `service stop` into a
1803/// fight with launchd.
1804#[must_use]
1805pub fn launchd_plist(plan: &InstallPlan) -> String {
1806    let identity = plan.identity();
1807    let directories = plan.directories();
1808    let mut out = String::new();
1809    out.push_str("<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n");
1810    out.push_str(
1811        "<!DOCTYPE plist PUBLIC \"-//Apple//DTD PLIST 1.0//EN\" \
1812         \"http://www.apple.com/DTDs/PropertyList-1.0.dtd\">\n",
1813    );
1814    out.push_str("<plist version=\"1.0\">\n<dict>\n");
1815    out.push_str(&plist_string("Label", &identity.launchd_label()));
1816
1817    out.push_str("  <key>ProgramArguments</key>\n  <array>\n");
1818    out.push_str(&format!(
1819        "    <string>{}</string>\n",
1820        xml_escape(&plan.binary().to_string_lossy())
1821    ));
1822    for argument in plan.arguments() {
1823        out.push_str(&format!(
1824            "    <string>{}</string>\n",
1825            xml_escape(&argument.to_string_lossy())
1826        ));
1827    }
1828    out.push_str("  </array>\n");
1829
1830    out.push_str("  <key>RunAtLoad</key>\n  <true/>\n");
1831    out.push_str("  <key>KeepAlive</key>\n  <dict>\n");
1832    out.push_str("    <key>SuccessfulExit</key>\n    <false/>\n");
1833    out.push_str("  </dict>\n");
1834    out.push_str(&format!(
1835        "  <key>ThrottleInterval</key>\n  <integer>{}</integer>\n",
1836        plan.restart().delay().as_secs()
1837    ));
1838    out.push_str(&plist_string("ProcessType", LAUNCHD_PROCESS_TYPE));
1839    out.push_str(&plist_string(
1840        "WorkingDirectory",
1841        &directories.state.to_string_lossy(),
1842    ));
1843    out.push_str(&plist_string(
1844        "StandardOutPath",
1845        &directories
1846            .logs
1847            .join("runner-manager.launchd.out.log")
1848            .to_string_lossy(),
1849    ));
1850    out.push_str(&plist_string(
1851        "StandardErrorPath",
1852        &directories
1853            .logs
1854            .join("runner-manager.launchd.err.log")
1855            .to_string_lossy(),
1856    ));
1857
1858    match plan.start_mode() {
1859        StartMode::Boot => {
1860            // A LaunchDaemon runs as root unless told otherwise, and here it
1861            // must: the System Keychain's master key is root-only.
1862            out.push_str(&plist_string(
1863                "UserName",
1864                ServiceAccount::for_definition(DefinitionKind::LaunchdPlist, StartMode::Boot)
1865                    .as_str(),
1866            ));
1867            // A daemon has no user session and must not be given one.
1868            out.push_str("  <key>SessionCreate</key>\n  <false/>\n");
1869        }
1870        StartMode::Login => {
1871            // A LaunchAgent already runs as the operator. Naming a `UserName`
1872            // here would be asking launchd for an account switch a login-mode
1873            // registration has no reason to want.
1874        }
1875    }
1876
1877    out.push_str("</dict>\n</plist>\n");
1878    out
1879}
1880
1881/// One `<key>`/`<string>` pair, indented as the rest of the plist is.
1882fn plist_string(key: &str, value: &str) -> String {
1883    format!(
1884        "  <key>{}</key>\n  <string>{}</string>\n",
1885        xml_escape(key),
1886        xml_escape(value)
1887    )
1888}
1889
1890// -- Windows Task Scheduler --------------------------------------------------
1891
1892/// The account a Task Scheduler task runs as.
1893///
1894/// Task Scheduler, unlike the other three managers, needs the principal spelled
1895/// out in the document. Isolating the one impure step keeps
1896/// [`windows_scheduled_task_xml`] a pure function that every CI leg can assert.
1897#[derive(Debug, Clone, PartialEq, Eq)]
1898pub struct TaskPrincipal {
1899    user_id: String,
1900}
1901
1902impl TaskPrincipal {
1903    /// The account running this call, as `DOMAIN\user`.
1904    ///
1905    /// # Errors
1906    ///
1907    /// [`ServiceError::BinaryPath`] is not the right shape here, so this
1908    /// reports [`ServiceError::Control`] naming what the environment failed to
1909    /// say.
1910    pub fn current() -> Result<Self, ServiceError> {
1911        let user = std::env::var("USERNAME")
1912            .ok()
1913            .filter(|value| !value.trim().is_empty());
1914        let Some(user) = user else {
1915            return Err(ServiceError::Control {
1916                operation: "identify the account for",
1917                name: SERVICE_NAME.to_string(),
1918                manager: "Windows Task Scheduler",
1919                detail: "this session reports no %USERNAME%, so there is no principal to \
1920                         register a logon-triggered task for"
1921                    .to_string(),
1922            });
1923        };
1924        let domain = std::env::var("USERDOMAIN")
1925            .ok()
1926            .filter(|value| !value.trim().is_empty());
1927        Ok(Self {
1928            user_id: match domain {
1929                Some(domain) => format!("{domain}\\{user}"),
1930                None => user,
1931            },
1932        })
1933    }
1934
1935    /// A named principal, for tests and for a caller that already knows.
1936    #[must_use]
1937    pub fn named(user_id: impl Into<String>) -> Self {
1938        Self {
1939            user_id: user_id.into(),
1940        }
1941    }
1942
1943    /// `DOMAIN\user`.
1944    #[must_use]
1945    pub fn user_id(&self) -> &str {
1946        &self.user_id
1947    }
1948}
1949
1950/// Renders the Task Scheduler document for a `--start-at login` registration.
1951///
1952/// `RunLevel` is `LeastPrivilege`, which is the whole of Windows' answer to
1953/// item 2 in this domain: the task runs with the operator's ordinary filtered
1954/// token and never with an elevated one, whatever the operator's group
1955/// membership.
1956#[must_use]
1957pub fn windows_scheduled_task_xml(plan: &InstallPlan, principal: &TaskPrincipal) -> String {
1958    let identity = plan.identity();
1959    let user = xml_escape(principal.user_id());
1960    let arguments = plan
1961        .arguments()
1962        .iter()
1963        .map(|argument| quote_argument(&argument.to_string_lossy()))
1964        .collect::<Vec<_>>()
1965        .join(" ");
1966    let mut out = String::new();
1967    out.push_str("<?xml version=\"1.0\" encoding=\"UTF-16\"?>\n");
1968    out.push_str(
1969        "<Task version=\"1.4\" \
1970         xmlns=\"http://schemas.microsoft.com/windows/2004/02/mit/task\">\n",
1971    );
1972    out.push_str("  <RegistrationInfo>\n");
1973    out.push_str(&format!(
1974        "    <Description>{}</Description>\n",
1975        xml_escape(identity.description())
1976    ));
1977    out.push_str(&format!(
1978        "    <URI>\\{}</URI>\n",
1979        xml_escape(identity.name())
1980    ));
1981    out.push_str("  </RegistrationInfo>\n");
1982
1983    out.push_str("  <Triggers>\n    <LogonTrigger>\n");
1984    out.push_str("      <Enabled>true</Enabled>\n");
1985    out.push_str(&format!("      <UserId>{user}</UserId>\n"));
1986    out.push_str("    </LogonTrigger>\n  </Triggers>\n");
1987
1988    out.push_str("  <Principals>\n    <Principal id=\"Author\">\n");
1989    out.push_str(&format!("      <UserId>{user}</UserId>\n"));
1990    out.push_str("      <LogonType>InteractiveToken</LogonType>\n");
1991    out.push_str("      <RunLevel>LeastPrivilege</RunLevel>\n");
1992    out.push_str("    </Principal>\n  </Principals>\n");
1993
1994    out.push_str("  <Settings>\n");
1995    // One agent per host is `d1`'s lock; saying so here means Task Scheduler
1996    // refuses the second start rather than starting a process that then loses
1997    // the race and exits.
1998    out.push_str("    <MultipleInstancesPolicy>IgnoreNew</MultipleInstancesPolicy>\n");
1999    out.push_str("    <DisallowStartIfOnBatteries>false</DisallowStartIfOnBatteries>\n");
2000    out.push_str("    <StopIfGoingOnBatteries>false</StopIfGoingOnBatteries>\n");
2001    out.push_str("    <AllowHardTerminate>true</AllowHardTerminate>\n");
2002    out.push_str("    <StartWhenAvailable>true</StartWhenAvailable>\n");
2003    out.push_str("    <RunOnlyIfNetworkAvailable>false</RunOnlyIfNetworkAvailable>\n");
2004    out.push_str("    <IdleSettings>\n");
2005    out.push_str("      <StopOnIdleEnd>false</StopOnIdleEnd>\n");
2006    out.push_str("      <RestartOnIdle>false</RestartOnIdle>\n");
2007    out.push_str("    </IdleSettings>\n");
2008    out.push_str("    <AllowStartOnDemand>true</AllowStartOnDemand>\n");
2009    out.push_str("    <Enabled>true</Enabled>\n");
2010    out.push_str("    <Hidden>false</Hidden>\n");
2011    out.push_str("    <RunOnlyIfIdle>false</RunOnlyIfIdle>\n");
2012    out.push_str("    <WakeToRun>false</WakeToRun>\n");
2013    // A daemon has no natural end, so any limit here would be a scheduled kill.
2014    out.push_str("    <ExecutionTimeLimit>PT0S</ExecutionTimeLimit>\n");
2015    out.push_str("    <Priority>7</Priority>\n");
2016    out.push_str("    <RestartOnFailure>\n");
2017    out.push_str(&format!(
2018        "      <Interval>{}</Interval>\n",
2019        iso8601_minutes(
2020            plan.restart()
2021                .effective_delay(DefinitionKind::WindowsScheduledTask)
2022        )
2023    ));
2024    out.push_str(&format!("      <Count>{START_LIMIT_BURST}</Count>\n"));
2025    out.push_str("    </RestartOnFailure>\n");
2026    out.push_str("  </Settings>\n");
2027
2028    out.push_str("  <Actions Context=\"Author\">\n    <Exec>\n");
2029    out.push_str(&format!(
2030        "      <Command>{}</Command>\n",
2031        xml_escape(&plan.binary().to_string_lossy())
2032    ));
2033    if !arguments.is_empty() {
2034        out.push_str(&format!(
2035            "      <Arguments>{}</Arguments>\n",
2036            xml_escape(&arguments)
2037        ));
2038    }
2039    out.push_str(&format!(
2040        "      <WorkingDirectory>{}</WorkingDirectory>\n",
2041        xml_escape(&plan.directories().state.to_string_lossy())
2042    ));
2043    out.push_str("    </Exec>\n  </Actions>\n");
2044    out.push_str("</Task>\n");
2045    out
2046}
2047
2048/// `PT<n>M`, which is the only shape Task Scheduler accepts for a restart
2049/// interval. See [`RestartPolicy::effective_delay`].
2050fn iso8601_minutes(duration: Duration) -> String {
2051    format!("PT{}M", duration.as_secs() / 60)
2052}
2053
2054/// Escapes the five XML entities. Applied to every value that reaches a plist
2055/// or a task document, because a Windows account name may legitimately contain
2056/// `&` and a path may contain `<`.
2057///
2058/// `pub(crate)` rather than private because [`crate::wsl::task`] renders a
2059/// second kind of Task Scheduler document and must escape it the same way; two
2060/// escapers in one crate is exactly how one of them ends up subtly different.
2061pub(crate) fn xml_escape(value: &str) -> String {
2062    let mut out = String::with_capacity(value.len());
2063    for c in value.chars() {
2064        match c {
2065            '&' => out.push_str("&amp;"),
2066            '<' => out.push_str("&lt;"),
2067            '>' => out.push_str("&gt;"),
2068            '"' => out.push_str("&quot;"),
2069            '\'' => out.push_str("&apos;"),
2070            other => out.push(other),
2071        }
2072    }
2073    out
2074}
2075
2076/// The inverse of [`xml_escape`], for reading a definition back off disk.
2077///
2078/// `&amp;` is replaced last, which is the whole subtlety: replacing it first
2079/// would turn a literal `&amp;amp;` into `&` in two passes instead of `&amp;`
2080/// in one.
2081fn xml_unescape(value: &str) -> String {
2082    value
2083        .replace("&lt;", "<")
2084        .replace("&gt;", ">")
2085        .replace("&quot;", "\"")
2086        .replace("&apos;", "'")
2087        .replace("&amp;", "&")
2088}
2089
2090// -- Windows Service Control Manager -----------------------------------------
2091
2092/// How `CreateServiceW` should be called, in a form that can be reviewed,
2093/// asserted, and printed on any platform.
2094///
2095/// The Windows backend builds its `ServiceInfo` from **this** value rather than
2096/// from the plan, so the descriptor a Linux CI leg asserts and the parameters a
2097/// Windows host registers cannot drift apart.
2098#[derive(Debug, Clone, PartialEq, Eq)]
2099pub struct WindowsServiceSpec {
2100    /// The service name.
2101    pub name: String,
2102    /// What `services.msc` shows.
2103    pub display_name: String,
2104    /// The one-line description.
2105    pub description: String,
2106    /// `true` for `SERVICE_AUTO_START`, `false` for `SERVICE_DEMAND_START`.
2107    pub automatic_start: bool,
2108    /// The account, or `None` for `LocalSystem`, which is what
2109    /// `CreateServiceW` takes a null `lpServiceStartName` to mean.
2110    pub account: Option<String>,
2111    /// The full launch command, quoted.
2112    pub command_line: String,
2113    /// The restart-on-failure policy.
2114    pub restart: RestartPolicy,
2115}
2116
2117/// Derives the Windows service parameters from a plan.
2118#[must_use]
2119pub fn windows_service_spec(plan: &InstallPlan) -> WindowsServiceSpec {
2120    WindowsServiceSpec {
2121        name: plan.identity().name().to_string(),
2122        display_name: plan.identity().display_name().to_string(),
2123        description: plan.identity().description().to_string(),
2124        // Only a boot-mode registration is a service at all; a login-mode one
2125        // is a scheduled task. So an automatic start is the only kind here, and
2126        // the field exists because the privileged tests register an on-demand
2127        // fixture rather than one that starts with the test machine.
2128        automatic_start: plan.start_mode() == StartMode::Boot && !plan.is_on_demand(),
2129        account: match ServiceAccount::for_definition(
2130            DefinitionKind::WindowsService,
2131            plan.start_mode(),
2132        ) {
2133            // `None` means LocalSystem to `CreateServiceW`, and naming it
2134            // explicitly would be one more string to spell right.
2135            ServiceAccount::LocalSystem => None,
2136            other => Some(other.as_str().to_string()),
2137        },
2138        command_line: plan.command_line(),
2139        restart: plan.restart(),
2140    }
2141}
2142
2143/// Renders the descriptor [`review_least_privilege`] reads and `service status`
2144/// can print.
2145#[must_use]
2146fn windows_service_descriptor(plan: &InstallPlan) -> String {
2147    let spec = windows_service_spec(plan);
2148    let mut out = String::new();
2149    out.push_str("[windows-service]\n");
2150    out.push_str(&format!("Name={}\n", spec.name));
2151    out.push_str(&format!("DisplayName={}\n", spec.display_name));
2152    out.push_str(&format!("Description={}\n", spec.description));
2153    // OWN_PROCESS and never INTERACTIVE_PROCESS: an interactive service would
2154    // put a process this product controls on the operator's desktop, which is
2155    // both deprecated by Windows and more than the requirement asks for.
2156    out.push_str("ServiceType=OWN_PROCESS\n");
2157    out.push_str(&format!(
2158        "StartType={}\n",
2159        if spec.automatic_start {
2160            "AutoStart"
2161        } else {
2162            "OnDemand"
2163        }
2164    ));
2165    out.push_str("ErrorControl=Normal\n");
2166    out.push_str(&format!(
2167        "Account={}\n",
2168        spec.account
2169            .as_deref()
2170            .unwrap_or(ServiceAccount::LocalSystem.as_str())
2171    ));
2172    out.push_str(&format!("CommandLine={}\n", spec.command_line));
2173    out.push_str(&format!(
2174        "FailureActionRestartDelaySecs={}\n",
2175        spec.restart.delay().as_secs()
2176    ));
2177    out.push_str(&format!(
2178        "FailureActionsResetPeriodSecs={}\n",
2179        spec.restart.reset_after().as_secs()
2180    ));
2181    // Without this flag the Service Control Manager applies failure actions
2182    // only to a crash, and a daemon that exits non-zero after failing to reach
2183    // GitHub is not a crash.
2184    out.push_str("FailureActionsOnNonCrashFailures=true\n");
2185    out.push_str(&format!(
2186        "ReadWritePaths={}\n",
2187        plan.directories()
2188            .all()
2189            .iter()
2190            .map(|path| quote_argument(&path.to_string_lossy()))
2191            .collect::<Vec<_>>()
2192            .join(" ")
2193    ));
2194    out
2195}
2196
2197// ---------------------------------------------------------------------------
2198// The least-privilege review
2199// ---------------------------------------------------------------------------
2200
2201/// Whether a finding is about too much authority or too little.
2202#[derive(Debug, Clone, Copy, PartialEq, Eq)]
2203pub enum FindingKind {
2204    /// The definition grants more than `05-infrastructure.md` item 2 asks for.
2205    /// This is what makes a review fail.
2206    Excess,
2207    /// The definition does not grant something the daemon needs, or does not
2208    /// state something the review cannot verify without.
2209    Shortfall,
2210}
2211
2212impl fmt::Display for FindingKind {
2213    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
2214        f.write_str(match self {
2215            Self::Excess => "excess",
2216            Self::Shortfall => "shortfall",
2217        })
2218    }
2219}
2220
2221/// One thing the review has to say about a definition.
2222#[derive(Debug, Clone, PartialEq, Eq)]
2223pub struct PrivilegeFinding {
2224    /// Excess or shortfall.
2225    pub kind: FindingKind,
2226    /// The directive, key, or element it is about.
2227    pub subject: String,
2228    /// What is wrong with it, in terms an operator can act on.
2229    pub detail: String,
2230}
2231
2232impl fmt::Display for PrivilegeFinding {
2233    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
2234        write!(f, "{}: {} -- {}", self.kind, self.subject, self.detail)
2235    }
2236}
2237
2238/// What a definition actually grants, measured against the requirement.
2239///
2240/// `05-infrastructure.md` item 2 and `07-security.md`'s release gate
2241/// (*"Service account permissions are documented and verified least
2242/// privilege"*) are the same requirement stated twice, and this is the
2243/// verification half. The documentation half is `docs/service-account.md`.
2244///
2245/// The review reads the **rendered text**, not the plan it came from. That is
2246/// deliberate and it is the only version of this check worth having: a review
2247/// that re-derived its expectations from the same values the renderer used
2248/// would agree with the renderer by construction and could never fail. Reading
2249/// the text means a hand-edited unit on a real host is reviewed as it is, and
2250/// means a test can widen a definition and watch this say so.
2251#[derive(Debug, Clone, PartialEq, Eq)]
2252pub struct PrivilegeReview {
2253    kind: DefinitionKind,
2254    account: ServiceAccount,
2255    controls: Vec<String>,
2256    findings: Vec<PrivilegeFinding>,
2257}
2258
2259impl PrivilegeReview {
2260    /// Whether the definition grants nothing beyond the requirement.
2261    #[must_use]
2262    pub fn is_least_privilege(&self) -> bool {
2263        !self
2264            .findings
2265            .iter()
2266            .any(|finding| finding.kind == FindingKind::Excess)
2267    }
2268
2269    /// Everything the review found, excesses and shortfalls together.
2270    #[must_use]
2271    pub fn findings(&self) -> &[PrivilegeFinding] {
2272        &self.findings
2273    }
2274
2275    /// Only the excesses — the findings that make [`Self::is_least_privilege`]
2276    /// false.
2277    #[must_use]
2278    pub fn excesses(&self) -> Vec<&PrivilegeFinding> {
2279        self.findings
2280            .iter()
2281            .filter(|finding| finding.kind == FindingKind::Excess)
2282            .collect()
2283    }
2284
2285    /// The controls the definition was confirmed to carry.
2286    ///
2287    /// Present so that a passing review says *what it checked* rather than only
2288    /// that it passed. A check that reports nothing when it succeeds is
2289    /// indistinguishable from a check that did not run.
2290    #[must_use]
2291    pub fn controls(&self) -> &[String] {
2292        &self.controls
2293    }
2294
2295    /// The account the registration runs under, and why it is the minimum.
2296    #[must_use]
2297    pub const fn account(&self) -> &ServiceAccount {
2298        &self.account
2299    }
2300
2301    /// Which platform's definition was reviewed.
2302    #[must_use]
2303    pub const fn kind(&self) -> DefinitionKind {
2304        self.kind
2305    }
2306}
2307
2308impl fmt::Display for PrivilegeReview {
2309    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
2310        writeln!(
2311            f,
2312            "{} runs as {} ({})",
2313            self.kind,
2314            self.account,
2315            self.account.justification()
2316        )?;
2317        for control in &self.controls {
2318            writeln!(f, "  confirmed  {control}")?;
2319        }
2320        for finding in &self.findings {
2321            writeln!(f, "  {finding}")?;
2322        }
2323        if self.is_least_privilege() {
2324            write!(f, "  verdict    least privilege")
2325        } else {
2326            write!(
2327                f,
2328                "  verdict    NOT least privilege: {} excess(es)",
2329                self.excesses().len()
2330            )
2331        }
2332    }
2333}
2334
2335/// Reads a definition back and reports what it grants.
2336///
2337/// `plan` supplies only the two things the text cannot: the four directories
2338/// the registration is *allowed* to write, and the start mode, which decides
2339/// which account is the minimum.
2340#[must_use]
2341pub fn review_least_privilege(
2342    definition: &ServiceDefinition,
2343    plan: &InstallPlan,
2344) -> PrivilegeReview {
2345    let mut controls = Vec::new();
2346    let mut findings = Vec::new();
2347    match definition.kind() {
2348        DefinitionKind::SystemdUnit => {
2349            review_systemd(definition.text(), plan, &mut controls, &mut findings);
2350        }
2351        DefinitionKind::LaunchdPlist => {
2352            review_launchd(definition.text(), plan, &mut controls, &mut findings);
2353        }
2354        DefinitionKind::WindowsScheduledTask => {
2355            review_scheduled_task(definition.text(), &mut controls, &mut findings);
2356        }
2357        DefinitionKind::WindowsService => {
2358            review_windows_service(definition.text(), plan, &mut controls, &mut findings);
2359        }
2360    }
2361    PrivilegeReview {
2362        kind: definition.kind(),
2363        // The definition's account, not the plan's: a Linux CI leg reviewing the
2364        // Windows descriptor must report `LocalSystem`, which is the account
2365        // that descriptor registers, and not `root`, which is the account this
2366        // leg's own host would use.
2367        account: ServiceAccount::for_definition(definition.kind(), plan.start_mode()),
2368        controls,
2369        findings,
2370    }
2371}
2372
2373/// Every writable path a definition may name, as the strings it names them by.
2374fn permitted_paths(kind: DefinitionKind, plan: &InstallPlan) -> Vec<String> {
2375    let mut permitted = plan
2376        .directories()
2377        .all()
2378        .iter()
2379        .map(|path| path.to_string_lossy().into_owned())
2380        .collect::<Vec<_>>();
2381    if kind == DefinitionKind::SystemdUnit
2382        && let Some(secret_directory) = plan.secret_guard().and_then(Path::parent)
2383    {
2384        permitted.push(secret_directory.to_string_lossy().into_owned());
2385    }
2386    permitted
2387}
2388
2389/// Whether two path spellings name the same place on **this host's**
2390/// filesystem. Case-insensitive on Windows, exact elsewhere.
2391///
2392/// For a rendered definition use [`same_path_for`] instead: which comparison is
2393/// right there follows the platform the definition is *for*, and every leg of
2394/// the CI matrix reviews all four.
2395fn same_path_text(left: &str, right: &str) -> bool {
2396    if cfg!(windows) {
2397        left.eq_ignore_ascii_case(right)
2398    } else {
2399        left == right
2400    }
2401}
2402
2403/// Whether two path spellings name the same place on the platform a given
2404/// definition targets.
2405fn same_path_for(kind: DefinitionKind, left: &str, right: &str) -> bool {
2406    match kind {
2407        DefinitionKind::WindowsService | DefinitionKind::WindowsScheduledTask => {
2408            left.eq_ignore_ascii_case(right)
2409        }
2410        DefinitionKind::LaunchdPlist | DefinitionKind::SystemdUnit => left == right,
2411    }
2412}
2413
2414/// Checks a `ReadWritePaths`-style list against the permitted directories.
2415fn review_writable_paths(
2416    kind: DefinitionKind,
2417    subject: &str,
2418    listed: &[String],
2419    plan: &InstallPlan,
2420    controls: &mut Vec<String>,
2421    findings: &mut Vec<PrivilegeFinding>,
2422) {
2423    let permitted = permitted_paths(kind, plan);
2424    for entry in listed {
2425        if !permitted
2426            .iter()
2427            .any(|allowed| same_path_for(kind, allowed, entry))
2428        {
2429            findings.push(PrivilegeFinding {
2430                kind: FindingKind::Excess,
2431                subject: subject.to_string(),
2432                detail: format!(
2433                    "{entry} is writable but is not one of this registration's required paths"
2434                ),
2435            });
2436        }
2437    }
2438    for allowed in &permitted {
2439        if !listed
2440            .iter()
2441            .any(|entry| same_path_for(kind, allowed, entry))
2442        {
2443            findings.push(PrivilegeFinding {
2444                kind: FindingKind::Shortfall,
2445                subject: subject.to_string(),
2446                detail: format!(
2447                    "{allowed} is one of this registration's directories but is not writable, \
2448                     so the daemon cannot use it"
2449                ),
2450            });
2451        }
2452    }
2453    if listed.len() == permitted.len() && findings.iter().all(|f| f.subject != subject) {
2454        controls.push(format!(
2455            "{subject} names exactly the required application-data and credential paths"
2456        ));
2457    }
2458}
2459
2460/// The one inbound-surface rule, applied to whichever platform names it.
2461///
2462/// `07-security.md` handling rule 2: *"The product exposes no inbound HTTP,
2463/// socket, or RPC surface anywhere, in any command."* A definition that asks the
2464/// service manager to open a socket or publish a Mach service on the daemon's
2465/// behalf would create exactly that surface without a line of product code
2466/// changing, which is why it is checked here and not only in review.
2467fn review_inbound_surface(
2468    text: &str,
2469    markers: &[(&str, &str)],
2470    controls: &mut Vec<String>,
2471    findings: &mut Vec<PrivilegeFinding>,
2472) {
2473    let mut clean = true;
2474    for (marker, detail) in markers {
2475        if text.contains(marker) {
2476            clean = false;
2477            findings.push(PrivilegeFinding {
2478                kind: FindingKind::Excess,
2479                subject: (*marker).to_string(),
2480                detail: (*detail).to_string(),
2481            });
2482        }
2483    }
2484    if clean {
2485        controls.push(
2486            "no socket, listener, or Mach service is published on the daemon's behalf".to_string(),
2487        );
2488    }
2489}
2490
2491fn review_systemd(
2492    text: &str,
2493    plan: &InstallPlan,
2494    controls: &mut Vec<String>,
2495    findings: &mut Vec<PrivilegeFinding>,
2496) {
2497    let directives = ini_directives(text, "Service");
2498    for expected in SYSTEMD_HARDENING {
2499        let (key, value) = expected
2500            .split_once('=')
2501            .expect("every hardening directive is written as key=value");
2502        match directives.get(key) {
2503            Some(actual) if actual == value => controls.push((*expected).to_string()),
2504            Some(actual) => findings.push(PrivilegeFinding {
2505                kind: FindingKind::Excess,
2506                subject: key.to_string(),
2507                detail: format!(
2508                    "is `{actual}`, not `{value}`, so the unit keeps authority the \
2509                                 requirement does not ask for"
2510                ),
2511            }),
2512            None => findings.push(PrivilegeFinding {
2513                kind: FindingKind::Excess,
2514                subject: key.to_string(),
2515                detail: format!(
2516                    "is absent, so the unit inherits systemd's default rather than `{value}`"
2517                ),
2518            }),
2519        }
2520    }
2521
2522    match directives.get("ReadWritePaths") {
2523        Some(value) => {
2524            let listed = split_quoted(value);
2525            review_writable_paths(
2526                DefinitionKind::SystemdUnit,
2527                "ReadWritePaths",
2528                &listed,
2529                plan,
2530                controls,
2531                findings,
2532            );
2533        }
2534        None => findings.push(PrivilegeFinding {
2535            kind: FindingKind::Shortfall,
2536            subject: "ReadWritePaths".to_string(),
2537            detail: "is absent, so `ProtectSystem=strict` leaves the daemon nowhere to write"
2538                .to_string(),
2539        }),
2540    }
2541
2542    // `User=` on a system unit is an escalation only in the other direction, so
2543    // what is checked is the pair systemd actually treats as privileged.
2544    if directives.contains_key("PrivateUsers")
2545        && directives.get("PrivateUsers") == Some(&"no".to_string())
2546    {
2547        findings.push(PrivilegeFinding {
2548            kind: FindingKind::Excess,
2549            subject: "PrivateUsers".to_string(),
2550            detail: "is explicitly disabled, which is broader than leaving it at systemd's default"
2551                .to_string(),
2552        });
2553    }
2554
2555    review_inbound_surface(
2556        text,
2557        &[
2558            (
2559                "ListenStream=",
2560                "asks systemd to open a listening socket for this service, which \
2561                 07-security.md rule 2 forbids the product to have",
2562            ),
2563            (
2564                "ListenDatagram=",
2565                "asks systemd to open a listening socket for this service, which \
2566                 07-security.md rule 2 forbids the product to have",
2567            ),
2568        ],
2569        controls,
2570        findings,
2571    );
2572}
2573
2574fn review_launchd(
2575    text: &str,
2576    plan: &InstallPlan,
2577    controls: &mut Vec<String>,
2578    findings: &mut Vec<PrivilegeFinding>,
2579) {
2580    // A scheduling class is not an authority, so no value here is an excess.
2581    // Anything but `Interactive` is a shortfall: launchd applies it to every
2582    // runner the daemon starts. See `LAUNCHD_PROCESS_TYPE`.
2583    match plist_string_value(text, "ProcessType") {
2584        Some(value) if value == LAUNCHD_PROCESS_TYPE => {
2585            controls.push(format!("ProcessType={LAUNCHD_PROCESS_TYPE}"));
2586        }
2587        found => findings.push(PrivilegeFinding {
2588            kind: FindingKind::Shortfall,
2589            subject: "ProcessType".to_string(),
2590            detail: format!(
2591                "is {}, and launchd applies it to every runner the daemon starts, so their jobs \
2592                 run below normal priority. The daemon rewrites it to `{LAUNCHD_PROCESS_TYPE}` \
2593                 when it next starts with no runner; `service status` says how to restart it",
2594                found.map_or_else(
2595                    || "absent (launchd's `Standard`)".to_string(),
2596                    |value| format!("`{value}`")
2597                )
2598            ),
2599        }),
2600    }
2601
2602    match plan.start_mode() {
2603        StartMode::Boot => {
2604            if plist_bool_value(text, "SessionCreate") == Some(true) {
2605                findings.push(PrivilegeFinding {
2606                    kind: FindingKind::Excess,
2607                    subject: "SessionCreate".to_string(),
2608                    detail: "asks launchd to create a security session for a job that runs \
2609                             outside every login session and has no use for one"
2610                        .to_string(),
2611                });
2612            } else {
2613                controls.push("SessionCreate is not requested".to_string());
2614            }
2615            match plist_string_value(text, "UserName").as_deref() {
2616                Some("root") => {
2617                    controls.push("UserName=root, stated rather than inherited".to_string())
2618                }
2619                Some(other) => findings.push(PrivilegeFinding {
2620                    kind: FindingKind::Shortfall,
2621                    subject: "UserName".to_string(),
2622                    detail: format!(
2623                        "is `{other}`, which cannot unlock the System Keychain: \
2624                         /var/db/SystemKey is root-only, so the daemon would start and then \
2625                         find no credential"
2626                    ),
2627                }),
2628                None => findings.push(PrivilegeFinding {
2629                    kind: FindingKind::Shortfall,
2630                    subject: "UserName".to_string(),
2631                    detail: "is absent, so the account is launchd's implicit default and this \
2632                             review cannot confirm it"
2633                        .to_string(),
2634                }),
2635            }
2636        }
2637        StartMode::Login => {
2638            if let Some(named) = plist_string_value(text, "UserName") {
2639                findings.push(PrivilegeFinding {
2640                    kind: FindingKind::Excess,
2641                    subject: "UserName".to_string(),
2642                    detail: format!(
2643                        "names `{named}` in a LaunchAgent, which already runs as the operator; \
2644                         naming an account here asks launchd for a switch a login-mode \
2645                         registration has no reason to want"
2646                    ),
2647                });
2648            } else {
2649                controls
2650                    .push("no UserName: the agent runs as the operator and no other".to_string());
2651            }
2652        }
2653    }
2654
2655    review_inbound_surface(
2656        text,
2657        &[
2658            (
2659                "<key>Sockets</key>",
2660                "asks launchd to open a socket for this job, which 07-security.md rule 2 \
2661                 forbids the product to have",
2662            ),
2663            (
2664                "<key>MachServices</key>",
2665                "publishes a Mach service, which is the RPC surface 07-security.md rule 2 \
2666                 forbids the product to have",
2667            ),
2668        ],
2669        controls,
2670        findings,
2671    );
2672}
2673
2674fn review_scheduled_task(
2675    text: &str,
2676    controls: &mut Vec<String>,
2677    findings: &mut Vec<PrivilegeFinding>,
2678) {
2679    match xml_value(text, "RunLevel").as_deref() {
2680        Some("LeastPrivilege") => controls.push("RunLevel=LeastPrivilege".to_string()),
2681        Some(other) => findings.push(PrivilegeFinding {
2682            kind: FindingKind::Excess,
2683            subject: "RunLevel".to_string(),
2684            detail: format!(
2685                "is `{other}`, so the task runs with an elevated token whenever the operator is \
2686                 an administrator"
2687            ),
2688        }),
2689        None => findings.push(PrivilegeFinding {
2690            kind: FindingKind::Excess,
2691            subject: "RunLevel".to_string(),
2692            detail: "is absent, so Task Scheduler decides the token rather than the definition"
2693                .to_string(),
2694        }),
2695    }
2696
2697    match xml_value(text, "LogonType").as_deref() {
2698        Some("InteractiveToken") => controls.push("LogonType=InteractiveToken".to_string()),
2699        Some(other) => findings.push(PrivilegeFinding {
2700            kind: FindingKind::Excess,
2701            subject: "LogonType".to_string(),
2702            detail: format!(
2703                "is `{other}`, which asks Windows to store or synthesise a credential for this \
2704                 task; an interactive token needs neither"
2705            ),
2706        }),
2707        None => findings.push(PrivilegeFinding {
2708            kind: FindingKind::Shortfall,
2709            subject: "LogonType".to_string(),
2710            detail: "is absent, so this review cannot confirm that no credential is stored"
2711                .to_string(),
2712        }),
2713    }
2714}
2715
2716fn review_windows_service(
2717    text: &str,
2718    plan: &InstallPlan,
2719    controls: &mut Vec<String>,
2720    findings: &mut Vec<PrivilegeFinding>,
2721) {
2722    let directives = ini_directives(text, "windows-service");
2723
2724    match directives.get("ServiceType").map(String::as_str) {
2725        Some("OWN_PROCESS") => controls.push("ServiceType=OWN_PROCESS".to_string()),
2726        Some(other) => findings.push(PrivilegeFinding {
2727            kind: FindingKind::Excess,
2728            subject: "ServiceType".to_string(),
2729            detail: format!(
2730                "is `{other}`; an interactive or shared-process service reaches further than a \
2731                 daemon that only talks to GitHub over HTTPS"
2732            ),
2733        }),
2734        None => findings.push(PrivilegeFinding {
2735            kind: FindingKind::Shortfall,
2736            subject: "ServiceType".to_string(),
2737            detail: "is absent, so this review cannot confirm the service is not interactive"
2738                .to_string(),
2739        }),
2740    }
2741
2742    // The account is not a free choice on Windows: `d2`'s DACL decides it. What
2743    // the review can check is that the definition names the one account that
2744    // DACL admits, and no broader one.
2745    match directives.get("Account").map(String::as_str) {
2746        Some(account) if account == ServiceAccount::LocalSystem.as_str() => {
2747            controls.push(format!(
2748                "Account={account}: the only stock account the machine-scoped store's DACL \
2749                 (SY, BA, OW) admits"
2750            ));
2751        }
2752        Some(other) => findings.push(PrivilegeFinding {
2753            kind: FindingKind::Shortfall,
2754            subject: "Account".to_string(),
2755            detail: format!(
2756                "is `{other}`, which the machine-scoped store's DACL does not name, so the \
2757                 daemon would start and then find no credential. Widening that DACL is not this \
2758                 registration's to do: an ACE reaching `{other}` would also reach every other \
2759                 service running under it"
2760            ),
2761        }),
2762        None => findings.push(PrivilegeFinding {
2763            kind: FindingKind::Shortfall,
2764            subject: "Account".to_string(),
2765            detail: "is absent, so this review cannot confirm which account was registered"
2766                .to_string(),
2767        }),
2768    }
2769
2770    match directives.get("ReadWritePaths") {
2771        Some(value) => {
2772            let listed = split_quoted(value);
2773            review_writable_paths(
2774                DefinitionKind::WindowsService,
2775                "ReadWritePaths",
2776                &listed,
2777                plan,
2778                controls,
2779                findings,
2780            );
2781        }
2782        None => findings.push(PrivilegeFinding {
2783            kind: FindingKind::Shortfall,
2784            subject: "ReadWritePaths".to_string(),
2785            detail: "is absent, so the directories the service was installed against are not \
2786                     recorded"
2787                .to_string(),
2788        }),
2789    }
2790}
2791
2792// -- parsing helpers ---------------------------------------------------------
2793
2794/// Reads `key=value` lines out of one `[section]` of an INI-shaped document.
2795///
2796/// systemd's unit format and this module's Windows descriptor are both this
2797/// shape. A repeated key takes its last value, which is systemd's own rule for
2798/// every directive here.
2799fn ini_directives(text: &str, section: &str) -> BTreeMap<String, String> {
2800    let mut out = BTreeMap::new();
2801    let mut inside = false;
2802    for line in text.lines() {
2803        let line = line.trim();
2804        if line.starts_with('[') && line.ends_with(']') {
2805            inside = &line[1..line.len() - 1] == section;
2806            continue;
2807        }
2808        if !inside || line.is_empty() || line.starts_with('#') || line.starts_with(';') {
2809            continue;
2810        }
2811        if let Some((key, value)) = line.split_once('=') {
2812            out.insert(key.trim().to_string(), value.trim().to_string());
2813        }
2814    }
2815    out
2816}
2817
2818/// Splits a whitespace-separated list that may contain arguments quoted by
2819/// [`quote_argument`].
2820fn split_quoted(value: &str) -> Vec<String> {
2821    let mut out = Vec::new();
2822    let mut rest = value.trim();
2823    while !rest.is_empty() {
2824        if rest.starts_with('"') {
2825            // `executable_from_command_line` already implements Windows' quoting
2826            // rules; reuse it rather than write a second, subtly different one.
2827            if let Some(parsed) = executable_from_command_line(rest) {
2828                out.push(parsed.to_string_lossy().into_owned());
2829            }
2830            // Advance past the closing quote.
2831            let mut depth = 0usize;
2832            let mut end = rest.len();
2833            for (index, c) in rest.char_indices() {
2834                match c {
2835                    '\\' => depth += 1,
2836                    '"' => {
2837                        if depth.is_multiple_of(2) && index > 0 {
2838                            end = index + 1;
2839                            break;
2840                        }
2841                        depth = 0;
2842                    }
2843                    _ => depth = 0,
2844                }
2845            }
2846            rest = rest[end..].trim_start();
2847        } else {
2848            let end = rest.find(char::is_whitespace).unwrap_or(rest.len());
2849            out.push(rest[..end].to_string());
2850            rest = rest[end..].trim_start();
2851        }
2852    }
2853    out
2854}
2855
2856/// The text between the **first** `<tag>` and its `</tag>`, unescaped, for the
2857/// small, known documents this module renders. Not a general XML parser and
2858/// does not pretend to be one.
2859///
2860/// "First" matters for a document that repeats a tag: a Task Scheduler task has
2861/// an `<Enabled>` in its trigger *and* one in its settings, so a caller after
2862/// the second one passes the slice that starts at `<Settings>`.
2863///
2864/// `pub(crate)` rather than private because [`crate::wsl::task`] reads back a
2865/// second kind of Task Scheduler document; one reader for both is the same
2866/// argument [`xml_escape`] makes for one escaper.
2867pub(crate) fn xml_value(text: &str, tag: &str) -> Option<String> {
2868    let open = format!("<{tag}>");
2869    let close = format!("</{tag}>");
2870    let start = text.find(&open)? + open.len();
2871    let end = text[start..].find(&close)? + start;
2872    Some(xml_unescape(text[start..end].trim()))
2873}
2874
2875/// Whether Task Scheduler will launch a login task automatically.
2876///
2877/// Its `/Query /XML` output normalises away values that equal the schema
2878/// default. In particular, an enabled task commonly has no `<Enabled>` node at
2879/// all, so only an explicit `false` means disabled.
2880#[cfg(any(windows, test))]
2881fn windows_login_task_starts_automatically(document: &str) -> bool {
2882    document.contains("<LogonTrigger>")
2883        && xml_value(document, "Enabled").as_deref() != Some("false")
2884}
2885
2886/// The `<string>` that follows `<key>key</key>` in a property list.
2887fn plist_value_after_key<'a>(text: &'a str, key: &str) -> Option<&'a str> {
2888    let marker = format!("<key>{key}</key>");
2889    let start = text.find(&marker)? + marker.len();
2890    Some(text[start..].trim_start())
2891}
2892
2893pub fn plist_string_value(text: &str, key: &str) -> Option<String> {
2894    let rest = plist_value_after_key(text, key)?;
2895    if !rest.starts_with("<string>") {
2896        return None;
2897    }
2898    xml_value(rest, "string")
2899}
2900
2901fn plist_bool_value(text: &str, key: &str) -> Option<bool> {
2902    let rest = plist_value_after_key(text, key)?;
2903    if rest.starts_with("<true/>") {
2904        Some(true)
2905    } else if rest.starts_with("<false/>") {
2906        Some(false)
2907    } else {
2908        None
2909    }
2910}
2911
2912// ---------------------------------------------------------------------------
2913// The install record
2914// ---------------------------------------------------------------------------
2915
2916/// The version of [`InstallRecord`] on disk.
2917///
2918/// Bumped when a record written by an older version can no longer be read.
2919/// `service status` reports an unreadable record as a problem with a remedy
2920/// rather than failing, because the remedy — uninstall and install again —
2921/// touches no configuration, no secret and no cache.
2922pub const RECORD_SCHEMA_VERSION: u32 = 1;
2923
2924/// What `service install` wrote down, and what `service status` reads back.
2925///
2926/// This is the *record* half of `05-infrastructure.md` item 6. The service
2927/// manager also knows the binary path, and the two are compared: a registration
2928/// whose command line no longer matches the record is a registration something
2929/// else has edited, and saying so is more useful than picking one of them to
2930/// believe.
2931///
2932/// It holds no credential. The nearest thing is the *location* of the secret
2933/// store, which `d2` publishes for `host show` to print, and even that is
2934/// derived rather than stored.
2935#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
2936pub struct InstallRecord {
2937    /// [`RECORD_SCHEMA_VERSION`] at the time of writing.
2938    pub schema_version: u32,
2939    /// What the operating system calls the registration.
2940    pub service_name: String,
2941    /// Which service manager holds it.
2942    pub manager: String,
2943    /// Boot or login.
2944    pub start_mode: StartMode,
2945    /// The account it runs under.
2946    pub account: ServiceAccount,
2947    /// **The resolved absolute path of the binary at install time.** Item 6.
2948    pub binary: PathBuf,
2949    /// Where [`Self::binary`] was copied from, when it is a copy.
2950    ///
2951    /// `#[serde(default)]` rather than a schema bump: a record written before
2952    /// this existed is still readable, and reads as `None`. That is not a
2953    /// silent downgrade — `None` means the registration names a package
2954    /// manager's own file, which is the layout that cannot be upgraded while
2955    /// the service runs, and `service status` says so by name.
2956    #[serde(default)]
2957    pub source_binary: Option<PathBuf>,
2958    /// The arguments it was registered with.
2959    pub arguments: Vec<String>,
2960    /// The restart-on-failure delay.
2961    pub restart_delay_secs: u64,
2962    /// How long it must run before the failure count resets.
2963    pub restart_reset_secs: u64,
2964    /// The rotating diagnostic log's stem. Item 4.
2965    pub log_file: PathBuf,
2966    /// Whether the manager was asked not to start this by itself. Always
2967    /// `false` for a registration an operator made; see
2968    /// [`InstallRequest::started_on_demand`].
2969    ///
2970    /// `#[serde(default)]` so that a record missing the field is rejected by
2971    /// the schema check below, with its remedy, rather than by a parse error
2972    /// that names a field an operator has never heard of.
2973    #[serde(default)]
2974    pub starts_on_demand: bool,
2975    /// Where the platform's definition file was written, when there is one.
2976    pub definition_path: Option<PathBuf>,
2977    /// When the registration was made.
2978    pub installed_at: DateTime<Utc>,
2979    /// Which build of the product made it.
2980    pub installed_by_version: String,
2981    // A TOML table has to follow every scalar at its level, so this field is
2982    // last by necessity rather than by taste.
2983    /// The four directories the registration was installed against. Item 2's
2984    /// *"configured cache and runtime directories"*.
2985    pub directories: ServiceDirectories,
2986}
2987
2988impl InstallRecord {
2989    /// Where the record lives, given this host's directories.
2990    #[must_use]
2991    pub fn path(paths: &AppPaths) -> PathBuf {
2992        paths.config_dir().join(RECORD_FILE)
2993    }
2994
2995    /// Builds the record for a plan that has just been applied.
2996    #[must_use]
2997    pub fn of(plan: &InstallPlan, definition: &ServiceDefinition, at: DateTime<Utc>) -> Self {
2998        Self {
2999            schema_version: RECORD_SCHEMA_VERSION,
3000            service_name: plan.identity().name().to_string(),
3001            manager: definition.kind().manager().to_string(),
3002            start_mode: plan.start_mode(),
3003            account: plan.account().clone(),
3004            binary: plan.binary().to_path_buf(),
3005            arguments: plan
3006                .arguments()
3007                .iter()
3008                .map(|argument| argument.to_string_lossy().into_owned())
3009                .collect(),
3010            restart_delay_secs: plan.restart().delay().as_secs(),
3011            restart_reset_secs: plan.restart().reset_after().as_secs(),
3012            starts_on_demand: plan.is_on_demand(),
3013            source_binary: plan.source_binary().map(Path::to_path_buf),
3014            log_file: plan.directories().log_file(),
3015            definition_path: definition.install_path().map(Path::to_path_buf),
3016            installed_at: at,
3017            installed_by_version: env!("CARGO_PKG_VERSION").to_string(),
3018            directories: plan.directories().clone(),
3019        }
3020    }
3021
3022    /// The plan this record describes, as this build would register it.
3023    ///
3024    /// The directories, binary, arguments and restart policy are the recorded
3025    /// ones; everything else -- hardening, the secret store's guard -- is what
3026    /// this build renders. That difference is the point: it is how a
3027    /// registration written by an older build is compared against, and
3028    /// repaired to, the current definition.
3029    #[must_use]
3030    pub fn plan(&self, identity: ServiceIdentity) -> InstallPlan {
3031        let plan = InstallPlan::unchecked(
3032            identity,
3033            self.start_mode,
3034            self.binary.clone(),
3035            self.directories.clone(),
3036        )
3037        .with_arguments(self.arguments.clone())
3038        .with_restart(self.restart());
3039        let plan = if self.starts_on_demand {
3040            plan.started_on_demand()
3041        } else {
3042            plan
3043        };
3044        match crate::secrets::PlatformSecretStore::for_start_mode(self.start_mode) {
3045            Ok(store) => plan.with_secret_guard(store.guard()),
3046            Err(_) => plan,
3047        }
3048    }
3049
3050    /// The restart policy this record describes, or [`RestartPolicy::default`]
3051    /// when the recorded numbers are outside the supported range — which can
3052    /// only happen to a record something else has edited.
3053    #[must_use]
3054    pub fn restart(&self) -> RestartPolicy {
3055        RestartPolicy::new(
3056            Duration::from_secs(self.restart_delay_secs),
3057            Duration::from_secs(self.restart_reset_secs),
3058        )
3059        .unwrap_or_default()
3060    }
3061
3062    /// Reads the record, or reports that there is none.
3063    ///
3064    /// `Ok(None)` means no registration was made by this product on this host.
3065    ///
3066    /// # Errors
3067    ///
3068    /// [`ServiceError::Record`] when the file exists and cannot be read,
3069    /// [`ServiceError::RecordNotPermitted`] when it exists and this account may
3070    /// not read it, and [`ServiceError::RecordUnreadable`] when it is not a
3071    /// record this version understands.
3072    pub fn read(paths: &AppPaths) -> Result<Option<Self>, ServiceError> {
3073        let path = Self::path(paths);
3074        let text = match std::fs::read_to_string(&path) {
3075            Ok(text) => text,
3076            Err(error) if error.kind() == std::io::ErrorKind::NotFound => return Ok(None),
3077            // Told apart from every other read failure because `status` carries
3078            // on through this one alone. See `ServiceError::RecordNotPermitted`.
3079            Err(error) if error.kind() == std::io::ErrorKind::PermissionDenied => {
3080                return Err(ServiceError::RecordNotPermitted {
3081                    path,
3082                    detail: error.to_string(),
3083                });
3084            }
3085            Err(error) => {
3086                return Err(ServiceError::Record {
3087                    operation: "read",
3088                    path,
3089                    detail: error.to_string(),
3090                });
3091            }
3092        };
3093        let record: Self =
3094            toml::from_str(&text).map_err(|error| ServiceError::RecordUnreadable {
3095                path: path.clone(),
3096                detail: error.to_string(),
3097            })?;
3098        if record.schema_version != RECORD_SCHEMA_VERSION {
3099            return Err(ServiceError::RecordUnreadable {
3100                path,
3101                detail: format!(
3102                    "it declares schema version {} and this build reads version {}",
3103                    record.schema_version, RECORD_SCHEMA_VERSION
3104                ),
3105            });
3106        }
3107        Ok(Some(record))
3108    }
3109
3110    /// Writes the record, replacing whatever was there.
3111    ///
3112    /// # Errors
3113    ///
3114    /// [`ServiceError::Record`].
3115    pub fn write(&self, paths: &AppPaths) -> Result<(), ServiceError> {
3116        use std::io::Write as _;
3117
3118        let path = Self::path(paths);
3119        let text = toml::to_string_pretty(self).map_err(|error| ServiceError::Record {
3120            operation: "encode",
3121            path: path.clone(),
3122            detail: error.to_string(),
3123        })?;
3124        if let Some(parent) = path.parent() {
3125            std::fs::create_dir_all(parent).map_err(|error| ServiceError::Record {
3126                operation: "write",
3127                path: path.clone(),
3128                detail: error.to_string(),
3129            })?;
3130        }
3131        let parent = path.parent().ok_or_else(|| ServiceError::Record {
3132            operation: "write",
3133            path: path.clone(),
3134            detail: "the record path has no parent directory".to_string(),
3135        })?;
3136        let mut temporary =
3137            tempfile::NamedTempFile::new_in(parent).map_err(|error| ServiceError::Record {
3138                operation: "write",
3139                path: path.clone(),
3140                detail: error.to_string(),
3141            })?;
3142        temporary
3143            .write_all(text.as_bytes())
3144            .and_then(|()| temporary.as_file().sync_all())
3145            .map_err(|error| ServiceError::Record {
3146                operation: "write",
3147                path: path.clone(),
3148                detail: error.to_string(),
3149            })?;
3150        // ------------------------------------------------------------------
3151        // THE RECORD IS READABLE BY THE ACCOUNT WHOSE DIRECTORY IT IS IN.
3152        // ------------------------------------------------------------------
3153        // `NamedTempFile` creates at `0600`, which is right for a temporary
3154        // file and wrong for this one. A boot-mode `service install` runs under
3155        // `sudo`, so `0600` made the record `root`-owned and unreadable by the
3156        // operator — and `service status`, which asks no privilege of anybody,
3157        // failed on their own host with `Permission denied` and a remedy that
3158        // named the command that had just failed.
3159        //
3160        // `0644` does not widen anything: `RECORD_FILE`'s own documentation is
3161        // that this is *"non-secret TOML"* holding no credential, and
3162        // `AppPaths::create_all` puts it in a `0700` directory, so no other
3163        // local account can reach it whatever its own mode says. It is the same
3164        // arrangement the rotating diagnostics already have.
3165        #[cfg(unix)]
3166        {
3167            use std::os::unix::fs::PermissionsExt as _;
3168
3169            temporary
3170                .as_file()
3171                .set_permissions(std::fs::Permissions::from_mode(0o644))
3172                .map_err(|error| ServiceError::Record {
3173                    operation: "write",
3174                    path: path.clone(),
3175                    detail: error.to_string(),
3176                })?;
3177        }
3178        temporary
3179            .persist(&path)
3180            .map(|_| ())
3181            .map_err(|error| ServiceError::Record {
3182                operation: "write",
3183                path,
3184                detail: error.error.to_string(),
3185            })
3186    }
3187
3188    /// Removes the record. Returns whether there was one.
3189    ///
3190    /// # Errors
3191    ///
3192    /// [`ServiceError::Record`].
3193    pub fn remove(paths: &AppPaths) -> Result<bool, ServiceError> {
3194        let path = Self::path(paths);
3195        match std::fs::remove_file(&path) {
3196            Ok(()) => Ok(true),
3197            Err(error) if error.kind() == std::io::ErrorKind::NotFound => Ok(false),
3198            Err(error) => Err(ServiceError::Record {
3199                operation: "remove",
3200                path,
3201                detail: error.to_string(),
3202            }),
3203        }
3204    }
3205}
3206
3207// ---------------------------------------------------------------------------
3208// The last successful GitHub contact
3209// ---------------------------------------------------------------------------
3210
3211/// The heartbeat file's own schema version.
3212const CONTACT_SCHEMA_VERSION: u32 = 1;
3213
3214#[derive(Debug, Clone, Serialize, Deserialize)]
3215struct ContactRecord {
3216    schema_version: u32,
3217    last_success: DateTime<Utc>,
3218}
3219
3220/// Records that GitHub was reached successfully, for `service status` to
3221/// report.
3222///
3223/// # The contract, for `f3` and for the agent
3224///
3225/// Journey 5 step 4 requires `service status` to report *"the last successful
3226/// GitHub contact"*, and `service status` runs in the operator's terminal while
3227/// the daemon runs in a service. They share no memory, so the fact has to be on
3228/// disk, and this is the file. **The daemon calls this after each successful
3229/// GitHub call; nothing else writes it.**
3230///
3231/// It is a single timestamp and deliberately not a log: an operator asking this
3232/// question wants to know whether the agent is alive and reaching GitHub *now*,
3233/// and a value that is minutes old answers it. Writing it is a whole-file
3234/// replace through a temporary in the same directory, so a status command can
3235/// never read a half-written timestamp.
3236///
3237/// # Errors
3238///
3239/// [`ServiceError::Record`] when `state/` cannot be written.
3240pub fn record_github_contact(paths: &AppPaths, at: DateTime<Utc>) -> Result<(), ServiceError> {
3241    write_state_record(
3242        &contact_path(paths),
3243        &ContactRecord {
3244            schema_version: CONTACT_SCHEMA_VERSION,
3245            last_success: at,
3246        },
3247    )
3248}
3249
3250/// Reads the last successful GitHub contact, or reports that none was recorded.
3251///
3252/// `Ok(None)` is the honest answer on a host whose agent has never run, and it
3253/// is what `service status` prints rather than a zero timestamp.
3254///
3255/// # Errors
3256///
3257/// [`ServiceError::Record`] when the file exists and cannot be read or parsed.
3258/// A malformed heartbeat is reported rather than treated as absence: absence
3259/// means *"the agent has never reached GitHub"*, and reporting a parse failure
3260/// as that would be a wrong answer to the question Journey 5 asks.
3261pub fn last_github_contact(paths: &AppPaths) -> Result<Option<DateTime<Utc>>, ServiceError> {
3262    Ok(read_state_record::<ContactRecord>(&contact_path(paths))?.map(|record| record.last_success))
3263}
3264
3265/// Where the heartbeat lives.
3266#[must_use]
3267pub fn contact_path(paths: &AppPaths) -> PathBuf {
3268    paths.state_dir().join(CONTACT_FILE)
3269}
3270
3271/// Writes a small TOML record under `state/`, through a temporary in the same
3272/// directory and a rename, so a status command never reads half of one.
3273pub(crate) fn write_state_record(path: &Path, record: &impl Serialize) -> Result<(), ServiceError> {
3274    let failed = |detail: String| ServiceError::Record {
3275        operation: "write",
3276        path: path.to_path_buf(),
3277        detail,
3278    };
3279    let text = toml::to_string_pretty(record).map_err(|error| failed(error.to_string()))?;
3280    let directory = path.parent().unwrap_or_else(|| Path::new("."));
3281    std::fs::create_dir_all(directory).map_err(|error| failed(error.to_string()))?;
3282    let temporary = path.with_extension("toml.new");
3283    std::fs::write(&temporary, text).map_err(|error| failed(error.to_string()))?;
3284    std::fs::rename(&temporary, path).map_err(|error| failed(error.to_string()))
3285}
3286
3287/// Reads a record [`write_state_record`] wrote. `Ok(None)` when there is none;
3288/// one that exists and does not parse is an error, never absence.
3289pub(crate) fn read_state_record<T: serde::de::DeserializeOwned>(
3290    path: &Path,
3291) -> Result<Option<T>, ServiceError> {
3292    let failed = |detail: String| ServiceError::Record {
3293        operation: "read",
3294        path: path.to_path_buf(),
3295        detail,
3296    };
3297    let text = match std::fs::read_to_string(path) {
3298        Ok(text) => text,
3299        Err(error) if error.kind() == std::io::ErrorKind::NotFound => return Ok(None),
3300        Err(error) => return Err(failed(error.to_string())),
3301    };
3302    toml::from_str(&text)
3303        .map(Some)
3304        .map_err(|error| failed(error.to_string()))
3305}
3306
3307/// Removes a record; one that is not there is already removed.
3308pub(crate) fn remove_state_record(path: &Path) -> Result<(), ServiceError> {
3309    match std::fs::remove_file(path) {
3310        Ok(()) => Ok(()),
3311        Err(error) if error.kind() == std::io::ErrorKind::NotFound => Ok(()),
3312        Err(error) => Err(ServiceError::Record {
3313            operation: "remove",
3314            path: path.to_path_buf(),
3315            detail: error.to_string(),
3316        }),
3317    }
3318}
3319
3320// ---------------------------------------------------------------------------
3321// The credential GitHub rejected
3322// ---------------------------------------------------------------------------
3323
3324#[derive(Debug, Serialize, Deserialize)]
3325#[serde(deny_unknown_fields)]
3326struct CredentialRejectionRecord {
3327    schema_version: u32,
3328    rejected_since: DateTime<Utc>,
3329}
3330
3331/// Records that GitHub rejected the daemon's credential, keeping the first
3332/// moment of an ongoing rejection.
3333///
3334/// # Why a file of its own
3335///
3336/// A rejected credential used to be visible only in the daemon's log: the
3337/// contact heartbeat simply stopped moving, `service status` still printed
3338/// `healthy`, and the TUI -- which authenticates with its own, perhaps freshly
3339/// renewed, copy -- showed nothing at all while the service started no runner
3340/// for hours. Separate from [`CONTACT_FILE`] so that an older build reading
3341/// that file is not handed a field it refuses.
3342///
3343/// # Errors
3344///
3345/// [`ServiceError::Record`] when `state/` cannot be written.
3346pub fn record_github_credential_rejected(
3347    paths: &AppPaths,
3348    at: DateTime<Utc>,
3349) -> Result<(), ServiceError> {
3350    if github_credential_rejected_since(paths)
3351        .ok()
3352        .flatten()
3353        .is_some()
3354    {
3355        return Ok(());
3356    }
3357    write_state_record(
3358        &credential_rejection_path(paths),
3359        &CredentialRejectionRecord {
3360            schema_version: CONTACT_SCHEMA_VERSION,
3361            rejected_since: at,
3362        },
3363    )
3364}
3365
3366/// Clears a recorded rejection once GitHub accepts the credential again.
3367///
3368/// # Errors
3369///
3370/// [`ServiceError::Record`] when the record exists and cannot be removed.
3371pub fn clear_github_credential_rejection(paths: &AppPaths) -> Result<(), ServiceError> {
3372    remove_state_record(&credential_rejection_path(paths))
3373}
3374
3375/// Since when GitHub has been rejecting the daemon's credential, if it is.
3376///
3377/// # Errors
3378///
3379/// [`ServiceError::Record`] when the file exists and cannot be read or parsed.
3380pub fn github_credential_rejected_since(
3381    paths: &AppPaths,
3382) -> Result<Option<DateTime<Utc>>, ServiceError> {
3383    Ok(
3384        read_state_record::<CredentialRejectionRecord>(&credential_rejection_path(paths))?
3385            .map(|record| record.rejected_since),
3386    )
3387}
3388
3389/// Where the rejection record lives.
3390#[must_use]
3391pub fn credential_rejection_path(paths: &AppPaths) -> PathBuf {
3392    paths.state_dir().join(CREDENTIAL_REJECTION_FILE)
3393}
3394
3395#[derive(Debug, Serialize, Deserialize)]
3396#[serde(deny_unknown_fields)]
3397struct CredentialUnreadableRecord {
3398    schema_version: u32,
3399    unreadable_since: DateTime<Utc>,
3400}
3401
3402/// Records that the daemon could not read its stored credential, keeping the
3403/// first moment of an ongoing failure.
3404///
3405/// # Why a file of its own
3406///
3407/// A daemon that cannot read its credential exits, and the service manager
3408/// starts it again a few seconds later, for ever. `service status` reads the
3409/// registration, which is fine, and the heartbeat, which merely stops moving --
3410/// so it said `healthy` while a macOS host that had just been upgraded was
3411/// locked out of its own keychain item and started no runner. The reason was
3412/// only in the daemon's log. The daemon is the one process whose view of the
3413/// store matters, and it shares no memory with `service status`, so it says
3414/// so on disk; [`clear_credential_unreadable`] removes it as soon as a read
3415/// succeeds.
3416///
3417/// # Errors
3418///
3419/// [`ServiceError::Record`] when `state/` cannot be written.
3420pub fn record_credential_unreadable(
3421    paths: &AppPaths,
3422    at: DateTime<Utc>,
3423) -> Result<(), ServiceError> {
3424    if credential_unreadable_since(paths).ok().flatten().is_some() {
3425        return Ok(());
3426    }
3427    write_state_record(
3428        &credential_unreadable_path(paths),
3429        &CredentialUnreadableRecord {
3430            schema_version: CONTACT_SCHEMA_VERSION,
3431            unreadable_since: at,
3432        },
3433    )
3434}
3435
3436/// Clears a recorded failure once the daemon reads its credential again.
3437///
3438/// # Errors
3439///
3440/// [`ServiceError::Record`] when the record exists and cannot be removed.
3441pub fn clear_credential_unreadable(paths: &AppPaths) -> Result<(), ServiceError> {
3442    remove_state_record(&credential_unreadable_path(paths))
3443}
3444
3445/// Since when the daemon has been unable to read its stored credential, if it
3446/// has.
3447///
3448/// # Errors
3449///
3450/// [`ServiceError::Record`] when the file exists and cannot be read or parsed.
3451pub fn credential_unreadable_since(
3452    paths: &AppPaths,
3453) -> Result<Option<DateTime<Utc>>, ServiceError> {
3454    Ok(
3455        read_state_record::<CredentialUnreadableRecord>(&credential_unreadable_path(paths))?
3456            .map(|record| record.unreadable_since),
3457    )
3458}
3459
3460/// Where the unreadable-credential record lives.
3461#[must_use]
3462pub fn credential_unreadable_path(paths: &AppPaths) -> PathBuf {
3463    paths.state_dir().join(CREDENTIAL_UNREADABLE_FILE)
3464}
3465
3466/// How to sign this host in again for `mode`: the exact command, in
3467/// backticks, and where to run it when that matters.
3468///
3469/// A machine-scoped store needs privilege: `sudo` on Unix, an elevated
3470/// terminal on Windows. A login-mode store on macOS is the login keychain,
3471/// which is unlocked for the desktop session only, so an SSH session cannot
3472/// write it.
3473#[must_use]
3474pub fn sign_in_instruction(mode: StartMode) -> String {
3475    let (command, place) = match mode {
3476        StartMode::Boot if cfg!(unix) => ("sudo runner-manager auth login --start-at boot", ""),
3477        StartMode::Boot => (
3478            "runner-manager auth login --start-at boot",
3479            " from an elevated terminal",
3480        ),
3481        StartMode::Login => (
3482            "runner-manager auth login --start-at login",
3483            MACOS_LOGIN_PLACE,
3484        ),
3485    };
3486    format!("`{command}`{place}")
3487}
3488
3489/// Where a login-mode sign-in has to happen on macOS, as a phrase to follow
3490/// the command; nothing elsewhere.
3491pub const MACOS_LOGIN_PLACE: &str = if cfg!(target_os = "macos") {
3492    " in a Terminal in this Mac's desktop session (an SSH session cannot unlock the login \
3493     keychain)"
3494} else {
3495    ""
3496};
3497
3498// ---------------------------------------------------------------------------
3499// A definition written by an older build
3500// ---------------------------------------------------------------------------
3501
3502/// An installed service definition that differs from what this build renders
3503/// for the same recorded installation.
3504///
3505/// # Why this exists
3506///
3507/// An upgrade replaces the binary a registration runs and nothing else. A
3508/// systemd unit written by 0.4.7 still carried `LoadCredential=`, which hands
3509/// the daemon a copy of the credential frozen at service start, and lacked
3510/// write access to the credential store. Every release after the fix kept
3511/// running under that unit: the daemon could neither renew the token nor see
3512/// one renewed by anybody else, and stopped starting runners the moment the
3513/// frozen access token expired.
3514#[derive(Debug, Clone, PartialEq, Eq)]
3515pub struct DefinitionDrift {
3516    /// The definition file on disk.
3517    pub path: PathBuf,
3518    /// The build that wrote it, as its install record says.
3519    pub installed_by_version: String,
3520    /// What this build renders in its place.
3521    pub rendered: String,
3522    /// The start mode it was installed for.
3523    pub start_mode: StartMode,
3524    /// Which manager reads the definition.
3525    pub manager: DefinitionKind,
3526}
3527
3528/// Compares the installed definition against this build's rendering.
3529///
3530/// A systemd unit is compared byte for byte. A launchd plist is compared on
3531/// `ProcessType` alone (see [`launchd_definition_drift`]). `None` for every
3532/// other manager, for a definition that cannot be read, and for one that
3533/// already matches. The file is read through `controls`, so a double never
3534/// compares against a real host's definition.
3535fn definition_drift(
3536    controls: &dyn ControlFactory,
3537    identity: &ServiceIdentity,
3538    record: &InstallRecord,
3539) -> Option<DefinitionDrift> {
3540    let compare = match host_definition_kind(record.start_mode) {
3541        DefinitionKind::SystemdUnit => systemd_definition_drift,
3542        DefinitionKind::LaunchdPlist => launchd_definition_drift,
3543        DefinitionKind::WindowsService | DefinitionKind::WindowsScheduledTask => return None,
3544    };
3545    let path = record.definition_path.clone()?;
3546    let installed = controls.installed_definition(&path)?;
3547    compare(identity, record, path, &installed)
3548}
3549
3550/// A plist whose `ProcessType` is not [`LAUNCHD_PROCESS_TYPE`], with the plist
3551/// this build renders in its place.
3552///
3553/// Only that one key, unlike the systemd comparison. The daemon repairs this
3554/// drift by itself (`daemon.rs`, `converge_launchd_definition`), so a
3555/// whole-file comparison would make it rewrite and reload over every edit an
3556/// operator made to the plist. A plist that does carry the wrong value is
3557/// replaced whole, as `service install` would replace it.
3558fn launchd_definition_drift(
3559    identity: &ServiceIdentity,
3560    record: &InstallRecord,
3561    path: PathBuf,
3562    installed: &str,
3563) -> Option<DefinitionDrift> {
3564    (plist_string_value(installed, "ProcessType").as_deref() != Some(LAUNCHD_PROCESS_TYPE)).then(
3565        || DefinitionDrift {
3566            path,
3567            installed_by_version: record.installed_by_version.clone(),
3568            rendered: launchd_plist(&record.plan(identity.clone())),
3569            start_mode: record.start_mode,
3570            manager: DefinitionKind::LaunchdPlist,
3571        },
3572    )
3573}
3574
3575fn systemd_definition_drift(
3576    identity: &ServiceIdentity,
3577    record: &InstallRecord,
3578    path: PathBuf,
3579    installed: &str,
3580) -> Option<DefinitionDrift> {
3581    let rendered = systemd_unit(&record.plan(identity.clone()));
3582    (installed != rendered).then(|| DefinitionDrift {
3583        path,
3584        installed_by_version: record.installed_by_version.clone(),
3585        rendered,
3586        start_mode: record.start_mode,
3587        manager: DefinitionKind::SystemdUnit,
3588    })
3589}
3590
3591// ---------------------------------------------------------------------------
3592// The last runner-root refusal
3593// ---------------------------------------------------------------------------
3594
3595const ROOT_REFUSAL_SCHEMA_VERSION: u32 = 1;
3596
3597#[derive(Debug, Clone, Serialize, Deserialize)]
3598struct RootRefusalFile {
3599    schema_version: u32,
3600    /// Keyed by policy, because the fact is per-policy and not per-host.
3601    #[serde(default)]
3602    refusals: BTreeMap<String, RootRefusalEntry>,
3603}
3604
3605#[derive(Debug, Clone, Serialize, Deserialize)]
3606struct RootRefusalEntry {
3607    at: DateTime<Utc>,
3608    kind: String,
3609    root: String,
3610    detail: String,
3611}
3612
3613/// One policy's last runner-root refusal, as `service status` reports it.
3614#[derive(Debug, Clone, PartialEq, Eq)]
3615pub struct RunnerRootRefusal {
3616    /// The policy that placed no runner.
3617    pub policy: String,
3618    /// When it was last refused.
3619    pub at: DateTime<Utc>,
3620    /// The closed-vocabulary cause, as `RunnerRootError::kind` names it.
3621    pub kind: String,
3622    /// The root it was refused.
3623    pub root: String,
3624    /// The refusal in full, including the remediation. Not redacted: this file
3625    /// is read by a command in the operator's terminal, not by the log sink.
3626    pub detail: String,
3627}
3628
3629/// Records that one policy could not use its runner root.
3630///
3631/// # The contract, and why a file
3632///
3633/// A launch the runner root refuses fails before any attempt row exists, so the
3634/// journal cannot carry it and no screen can show it. The daemon's own log
3635/// cannot either: it redacts every field it does not allow-list and scrubs
3636/// anything path-shaped out of the ones it does, which is most of a
3637/// `RunnerRootError`. So the sentence an operator needs — the directory and the
3638/// remediation, verbatim — has nowhere to go, and the failure reads
3639/// `runner_start_failed reason=other` once per poll for as long as it lasts.
3640/// That is not hypothetical: it cost three hours.
3641///
3642/// **Keyed by policy, and that is load-bearing.** A host runs several policies
3643/// and they do not share a fate: `allocate_persistent_slot` tolerates an
3644/// unresolvable host default, so a repository with its own persistent root
3645/// places runners happily while an ephemeral policy on a withheld volume places
3646/// none. A single host-wide slot would have the working policy clear the broken
3647/// one's record on the same reconcile pass, and `service status` would report a
3648/// healthy host that had started zero runners for an entire target.
3649///
3650/// **The daemon writes this when a root refuses a launch and clears the policy's
3651/// entry when one succeeds; nothing else writes it.** It keeps the last refusal
3652/// per policy rather than a history, for the reason [`record_github_contact`] is
3653/// a single timestamp: the operator is asking whether runners can be placed
3654/// *now*.
3655///
3656/// Written whole through a temporary in the same directory, so a status command
3657/// cannot read half a record.
3658///
3659/// # Errors
3660///
3661/// [`ServiceError::Record`] when `state/` cannot be read or written.
3662pub fn record_runner_root_refusal(
3663    paths: &AppPaths,
3664    policy: &str,
3665    at: DateTime<Utc>,
3666    kind: &str,
3667    root: &str,
3668    detail: &str,
3669) -> Result<(), ServiceError> {
3670    let mut file = read_refusal_file(paths)?.unwrap_or(RootRefusalFile {
3671        schema_version: ROOT_REFUSAL_SCHEMA_VERSION,
3672        refusals: BTreeMap::new(),
3673    });
3674    file.schema_version = ROOT_REFUSAL_SCHEMA_VERSION;
3675    file.refusals.insert(
3676        policy.to_owned(),
3677        RootRefusalEntry {
3678            at,
3679            kind: kind.to_owned(),
3680            root: root.to_owned(),
3681            detail: detail.to_owned(),
3682        },
3683    );
3684    write_refusal_file(paths, &file)
3685}
3686
3687/// Clears one policy's entry, because that policy placed a runner.
3688///
3689/// Removes the file once the last entry goes, so a host that is working leaves
3690/// nothing behind for a later `service status` to report. A policy with no entry
3691/// is not an error: this is called on every successful placement, and almost
3692/// every one of those follows another success.
3693///
3694/// # Errors
3695///
3696/// [`ServiceError::Record`] when `state/` cannot be read or written.
3697pub fn clear_runner_root_refusal(paths: &AppPaths, policy: &str) -> Result<(), ServiceError> {
3698    let Some(mut file) = read_refusal_file(paths)? else {
3699        return Ok(());
3700    };
3701    if file.refusals.remove(policy).is_none() {
3702        return Ok(());
3703    }
3704    if file.refusals.is_empty() {
3705        let path = root_refusal_path(paths);
3706        return match std::fs::remove_file(&path) {
3707            Ok(()) => Ok(()),
3708            Err(error) if error.kind() == std::io::ErrorKind::NotFound => Ok(()),
3709            Err(error) => Err(ServiceError::Record {
3710                operation: "remove",
3711                path,
3712                detail: error.to_string(),
3713            }),
3714        };
3715    }
3716    write_refusal_file(paths, &file)
3717}
3718
3719/// Every policy whose runner root refused it and has not since succeeded.
3720///
3721/// Ordered by policy, so two runs of `service status` on an unchanged host
3722/// print the same thing in the same order.
3723///
3724/// # Errors
3725///
3726/// [`ServiceError::Record`] when the file exists and cannot be read or parsed.
3727/// Reported rather than treated as absence, for the reason
3728/// [`last_github_contact`] gives: absence means "every policy is placing
3729/// runners", and answering a parse failure with that would be a wrong answer.
3730pub fn runner_root_refusals(paths: &AppPaths) -> Result<Vec<RunnerRootRefusal>, ServiceError> {
3731    Ok(read_refusal_file(paths)?
3732        .map(|file| {
3733            file.refusals
3734                .into_iter()
3735                .map(|(policy, entry)| RunnerRootRefusal {
3736                    policy,
3737                    at: entry.at,
3738                    kind: entry.kind,
3739                    root: entry.root,
3740                    detail: entry.detail,
3741                })
3742                .collect()
3743        })
3744        .unwrap_or_default())
3745}
3746
3747fn read_refusal_file(paths: &AppPaths) -> Result<Option<RootRefusalFile>, ServiceError> {
3748    read_state_record(&root_refusal_path(paths))
3749}
3750
3751fn write_refusal_file(paths: &AppPaths, file: &RootRefusalFile) -> Result<(), ServiceError> {
3752    write_state_record(&root_refusal_path(paths), file)
3753}
3754
3755/// Where the refusals live.
3756#[must_use]
3757pub fn root_refusal_path(paths: &AppPaths) -> PathBuf {
3758    paths.state_dir().join(ROOT_REFUSAL_FILE)
3759}
3760
3761// ---------------------------------------------------------------------------
3762// The recorded binary path
3763// ---------------------------------------------------------------------------
3764
3765/// What became of the absolute path `install` recorded.
3766///
3767/// `05-infrastructure.md` item 6 in one type. Three of the four variants are
3768/// errors, and `service status` reports them as errors rather than as health —
3769/// which is the whole point, because the npm case produces a service that looks
3770/// installed, is registered, and cannot start.
3771#[derive(Debug, Clone, PartialEq, Eq)]
3772pub enum BinaryPath {
3773    /// The recorded path is a file, and the service manager still starts it.
3774    Current {
3775        /// The path.
3776        path: PathBuf,
3777    },
3778    /// **Nothing is at the recorded path.**
3779    ///
3780    /// This is the npm upgrade: `npm i -g @ivan-murzak/runner-manager` puts the binary under
3781    /// the active Node installation's global prefix, and switching Node versions
3782    /// with `nvm`, `fnm`, or `volta` moves that prefix. The service is still
3783    /// registered, still set to start at boot, and starts nothing.
3784    Missing {
3785        /// The path the record names.
3786        recorded: PathBuf,
3787    },
3788    /// Something is at the recorded path, but it is not a file the service
3789    /// manager could start.
3790    NotExecutable {
3791        /// The path the record names.
3792        recorded: PathBuf,
3793        /// What is there instead.
3794        detail: String,
3795    },
3796    /// The record and the service manager name different binaries.
3797    Diverged {
3798        /// What the record says.
3799        recorded: PathBuf,
3800        /// What the service manager is registered to start.
3801        registered: PathBuf,
3802    },
3803}
3804
3805impl BinaryPath {
3806    /// Whether this is a state `service status` must report as an error.
3807    #[must_use]
3808    pub const fn is_error(&self) -> bool {
3809        !matches!(self, Self::Current { .. })
3810    }
3811
3812    /// The path the record names, whatever state it is in.
3813    #[must_use]
3814    pub fn recorded(&self) -> &Path {
3815        match self {
3816            Self::Current { path } => path,
3817            Self::Missing { recorded }
3818            | Self::NotExecutable { recorded, .. }
3819            | Self::Diverged { recorded, .. } => recorded,
3820        }
3821    }
3822}
3823
3824impl fmt::Display for BinaryPath {
3825    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
3826        match self {
3827            Self::Current { path } => write!(f, "{}", path.display()),
3828            Self::Missing { recorded } => write!(
3829                f,
3830                "{} -- STALE: nothing is at the recorded path, so the service cannot start. A \
3831                 package manager that moved the binary is the usual cause; an `npm i -g` \
3832                 installation moves with the active Node version. Run `service install` again \
3833                 from the binary that is now installed.",
3834                recorded.display()
3835            ),
3836            Self::NotExecutable { recorded, detail } => write!(
3837                f,
3838                "{} -- STALE: {detail}, so the service cannot start. Run `service install` again \
3839                 from the installed binary.",
3840                recorded.display()
3841            ),
3842            Self::Diverged {
3843                recorded,
3844                registered,
3845            } => write!(
3846                f,
3847                "{} -- STALE: the service manager is registered to start {} instead. Something \
3848                 has edited the registration since it was installed. Run `service uninstall` and \
3849                 `service install`; neither touches configuration, secrets, or the cache.",
3850                recorded.display(),
3851                registered.display()
3852            ),
3853        }
3854    }
3855}
3856
3857/// Decides what became of a recorded path.
3858///
3859/// `registered` is what the service manager says it will start, when the
3860/// platform can be asked. Order matters and is deliberate: **absence is checked
3861/// first**, because a missing binary is the failure item 6 exists for and an
3862/// operator hearing "the registration disagrees with the record" about a file
3863/// that is not there would be sent to the wrong problem.
3864#[must_use]
3865pub fn inspect_binary(recorded: &Path, registered: Option<&Path>) -> BinaryPath {
3866    match std::fs::metadata(recorded) {
3867        Err(error) if error.kind() == std::io::ErrorKind::NotFound => {
3868            return BinaryPath::Missing {
3869                recorded: recorded.to_path_buf(),
3870            };
3871        }
3872        Err(error) => {
3873            return BinaryPath::NotExecutable {
3874                recorded: recorded.to_path_buf(),
3875                detail: format!("it cannot be inspected ({error})"),
3876            };
3877        }
3878        Ok(metadata) if !metadata.is_file() => {
3879            return BinaryPath::NotExecutable {
3880                recorded: recorded.to_path_buf(),
3881                detail: "what is there is not a file".to_string(),
3882            };
3883        }
3884        Ok(_) => {}
3885    }
3886    if let Some(registered) = registered
3887        && !same_path_text(&recorded.to_string_lossy(), &registered.to_string_lossy())
3888    {
3889        return BinaryPath::Diverged {
3890            recorded: recorded.to_path_buf(),
3891            registered: registered.to_path_buf(),
3892        };
3893    }
3894    BinaryPath::Current {
3895        path: recorded.to_path_buf(),
3896    }
3897}
3898
3899// ---------------------------------------------------------------------------
3900// The control seam
3901// ---------------------------------------------------------------------------
3902
3903/// What a service manager says about a registration it holds.
3904#[derive(Debug, Clone, PartialEq, Eq)]
3905pub struct Registration {
3906    /// Which manager holds it, which is also what decides the start mode:
3907    /// a Windows service is a boot registration and a Task Scheduler task is a
3908    /// login one; a LaunchDaemon and a system unit are boot, a LaunchAgent and
3909    /// a user unit are login.
3910    pub manager: DefinitionKind,
3911    /// The start mode the *domain* it was found in implies.
3912    pub start_mode: StartMode,
3913    /// The full command line, as the manager stores it.
3914    pub command_line: String,
3915    /// The account it runs under, when the manager reports one.
3916    pub account: Option<String>,
3917    /// Whether it is running now.
3918    pub running: bool,
3919    /// Whether it will start by itself.
3920    ///
3921    /// Distinct from [`Registration::start_mode`]: a Windows service can be
3922    /// registered in the boot domain and still be set to `demand` start, which
3923    /// is a service that exists, looks installed, and does not come back after
3924    /// a reboot.
3925    pub starts_automatically: bool,
3926    /// The restart-on-failure delay the manager reports, when it reports one.
3927    ///
3928    /// The *delay* rather than the whole [`RestartPolicy`], because that is the
3929    /// half every one of the four managers can be asked for. The failure-count
3930    /// reset window has no representation at all in Task Scheduler or launchd,
3931    /// so a `RestartPolicy` read back from them would carry one number the
3932    /// manager reported and one this module invented — and a comparison against
3933    /// an invented number is a comparison that cannot fail.
3934    pub restart_delay: Option<Duration>,
3935}
3936
3937impl Registration {
3938    /// The executable the manager will start, parsed out of the command line.
3939    #[must_use]
3940    pub fn binary(&self) -> Option<PathBuf> {
3941        executable_from_command_line(&self.command_line)
3942    }
3943}
3944
3945/// One platform's service manager, for one start-mode domain.
3946///
3947/// Every method takes the identity rather than storing it, so one control can
3948/// answer about the product's registration and about a test fixture's without
3949/// either being able to reach the other by accident.
3950pub trait ServiceControl: fmt::Debug {
3951    /// Which manager this is.
3952    fn manager(&self) -> DefinitionKind;
3953
3954    /// Registers the plan, returning the definition that was applied.
3955    ///
3956    /// # Errors
3957    ///
3958    /// [`ServiceError::Control`] or [`ServiceError::NeedsElevation`].
3959    fn install(&self, plan: &InstallPlan) -> Result<ServiceDefinition, ServiceError>;
3960
3961    /// Deregisters. Returns whether there was a registration to remove.
3962    ///
3963    /// **Removes the registration and nothing else.** No implementation of this
3964    /// method may delete a directory, a database, a secret, or a cache;
3965    /// `05-infrastructure.md` item 5 is a property of every backend, not a
3966    /// check somewhere above them.
3967    ///
3968    /// # Errors
3969    ///
3970    /// [`ServiceError::Control`] or [`ServiceError::NeedsElevation`].
3971    fn uninstall(&self, identity: &ServiceIdentity) -> Result<bool, ServiceError>;
3972
3973    /// What the manager knows about this registration, if anything.
3974    ///
3975    /// # Errors
3976    ///
3977    /// [`ServiceError::Control`].
3978    fn query(&self, identity: &ServiceIdentity) -> Result<Option<Registration>, ServiceError>;
3979
3980    /// Starts it now, without waiting for the next boot or logon.
3981    ///
3982    /// # Errors
3983    ///
3984    /// [`ServiceError::Control`], [`ServiceError::NeedsElevation`], or
3985    /// [`ServiceError::NotInstalled`].
3986    fn start(&self, identity: &ServiceIdentity) -> Result<(), ServiceError>;
3987
3988    /// Stops it. Returns whether it was running.
3989    ///
3990    /// # Errors
3991    ///
3992    /// As [`ServiceControl::start`].
3993    fn stop(&self, identity: &ServiceIdentity) -> Result<bool, ServiceError>;
3994}
3995
3996/// Chooses the control for a start mode.
3997///
3998/// The indirection is what makes [`ServiceOperations`] testable: production
3999/// hands it [`HostControls`], every test in this file hands it
4000/// [`RecordingControls`], and neither knows the difference.
4001pub trait ControlFactory: fmt::Debug + Send + Sync {
4002    /// The control for one start-mode domain.
4003    ///
4004    /// # Errors
4005    ///
4006    /// [`ServiceError::Control`] when this host has no manager for that domain.
4007    fn control(&self, mode: StartMode) -> Result<Box<dyn ServiceControl>, ServiceError>;
4008
4009    /// The text of an installed definition file, for drift detection. `None`
4010    /// when there is none to read. The default reads nothing, so a double
4011    /// never reports the real host's unit as its own.
4012    fn installed_definition(&self, _path: &Path) -> Option<String> {
4013        None
4014    }
4015
4016    /// Whether this Mac signs somebody in after a restart, and whether
4017    /// FileVault holds it at the unlock screen. `None` where the platform has
4018    /// no such probe; the default probes nothing, for the same reason as
4019    /// [`Self::installed_definition`].
4020    fn unattended_login(&self) -> Option<crate::unattended_login::UnattendedLogin> {
4021        None
4022    }
4023}
4024
4025/// The real service managers of the host this binary was built for.
4026#[derive(Debug, Clone, Copy, Default)]
4027pub struct HostControls;
4028
4029// ---------------------------------------------------------------------------
4030// The operations
4031// ---------------------------------------------------------------------------
4032
4033/// What `install` did.
4034#[derive(Debug, Clone)]
4035pub struct Installed {
4036    /// The plan that was applied, including the resolved absolute binary path.
4037    pub plan: InstallPlan,
4038    /// The definition the platform was given.
4039    pub definition: ServiceDefinition,
4040    /// The record that was written.
4041    pub record: InstallRecord,
4042    /// What the definition grants, measured against the requirement.
4043    pub review: PrivilegeReview,
4044    /// What was done to the runner root the registration will run jobs under.
4045    ///
4046    /// Named trustees rather than SIDs, so printing it adds no identity to the
4047    /// output. See [`crate::runner_root_access`].
4048    pub runner_root: RootAccessSummary,
4049    /// Whether this replaced a registration that was already there, rather than
4050    /// making a new one.
4051    ///
4052    /// Reported rather than glossed over: "Service installed" on a host that
4053    /// had one already reads as though nothing had been taken away and put
4054    /// back, and what was taken away and put back is the thing an operator
4055    /// watching a running agent wants to know about.
4056    pub replaced_existing: bool,
4057}
4058
4059/// What `uninstall` did — and, as importantly, what it did not.
4060#[derive(Debug, Clone, PartialEq, Eq)]
4061pub struct Uninstalled {
4062    /// Whether a registration was removed.
4063    pub removed_registration: bool,
4064    /// Whether the install record was removed.
4065    pub removed_record: bool,
4066    /// Whether a definition file was removed, and which.
4067    pub removed_definition: Option<PathBuf>,
4068    /// The directories that were **left exactly as they were**.
4069    ///
4070    /// `05-infrastructure.md` item 5 stated as a value rather than as a
4071    /// promise: `service uninstall` prints this list, so an operator can see
4072    /// that the configuration, the SQLite database, the stored token and the
4073    /// runner package cache are all still there.
4074    pub preserved: Vec<PathBuf>,
4075}
4076
4077impl fmt::Display for Uninstalled {
4078    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
4079        if self.removed_registration {
4080            writeln!(f, "The service registration was removed.")?;
4081        } else {
4082            writeln!(f, "There was no service registration to remove.")?;
4083        }
4084        writeln!(f, "Nothing else was deleted. These are untouched:")?;
4085        for path in &self.preserved {
4086            writeln!(f, "  {}", path.display())?;
4087        }
4088        write!(
4089            f,
4090            "The stored GitHub token is untouched too; `auth logout` is what purges it."
4091        )
4092    }
4093}
4094
4095/// What `set_start_mode` did.
4096#[derive(Debug, Clone, PartialEq, Eq)]
4097pub struct StartModeChange {
4098    /// What it was.
4099    pub from: StartMode,
4100    /// What it is now.
4101    pub to: StartMode,
4102    /// `false` when the registration was already in the requested mode.
4103    pub changed: bool,
4104    /// The secret store the new mode obliges, so a caller can tell an operator
4105    /// whether the token has to move too.
4106    pub store_scope: crate::secrets::SecretScope,
4107    /// What the mode change did to the runner root's access control.
4108    ///
4109    /// A mode change moves the account the daemon runs as, and the runner root
4110    /// admits that account by name — so the two move together or the new
4111    /// registration cannot write its own workspaces.
4112    pub runner_root: RootAccessSummary,
4113}
4114
4115impl fmt::Display for StartModeChange {
4116    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
4117        if !self.changed {
4118            return write!(f, "The service already starts at {}.", self.to);
4119        }
4120        write!(
4121            f,
4122            "The service now starts at {} instead of {}. It reads the {}-scoped secret store; \
4123             if the token was stored under the other scope, run `auth login` again. {}",
4124            self.to, self.from, self.store_scope, self.runner_root
4125        )
4126    }
4127}
4128
4129/// Install, uninstall, inspect, and switch start mode.
4130///
4131/// This is the whole of the library contract `f3` builds `service install`,
4132/// `service uninstall` and `service status` on. Nothing here prints; every
4133/// operation returns a value with a [`fmt::Display`] a command can write, so the
4134/// same facts are available to the TUI and to a JSON document without being
4135/// re-derived from text.
4136#[derive(Debug, Clone)]
4137pub struct ServiceOperations {
4138    paths: AppPaths,
4139    identity: ServiceIdentity,
4140    controls: std::sync::Arc<dyn ControlFactory>,
4141    runner_root: Option<LocalAbsolutePath>,
4142}
4143
4144impl ServiceOperations {
4145    /// Operates on this host's real service managers.
4146    #[must_use]
4147    pub fn on_this_host(paths: AppPaths) -> Self {
4148        Self::with_controls(
4149            paths,
4150            ServiceIdentity::product(),
4151            std::sync::Arc::new(HostControls),
4152        )
4153    }
4154
4155    /// Operates on the controls a caller supplies.
4156    ///
4157    /// Both other arguments are explicit for the same reason: the privileged
4158    /// installer tests register a real service under a
4159    /// [`ServiceIdentity::fixture`] name, against a disposable
4160    /// [`AppPaths::rooted_at`] tree, and must not be able to touch an operator's
4161    /// installation even by mistake.
4162    #[must_use]
4163    pub fn with_controls(
4164        paths: AppPaths,
4165        identity: ServiceIdentity,
4166        controls: std::sync::Arc<dyn ControlFactory>,
4167    ) -> Self {
4168        Self {
4169            paths,
4170            identity,
4171            controls,
4172            runner_root: None,
4173        }
4174    }
4175
4176    /// Points runner-root preparation at a directory the caller owns, for the
4177    /// **current account**.
4178    ///
4179    /// **Only a [`ServiceIdentity::fixture`] registration is allowed to move
4180    /// it, and the guard is here rather than at the call site.** A product
4181    /// registration ignores the override entirely and resolves
4182    /// [`crate::runner_root::default_runner_root`] itself, which is what keeps
4183    /// "custom roots are never re-ACLed" true no matter what a caller passes.
4184    ///
4185    /// It exists because the directory the product uses is
4186    /// `%SystemDrive%\rman`, and a smoke test that created and re-permissioned
4187    /// *that* would be editing the machine it runs on from outside its own
4188    /// fixture. So a fixture aims this at a temporary directory it created and
4189    /// will delete — and "and will delete" is why the override is always
4190    /// reconciled for the calling account rather than for the account the start
4191    /// mode obliges. A boot-mode root admits `SY` and `BA` only, which an
4192    /// ordinary filtered token is neither, so a test that supplied one could
4193    /// not then inspect or remove its own temporary directory.
4194    #[must_use]
4195    pub fn with_runner_root(mut self, root: LocalAbsolutePath) -> Self {
4196        self.runner_root = Some(root);
4197        self
4198    }
4199
4200    /// The directories this operates against.
4201    #[must_use]
4202    pub const fn paths(&self) -> &AppPaths {
4203        &self.paths
4204    }
4205
4206    /// The registration this operates on.
4207    #[must_use]
4208    pub const fn identity(&self) -> &ServiceIdentity {
4209        &self.identity
4210    }
4211
4212    /// Registers `daemon run` with the operating system.
4213    ///
4214    /// The order is the contract, and each step is a requirement:
4215    ///
4216    /// 1. the four directories are created, because item 2 is about writing them;
4217    /// 2. **the single-instance lock is taken** — item 1. It is held for the
4218    ///    whole of the install rather than probed and released, so a daemon that
4219    ///    starts halfway through cannot end up racing a registration;
4220    /// 3. neither start-mode domain already holds a registration — item 6's
4221    ///    record would otherwise describe one of two;
4222    /// 4. the binary path is resolved and confirmed to be a file — item 6;
4223    /// 5. the platform registers it;
4224    /// 6. the record is written.
4225    ///
4226    /// # Errors
4227    ///
4228    /// [`ServiceError::LockHeld`] when an agent is already running,
4229    /// [`ServiceError::AlreadyInstalled`], [`ServiceError::BinaryMissing`], and
4230    /// whatever the platform reports.
4231    pub fn install(&self, request: &InstallRequest) -> Result<Installed, ServiceError> {
4232        self.paths
4233            .create_all()
4234            .map_err(|source| ServiceError::Paths {
4235                source: Box::new(source),
4236            })?;
4237
4238        // Item 1. The guard lives until the end of this function; dropping it is
4239        // the release, so an early return releases it too.
4240        let _guard = self.refuse_while_an_agent_runs()?;
4241        // Replacing applies to this account's own registration only: over
4242        // another installation's it would re-register that service against
4243        // this account's directories.
4244        self.refuse_foreign("replace")?;
4245
4246        // --------------------------------------------------------------------
4247        // AN INSTALL OVER THE SAME START MODE REPLACES; IT DOES NOT REFUSE.
4248        // --------------------------------------------------------------------
4249        // This used to refuse any existing registration, and that was wrong in
4250        // a way that took a real host down.
4251        //
4252        // `service install` is *also* how an operator moves to a new version by
4253        // hand — `install_owned_copy` says so, and the `.old` rename it does
4254        // exists precisely so that reinstalling over a **running** service
4255        // works. So the caller replaces the copy the service runs and *then*
4256        // asks to register it. Refusing at that point does not undo the swap:
4257        // the daemon is left running a binary nobody registered, the command
4258        // reports failure, and on macOS the new binary is a program the stored
4259        // credential's keychain grant does not name. Watched on 0.1.17: the
4260        // copy under `state/bin` was replaced at 00:16, `service install` said
4261        // "already registered", and the daemon crash-looped on `-25293` from
4262        // then on.
4263        //
4264        // Registering the same service twice is not what an operator is asking
4265        // for here and is not what they get: the platform is asked to drop the
4266        // registration and take it again, which is a deregister followed by a
4267        // register because that is the only thing launchd, systemd and the SCM
4268        // all support — `launchctl bootstrap` refuses a label already loaded.
4269        //
4270        // The *other* mode is still refused. Moving between boot and login
4271        // moves the registration between two service managers and changes the
4272        // account and the secret store with it; `set_start_mode` does that,
4273        // carefully, and an install is not the place for it.
4274        let (replacing, previous_was_running) = match self.find_registration()? {
4275            Some((existing, registration)) if existing == request.start_mode() => {
4276                (true, registration.running)
4277            }
4278            Some((existing, _)) => {
4279                return Err(ServiceError::AlreadyInstalled {
4280                    name: self.identity.name().to_string(),
4281                    existing,
4282                    requested: request.start_mode(),
4283                });
4284            }
4285            None => (false, false),
4286        };
4287
4288        let plan = InstallPlan::resolve(
4289            self.identity.clone(),
4290            request,
4291            ServiceDirectories::of(&self.paths),
4292        )?;
4293
4294        // Resolved before the runner root, and used after it: every step from
4295        // here on has to be able to undo the directory, and a `?` between the
4296        // preparation and the first fallible use would leave one behind with
4297        // nothing to say so.
4298        let control = self.controls.control(plan.start_mode())?;
4299
4300        // Item 8, and the one step that can refuse an otherwise valid install.
4301        // Before the platform is asked to register anything, because a
4302        // registration whose workspaces would land in a directory ordinary local
4303        // users can write is one that should never have existed — and because
4304        // undoing a directory is cheaper than undoing a service.
4305        let root = self.prepare_runner_root(plan.start_mode())?;
4306
4307        // Read before the registration is dropped, so that a replace which then
4308        // fails to take can put back what was there. A record this account
4309        // cannot read leaves nothing to reinstate, which is reported rather
4310        // than pretended: see `reinstate`.
4311        let previous = if replacing {
4312            InstallRecord::read(&self.paths).ok().flatten()
4313        } else {
4314            None
4315        };
4316        if replacing && let Err(cause) = control.uninstall(&self.identity) {
4317            return Err(undo_runner_root(&root, "install", &self.identity, cause));
4318        }
4319
4320        let definition = match control.install(&plan) {
4321            Ok(definition) => definition,
4322            Err(cause) => {
4323                // Nothing was deregistered on a first install, so there is
4324                // nothing to put back and `reinstate` says so with `Ok(())`.
4325                let restored =
4326                    self.reinstate(control.as_ref(), previous.as_ref(), previous_was_running);
4327                return Err(rolled_back(
4328                    retained_runner_root(&root),
4329                    restored,
4330                    "install",
4331                    &self.identity,
4332                    cause,
4333                ));
4334            }
4335        };
4336        let review = review_least_privilege(&definition, &plan);
4337        let record = InstallRecord::of(&plan, &definition, Utc::now());
4338        if let Err(cause) = record.write(&self.paths) {
4339            let rollback = control
4340                .uninstall(&self.identity)
4341                .map(|_| ())
4342                .and_then(|()| {
4343                    self.reinstate(control.as_ref(), previous.as_ref(), previous_was_running)
4344                });
4345            return Err(rolled_back(
4346                retained_runner_root(&root),
4347                rollback,
4348                "install",
4349                &self.identity,
4350                cause,
4351            ));
4352        }
4353
4354        // Registration is not the postcondition of an install: a service the
4355        // current session still has to start by hand is only half-installed.
4356        // This matters most for a logon-triggered Windows task because its
4357        // trigger has already passed by the time an operator reinstalls it.
4358        // Start through the same control that registered the definition and
4359        // roll the whole replacement back if the manager refuses it.
4360        if let Err(cause) = control.start(&self.identity) {
4361            let remove_new = control.uninstall(&self.identity).map(|_| ());
4362            let restore_registration = remove_new.and_then(|()| {
4363                self.reinstate(control.as_ref(), previous.as_ref(), previous_was_running)
4364            });
4365            let restore_record = restore_registration.and_then(|()| match &previous {
4366                Some(previous) => previous.write(&self.paths),
4367                None => InstallRecord::remove(&self.paths).map(|_| ()),
4368            });
4369            return Err(rolled_back(
4370                retained_runner_root(&root),
4371                restore_record,
4372                "install and start",
4373                &self.identity,
4374                cause,
4375            ));
4376        }
4377        Ok(Installed {
4378            plan,
4379            definition,
4380            record,
4381            review,
4382            runner_root: root.summary().clone(),
4383            replaced_existing: replacing,
4384        })
4385    }
4386
4387    /// Puts back a registration this call deregistered in order to replace it.
4388    ///
4389    /// `Ok(())` when there was nothing to put back, which is the ordinary case:
4390    /// a first install deregisters nothing. When there *was* a record, the plan
4391    /// is rebuilt from it rather than from the request, because the request is
4392    /// the one that just failed.
4393    ///
4394    /// Best effort by construction — the caller passes the result to
4395    /// [`rolled_back`], which reports a failed rollback alongside the original
4396    /// cause rather than replacing it.
4397    fn reinstate(
4398        &self,
4399        control: &dyn ServiceControl,
4400        previous: Option<&InstallRecord>,
4401        was_running: bool,
4402    ) -> Result<(), ServiceError> {
4403        let Some(record) = previous else {
4404            return Ok(());
4405        };
4406        control.install(&record.plan(self.identity.clone()))?;
4407        if was_running {
4408            control.start(&self.identity)?;
4409        }
4410        Ok(())
4411    }
4412
4413    /// Deregisters, and deletes nothing else.
4414    ///
4415    /// # Errors
4416    ///
4417    /// Whatever the platform reports. A missing registration is **not** an
4418    /// error: `service uninstall` on a host that has none should say so and
4419    /// exit cleanly, because an operator running it twice has not made a
4420    /// mistake.
4421    pub fn uninstall(&self) -> Result<Uninstalled, ServiceError> {
4422        let record = InstallRecord::read(&self.paths).ok().flatten();
4423        let removed_definition = record
4424            .as_ref()
4425            .and_then(|record| record.definition_path.clone());
4426
4427        let mut removed_registration = false;
4428        // Both domains, not only the recorded one. A record can be lost while a
4429        // registration survives, and leaving a registration behind because a
4430        // TOML file was deleted is exactly the state `uninstall` exists to end.
4431        //
4432        // Except another installation's: it is skipped, and when it is all
4433        // there is, the uninstall is refused and this account's record kept.
4434        let mut foreign = None;
4435        for mode in [StartMode::Boot, StartMode::Login] {
4436            let control = self.controls.control(mode)?;
4437            if let Some(directories) = control
4438                .query(&self.identity)
4439                .ok()
4440                .flatten()
4441                .and_then(|registration| self.foreign_directories(&registration))
4442            {
4443                foreign = Some(directories);
4444                continue;
4445            }
4446            if control.uninstall(&self.identity)? {
4447                removed_registration = true;
4448            }
4449        }
4450        if !removed_registration && let Some(directories) = foreign {
4451            return Err(self.foreign_refusal("uninstall", directories));
4452        }
4453        let removed_record = InstallRecord::remove(&self.paths)?;
4454        Ok(Uninstalled {
4455            removed_registration,
4456            removed_record,
4457            removed_definition: removed_definition.filter(|_| removed_registration),
4458            preserved: self
4459                .paths
4460                .all()
4461                .iter()
4462                .map(|(_, path)| (*path).to_path_buf())
4463                .collect(),
4464        })
4465    }
4466
4467    /// Switches between `boot` and `login` **without re-resolving anything**.
4468    ///
4469    /// `05-infrastructure.md` item 7. Everything the new registration carries —
4470    /// the absolute binary path, the arguments, the restart policy, the four
4471    /// directories — comes from the existing record, so the operator is not
4472    /// asked to reinstall the product and a binary that has since moved is not
4473    /// silently swapped for whichever one happens to be running this command.
4474    ///
4475    /// On Windows the registration necessarily moves between two different
4476    /// Windows facilities, because Windows has no service that starts at logon;
4477    /// see this module's documentation. That is still not a reinstall: no file
4478    /// is downloaded, replaced, or re-resolved.
4479    ///
4480    /// The caller must persist the new mode onto this host's `Host` record so
4481    /// that `host show` reports it — [`StartModeChange::to`] is the value, and
4482    /// `f1` already emits `host.service_start_mode` from that field.
4483    ///
4484    /// # Errors
4485    ///
4486    /// [`ServiceError::NotInstalled`] when there is no record to switch, and
4487    /// whatever the platform reports.
4488    pub fn set_start_mode(&self, to: StartMode) -> Result<StartModeChange, ServiceError> {
4489        let Some(record) = InstallRecord::read(&self.paths)? else {
4490            return Err(ServiceError::NotInstalled {
4491                name: self.identity.name().to_string(),
4492                operation: "switch the start mode of",
4493            });
4494        };
4495        let from = record.start_mode;
4496        if from == to {
4497            // Nothing moves, so nothing about the root's access control has to.
4498            // Reconciling it here would turn a no-op command into one that can
4499            // fail on a permission it does not need.
4500            return Ok(StartModeChange {
4501                from,
4502                to,
4503                changed: false,
4504                store_scope: crate::secrets::SecretScope::for_start_mode(to),
4505                runner_root: RootAccessSummary::NotApplicable,
4506            });
4507        }
4508
4509        // A real switch only: a no-op touches no service manager at all. Over
4510        // another installation's registration it would move that service.
4511        self.refuse_foreign("switch the start mode of")?;
4512
4513        #[cfg(windows)]
4514        let arguments = {
4515            let mut arguments = record.arguments.clone();
4516            arguments.retain(|argument| argument != WINDOWS_SCM_HOST_ARGUMENT);
4517            if to == StartMode::Boot {
4518                arguments.push(WINDOWS_SCM_HOST_ARGUMENT.to_string());
4519            }
4520            arguments
4521        };
4522        #[cfg(not(windows))]
4523        let arguments = record.arguments.clone();
4524        let plan = InstallPlan::unchecked(
4525            self.identity.clone(),
4526            to,
4527            record.binary.clone(),
4528            record.directories.clone(),
4529        )
4530        .with_arguments(arguments)
4531        .with_restart(record.restart());
4532        let plan = if record.starts_on_demand {
4533            plan.started_on_demand()
4534        } else {
4535            plan
4536        };
4537        let plan = match crate::secrets::PlatformSecretStore::for_start_mode(to) {
4538            Ok(store) => plan.with_secret_guard(store.guard()),
4539            Err(_) => plan,
4540        };
4541
4542        // Install the target domain before touching the live one. This makes a
4543        // failed target install a no-op from the operator's point of view and,
4544        // unlike uninstall-first ordering, never trades a working service for
4545        // an error message. Resolved before the runner root and used after it,
4546        // so that no `?` sits between preparing the directory and the first
4547        // step that knows how to undo it.
4548        let target = self.controls.control(to)?;
4549        // The domain being left, resolved here rather than where it is used for
4550        // the same reason: the only step that removes it runs after the runner
4551        // root has been prepared, and a `?` there would abandon the target
4552        // registration, the record and the directory without a word.
4553        let previous = self.controls.control(from)?;
4554
4555        // The account changes with the mode, and so must the account the runner
4556        // root admits: `04-security-recovery.md` requires the selected identity
4557        // to be *reconciled* when service mode changes, which means adding the
4558        // operator's on the way to login and dropping it again on the way back.
4559        // Before the target install, for the same reason `install` does it
4560        // first — a root that cannot be made safe must not produce a working
4561        // registration.
4562        let root = self.prepare_runner_root(to)?;
4563
4564        let definition = match target.install(&plan) {
4565            Ok(definition) => definition,
4566            Err(cause) => {
4567                return Err(undo_runner_root(
4568                    &root,
4569                    "switch start mode",
4570                    &self.identity,
4571                    cause,
4572                ));
4573            }
4574        };
4575        let next_record = InstallRecord::of(&plan, &definition, record.installed_at);
4576        if let Err(cause) = next_record.write(&self.paths) {
4577            return Err(rolled_back(
4578                retained_runner_root(&root),
4579                target.uninstall(&self.identity),
4580                "switch start mode",
4581                &self.identity,
4582                cause,
4583            ));
4584        }
4585
4586        // Only after the target registration and its durable record exist is
4587        // it safe to remove the old domain. If that last step fails, remove the
4588        // target and put the old record back so status and reality agree.
4589        if let Err(cause) = previous.uninstall(&self.identity) {
4590            let target_rollback = target.uninstall(&self.identity);
4591            let record_rollback = record.write(&self.paths);
4592            return Err(rolled_back(
4593                retained_runner_root(&root),
4594                target_rollback.and(record_rollback),
4595                "switch start mode",
4596                &self.identity,
4597                cause,
4598            ));
4599        }
4600        Ok(StartModeChange {
4601            from,
4602            to,
4603            changed: true,
4604            store_scope: crate::secrets::SecretScope::for_start_mode(to),
4605            runner_root: root.summary().clone(),
4606        })
4607    }
4608
4609    /// Starts the registration now.
4610    ///
4611    /// # Errors
4612    ///
4613    /// [`ServiceError::NotInstalled`], or whatever the platform reports.
4614    pub fn start(&self) -> Result<(), ServiceError> {
4615        let mode = self.own_registration_mode("start")?;
4616        self.controls.control(mode)?.start(&self.identity)
4617    }
4618
4619    /// The domain holding this account's own registration, for `operation`.
4620    ///
4621    /// # Errors
4622    ///
4623    /// [`ServiceError::NotInstalled`] when there is none, and
4624    /// [`ServiceError::ForeignInstallation`] when the only one is another
4625    /// installation's.
4626    fn own_registration_mode(&self, operation: &'static str) -> Result<StartMode, ServiceError> {
4627        match self.registration_for_status()? {
4628            None => Err(ServiceError::NotInstalled {
4629                name: self.identity.name().to_string(),
4630                operation,
4631            }),
4632            Some((_, Some(directories))) => Err(self.foreign_refusal(operation, directories)),
4633            Some((registration, None)) => Ok(registration.start_mode),
4634        }
4635    }
4636
4637    /// Stops the registration. Returns whether it was running.
4638    ///
4639    /// # Errors
4640    ///
4641    /// As [`ServiceOperations::start`].
4642    pub fn stop(&self) -> Result<bool, ServiceError> {
4643        let mode = self.own_registration_mode("stop")?;
4644        self.controls.control(mode)?.stop(&self.identity)
4645    }
4646
4647    /// Everything Journey 5 step 4 asks `service status` to report.
4648    ///
4649    /// # Errors
4650    ///
4651    /// [`ServiceError::Record`] when local state cannot be read, and whatever
4652    /// the platform reports. A *stale binary path* is deliberately **not** an
4653    /// error here: it is a reported state, because a status command that
4654    /// refused to print anything else would hide the very facts an operator
4655    /// needs in order to fix it.
4656    ///
4657    /// A record **this account may not read** is reported for the same reason
4658    /// and is the second exception. On a boot-mode host the record was written
4659    /// by `sudo service install`, and a version before this one wrote it `0600`
4660    /// — so the operator's own `service status` ended with `Permission denied`
4661    /// and printed nothing at all, on a host whose registration the service
4662    /// manager would have described perfectly well. What launchd, systemd or
4663    /// the SCM says is a separate fact from the record, and it is still worth
4664    /// having.
4665    ///
4666    /// # A registration made from another data root
4667    ///
4668    /// The service manager holds one registration per machine under one name,
4669    /// and every account on the machine sees it. When its command line names
4670    /// application-data directories that are not this account's, it is
4671    /// **another installation's service**, and comparing this account's
4672    /// install record against it would report a stale binary, a mismatched
4673    /// start mode and a secret store in the wrong scope, each with a remedy
4674    /// (`service uninstall`, `service install`, `auth login`) that, followed
4675    /// from this account, breaks a healthy service. So it is reported as what
4676    /// it is: see [`ForeignService`].
4677    pub fn status(&self) -> Result<ServiceStatus, ServiceError> {
4678        match self.registration_for_status()? {
4679            Some((registration, Some(directories))) => {
4680                Ok(self.foreign_status(registration, directories))
4681            }
4682            found => {
4683                let (record, record_refused) = match InstallRecord::read(&self.paths) {
4684                    Ok(record) => (record, None),
4685                    Err(refusal @ ServiceError::RecordNotPermitted { .. }) => (None, Some(refusal)),
4686                    Err(error) => return Err(error),
4687                };
4688                self.status_at(
4689                    &self.paths,
4690                    record,
4691                    record_refused.as_ref(),
4692                    found.map(|(registration, _)| registration),
4693                )
4694            }
4695        }
4696    }
4697
4698    /// [`Self::status`] for the installation whose directories are `paths`,
4699    /// from the record already read there.
4700    fn status_at(
4701        &self,
4702        paths: &AppPaths,
4703        record: Option<InstallRecord>,
4704        record_refused: Option<&ServiceError>,
4705        found: Option<Registration>,
4706    ) -> Result<ServiceStatus, ServiceError> {
4707        let last_github_contact = last_github_contact(paths)?;
4708        let drift = record
4709            .as_ref()
4710            .and_then(|record| definition_drift(self.controls.as_ref(), &self.identity, record));
4711        let credential_rejected_since = github_credential_rejected_since(paths).ok().flatten();
4712        let credential_unreadable_since = credential_unreadable_since(paths).ok().flatten();
4713        let probed = self.controls.unattended_login();
4714        let unattended = record.as_ref().and_then(|record| {
4715            // The account is only read where something was probed to judge
4716            // it against.
4717            let account = probed
4718                .as_ref()
4719                .map_or_else(String::new, |_| service_account_name(&record.account));
4720            unattended_gap(record.start_mode, &account, probed.as_ref())
4721        });
4722        let now = Utc::now();
4723        let installed = record.is_some() || found.is_some();
4724        let status = ServiceStatus::compose(
4725            self.identity.clone(),
4726            record,
4727            record_refused,
4728            found,
4729            last_github_contact,
4730            paths,
4731        )
4732        .with_definition_drift(drift)
4733        .with_credential_rejection(credential_rejected_since)
4734        .with_credential_unreadable(credential_unreadable_since)
4735        .with_launches_blocked(crate::launch_health::launches_blocked(paths, now))
4736        .with_unattended_gap(unattended)
4737        .with_host_unfit(
4738            crate::host_fitness::host_unfit_in_force(paths, now).filter(|_| installed),
4739        );
4740        let restart = status.restart_command();
4741        Ok(status.with_stall(
4742            crate::daemon_heartbeat::liveness(paths, now).stall(last_github_contact, &restart),
4743        ))
4744    }
4745
4746    /// The installed definition, when it is not what this build renders for
4747    /// the recorded installation. See [`definition_drift`].
4748    ///
4749    /// # Errors
4750    ///
4751    /// Whatever reading the install record reports.
4752    pub fn definition_drift(&self) -> Result<Option<DefinitionDrift>, ServiceError> {
4753        Ok(InstallRecord::read(&self.paths)?
4754            .and_then(|record| definition_drift(self.controls.as_ref(), &self.identity, &record)))
4755    }
4756
4757    /// Writes the plist this build renders over an outdated one, then has a
4758    /// detached helper boot the job out and bootstrap it again from the new
4759    /// file.
4760    ///
4761    /// launchd reads a plist only when the job is bootstrapped. Rewriting the
4762    /// file changes nothing for a job that is already loaded, and a `KeepAlive`
4763    /// restart reuses the loaded copy. So the job has to be booted out, and
4764    /// that terminates the daemon that asked for it. The helper runs in a
4765    /// session of its own, which launchd does not kill with the job, and
4766    /// retries the bootstrap: launchd refuses it for a few seconds while the
4767    /// old job is still exiting (`5: Input/output error`, measured).
4768    ///
4769    /// Call this only from the daemon the job runs, and only while it holds no
4770    /// runner. It is an operation on the registration the daemon itself runs
4771    /// under, and it never changes the start mode.
4772    ///
4773    /// # Errors
4774    ///
4775    /// [`ServiceError::Control`] or [`ServiceError::NeedsElevation`] when the
4776    /// plist cannot be written or the helper cannot be started. The loaded job
4777    /// is untouched in both cases.
4778    #[cfg(target_os = "macos")]
4779    pub fn reload_launchd_definition(&self, drift: &DefinitionDrift) -> Result<(), ServiceError> {
4780        // The record names the start mode; the account this process runs as has
4781        // to agree, or the record describes a registration other than the one
4782        // running, and root would end up rewriting an operator's LaunchAgent.
4783        // SAFETY: `getuid` reads the calling process's real user id and cannot
4784        // fail.
4785        let root = unsafe { libc::getuid() } == 0;
4786        if root != (drift.start_mode == StartMode::Boot) {
4787            return Err(ServiceError::Control {
4788                operation: "rewrite",
4789                name: self.identity.name().to_string(),
4790                manager: "launchd",
4791                detail: format!(
4792                    "the install record says --start-at {}, but this daemon runs as {}; the plist \
4793                     is left as it is",
4794                    drift.start_mode,
4795                    if root { "root" } else { "the operator" }
4796                ),
4797            });
4798        }
4799        replace_definition(self.identity.name(), drift)?;
4800        // The definition is now this build's, as after the WSL unit rewrite.
4801        // Best effort: the record says who wrote the plist and nothing reads it
4802        // to decide whether to repair.
4803        if let Ok(Some(mut record)) = InstallRecord::read(&self.paths) {
4804            record.installed_by_version = env!("CARGO_PKG_VERSION").to_string();
4805            let _ = record.write(&self.paths);
4806        }
4807        sys::reload_detached(&self.identity, drift.start_mode, &drift.path)
4808    }
4809
4810    /// Creates or reconciles the runner root this start mode's account needs.
4811    ///
4812    /// The account is not an argument: it is [`ServiceAccount::for_start_mode`],
4813    /// the same function the registration's own principal comes from, so the
4814    /// directory admits exactly the identity the definition registers and a mode
4815    /// change reconciles both together or neither.
4816    ///
4817    /// On macOS and Linux this is a no-op that returns
4818    /// [`RootAccessSummary::NotApplicable`]; see [`crate::runner_root_access`].
4819    fn prepare_runner_root(&self, mode: StartMode) -> Result<RootAccessChange, ServiceError> {
4820        #[cfg(not(windows))]
4821        {
4822            // `runner_root` is consumed here as well as in the Windows arm: a
4823            // field only one platform reads is a dead field on the other.
4824            let _ = (mode, &self.runner_root);
4825            Ok(RootAccessChange::not_applicable())
4826        }
4827        #[cfg(windows)]
4828        {
4829            let wrap = |source| ServiceError::RunnerRoot {
4830                source: Box::new(source),
4831            };
4832            // The test seam, and the two things allowed through it. `cfg!(test)`
4833            // is false in every shipped binary, so what a released build honours
4834            // is the fixture name alone — and a fixture name cannot be the
4835            // product's, which is what keeps a released `service install`
4836            // pointed at the platform default whatever a caller passes.
4837            //
4838            // An overridden root is always reconciled **for this account**,
4839            // which is the foreground admission — a real mode of the product
4840            // rather than a concession invented here. It has to be: a boot-mode
4841            // root admits `SY` and `BA` only, and a test process holding an
4842            // ordinary filtered token is neither, so it could not inspect the
4843            // temporary directory it just supplied nor delete it afterwards.
4844            // What that costs is that the *boot* descriptor is not proved
4845            // through this path; it is proved purely, by this module's
4846            // `the_runner_root_a_boot_registration_needs_admits_only_the_service`
4847            // and by `runner_root_access`'s own tests, and for real by the
4848            // privileged installer test, which runs elevated.
4849            if let Some(root) = self
4850                .runner_root
4851                .as_ref()
4852                .filter(|_| self.identity.is_fixture() || cfg!(test))
4853            {
4854                let admission = RootAdmission::of_this_account().map_err(wrap)?;
4855                return crate::runner_root_access::reconcile(&self.paths, root, &admission)
4856                    .map_err(wrap);
4857            }
4858
4859            let admission = match ServiceAccount::for_start_mode(mode) {
4860                // A boot registration runs as LocalSystem, which the constant
4861                // `SY` ace already names.
4862                ServiceAccount::LocalSystem => RootAdmission::LocalSystem,
4863                // A login registration runs as this account, under a filtered
4864                // token in which Administrators is deny-only — so without an
4865                // ace of its own it would be admitted by nothing.
4866                ServiceAccount::InvokingUser | ServiceAccount::Root => {
4867                    RootAdmission::of_this_account().map_err(wrap)?
4868                }
4869            };
4870            crate::runner_root_access::ensure_default_root(&self.paths, &admission).map_err(wrap)
4871        }
4872    }
4873
4874    /// Takes the single-instance lock, or refuses with `d1`'s own message.
4875    fn refuse_while_an_agent_runs(&self) -> Result<HostLock, ServiceError> {
4876        HostLock::try_acquire(&self.paths, LockKind::SingleInstance).map_err(
4877            |source| match source {
4878                held @ LockError::Held { .. } => ServiceError::LockHeld {
4879                    source: Box::new(held),
4880                },
4881                other => ServiceError::LockUnreadable {
4882                    source: Box::new(other),
4883                },
4884            },
4885        )
4886    }
4887
4888    /// The registration, in whichever domain holds it.
4889    fn find_registration(&self) -> Result<Option<(StartMode, Registration)>, ServiceError> {
4890        for mode in [StartMode::Boot, StartMode::Login] {
4891            let control = self.controls.control(mode)?;
4892            if let Some(registration) = control.query(&self.identity)? {
4893                return Ok(Some((registration.start_mode, registration)));
4894            }
4895        }
4896        Ok(None)
4897    }
4898
4899    /// The registration [`Self::status`] describes, with the directories it
4900    /// runs against when they are not this account's: this account's own
4901    /// registration when either domain holds one, else whichever there is.
4902    ///
4903    /// [`Self::find_registration`] stops at the boot domain, so a boot service
4904    /// made from another data root would hide this account's own login
4905    /// registration. The other domain is asked only in that case, and a
4906    /// failure to ask it is not this status's failure.
4907    fn registration_for_status(
4908        &self,
4909    ) -> Result<Option<(Registration, Option<ServiceDirectories>)>, ServiceError> {
4910        let Some((mode, found)) = self.find_registration()? else {
4911            return Ok(None);
4912        };
4913        let Some(directories) = self.foreign_directories(&found) else {
4914            return Ok(Some((found, None)));
4915        };
4916        let other = match mode {
4917            StartMode::Boot => StartMode::Login,
4918            StartMode::Login => StartMode::Boot,
4919        };
4920        let own = self
4921            .controls
4922            .control(other)
4923            .ok()
4924            .and_then(|control| control.query(&self.identity).ok().flatten())
4925            .filter(|registration| self.foreign_directories(registration).is_none());
4926        Ok(Some(match own {
4927            Some(own) => (own, None),
4928            None => (found, Some(directories)),
4929        }))
4930    }
4931
4932    /// Refuses `operation` when the machine's registration is another
4933    /// installation's. See [`ServiceError::ForeignInstallation`].
4934    fn refuse_foreign(&self, operation: &'static str) -> Result<(), ServiceError> {
4935        match self.registration_for_status()? {
4936            Some((_, Some(directories))) => Err(self.foreign_refusal(operation, directories)),
4937            _ => Ok(()),
4938        }
4939    }
4940
4941    /// [`ServiceError::ForeignInstallation`] for the installation at
4942    /// `directories`.
4943    fn foreign_refusal(
4944        &self,
4945        operation: &'static str,
4946        directories: ServiceDirectories,
4947    ) -> ServiceError {
4948        let foreign = self.foreign_service_at(directories);
4949        ServiceError::ForeignInstallation {
4950            name: self.identity.name().to_string(),
4951            operation,
4952            installed_by: foreign.installed_by(),
4953            act_as: foreign.act_as(),
4954        }
4955    }
4956
4957    /// The directories `registration` runs against, when they are not this
4958    /// account's. See [`ForeignService`].
4959    fn foreign_directories(&self, registration: &Registration) -> Option<ServiceDirectories> {
4960        ServiceDirectories::from_command_line(&registration.command_line)
4961            .filter(|directories| !same_directory(&directories.config, self.paths.config_dir()))
4962    }
4963
4964    /// The machine's service and the installation it belongs to, when that is
4965    /// not this account's, without reading anything of that installation's
4966    /// beyond its path. `None` when the service is this account's, when there
4967    /// is none, or when the service manager cannot be asked.
4968    #[must_use]
4969    pub fn foreign_service(&self) -> Option<ForeignService> {
4970        let (_, directories) = self.registration_for_status().ok()??;
4971        Some(self.foreign_service_at(directories?))
4972    }
4973
4974    /// The [`ForeignService`] for an installation at `directories`, before
4975    /// anything of its own is read.
4976    fn foreign_service_at(&self, directories: ServiceDirectories) -> ForeignService {
4977        let root = directories.root();
4978        let home = host_home();
4979        // This account's own installation under other directories, when the
4980        // four are exactly a layout a command of this account resolves.
4981        let this_account = home
4982            .as_deref()
4983            .filter(|home| contains_directory(home, &root))
4984            .and_then(|_| {
4985                if directories == ServiceDirectories::of(&AppPaths::rooted_at(&root)) {
4986                    Some(format!(
4987                        "with `--data-dir {}`",
4988                        quote_argument(&root.to_string_lossy())
4989                    ))
4990                } else {
4991                    AppPaths::discover()
4992                        .ok()
4993                        .filter(|standard| directories == ServiceDirectories::of(standard))
4994                        .map(|_| "without `--data-dir`".to_owned())
4995                }
4996            });
4997        ForeignService {
4998            owner: data_root_owner(&root, home.as_deref()),
4999            this_account,
5000            root,
5001            directories,
5002            local_root: ServiceDirectories::of(&self.paths).root(),
5003            local_record: InstallRecord::path(&self.paths),
5004            unused_local_record: InstallRecord::read(&self.paths).ok().flatten(),
5005            unreadable: None,
5006        }
5007    }
5008
5009    /// [`Self::status`] for a registration made from another data root.
5010    ///
5011    /// What that installation recorded is read when this account may read it,
5012    /// and the status is then exactly the one its owner would see. When it may
5013    /// not, the status holds what the service manager says and nothing it
5014    /// would have to guess.
5015    fn foreign_status(
5016        &self,
5017        registration: Registration,
5018        directories: ServiceDirectories,
5019    ) -> ServiceStatus {
5020        let service_paths = directories.paths();
5021        let mut foreign = self.foreign_service_at(directories);
5022        let unreadable = match InstallRecord::read(&service_paths) {
5023            Ok(Some(record)) => {
5024                match self.status_at(
5025                    &service_paths,
5026                    Some(record),
5027                    None,
5028                    Some(registration.clone()),
5029                ) {
5030                    Ok(status) => return status.with_foreign(foreign),
5031                    Err(error) => error.to_string(),
5032                }
5033            }
5034            Ok(None) => format!(
5035                "there is no install record at {}",
5036                InstallRecord::path(&service_paths).display()
5037            ),
5038            // Not the error's own words: `RecordNotPermitted` and
5039            // `RecordUnreadable` both end in a `service install` remedy, which
5040            // from this account would re-register the machine's service
5041            // against the wrong directories.
5042            Err(
5043                ServiceError::RecordNotPermitted { path, detail }
5044                | ServiceError::RecordUnreadable { path, detail }
5045                | ServiceError::Record { path, detail, .. },
5046            ) => format!("this account cannot read {} ({detail})", path.display()),
5047            Err(error) => error.to_string(),
5048        };
5049        foreign.unreadable = Some(unreadable);
5050        ServiceStatus::foreign_unread(self.identity.clone(), registration, foreign)
5051    }
5052
5053    /// Moves this account's own configuration aside, when no registration on
5054    /// this machine runs against it: `host forget-local`.
5055    ///
5056    /// Everything in the configuration directory -- the install record, the
5057    /// database with its host row and policies, `runner.env` and the rest -- is
5058    /// **renamed** into a `forgotten-<timestamp>` directory beside it, never
5059    /// deleted, so moving it back restores it exactly. The state, runtime and
5060    /// log directories are left alone even where the platform puts them inside
5061    /// the configuration directory (macOS does), and so is every secret store
5062    /// and every directory a registration names.
5063    ///
5064    /// Both domains are asked, strictly: a domain that cannot be asked might
5065    /// hold this account's own registration, so it refuses rather than reads
5066    /// as empty.
5067    ///
5068    /// # Errors
5069    ///
5070    /// [`ServiceError::LocalConfigurationInUse`] when a registration on this
5071    /// machine runs against these directories, when a registration from
5072    /// another data root names a directory overlapping them, or when an agent
5073    /// holds this account's single-instance lock. [`ServiceError::Record`] when
5074    /// the move fails; whatever had already moved is put back first, and
5075    /// anything that could not be put back is named.
5076    pub fn forget_local(&self, at: DateTime<Utc>) -> Result<ForgottenLocal, ServiceError> {
5077        let config = self.paths.config_dir();
5078        let in_use = |detail: String| ServiceError::LocalConfigurationInUse { detail };
5079        let mut service_root = None;
5080        for mode in [StartMode::Boot, StartMode::Login] {
5081            let registration = self
5082                .controls
5083                .control(mode)
5084                .and_then(|control| control.query(&self.identity))
5085                .map_err(|error| {
5086                    in_use(format!(
5087                        "whether a {mode} registration runs against it could not be read ({error})"
5088                    ))
5089                })?;
5090            let Some(registration) = registration else {
5091                continue;
5092            };
5093            let Some(directories) = self.foreign_directories(&registration) else {
5094                return Err(in_use(format!(
5095                    "{} holds a registration that runs against {}. `service status` describes it",
5096                    registration.manager,
5097                    config.display()
5098                )));
5099            };
5100            if let Some(overlap) = directories.all().into_iter().find(|directory| {
5101                contains_directory(directory, config) || contains_directory(config, directory)
5102            }) {
5103                return Err(in_use(format!(
5104                    "the machine's service runs against {}, which overlaps {}",
5105                    overlap.display(),
5106                    config.display()
5107                )));
5108            }
5109            service_root.get_or_insert_with(|| directories.root());
5110        }
5111        let _lock =
5112            HostLock::try_acquire(&self.paths, LockKind::SingleInstance).map_err(|source| {
5113                in_use(format!(
5114                    "an agent may be running against these directories: {source}"
5115                ))
5116            })?;
5117        let failed = |operation: &'static str, path: &Path| {
5118            let path = path.to_path_buf();
5119            move |error: std::io::Error| ServiceError::Record {
5120                operation,
5121                path,
5122                detail: error.to_string(),
5123            }
5124        };
5125        // Never the agent's own directories, wherever the platform nests them.
5126        let kept = [
5127            self.paths.state_dir(),
5128            self.paths.runtime_dir(),
5129            self.paths.logs_dir(),
5130        ];
5131        let mut entries = Vec::new();
5132        for entry in std::fs::read_dir(config).map_err(failed("list", config))? {
5133            let entry = entry.map_err(failed("list", config))?.path();
5134            let archived = entry
5135                .file_name()
5136                .is_some_and(|name| name.to_string_lossy().starts_with(FORGOTTEN_PREFIX));
5137            if !archived && !kept.iter().any(|kept| contains_directory(&entry, kept)) {
5138                entries.push(entry);
5139            }
5140        }
5141        if entries.is_empty() {
5142            return Ok(ForgottenLocal {
5143                archive: None,
5144                moved: entries,
5145                service_root,
5146            });
5147        }
5148        entries.sort();
5149        let archive = config.join(format!("{FORGOTTEN_PREFIX}{}", at.format("%Y%m%dT%H%M%SZ")));
5150        std::fs::create_dir(&archive).map_err(failed("create", &archive))?;
5151        let inside = |entry: &Path| archive.join(entry.file_name().unwrap_or_default());
5152        for (index, entry) in entries.iter().enumerate() {
5153            if let Err(error) = std::fs::rename(entry, inside(entry)) {
5154                let stranded: Vec<String> = entries[..index]
5155                    .iter()
5156                    .rev()
5157                    .filter(|done| std::fs::rename(inside(done), done).is_err())
5158                    .map(|done| inside(done).display().to_string())
5159                    .collect();
5160                let _ = std::fs::remove_dir(&archive);
5161                let mut detail = error.to_string();
5162                if !stranded.is_empty() {
5163                    detail.push_str(&format!(
5164                        "; these could not be put back and are still in the archive: {}",
5165                        stranded.join(", ")
5166                    ));
5167                }
5168                return Err(ServiceError::Record {
5169                    operation: "archive",
5170                    path: entry.clone(),
5171                    detail,
5172                });
5173            }
5174        }
5175        Ok(ForgottenLocal {
5176            archive: Some(archive),
5177            moved: entries,
5178            service_root,
5179        })
5180    }
5181}
5182
5183/// The name every `host forget-local` archive starts with. A later run leaves
5184/// earlier archives where they are.
5185pub const FORGOTTEN_PREFIX: &str = "forgotten-";
5186
5187/// What [`ServiceOperations::forget_local`] did.
5188#[derive(Debug, Clone, PartialEq, Eq)]
5189pub struct ForgottenLocal {
5190    /// Where the configuration went, or `None` when there was nothing to move.
5191    pub archive: Option<PathBuf>,
5192    /// What was moved, by its old path.
5193    pub moved: Vec<PathBuf>,
5194    /// The data root of the machine's service, when one runs from another
5195    /// installation. Named so the operator can see it was not touched.
5196    pub service_root: Option<PathBuf>,
5197}
5198
5199impl fmt::Display for ForgottenLocal {
5200    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
5201        let Some(archive) = &self.archive else {
5202            return write!(
5203                f,
5204                "Nothing to archive: this account's configuration directory is empty."
5205            );
5206        };
5207        writeln!(f, "Archived this account's unused local configuration")?;
5208        writeln!(f, "  archive                   {}", archive.display())?;
5209        for path in &self.moved {
5210            writeln!(f, "  moved                     {}", path.display())?;
5211        }
5212        if let Some(root) = &self.service_root {
5213            writeln!(
5214                f,
5215                "  service data root         {} (untouched)",
5216                root.display()
5217            )?;
5218        }
5219        write!(
5220            f,
5221            "  restore                   move everything in the archive back into its parent \
5222             directory"
5223        )
5224    }
5225}
5226
5227/// Whether two directory spellings name the same place: the same text on
5228/// this platform's terms, or the same canonical directory.
5229fn same_directory(left: &Path, right: &Path) -> bool {
5230    let text = |path: &Path| {
5231        path.to_string_lossy()
5232            .trim_end_matches(['/', '\\'])
5233            .to_owned()
5234    };
5235    same_path_text(&text(left), &text(right))
5236        || matches!(
5237            (std::fs::canonicalize(left), std::fs::canonicalize(right)),
5238            (Ok(left), Ok(right)) if left == right
5239        )
5240}
5241
5242/// Where the remedy starts in a [`StatusProblem`]'s detail: the sentence
5243/// openings every builder above uses for it.
5244const REMEDY_OPENINGS: [&str; 8] = [
5245    ". Run ",
5246    ". If nothing is waiting",
5247    ". Re-register",
5248    ". Switch the start mode",
5249    " Sign in again",
5250    ". Fix: ",
5251    "; to do it now",
5252    "; if the service manager denies",
5253];
5254
5255/// A problem's diagnosis without its remedy. See
5256/// [`ServiceStatus::with_foreign`].
5257///
5258/// A detail that still names a command that changes the registration or the
5259/// credential after the cut -- a builder this list does not know yet -- is
5260/// reduced to the first clause rather than passed on, because the cost of
5261/// showing too little is a shorter sentence and the cost of showing too much
5262/// is a broken service.
5263fn diagnosis_only(detail: &str) -> String {
5264    let cut = REMEDY_OPENINGS
5265        .iter()
5266        .filter_map(|opening| detail.find(opening))
5267        .min()
5268        .unwrap_or(detail.len());
5269    let diagnosis = detail[..cut].trim_end_matches(['.', ' ']);
5270    let names_a_change = |text: &str| {
5271        [
5272            "service install",
5273            "service uninstall",
5274            "auth login",
5275            "launchctl",
5276            "systemctl",
5277        ]
5278        .iter()
5279        .any(|command| text.contains(command))
5280    };
5281    if !names_a_change(diagnosis) {
5282        return diagnosis.to_owned();
5283    }
5284    // Clause ends followed by a space, so a `.` inside a path does not cut.
5285    let end = [", ", "; ", ". "]
5286        .iter()
5287        .filter_map(|separator| diagnosis.find(separator))
5288        .min()
5289        .unwrap_or(diagnosis.len());
5290    let first = &diagnosis[..end];
5291    if names_a_change(first) {
5292        "the installation's own records report this".to_owned()
5293    } else {
5294        first.to_owned()
5295    }
5296}
5297
5298/// A path's components as this platform compares them: case-insensitively
5299/// on Windows.
5300fn path_key(path: &Path) -> Vec<String> {
5301    path.components()
5302        .map(|component| {
5303            let text = component.as_os_str().to_string_lossy();
5304            if cfg!(windows) {
5305                text.to_lowercase()
5306            } else {
5307                text.into_owned()
5308            }
5309        })
5310        .collect()
5311}
5312
5313/// Whether `inner` is `outer` or a directory under it, by the rule
5314/// [`same_directory`] uses: on this platform's terms, or canonically.
5315fn contains_directory(outer: &Path, inner: &Path) -> bool {
5316    path_key(inner).starts_with(&path_key(outer))
5317        || matches!(
5318            (std::fs::canonicalize(outer), std::fs::canonicalize(inner)),
5319            (Ok(outer), Ok(inner)) if inner.starts_with(&outer)
5320        )
5321}
5322
5323/// The account whose data root `root` is, when the path says: the folder
5324/// under the directory that holds this account's `home`, `root` for root's
5325/// home on Unix, and `LocalSystem` for its profile on Windows. `None` when
5326/// the path names nobody, which is reported as the path alone.
5327#[must_use]
5328pub fn data_root_owner(root: &Path, home: Option<&Path>) -> Option<String> {
5329    let target = path_key(root);
5330    if cfg!(windows) {
5331        if target
5332            .windows(3)
5333            .any(|run| run == ["system32", "config", "systemprofile"])
5334        {
5335            return Some("LocalSystem".to_owned());
5336        }
5337    } else if ["/root", "/var/root", "/private/var/root"]
5338        .iter()
5339        .any(|home| root.starts_with(home))
5340    {
5341        return Some("root".to_owned());
5342    }
5343    let users = path_key(home?.parent()?);
5344    // A home directly under the root (`/root` under `sudo`) names no directory
5345    // of homes: every path would then "belong" to its first component.
5346    if users.len() < 2 {
5347        return None;
5348    }
5349    if target.len() > users.len() && target[..users.len()] == users[..] {
5350        root.components()
5351            .nth(users.len())
5352            .map(|name| name.as_os_str().to_string_lossy().into_owned())
5353    } else {
5354        None
5355    }
5356}
5357
5358/// A registration whose command line names application-data directories that
5359/// are not this account's: **another installation's service**.
5360///
5361/// Every account on a machine sees the one registration its service manager
5362/// holds under the product's name. A second account -- one that ran a
5363/// login-mode install once, say -- has an install record and a database of its
5364/// own, and comparing them against the other installation's registration used
5365/// to report a stale binary, a mismatched start mode and a secret store in the
5366/// wrong scope, with `service uninstall`, `service install` and `auth login`
5367/// as the remedies. Followed from that account, each of them breaks or
5368/// replaces a service that is healthy. So the registration is reported as the
5369/// other installation's, with what can be read of it, and this account's own
5370/// record as the unused configuration it is.
5371#[derive(Debug, Clone, PartialEq, Eq)]
5372pub struct ForeignService {
5373    /// The directories its command line names.
5374    pub directories: ServiceDirectories,
5375    /// The data root they sit under. See [`ServiceDirectories::root`].
5376    pub root: PathBuf,
5377    /// The account that data root belongs to, when its path says.
5378    pub owner: Option<String>,
5379    /// How a command run by this account reaches the installation, when it is
5380    /// this account's own under other directories -- an install made with
5381    /// `--data-dir`, seen from a command run without it, or the other way
5382    /// round. `None` for another account's.
5383    pub this_account: Option<String>,
5384    /// This account's own data root, which no registration runs against.
5385    pub local_root: PathBuf,
5386    /// Where this account's own install record is, or would be.
5387    pub local_record: PathBuf,
5388    /// This account's own install record, from an earlier install.
5389    pub unused_local_record: Option<InstallRecord>,
5390    /// Why the service's own records could not be read, or `None` when they
5391    /// were and the status describes them.
5392    pub unreadable: Option<String>,
5393}
5394
5395impl ForeignService {
5396    /// How to see the service's own details: as its owner, or elevated.
5397    #[must_use]
5398    pub fn where_to_look(&self) -> String {
5399        if let Some(reach) = &self.this_account {
5400            return reach.clone();
5401        }
5402        let elevated = if cfg!(windows) {
5403            "from an elevated terminal (Run as administrator)"
5404        } else {
5405            "with sudo"
5406        };
5407        match &self.owner {
5408            Some(owner) if owner != "LocalSystem" && owner != "root" => {
5409                format!("{elevated}, or signed in as {owner}")
5410            }
5411            _ => elevated.to_owned(),
5412        }
5413    }
5414
5415    /// Where a change to the service has to be made from: the installation's
5416    /// own account, because every remedy a command prints acts on the
5417    /// directories of whoever runs it. Elevation alone reads the service's
5418    /// records but is still this account.
5419    #[must_use]
5420    pub fn act_as(&self) -> String {
5421        if let Some(reach) = &self.this_account {
5422            return reach.clone();
5423        }
5424        match &self.owner {
5425            Some(owner) if owner != "LocalSystem" && owner != "root" => {
5426                format!("signed in as {owner}")
5427            }
5428            _ => format!(
5429                "from the account whose data root {} is",
5430                self.root.display()
5431            ),
5432        }
5433    }
5434
5435    /// Whether the service's own records could not be read, so only what the
5436    /// service manager says is known.
5437    #[must_use]
5438    pub const fn is_unread(&self) -> bool {
5439        self.unreadable.is_some()
5440    }
5441
5442    /// Who installed it, for a sentence.
5443    #[must_use]
5444    pub fn installed_by(&self) -> String {
5445        match &self.owner {
5446            Some(owner) => format!("{owner}'s data root {}", self.root.display()),
5447            None => format!("the data root {}", self.root.display()),
5448        }
5449    }
5450
5451    /// The data root and its owner, for a field: `<root> (<owner>)`.
5452    #[must_use]
5453    pub fn label(&self) -> String {
5454        match &self.owner {
5455            Some(owner) => format!("{} ({owner})", self.root.display()),
5456            None => self.root.display().to_string(),
5457        }
5458    }
5459
5460    /// The note describing this account's own record, when there is one.
5461    #[must_use]
5462    pub fn unused_local_note(&self) -> Option<String> {
5463        let record = self.unused_local_record.as_ref()?;
5464        Some(format!(
5465            "unused local configuration from an earlier install: {} records a {} registration \
5466             made by runner-manager {} on {}, and no service on this machine runs against it. \
5467             `runner-manager host forget-local` moves it aside; it never touches the service's \
5468             own directories or any secret.",
5469            self.local_record.display(),
5470            record.start_mode,
5471            record.installed_by_version,
5472            record.installed_at.format("%Y-%m-%d"),
5473        ))
5474    }
5475}
5476
5477// ---------------------------------------------------------------------------
5478// Status
5479// ---------------------------------------------------------------------------
5480
5481/// One thing `service status` has to report as wrong.
5482#[derive(Debug, Clone, PartialEq, Eq)]
5483pub struct StatusProblem {
5484    /// What it is about.
5485    pub subject: &'static str,
5486    /// What is wrong, and what to do.
5487    pub detail: String,
5488}
5489
5490impl fmt::Display for StatusProblem {
5491    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
5492        write!(f, "{}: {}", self.subject, self.detail)
5493    }
5494}
5495
5496/// Everything `service status` reports.
5497///
5498/// Journey 5 step 4 names four of these — the start mode, the resolved binary
5499/// path, the diagnostic log path, and the last successful GitHub contact — and
5500/// the release gate adds the fifth: *"`service status` reports a stale binary
5501/// path as an error rather than appearing healthy"*. [`Self::is_healthy`] is
5502/// what an exit code should be derived from, and it is false whenever
5503/// [`Self::problems`] is non-empty.
5504#[derive(Debug, Clone)]
5505pub struct ServiceStatus {
5506    identity: ServiceIdentity,
5507    record: Option<InstallRecord>,
5508    registration: Option<Registration>,
5509    binary: Option<BinaryPath>,
5510    log_file: PathBuf,
5511    store: Option<crate::secrets::ActiveStore>,
5512    last_github_contact: Option<DateTime<Utc>>,
5513    runner_root: Option<(PathBuf, RootAccessReport)>,
5514    credential_rejected_since: Option<DateTime<Utc>>,
5515    launches_blocked: Option<crate::launch_health::LaunchesBlocked>,
5516    unattended_gap: Option<UnattendedGap>,
5517    foreign: Option<ForeignService>,
5518    problems: Vec<StatusProblem>,
5519    notes: Vec<String>,
5520}
5521
5522/// Why a registration does not come back by itself after an unattended
5523/// restart, and what an operator can do about it.
5524#[derive(Debug, Clone, PartialEq, Eq)]
5525pub struct UnattendedGap {
5526    /// One or more sentences, the way `service status` prints them.
5527    pub reason: String,
5528    /// What changes it. Never something this product does by itself.
5529    pub remedy: String,
5530}
5531
5532/// The name of the account a registration runs as: the invoking user's own
5533/// for a login-mode one.
5534#[must_use]
5535pub fn service_account_name(account: &ServiceAccount) -> String {
5536    match account {
5537        // Under `sudo` the invoking user is the one sudo names, not root.
5538        ServiceAccount::InvokingUser => std::env::var("SUDO_USER")
5539            .ok()
5540            .filter(|user| !user.is_empty() && crate::host_fitness::is_elevated())
5541            .or_else(crate::unattended_login::current_account)
5542            .unwrap_or_else(|| "the operator".to_owned()),
5543        other => other.as_str().to_owned(),
5544    }
5545}
5546
5547/// Works out the gap for a registration in `mode` running as `account`.
5548///
5549/// `found` is `None` where nothing could be probed (every platform but macOS):
5550/// a login registration then keeps the general statement that it waits for a
5551/// sign-in. On a Mac the statement depends on automatic login and FileVault,
5552/// and a login registration that automatic login brings back has no gap. The
5553/// earlier statement was printed for every login registration, so a Mac set to
5554/// sign its runner account in automatically was told it would not resume.
5555#[must_use]
5556pub fn unattended_gap(
5557    mode: StartMode,
5558    account: &str,
5559    found: Option<&crate::unattended_login::UnattendedLogin>,
5560) -> Option<UnattendedGap> {
5561    let Some(found) = found else {
5562        return (mode == StartMode::Login).then(|| UnattendedGap {
5563            reason: "This registration starts at login, so the agent does not run until the \
5564                     operator signs in; this host does not resume work after an unattended \
5565                     reboot."
5566                .to_owned(),
5567            remedy: "For machine-on operation, move the service and credential to `--start-at \
5568                     boot` from an elevated terminal."
5569                .to_owned(),
5570        });
5571    };
5572    let resume = crate::unattended_login::resume(found, mode, account);
5573    if matches!(resume, crate::unattended_login::Resume::Unknown(_)) {
5574        // `host doctor` says what could not be read; a note here would claim
5575        // a gap nobody has seen.
5576        return None;
5577    }
5578    Some(UnattendedGap {
5579        reason: resume.explain(account)?,
5580        remedy: resume.remedy().unwrap_or_else(|| {
5581            "Run `runner-manager host doctor` on the Mac's desktop, where automatic login and \
5582             FileVault can be read."
5583                .to_owned()
5584        }),
5585    })
5586}
5587
5588impl ServiceStatus {
5589    /// `record_refused` carries a [`ServiceError::RecordNotPermitted`] when
5590    /// there **is** a record and this account may not read it, which is not the
5591    /// same state as having none and must not be reported as one. See
5592    /// [`Operations::status`].
5593    fn compose(
5594        identity: ServiceIdentity,
5595        record: Option<InstallRecord>,
5596        record_refused: Option<&ServiceError>,
5597        registration: Option<Registration>,
5598        last_github_contact: Option<DateTime<Utc>>,
5599        paths: &AppPaths,
5600    ) -> Self {
5601        let mut problems = Vec::new();
5602        let mut notes = Vec::new();
5603
5604        if let Some(refusal) = record_refused {
5605            // A problem rather than a note: `is_healthy` stays false, because
5606            // an operator who cannot read their own install record does have
5607            // something to fix. It is just not a reason to withhold everything
5608            // the service manager knows.
5609            problems.push(StatusProblem {
5610                subject: "install record",
5611                detail: refusal.to_string(),
5612            });
5613        }
5614
5615        let log_file = record.as_ref().map_or_else(
5616            || ServiceDirectories::of(paths).log_file(),
5617            |record| record.log_file.clone(),
5618        );
5619
5620        let binary = record.as_ref().map(|record| {
5621            inspect_binary(
5622                &record.binary,
5623                registration
5624                    .as_ref()
5625                    .and_then(Registration::binary)
5626                    .as_deref(),
5627            )
5628        });
5629        if let Some(state) = &binary
5630            && state.is_error()
5631        {
5632            problems.push(StatusProblem {
5633                subject: "binary",
5634                detail: state.to_string(),
5635            });
5636        }
5637
5638        match (&record, &registration) {
5639            (Some(_), None) => problems.push(StatusProblem {
5640                subject: "registration",
5641                detail: "this host has a service record but no service manager knows the \
5642                         registration. Run `service install` again; it deletes nothing."
5643                    .to_string(),
5644            }),
5645            // Not reported when the record was refused rather than absent: the
5646            // problem above already names the real state, and "there is no
5647            // install record" would be a second, wrong diagnosis of the same
5648            // file, with a `service uninstall` in its remedy.
5649            (None, Some(found)) if record_refused.is_none() => problems.push(StatusProblem {
5650                subject: "record",
5651                detail: format!(
5652                    "{} holds a registration for this service but there is no install record, so \
5653                     the path it was installed from and the directories it was installed against \
5654                     are unknown. Run `service uninstall` and `service install`.",
5655                    found.manager
5656                ),
5657            }),
5658            (None, Some(_)) => {}
5659            (Some(record), Some(found)) => {
5660                if record.start_mode != found.start_mode {
5661                    problems.push(StatusProblem {
5662                        subject: "start mode",
5663                        detail: format!(
5664                            "the record says {} and {} holds a {} registration. Switch the start \
5665                             mode again to make them agree.",
5666                            record.start_mode, found.manager, found.start_mode
5667                        ),
5668                    });
5669                }
5670                if record.start_mode == StartMode::Boot && !found.starts_automatically {
5671                    problems.push(StatusProblem {
5672                        subject: "start mode",
5673                        detail: format!(
5674                            "{} holds the registration but will not start it by itself, so this \
5675                             host does not resume work after a reboot.",
5676                            found.manager
5677                        ),
5678                    });
5679                }
5680                if let Some(actual) = found.restart_delay {
5681                    let expected = record.restart().effective_delay(found.manager);
5682                    if actual != expected {
5683                        problems.push(StatusProblem {
5684                            subject: "restart policy",
5685                            detail: format!(
5686                                "the record says the service restarts after {}s and {} reports \
5687                                 {}s. Something has edited the registration since it was \
5688                                 installed.",
5689                                expected.as_secs(),
5690                                found.manager,
5691                                actual.as_secs()
5692                            ),
5693                        });
5694                    } else if expected != record.restart().delay() {
5695                        // Not a fault. Task Scheduler expresses this in whole
5696                        // minutes, so the delay in force is longer than the one
5697                        // asked for -- never shorter, which is the direction
5698                        // the requirement cares about.
5699                        notes.push(format!(
5700                            "{} expresses the restart delay in whole minutes, so the {}s asked \
5701                             for is enforced as {}s. The service therefore never restarts faster \
5702                             than the configured bound.",
5703                            found.manager,
5704                            record.restart().delay().as_secs(),
5705                            expected.as_secs()
5706                        ));
5707                    }
5708                }
5709                if found.starts_automatically && !found.running {
5710                    problems.push(StatusProblem {
5711                        subject: "runtime",
5712                        detail: format!(
5713                            "{} holds an automatic registration, but the daemon is stopped. Run \
5714                             `runner-manager service install --start-at {}`; if the service \
5715                             manager denies replacement, retry from an elevated terminal. If it \
5716                             stops again, inspect {}.",
5717                            found.manager,
5718                            found.start_mode,
5719                            log_file.display()
5720                        ),
5721                    });
5722                }
5723            }
5724            (None, None) => {}
5725        }
5726
5727        // The two facts here really are independent, which is what `f1`'s
5728        // `status.rs` says this check needs and could not have there: the scope
5729        // is derived from the mode the **record** carries, and it is compared
5730        // against the mode the **service manager** actually holds.
5731        let store = record.as_ref().and_then(|record| {
5732            let registered_mode = registration
5733                .as_ref()
5734                .map_or(record.start_mode, |found| found.start_mode);
5735            crate::secrets::PlatformSecretStore::for_start_mode(record.start_mode)
5736                .ok()
5737                .map(|store| crate::secrets::ActiveStore::of(&store, registered_mode))
5738        });
5739        if let Some(store) = &store
5740            && !store.agrees_with_start_mode()
5741        {
5742            problems.push(StatusProblem {
5743                subject: "secret store",
5744                detail: format!(
5745                    "{store}. Run `auth login` again so the token is stored where the registered \
5746                     start mode can read it."
5747                ),
5748            });
5749        }
5750
5751        if last_github_contact.is_none() {
5752            notes.push(
5753                "GitHub has not been reached successfully since this host's state directory was \
5754                 created."
5755                    .to_string(),
5756            );
5757        }
5758
5759        // The privileged inspection output. Read-only, tolerant of an account
5760        // that may not read the descriptor at all, and never a `problem`: a
5761        // root this account cannot inspect is a true statement about this
5762        // account's rights, not a fault in the registration. What refuses an
5763        // install is the preflight in `runner_root_access`, which runs as the
5764        // installing account and has the authority to know.
5765        let runner_root = crate::runner_root::default_runner_root(paths)
5766            .ok()
5767            .map(|root| {
5768                let path = root.as_path().to_path_buf();
5769                let report = crate::runner_root_access::report(&path);
5770                (path, report)
5771            });
5772        // Reported, and deliberately not a `problem`. The Definition of Done
5773        // asks for a broad root to be "reported and fail the security
5774        // preflight", and the security preflight is `install`'s -- which runs
5775        // as the installing account and refuses outright. `service status` runs
5776        // as whoever typed it, is expected to be readable on a host with
5777        // nothing installed at all, and drives an exit code; turning a
5778        // directory that predates this feature into a non-zero exit for a
5779        // machine that has never installed the service would report a fault
5780        // that is not this registration's.
5781        if let Some((
5782            path,
5783            RootAccessReport::Present {
5784                broad_write: true, ..
5785            },
5786        )) = &runner_root
5787        {
5788            notes.push(format!(
5789                "the platform default runner root {} can be written by ordinary local users, so \
5790                 it is not a safe place to run jobs. `service install` refuses it rather than \
5791                 tightening it, because the contents of a directory anybody could write cannot \
5792                 be trusted: remove or empty it, or choose another root with `runner-manager \
5793                 host set-runtime-root --path <PATH>`.",
5794                path.display()
5795            ));
5796        }
5797
5798        // The surface the agent has nowhere else to reach. A root that refuses
5799        // a launch does so before any attempt row exists, and the daemon's log
5800        // redacts the paths out of the sentence, so without this the operator
5801        // sees `runner_start_failed reason=other` once per poll and nothing
5802        // that names the directory or the fix.
5803        //
5804        // A **note** and not a problem, by the same rule the broad-write root
5805        // above obeys: `service status` runs as whoever typed it, is expected
5806        // to be readable on a host with nothing installed at all, and drives an
5807        // exit code. A problem here would fail that exit code on three hosts
5808        // that are not broken -- one whose root was fixed but which has had no
5809        // queued job since, one whose service was uninstalled without clearing
5810        // `state/`, and one that only ever ran the agent in the foreground --
5811        // and the printed remedy (`service uninstall && service install`) does
5812        // not touch `state/`, so following it would not clear the error either.
5813        // A note is read by the same operator at the same moment and traps
5814        // nobody in a loop.
5815        //
5816        // A read failure is reported and not swallowed, for the reason
5817        // `runner_root_refusals` documents: absence means "every policy is
5818        // placing runners", and answering a parse failure with that would be
5819        // the wrong answer to the question this row exists for.
5820        match runner_root_refusals(paths) {
5821            Ok(refusals) => {
5822                for refusal in refusals {
5823                    notes.push(format!(
5824                        "policy {} started no runner: its runner root {} refused the launch \
5825                         ({}), last at {}. {} This clears when that policy next places a \
5826                         runner.",
5827                        refusal.policy,
5828                        refusal.root,
5829                        refusal.kind,
5830                        refusal.at.to_rfc3339(),
5831                        refusal.detail,
5832                    ));
5833                }
5834            }
5835            Err(error) => notes.push(format!(
5836                "whether the agent could use its runner roots could not be read: {error}"
5837            )),
5838        }
5839
5840        Self {
5841            identity,
5842            record,
5843            registration,
5844            binary,
5845            log_file,
5846            store,
5847            last_github_contact,
5848            runner_root,
5849            credential_rejected_since: None,
5850            launches_blocked: None,
5851            unattended_gap: None,
5852            foreign: None,
5853            problems,
5854            notes,
5855        }
5856    }
5857
5858    /// Marks a status read from another installation's directories as that
5859    /// installation's. See [`ForeignService`].
5860    ///
5861    /// Its problems were written for its owner, and their remedies -- `service
5862    /// install`, `auth login`, re-registering a drifted definition -- act on
5863    /// the directories of whoever runs them. Printed to this account, which
5864    /// reaches here precisely by following "run it elevated", they would
5865    /// re-register the machine's service against this account's data root.
5866    /// So each keeps its diagnosis and loses its remedy, and says where the
5867    /// remedy has to be run instead.
5868    fn with_foreign(mut self, foreign: ForeignService) -> Self {
5869        let act_as = foreign.act_as();
5870        for problem in &mut self.problems {
5871            problem.detail = format!(
5872                "{}. Act on it {act_as}: a remedy run from this account acts on this account's \
5873                 own directories, not the service's.",
5874                diagnosis_only(&problem.detail)
5875            );
5876        }
5877        if let Some(note) = foreign.unused_local_note() {
5878            self.notes.push(note);
5879        }
5880        self.foreign = Some(foreign);
5881        self
5882    }
5883
5884    /// A registration made from another data root whose records this account
5885    /// cannot read: what the service manager reports, and nothing else.
5886    ///
5887    /// The only problems are ones the service manager alone settles -- an
5888    /// automatic registration that is stopped, a boot registration that will
5889    /// not start by itself -- and their remedy is to look as the owner, never
5890    /// to reinstall from here.
5891    fn foreign_unread(
5892        identity: ServiceIdentity,
5893        registration: Registration,
5894        foreign: ForeignService,
5895    ) -> Self {
5896        let mut problems = Vec::new();
5897        let look = foreign.where_to_look();
5898        if registration.starts_automatically && !registration.running {
5899            problems.push(StatusProblem {
5900                subject: "runtime",
5901                detail: format!(
5902                    "{} holds an automatic registration for the service installed from {}, but \
5903                     it is stopped. Run `runner-manager service status` {look} to see why; do not \
5904                     reinstall it from this account.",
5905                    registration.manager,
5906                    foreign.installed_by(),
5907                ),
5908            });
5909        }
5910        if registration.start_mode == StartMode::Boot && !registration.starts_automatically {
5911            problems.push(StatusProblem {
5912                subject: "start mode",
5913                detail: format!(
5914                    "{} holds the boot registration installed from {} but will not start it by \
5915                     itself. Run `runner-manager service status` {look}.",
5916                    registration.manager,
5917                    foreign.installed_by(),
5918                ),
5919            });
5920        }
5921        let mut notes = Vec::new();
5922        if let Some(reason) = &foreign.unreadable {
5923            notes.push(format!(
5924                "its health is not known here: {reason}. Run `runner-manager service status` \
5925                 {look} to see it."
5926            ));
5927        }
5928        notes.extend(foreign.unused_local_note());
5929        Self {
5930            identity,
5931            record: None,
5932            log_file: foreign.directories.log_file(),
5933            registration: Some(registration),
5934            binary: None,
5935            store: None,
5936            last_github_contact: None,
5937            runner_root: None,
5938            credential_rejected_since: None,
5939            launches_blocked: None,
5940            unattended_gap: None,
5941            foreign: Some(foreign),
5942            problems,
5943            notes,
5944        }
5945    }
5946
5947    /// The other installation this machine's service belongs to, when it is
5948    /// not this account's. See [`ForeignService`].
5949    #[must_use]
5950    pub const fn foreign(&self) -> Option<&ForeignService> {
5951        self.foreign.as_ref()
5952    }
5953
5954    /// Whether this is another installation's service whose own records this
5955    /// account could not read, so only the service manager's facts are here.
5956    #[must_use]
5957    pub fn is_unread(&self) -> bool {
5958        self.foreign.as_ref().is_some_and(ForeignService::is_unread)
5959    }
5960
5961    /// The one-word-or-so verdict `service status` ends with.
5962    #[must_use]
5963    pub fn verdict(&self) -> String {
5964        if !self.is_healthy() {
5965            return "NOT healthy".to_owned();
5966        }
5967        match &self.foreign {
5968            Some(foreign) if foreign.is_unread() => format!(
5969                "{}; installed from {}, so its health is read {}",
5970                if self.is_running() {
5971                    "running"
5972                } else {
5973                    "not running"
5974                },
5975                foreign.installed_by(),
5976                foreign.where_to_look()
5977            ),
5978            _ => "healthy".to_owned(),
5979        }
5980    }
5981
5982    /// Reports a definition this build would render differently.
5983    #[must_use]
5984    pub fn with_definition_drift(mut self, drift: Option<DefinitionDrift>) -> Self {
5985        if let Some(drift) = drift {
5986            if drift.manager == DefinitionKind::LaunchdPlist {
5987                // A restart, not `service install`: the running daemon holds the
5988                // single-instance lock, which refuses an install, and a daemon
5989                // that starts with no runner repairs the plist itself.
5990                let restart = match drift.start_mode {
5991                    StartMode::Boot => format!(
5992                        "sudo launchctl kickstart -k system/{}",
5993                        self.identity.launchd_label()
5994                    ),
5995                    StartMode::Login => format!(
5996                        "launchctl kickstart -k gui/$(id -u)/{}",
5997                        self.identity.launchd_label()
5998                    ),
5999                };
6000                self.problems.push(StatusProblem {
6001                    subject: "definition",
6002                    detail: format!(
6003                        "{} was written by runner-manager {} with a `ProcessType` other than \
6004                         `{LAUNCHD_PROCESS_TYPE}`, which launchd applies to every runner the \
6005                         daemon starts, so their jobs run below normal priority. The daemon \
6006                         rewrites it and reloads itself when it starts with no runner, which it \
6007                         does after every upgrade; to do it now, restart the service while no \
6008                         job is running: `{restart}`. Configuration and credentials are kept.",
6009                        drift.path.display(),
6010                        drift.installed_by_version,
6011                    ),
6012                });
6013                return self;
6014            }
6015            let repair = match drift.start_mode {
6016                StartMode::Boot => {
6017                    "`sudo systemctl stop runner-manager && sudo runner-manager service install \
6018                     --start-at boot` (for a managed WSL host, run `runner-manager wsl install \
6019                     --distribution <name>` from Windows)"
6020                }
6021                StartMode::Login => {
6022                    "`systemctl --user stop runner-manager && runner-manager service install \
6023                     --start-at login`"
6024                }
6025            };
6026            self.problems.push(StatusProblem {
6027                subject: "definition",
6028                detail: format!(
6029                    "{} was written by runner-manager {} and differs from what {} renders, so \
6030                     the daemon runs under outdated settings (for example a credential frozen \
6031                     at service start). Re-register it with {repair}; configuration and \
6032                     credentials are kept.",
6033                    drift.path.display(),
6034                    drift.installed_by_version,
6035                    env!("CARGO_PKG_VERSION"),
6036                ),
6037            });
6038        }
6039        self
6040    }
6041
6042    /// Reports that GitHub is rejecting the daemon's credential.
6043    #[must_use]
6044    pub fn with_credential_rejection(mut self, since: Option<DateTime<Utc>>) -> Self {
6045        if let Some(since) = since {
6046            self.credential_rejected_since = Some(since);
6047            self.problems.push(StatusProblem {
6048                subject: "github credential",
6049                detail: format!(
6050                    "GitHub has rejected the service's credential since {}, so the service \
6051                     cannot see queued jobs or start runners. Run `runner-manager auth login` \
6052                     (for a managed WSL host, `runner-manager --host wsl:<name> auth login` \
6053                     from Windows).",
6054                    since.to_rfc3339()
6055                ),
6056            });
6057        }
6058        self
6059    }
6060
6061    /// Since when GitHub has rejected the service's credential, if it has.
6062    #[must_use]
6063    pub const fn credential_rejected_since(&self) -> Option<DateTime<Utc>> {
6064        self.credential_rejected_since
6065    }
6066
6067    /// Reports that the daemon starts no runner because its launches are
6068    /// blocked. See [`crate::launch_health`].
6069    #[must_use]
6070    pub fn with_launches_blocked(
6071        mut self,
6072        blocked: Option<crate::launch_health::LaunchesBlocked>,
6073    ) -> Self {
6074        if let Some(blocked) = &blocked {
6075            self.problems.push(StatusProblem {
6076                subject: LAUNCHES_BLOCKED_SUBJECT,
6077                detail: format!(
6078                    "the service has started no runner {blocked}. Fix: {}.",
6079                    blocked.remedy
6080                ),
6081            });
6082        }
6083        self.launches_blocked = blocked;
6084        self
6085    }
6086
6087    /// Why the service starts no runner, when its launches are blocked.
6088    #[must_use]
6089    pub const fn launches_blocked(&self) -> Option<&crate::launch_health::LaunchesBlocked> {
6090        self.launches_blocked.as_ref()
6091    }
6092
6093    /// Reports that the daemon cannot read its own stored credential, and the
6094    /// exact command that ends it. See [`record_credential_unreadable`].
6095    #[must_use]
6096    pub fn with_credential_unreadable(mut self, since: Option<DateTime<Utc>>) -> Self {
6097        if let Some(since) = since {
6098            let mode = self.start_mode().unwrap_or_default();
6099            self.problems.push(StatusProblem {
6100                subject: CREDENTIAL_UNREADABLE_SUBJECT,
6101                detail: format!(
6102                    "the service has been unable to read its stored GitHub credential since {}, \
6103                     so it starts no runner; the diagnostic log has the store's own answer.{} \
6104                     Sign in again: {}.",
6105                    since.to_rfc3339(),
6106                    if cfg!(target_os = "macos") {
6107                        " That is what a keychain item written by a different build looks like."
6108                    } else {
6109                        ""
6110                    },
6111                    sign_in_instruction(mode)
6112                ),
6113            });
6114        }
6115        self
6116    }
6117
6118    /// What this host's default runner root grants, and to whom.
6119    ///
6120    /// `None` when the platform default could not even be resolved. The
6121    /// descriptor inside has been through
6122    /// [`crate::runner_root_access::redact`], so it names the well-known
6123    /// trustees and says "an account" for everything else — no more identity
6124    /// than the `account` line above it already prints.
6125    #[must_use]
6126    pub fn runner_root(&self) -> Option<(&Path, &RootAccessReport)> {
6127        self.runner_root
6128            .as_ref()
6129            .map(|(path, report)| (path.as_path(), report))
6130    }
6131
6132    /// Whether a registration exists at all.
6133    #[must_use]
6134    pub const fn is_installed(&self) -> bool {
6135        self.registration.is_some() || self.record.is_some()
6136    }
6137
6138    /// Whether the daemon is running now.
6139    #[must_use]
6140    pub fn is_running(&self) -> bool {
6141        self.registration
6142            .as_ref()
6143            .is_some_and(|registration| registration.running)
6144    }
6145
6146    /// **False whenever anything is wrong**, including a stale binary path.
6147    #[must_use]
6148    pub fn is_healthy(&self) -> bool {
6149        self.problems.is_empty()
6150    }
6151
6152    /// Everything that is wrong.
6153    #[must_use]
6154    pub fn problems(&self) -> &[StatusProblem] {
6155        &self.problems
6156    }
6157
6158    /// True statements that are not faults — a login-mode registration not
6159    /// running unattended, or an agent that has not yet reached GitHub.
6160    #[must_use]
6161    pub fn notes(&self) -> &[String] {
6162        &self.notes
6163    }
6164
6165    /// Reports that the daemon starts no runner because a required host check
6166    /// fails from its own session. That is the service not doing its job, so
6167    /// it is a problem: a host whose daemon refused every runner used to read
6168    /// `verdict healthy`.
6169    #[must_use]
6170    pub fn with_host_unfit(mut self, unfit: Option<crate::host_fitness::HostUnfitRecord>) -> Self {
6171        if let Some(unfit) = unfit {
6172            self.problems.push(StatusProblem {
6173                subject: HOST_UNFIT_SUBJECT,
6174                detail: format!(
6175                    "the service has started no runner since {}, because a required host check \
6176                     fails from its own session: {}",
6177                    unfit.since.to_rfc3339(),
6178                    unfit.describe()
6179                ),
6180            });
6181        }
6182        self
6183    }
6184
6185    /// Reports a daemon that is running and has stopped making progress. See
6186    /// [`crate::daemon_heartbeat`]: launchd, systemd and the SCM all report
6187    /// such a daemon running, so this used to read `verdict healthy`.
6188    #[must_use]
6189    pub fn with_stall(mut self, stall: Option<String>) -> Self {
6190        if let Some(detail) = stall {
6191            self.problems.push(StatusProblem {
6192                subject: DAEMON_STALLED_SUBJECT,
6193                detail,
6194            });
6195        }
6196        self
6197    }
6198
6199    /// Records whether this registration comes back by itself after an
6200    /// unattended restart, as a note. See [`unattended_gap`].
6201    #[must_use]
6202    pub fn with_unattended_gap(mut self, gap: Option<UnattendedGap>) -> Self {
6203        if let Some(gap) = &gap {
6204            self.notes.push(gap.reason.clone());
6205        }
6206        self.unattended_gap = gap;
6207        self
6208    }
6209
6210    /// Why this registration does not come back by itself after an
6211    /// unattended restart, when it does not.
6212    #[must_use]
6213    pub const fn unattended_gap(&self) -> Option<&UnattendedGap> {
6214        self.unattended_gap.as_ref()
6215    }
6216
6217    /// The service manager's command that restarts this registration, for its
6218    /// start mode (boot when unknown). See [`ServiceIdentity::restart_command`].
6219    #[must_use]
6220    pub fn restart_command(&self) -> String {
6221        self.identity
6222            .restart_command(self.start_mode().unwrap_or(StartMode::Boot))
6223    }
6224
6225    /// The recorded start mode. Journey 5 step 4. For another installation's
6226    /// service whose record this account cannot read, the mode the service
6227    /// manager's domain implies.
6228    #[must_use]
6229    pub fn start_mode(&self) -> Option<StartMode> {
6230        self.record
6231            .as_ref()
6232            .map(|record| record.start_mode)
6233            .or_else(|| {
6234                self.registration
6235                    .as_ref()
6236                    .filter(|_| self.is_unread())
6237                    .map(|registration| registration.start_mode)
6238            })
6239    }
6240
6241    /// The binary the service runs: the recorded one, or for another
6242    /// installation's service whose record this account cannot read, the one
6243    /// the service manager names.
6244    #[must_use]
6245    pub fn service_binary(&self) -> Option<PathBuf> {
6246        self.record
6247            .as_ref()
6248            .map(|record| record.binary.clone())
6249            .or_else(|| {
6250                self.registration
6251                    .as_ref()
6252                    .filter(|_| self.is_unread())
6253                    .and_then(Registration::binary)
6254            })
6255    }
6256
6257    /// The resolved absolute binary path and what became of it. Journey 5
6258    /// step 4 and `05-infrastructure.md` item 6.
6259    #[must_use]
6260    pub const fn binary(&self) -> Option<&BinaryPath> {
6261        self.binary.as_ref()
6262    }
6263
6264    /// The diagnostic log path. `05-infrastructure.md` item 4.
6265    #[must_use]
6266    pub fn log_file(&self) -> &Path {
6267        &self.log_file
6268    }
6269
6270    /// The last successful GitHub contact. Journey 5 step 4.
6271    #[must_use]
6272    pub const fn last_github_contact(&self) -> Option<DateTime<Utc>> {
6273        self.last_github_contact
6274    }
6275
6276    /// Which store the daemon reads, and whether that agrees with the
6277    /// registration.
6278    #[must_use]
6279    pub const fn secret_store(&self) -> Option<&crate::secrets::ActiveStore> {
6280        self.store.as_ref()
6281    }
6282
6283    /// The record `install` wrote.
6284    #[must_use]
6285    pub const fn record(&self) -> Option<&InstallRecord> {
6286        self.record.as_ref()
6287    }
6288
6289    /// What the service manager says.
6290    #[must_use]
6291    pub const fn registration(&self) -> Option<&Registration> {
6292        self.registration.as_ref()
6293    }
6294}
6295
6296impl fmt::Display for ServiceStatus {
6297    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
6298        writeln!(f, "Service: {}", self.identity)?;
6299        match (&self.record, &self.registration) {
6300            (None, None) => {
6301                writeln!(
6302                    f,
6303                    "  installed                 no. `service install` registers `{} {}`.",
6304                    SERVICE_NAME,
6305                    DAEMON_ARGUMENTS.join(" ")
6306                )?;
6307            }
6308            _ => {
6309                let manager = self
6310                    .registration
6311                    .as_ref()
6312                    .map(|registration| registration.manager.manager());
6313                writeln!(
6314                    f,
6315                    "  installed                 {}",
6316                    manager.unwrap_or("yes, but no service manager knows it")
6317                )?;
6318                writeln!(
6319                    f,
6320                    "  state                     {}",
6321                    if self.is_running() {
6322                        "running"
6323                    } else {
6324                        "not running"
6325                    }
6326                )?;
6327            }
6328        }
6329        if let Some(foreign) = &self.foreign {
6330            writeln!(
6331                f,
6332                "  installed from            {} -- another installation's, not this account's ({})",
6333                foreign.installed_by(),
6334                foreign.local_root.display()
6335            )?;
6336            // Without the record, the service manager's own facts.
6337            if foreign.is_unread()
6338                && let Some(registration) = &self.registration
6339            {
6340                writeln!(f, "  start mode                {}", registration.start_mode)?;
6341                if let Some(account) = &registration.account {
6342                    writeln!(f, "  account                   {account}")?;
6343                }
6344                if let Some(binary) = self.service_binary() {
6345                    writeln!(f, "  binary                    {}", binary.display())?;
6346                }
6347            }
6348        }
6349        if let Some(record) = &self.record {
6350            writeln!(f, "  start mode                {}", record.start_mode)?;
6351            writeln!(f, "  account                   {}", record.account)?;
6352            writeln!(f, "  restart on failure        {}", record.restart())?;
6353            writeln!(
6354                f,
6355                "  arguments                 {}",
6356                record.arguments.join(" ")
6357            )?;
6358        }
6359        if let Some(binary) = &self.binary {
6360            writeln!(f, "  binary                    {binary}")?;
6361        }
6362        writeln!(f, "  diagnostic log            {}", self.log_file.display())?;
6363        if let Some((path, report)) = &self.runner_root {
6364            // Named as the *default* rather than as "the runner root", because
6365            // it is only the effective one until an operator runs
6366            // `host set-runtime-root`. This crate cannot see that setting — it
6367            // lives in the application's store — so an unqualified label here
6368            // would contradict the `runner root … (host-configured)` row that
6369            // `status` and `host show` print from the value that is in force.
6370            writeln!(f, "  default runner root       {}", path.display())?;
6371            if *report != RootAccessReport::NotApplicable {
6372                writeln!(f, "  default root access       {report}")?;
6373            }
6374        }
6375        if let Some(store) = &self.store {
6376            writeln!(f, "  secret store              {store}")?;
6377        }
6378        // Not known, rather than "never", for a service whose records were not
6379        // read: its contact file is in a directory this account cannot open.
6380        if !self.is_unread() {
6381            writeln!(
6382                f,
6383                "  last GitHub contact       {}",
6384                match self.last_github_contact {
6385                    Some(at) => at.to_rfc3339(),
6386                    None => "never".to_string(),
6387                }
6388            )?;
6389        }
6390        for note in &self.notes {
6391            writeln!(f, "  note                      {note}")?;
6392        }
6393        for problem in &self.problems {
6394            writeln!(f, "  ERROR                     {problem}")?;
6395        }
6396        write!(f, "  verdict                   {}", self.verdict())
6397    }
6398}
6399
6400// ---------------------------------------------------------------------------
6401// The in-memory double
6402// ---------------------------------------------------------------------------
6403
6404/// A [`ControlFactory`] that registers nothing and remembers everything.
6405///
6406/// Public rather than `#[cfg(test)]` on purpose. `f3` builds three commands on
6407/// [`ServiceOperations`] and has to be able to test them without a service
6408/// manager, and a double that lived behind `#[cfg(test)]` here would be
6409/// invisible from another crate — which would leave `f3` either untested or
6410/// writing a second double that drifts from this one.
6411///
6412/// It is not a simulator. It stores what it was asked to register and reports it
6413/// back, which is exactly enough to exercise the logic that sits *above* a
6414/// service manager: the lock refusal, the record, the stale-path detection, the
6415/// start-mode switch, and the promise that uninstall deletes nothing else.
6416#[derive(Debug, Clone, Default)]
6417pub struct RecordingControls {
6418    state: std::sync::Arc<std::sync::Mutex<RecordingState>>,
6419}
6420
6421#[derive(Debug, Default)]
6422struct RecordingState {
6423    registrations: BTreeMap<(StartMode, String), Registration>,
6424    definitions: BTreeMap<String, ServiceDefinition>,
6425    calls: Vec<String>,
6426    #[cfg(test)]
6427    install_failures: BTreeMap<StartMode, String>,
6428    #[cfg(test)]
6429    after_install: BTreeMap<StartMode, TestInstallSideEffect>,
6430}
6431
6432#[cfg(test)]
6433#[derive(Debug, Clone)]
6434enum TestInstallSideEffect {
6435    HideDirectory { directory: PathBuf, hidden: PathBuf },
6436}
6437
6438impl RecordingControls {
6439    /// A factory holding no registrations.
6440    #[must_use]
6441    pub fn new() -> Self {
6442        Self::default()
6443    }
6444
6445    /// Every call made through this factory, in order, as
6446    /// `"<operation> <name> (<mode>)"`.
6447    #[must_use]
6448    pub fn calls(&self) -> Vec<String> {
6449        self.state.lock().expect("not poisoned").calls.clone()
6450    }
6451
6452    /// Every registration currently held, with the domain it is in.
6453    #[must_use]
6454    pub fn registrations(&self) -> Vec<(StartMode, String, Registration)> {
6455        self.state
6456            .lock()
6457            .expect("not poisoned")
6458            .registrations
6459            .iter()
6460            .map(|((mode, name), registration)| (*mode, name.clone(), registration.clone()))
6461            .collect()
6462    }
6463
6464    /// The definition applied for a registration, if it is held.
6465    #[must_use]
6466    pub fn definition(&self, name: &str) -> Option<ServiceDefinition> {
6467        self.state
6468            .lock()
6469            .expect("not poisoned")
6470            .definitions
6471            .get(name)
6472            .cloned()
6473    }
6474
6475    /// Edits a held registration, as something outside this product would.
6476    ///
6477    /// This is how the divergence and start-type problems are made reachable
6478    /// from a test: `sc config`, `launchctl` and `systemctl` can all change a
6479    /// registration after installation, and a status command that could not be
6480    /// shown detecting that would be a status command nobody had tested against
6481    /// the case it exists for.
6482    ///
6483    /// Does nothing when no registration of that name is held.
6484    pub fn edit(&self, name: &str, edit: impl FnOnce(&mut Registration)) {
6485        let mut state = self.state.lock().expect("not poisoned");
6486        if let Some((_, registration)) = state
6487            .registrations
6488            .iter_mut()
6489            .find(|((_, held), _)| held == name)
6490        {
6491            edit(registration);
6492        }
6493    }
6494
6495    #[cfg(test)]
6496    fn fail_next_install(&self, mode: StartMode, detail: &str) {
6497        self.state
6498            .lock()
6499            .expect("not poisoned")
6500            .install_failures
6501            .insert(mode, detail.to_string());
6502    }
6503
6504    #[cfg(test)]
6505    fn hide_directory_after_install(&self, mode: StartMode, directory: PathBuf, hidden: PathBuf) {
6506        self.state
6507            .lock()
6508            .expect("not poisoned")
6509            .after_install
6510            .insert(
6511                mode,
6512                TestInstallSideEffect::HideDirectory { directory, hidden },
6513            );
6514    }
6515}
6516
6517impl ControlFactory for RecordingControls {
6518    fn control(&self, mode: StartMode) -> Result<Box<dyn ServiceControl>, ServiceError> {
6519        Ok(Box::new(RecordingControl {
6520            mode,
6521            state: std::sync::Arc::clone(&self.state),
6522        }))
6523    }
6524}
6525
6526#[derive(Debug)]
6527struct RecordingControl {
6528    mode: StartMode,
6529    state: std::sync::Arc<std::sync::Mutex<RecordingState>>,
6530}
6531
6532impl RecordingControl {
6533    fn note(&self, operation: &str, name: &str) {
6534        self.state
6535            .lock()
6536            .expect("not poisoned")
6537            .calls
6538            .push(format!("{operation} {name} ({})", self.mode));
6539    }
6540}
6541
6542impl ServiceControl for RecordingControl {
6543    fn manager(&self) -> DefinitionKind {
6544        // The double reports the kind this host's real backend would, so a test
6545        // that asserts on the manager name asserts something true of the
6546        // platform it is running on.
6547        host_definition_kind(self.mode)
6548    }
6549
6550    fn install(&self, plan: &InstallPlan) -> Result<ServiceDefinition, ServiceError> {
6551        self.note("install", plan.identity().name());
6552        #[cfg(test)]
6553        if let Some(detail) = self
6554            .state
6555            .lock()
6556            .expect("not poisoned")
6557            .install_failures
6558            .remove(&self.mode)
6559        {
6560            return Err(ServiceError::Control {
6561                operation: "install",
6562                name: plan.identity().name().to_string(),
6563                manager: "recording control",
6564                detail,
6565            });
6566        }
6567        // The real definition, not a stub. Rendering is pure, so the double can
6568        // afford it -- and a stub would make `Installed::review` report that the
6569        // definition confirms nothing, which is exactly the shape of false
6570        // assurance a test double must never hand back.
6571        let definition = ServiceDefinition::for_host(plan)?;
6572        let mut state = self.state.lock().expect("not poisoned");
6573        state.registrations.insert(
6574            (self.mode, plan.identity().name().to_string()),
6575            Registration {
6576                manager: host_definition_kind(self.mode),
6577                start_mode: self.mode,
6578                command_line: plan.command_line(),
6579                account: Some(plan.account().as_str().to_string()),
6580                running: false,
6581                starts_automatically: true,
6582                restart_delay: Some(plan.restart().delay()),
6583            },
6584        );
6585        state
6586            .definitions
6587            .insert(plan.identity().name().to_string(), definition.clone());
6588        #[cfg(test)]
6589        let side_effect = state.after_install.remove(&self.mode);
6590        drop(state);
6591        #[cfg(test)]
6592        if let Some(TestInstallSideEffect::HideDirectory { directory, hidden }) = side_effect {
6593            std::fs::rename(&directory, &hidden).expect("test fault can hide the record directory");
6594            std::fs::write(&directory, b"blocks recreation")
6595                .expect("test fault can block record directory recreation");
6596        }
6597        Ok(definition)
6598    }
6599
6600    fn uninstall(&self, identity: &ServiceIdentity) -> Result<bool, ServiceError> {
6601        self.note("uninstall", identity.name());
6602        let mut state = self.state.lock().expect("not poisoned");
6603        state.definitions.remove(identity.name());
6604        Ok(state
6605            .registrations
6606            .remove(&(self.mode, identity.name().to_string()))
6607            .is_some())
6608    }
6609
6610    fn query(&self, identity: &ServiceIdentity) -> Result<Option<Registration>, ServiceError> {
6611        self.note("query", identity.name());
6612        Ok(self
6613            .state
6614            .lock()
6615            .expect("not poisoned")
6616            .registrations
6617            .get(&(self.mode, identity.name().to_string()))
6618            .cloned())
6619    }
6620
6621    fn start(&self, identity: &ServiceIdentity) -> Result<(), ServiceError> {
6622        self.note("start", identity.name());
6623        let mut state = self.state.lock().expect("not poisoned");
6624        match state
6625            .registrations
6626            .get_mut(&(self.mode, identity.name().to_string()))
6627        {
6628            Some(registration) => {
6629                registration.running = true;
6630                Ok(())
6631            }
6632            None => Err(ServiceError::NotInstalled {
6633                name: identity.name().to_string(),
6634                operation: "start",
6635            }),
6636        }
6637    }
6638
6639    fn stop(&self, identity: &ServiceIdentity) -> Result<bool, ServiceError> {
6640        self.note("stop", identity.name());
6641        let mut state = self.state.lock().expect("not poisoned");
6642        match state
6643            .registrations
6644            .get_mut(&(self.mode, identity.name().to_string()))
6645        {
6646            Some(registration) => Ok(std::mem::replace(&mut registration.running, false)),
6647            None => Err(ServiceError::NotInstalled {
6648                name: identity.name().to_string(),
6649                operation: "stop",
6650            }),
6651        }
6652    }
6653}
6654
6655impl ControlFactory for HostControls {
6656    fn control(&self, mode: StartMode) -> Result<Box<dyn ServiceControl>, ServiceError> {
6657        sys::control(mode)
6658    }
6659
6660    fn installed_definition(&self, path: &Path) -> Option<String> {
6661        std::fs::read_to_string(path).ok()
6662    }
6663
6664    fn unattended_login(&self) -> Option<crate::unattended_login::UnattendedLogin> {
6665        crate::unattended_login::probe()
6666    }
6667}
6668
6669/// The operator's home directory, when the platform reports one.
6670///
6671/// Only login-mode registrations need it — a LaunchAgent and a systemd user
6672/// unit both live under it — and it is resolved lazily so that a boot-mode
6673/// operation on an account with no profile is not refused for wanting something
6674/// it never uses.
6675#[must_use]
6676pub fn host_home() -> Option<PathBuf> {
6677    directories::BaseDirs::new().map(|dirs| dirs.home_dir().to_path_buf())
6678}
6679
6680/// The Unix backends' name for the same thing.
6681#[cfg(unix)]
6682fn home_directory() -> Option<PathBuf> {
6683    host_home()
6684}
6685
6686/// Runs a command and returns its exit status with both streams as text.
6687///
6688/// The three Unix backends and the Windows Task Scheduler backend all drive a
6689/// stock command-line tool, and all four want the same three things back. The
6690/// tools are chosen for machine-readable, locale-independent output wherever one
6691/// exists — `systemctl is-active`, `launchctl print`, `schtasks /XML` — because
6692/// parsing a localised human-facing table is how a status command starts lying
6693/// on somebody else's machine.
6694fn run(program: &str, arguments: &[&std::ffi::OsStr]) -> std::io::Result<(bool, String, String)> {
6695    let output = std::process::Command::new(program)
6696        .args(arguments)
6697        .output()?;
6698    Ok((
6699        output.status.success(),
6700        String::from_utf8_lossy(&output.stdout).into_owned(),
6701        String::from_utf8_lossy(&output.stderr).into_owned(),
6702    ))
6703}
6704
6705/// Which manager holds a given start-mode domain on the platform this binary
6706/// was built for.
6707#[must_use]
6708pub const fn host_definition_kind(mode: StartMode) -> DefinitionKind {
6709    if cfg!(windows) {
6710        match mode {
6711            StartMode::Boot => DefinitionKind::WindowsService,
6712            StartMode::Login => DefinitionKind::WindowsScheduledTask,
6713        }
6714    } else if cfg!(target_os = "macos") {
6715        DefinitionKind::LaunchdPlist
6716    } else {
6717        DefinitionKind::SystemdUnit
6718    }
6719}
6720
6721/// Enables a launchd label after it has been bootstrapped.
6722///
6723/// `launchctl bootstrap` deliberately preserves a label's disabled bit. An
6724/// `enable` failure therefore means the new registration cannot satisfy its
6725/// start policy. Compensate by booting it out and removing the plist so a
6726/// failed install cannot look installed to `service status`.
6727#[cfg(any(target_os = "macos", test))]
6728fn enable_launchd_registration(
6729    mut launchctl: impl FnMut(&[&std::ffi::OsStr]) -> (bool, String),
6730    domain: &str,
6731    service_target: &str,
6732    plist: &Path,
6733    name: &str,
6734    elevation_remedy: &'static str,
6735) -> Result<(), ServiceError> {
6736    let (enabled, cause) = launchctl(&[
6737        std::ffi::OsStr::new("enable"),
6738        std::ffi::OsStr::new(service_target),
6739    ]);
6740    if enabled {
6741        return Ok(());
6742    }
6743
6744    let (booted_out, bootout_detail) = launchctl(&[
6745        std::ffi::OsStr::new("bootout"),
6746        std::ffi::OsStr::new(service_target),
6747    ]);
6748    let removed = std::fs::remove_file(plist);
6749    if !booted_out || removed.is_err() {
6750        return Err(ServiceError::Rollback {
6751            operation: "enable launchd registration",
6752            name: name.to_string(),
6753            cause,
6754            rollback: format!(
6755                "launchctl bootout {domain}: {}; remove {}: {}",
6756                if booted_out {
6757                    "succeeded".to_string()
6758                } else {
6759                    bootout_detail
6760                },
6761                plist.display(),
6762                removed
6763                    .err()
6764                    .map_or_else(|| "succeeded".to_string(), |error| error.to_string())
6765            ),
6766        });
6767    }
6768
6769    if cause.to_ascii_lowercase().contains("permission denied") {
6770        Err(ServiceError::NeedsElevation {
6771            operation: "enable",
6772            name: name.to_string(),
6773            detail: cause,
6774            remedy: elevation_remedy,
6775        })
6776    } else {
6777        Err(ServiceError::Control {
6778            operation: "enable",
6779            name: name.to_string(),
6780            manager: "launchd",
6781            detail: cause,
6782        })
6783    }
6784}
6785
6786// ---------------------------------------------------------------------------
6787// Windows
6788// ---------------------------------------------------------------------------
6789
6790/// A stop request delivered by the Windows Service Control Manager.
6791///
6792/// The application owns the drain policy; this platform boundary only turns
6793/// `SERVICE_CONTROL_STOP`/`SHUTDOWN` into an awaitable notification.
6794#[derive(Debug, Clone)]
6795pub struct ServiceShutdown(tokio::sync::watch::Receiver<bool>);
6796
6797impl ServiceShutdown {
6798    /// Waits until SCM asks the service to stop or shut down.
6799    pub async fn wait(mut self) {
6800        if !*self.0.borrow() {
6801            let _ = self.0.changed().await;
6802        }
6803    }
6804}
6805
6806/// Runs the production process as a real Windows service host.
6807///
6808/// `StartServiceCtrlDispatcher` must run in the service process's main thread,
6809/// so the callback is handed through a process-global slot to the entrypoint
6810/// invoked by SCM. The slot is single-use by design: one process hosts exactly
6811/// one own-process service.
6812#[cfg(windows)]
6813pub fn run_windows_service_host<F>(run: F) -> Result<u8, ServiceError>
6814where
6815    F: FnOnce(ServiceShutdown) -> u8 + Send + 'static,
6816{
6817    windows_host::run(Box::new(run))
6818}
6819
6820#[cfg(windows)]
6821mod windows_host {
6822    use std::ffi::OsString;
6823    use std::sync::{Arc, Mutex, OnceLock, mpsc};
6824    use std::time::Duration;
6825
6826    use windows_service::service::{
6827        ServiceControl, ServiceControlAccept, ServiceExitCode, ServiceState, ServiceStatus,
6828        ServiceType,
6829    };
6830    use windows_service::service_control_handler::{
6831        self, ServiceControlHandlerResult, ServiceStatusHandle,
6832    };
6833
6834    use super::{SERVICE_NAME, ServiceError, ServiceShutdown};
6835
6836    type Runner = Box<dyn FnOnce(ServiceShutdown) -> u8 + Send>;
6837
6838    struct Invocation {
6839        run: Runner,
6840        result: mpsc::SyncSender<Result<u8, String>>,
6841    }
6842
6843    static INVOCATION: OnceLock<Mutex<Option<Invocation>>> = OnceLock::new();
6844
6845    windows_service::define_windows_service!(ffi_service_main, service_main);
6846
6847    pub(super) fn run(run: Runner) -> Result<u8, ServiceError> {
6848        let (result_tx, result_rx) = mpsc::sync_channel(1);
6849        let slot = INVOCATION.get_or_init(|| Mutex::new(None));
6850        let mut invocation = slot
6851            .lock()
6852            .map_err(|_| host_error("prepare", "the service-host slot is poisoned"))?;
6853        if invocation.is_some() {
6854            return Err(host_error(
6855                "prepare",
6856                "the service-host slot was already used",
6857            ));
6858        }
6859        *invocation = Some(Invocation {
6860            run,
6861            result: result_tx,
6862        });
6863        drop(invocation);
6864
6865        windows_service::service_dispatcher::start("", ffi_service_main)
6866            .map_err(|error| host_error("connect", &error.to_string()))?;
6867        result_rx
6868            .recv()
6869            .map_err(|error| host_error("finish", &error.to_string()))?
6870            .map_err(|detail| host_error("run", &detail))
6871    }
6872
6873    fn service_main(_arguments: Vec<OsString>) {
6874        let Some(invocation) = INVOCATION.get().and_then(|slot| slot.lock().ok()?.take()) else {
6875            return;
6876        };
6877        let result = run_service(invocation.run);
6878        let _ = invocation.result.send(result);
6879    }
6880
6881    fn run_service(run: Runner) -> Result<u8, String> {
6882        let (stop_tx, stop_rx) = tokio::sync::watch::channel(false);
6883        let status: Arc<Mutex<Option<ServiceStatusHandle>>> = Arc::new(Mutex::new(None));
6884        let handler_status = Arc::clone(&status);
6885        let handler = move |control| match control {
6886            ServiceControl::Interrogate => ServiceControlHandlerResult::NoError,
6887            ServiceControl::Stop | ServiceControl::Shutdown => {
6888                if let Some(handle) = handler_status.lock().ok().and_then(|guard| *guard) {
6889                    let _ = handle.set_service_status(service_status(
6890                        ServiceState::StopPending,
6891                        ServiceControlAccept::empty(),
6892                        1,
6893                        Duration::from_secs(300),
6894                        0,
6895                    ));
6896                }
6897                let _ = stop_tx.send(true);
6898                ServiceControlHandlerResult::NoError
6899            }
6900            _ => ServiceControlHandlerResult::NotImplemented,
6901        };
6902        let handle = service_control_handler::register("", handler)
6903            .map_err(|error| format!("cannot register the service control handler: {error}"))?;
6904        *status
6905            .lock()
6906            .map_err(|_| "the service status handle is poisoned".to_string())? = Some(handle);
6907        handle
6908            .set_service_status(service_status(
6909                ServiceState::Running,
6910                ServiceControlAccept::STOP | ServiceControlAccept::SHUTDOWN,
6911                0,
6912                Duration::default(),
6913                0,
6914            ))
6915            .map_err(|error| format!("cannot report SERVICE_RUNNING: {error}"))?;
6916
6917        let exit = run(ServiceShutdown(stop_rx));
6918        handle
6919            .set_service_status(service_status(
6920                ServiceState::Stopped,
6921                ServiceControlAccept::empty(),
6922                0,
6923                Duration::default(),
6924                u32::from(exit),
6925            ))
6926            .map_err(|error| format!("cannot report SERVICE_STOPPED: {error}"))?;
6927        Ok(exit)
6928    }
6929
6930    fn service_status(
6931        state: ServiceState,
6932        accepted: ServiceControlAccept,
6933        checkpoint: u32,
6934        wait_hint: Duration,
6935        exit: u32,
6936    ) -> ServiceStatus {
6937        ServiceStatus {
6938            service_type: ServiceType::OWN_PROCESS,
6939            current_state: state,
6940            controls_accepted: accepted,
6941            exit_code: if exit == 0 {
6942                ServiceExitCode::Win32(0)
6943            } else {
6944                ServiceExitCode::ServiceSpecific(exit)
6945            },
6946            checkpoint,
6947            wait_hint,
6948            process_id: None,
6949        }
6950    }
6951
6952    fn host_error(operation: &'static str, detail: &str) -> ServiceError {
6953        ServiceError::Control {
6954            operation,
6955            name: SERVICE_NAME.to_string(),
6956            manager: "the Windows Service Control Manager",
6957            detail: detail.to_string(),
6958        }
6959    }
6960}
6961
6962#[cfg(windows)]
6963mod sys {
6964    //! Two managers, because Windows has two answers.
6965    //!
6966    //! `--start-at boot` is a service in the Service Control Manager, which is
6967    //! the only Windows facility that starts something before anybody logs in.
6968    //! `--start-at login` is a Task Scheduler task with a logon trigger, which
6969    //! is the only Windows facility that starts something when somebody does.
6970    //! There is no single mechanism that does both: service trigger-start has no
6971    //! logon trigger, and a scheduled task cannot run before a session exists.
6972
6973    use std::ffi::{OsStr, OsString};
6974    use std::time::{Duration, Instant};
6975
6976    use runner_manager_domain::model::StartMode;
6977    use windows_service::service::{
6978        ServiceAccess, ServiceAction, ServiceActionType, ServiceErrorControl,
6979        ServiceFailureActions, ServiceFailureResetPeriod, ServiceInfo, ServiceStartType,
6980        ServiceState, ServiceType,
6981    };
6982    use windows_service::service_manager::{ServiceManager, ServiceManagerAccess};
6983
6984    use super::{
6985        DefinitionKind, InstallPlan, Registration, ServiceControl, ServiceDefinition, ServiceError,
6986        ServiceIdentity, TaskPrincipal, run, windows_service_spec, xml_value,
6987    };
6988
6989    /// `ERROR_SERVICE_DOES_NOT_EXIST`. Not an error here: it is the answer to
6990    /// "is this registered".
6991    const SERVICE_DOES_NOT_EXIST: i32 = 1060;
6992    /// `ERROR_SERVICE_MARKED_FOR_DELETE`. `DeleteService` is asynchronous: the
6993    /// registration remains in this state until every open service handle is
6994    /// closed, and callers must not mistake that short window for a leak.
6995    const SERVICE_MARKED_FOR_DELETE: i32 = 1072;
6996    /// `ERROR_ACCESS_DENIED`.
6997    const ACCESS_DENIED: i32 = 5;
6998    const DELETE_TIMEOUT: Duration = Duration::from_secs(30);
6999    const DELETE_POLL_INTERVAL: Duration = Duration::from_millis(200);
7000
7001    const ELEVATION_REMEDY: &str = "Run the command from an elevated prompt: right-click Windows Terminal or PowerShell and \
7002         choose \"Run as administrator\".";
7003
7004    pub(super) fn control(mode: StartMode) -> Result<Box<dyn ServiceControl>, ServiceError> {
7005        Ok(match mode {
7006            StartMode::Boot => Box::new(ScmControl),
7007            StartMode::Login => Box::new(TaskControl),
7008        })
7009    }
7010
7011    // -- the Service Control Manager -----------------------------------------
7012
7013    #[derive(Debug)]
7014    struct ScmControl;
7015
7016    /// Turns a `windows-service` failure into this module's error, keeping the
7017    /// "needs elevation" case separate: an operator told "access is denied" and
7018    /// an operator told "run this elevated" are not equally well served.
7019    fn scm_error(
7020        operation: &'static str,
7021        name: &str,
7022        error: &windows_service::Error,
7023    ) -> ServiceError {
7024        let raw = match error {
7025            windows_service::Error::Winapi(io) => io.raw_os_error(),
7026            _ => None,
7027        };
7028        let detail = match error {
7029            windows_service::Error::Winapi(io) => io.to_string(),
7030            other => other.to_string(),
7031        };
7032        if raw == Some(ACCESS_DENIED) {
7033            return ServiceError::NeedsElevation {
7034                operation,
7035                name: name.to_string(),
7036                detail,
7037                remedy: ELEVATION_REMEDY,
7038            };
7039        }
7040        ServiceError::Control {
7041            operation,
7042            name: name.to_string(),
7043            manager: "the Windows Service Control Manager",
7044            detail,
7045        }
7046    }
7047
7048    fn open_manager(
7049        access: ServiceManagerAccess,
7050        operation: &'static str,
7051        name: &str,
7052    ) -> Result<ServiceManager, ServiceError> {
7053        ServiceManager::local_computer(None::<&OsStr>, access)
7054            .map_err(|error| scm_error(operation, name, &error))
7055    }
7056
7057    impl ServiceControl for ScmControl {
7058        fn manager(&self) -> DefinitionKind {
7059            DefinitionKind::WindowsService
7060        }
7061
7062        fn install(&self, plan: &InstallPlan) -> Result<ServiceDefinition, ServiceError> {
7063            let spec = windows_service_spec(plan);
7064            let manager = open_manager(
7065                ServiceManagerAccess::CONNECT | ServiceManagerAccess::CREATE_SERVICE,
7066                "install",
7067                &spec.name,
7068            )?;
7069            let info = ServiceInfo {
7070                name: OsString::from(&spec.name),
7071                display_name: OsString::from(&spec.display_name),
7072                // Never INTERACTIVE_PROCESS: see `windows_service_descriptor`.
7073                service_type: ServiceType::OWN_PROCESS,
7074                start_type: if spec.automatic_start {
7075                    ServiceStartType::AutoStart
7076                } else {
7077                    ServiceStartType::OnDemand
7078                },
7079                error_control: ServiceErrorControl::Normal,
7080                executable_path: plan.binary().to_path_buf(),
7081                launch_arguments: plan.arguments().to_vec(),
7082                dependencies: Vec::new(),
7083                // `None` is LocalSystem, which is the account `d2`'s DACL admits.
7084                account_name: spec.account.as_ref().map(OsString::from),
7085                // No password is ever taken, asked for, or stored. An account
7086                // that needed one would be an account this installer refuses.
7087                account_password: None,
7088            };
7089            let service = manager
7090                .create_service(
7091                    &info,
7092                    ServiceAccess::CHANGE_CONFIG
7093                        | ServiceAccess::QUERY_CONFIG
7094                        | ServiceAccess::QUERY_STATUS
7095                        | ServiceAccess::START
7096                        | ServiceAccess::STOP
7097                        | ServiceAccess::DELETE,
7098                )
7099                .map_err(|error| scm_error("install", &spec.name, &error))?;
7100            service
7101                .set_description(&spec.description)
7102                .map_err(|error| scm_error("describe", &spec.name, &error))?;
7103            service
7104                .update_failure_actions(ServiceFailureActions {
7105                    reset_period: ServiceFailureResetPeriod::After(spec.restart.reset_after()),
7106                    reboot_msg: None,
7107                    command: None,
7108                    // Three identical restart actions rather than one: the
7109                    // Service Control Manager applies the first action to the
7110                    // first failure, the second to the second, and the last to
7111                    // every failure after that. One action would leave the
7112                    // second and later failures unhandled, which is a service
7113                    // that comes back once and then stays down.
7114                    actions: Some(vec![
7115                        ServiceAction {
7116                            action_type: ServiceActionType::Restart,
7117                            delay: spec.restart.delay(),
7118                        };
7119                        3
7120                    ]),
7121                })
7122                .map_err(|error| scm_error("set the restart policy of", &spec.name, &error))?;
7123            // Without this the failure actions apply only to a crash, and a
7124            // daemon that exits non-zero is not a crash.
7125            service
7126                .set_failure_actions_on_non_crash_failures(true)
7127                .map_err(|error| scm_error("set the restart policy of", &spec.name, &error))?;
7128            Ok(ServiceDefinition::windows_service(plan))
7129        }
7130
7131        fn uninstall(&self, identity: &ServiceIdentity) -> Result<bool, ServiceError> {
7132            let manager =
7133                open_manager(ServiceManagerAccess::CONNECT, "uninstall", identity.name())?;
7134            let service = match manager.open_service(
7135                identity.name(),
7136                ServiceAccess::QUERY_STATUS | ServiceAccess::STOP | ServiceAccess::DELETE,
7137            ) {
7138                Ok(service) => Some(service),
7139                Err(error) if is_missing(&error) => return Ok(false),
7140                // A previous caller has already asked SCM to delete this
7141                // registration. That is the desired state, not an uninstall
7142                // failure. Wait for SCM to release the name so a following
7143                // `service install` can use it immediately.
7144                Err(error) if is_marked_for_delete(&error) => None,
7145                Err(error) => return Err(scm_error("uninstall", identity.name(), &error)),
7146            };
7147            if let Some(service) = service {
7148                // A running service can be deleted, but it lingers until it stops.
7149                // Stopping first is what makes `uninstall` followed by `install`
7150                // work in one sitting, which is what the start-mode switch needs.
7151                if let Ok(status) = service.query_status()
7152                    && status.current_state != ServiceState::Stopped
7153                {
7154                    let _ = service.stop();
7155                }
7156                match service.delete() {
7157                    Ok(()) => {}
7158                    // DeleteService is idempotent from the operator's point of
7159                    // view. Another process can mark the service between our
7160                    // OpenService and DeleteService calls.
7161                    Err(error) if is_marked_for_delete(&error) => {}
7162                    Err(error) => {
7163                        return Err(scm_error("uninstall", identity.name(), &error));
7164                    }
7165                }
7166                drop(service);
7167            }
7168
7169            // `DeleteService` marks a registration for deletion and returns;
7170            // SCM removes it only after the last service handle closes. Drop
7171            // ours before polling, then wait for the observable postcondition
7172            // promised by `uninstall`: an immediate status check must not see
7173            // a registration that is merely on its way out. This also keeps a
7174            // stop/uninstall/install sequence deterministic on busy hosts.
7175            let absent = wait_until_scm_absent(DELETE_TIMEOUT, DELETE_POLL_INTERVAL, || {
7176                match manager.open_service(identity.name(), ServiceAccess::QUERY_STATUS) {
7177                    Ok(service) => {
7178                        drop(service);
7179                        Ok(false)
7180                    }
7181                    Err(error) if is_missing(&error) => Ok(true),
7182                    Err(error) if is_marked_for_delete(&error) => Ok(false),
7183                    Err(error) => Err(scm_error("verify uninstall of", identity.name(), &error)),
7184                }
7185            })?;
7186            if !absent {
7187                return Err(ServiceError::Control {
7188                    operation: "verify uninstall of",
7189                    name: identity.name().to_string(),
7190                    manager: "the Windows Service Control Manager",
7191                    detail: format!(
7192                        "the registration was still visible {} seconds after DeleteService; \
7193                         retry `service uninstall` from an elevated prompt",
7194                        DELETE_TIMEOUT.as_secs()
7195                    ),
7196                });
7197            }
7198            Ok(true)
7199        }
7200
7201        fn query(&self, identity: &ServiceIdentity) -> Result<Option<Registration>, ServiceError> {
7202            let manager = open_manager(ServiceManagerAccess::CONNECT, "inspect", identity.name())?;
7203            let service = match manager.open_service(
7204                identity.name(),
7205                ServiceAccess::QUERY_CONFIG | ServiceAccess::QUERY_STATUS,
7206            ) {
7207                Ok(service) => service,
7208                Err(error) if is_missing(&error) => return Ok(None),
7209                Err(error) => return Err(scm_error("inspect", identity.name(), &error)),
7210            };
7211            let config = service
7212                .query_config()
7213                .map_err(|error| scm_error("inspect", identity.name(), &error))?;
7214            let status = service
7215                .query_status()
7216                .map_err(|error| scm_error("inspect", identity.name(), &error))?;
7217            let restart_delay = service.get_failure_actions().ok().and_then(|actions| {
7218                actions
7219                    .actions
7220                    .and_then(|actions| actions.into_iter().next())
7221                    .filter(|action| action.action_type == ServiceActionType::Restart)
7222                    .map(|action| action.delay)
7223            });
7224            Ok(Some(Registration {
7225                manager: DefinitionKind::WindowsService,
7226                start_mode: StartMode::Boot,
7227                // `lpBinaryPathName` holds the executable *and* its arguments
7228                // as one string, which is why this module carries its own
7229                // parser rather than trusting `executable_path`'s name.
7230                command_line: config.executable_path.to_string_lossy().into_owned(),
7231                account: config
7232                    .account_name
7233                    .map(|account| account.to_string_lossy().into_owned()),
7234                running: status.current_state == ServiceState::Running,
7235                starts_automatically: config.start_type == ServiceStartType::AutoStart,
7236                restart_delay,
7237            }))
7238        }
7239
7240        fn start(&self, identity: &ServiceIdentity) -> Result<(), ServiceError> {
7241            let manager = open_manager(ServiceManagerAccess::CONNECT, "start", identity.name())?;
7242            let service = manager
7243                .open_service(identity.name(), ServiceAccess::START)
7244                .map_err(|error| scm_error("start", identity.name(), &error))?;
7245            service
7246                .start::<&OsStr>(&[])
7247                .map_err(|error| scm_error("start", identity.name(), &error))
7248        }
7249
7250        fn stop(&self, identity: &ServiceIdentity) -> Result<bool, ServiceError> {
7251            let manager = open_manager(ServiceManagerAccess::CONNECT, "stop", identity.name())?;
7252            let service = manager
7253                .open_service(
7254                    identity.name(),
7255                    ServiceAccess::STOP | ServiceAccess::QUERY_STATUS,
7256                )
7257                .map_err(|error| scm_error("stop", identity.name(), &error))?;
7258            let status = service
7259                .query_status()
7260                .map_err(|error| scm_error("stop", identity.name(), &error))?;
7261            if status.current_state == ServiceState::Stopped {
7262                return Ok(false);
7263            }
7264            service
7265                .stop()
7266                .map_err(|error| scm_error("stop", identity.name(), &error))?;
7267            Ok(true)
7268        }
7269    }
7270
7271    fn is_missing(error: &windows_service::Error) -> bool {
7272        matches!(error, windows_service::Error::Winapi(io)
7273            if io.raw_os_error() == Some(SERVICE_DOES_NOT_EXIST))
7274    }
7275
7276    pub(super) fn is_marked_for_delete(error: &windows_service::Error) -> bool {
7277        matches!(error, windows_service::Error::Winapi(io)
7278            if io.raw_os_error() == Some(SERVICE_MARKED_FOR_DELETE))
7279    }
7280
7281    pub(super) fn wait_until_scm_absent(
7282        timeout: Duration,
7283        poll_interval: Duration,
7284        mut probe_absent: impl FnMut() -> Result<bool, ServiceError>,
7285    ) -> Result<bool, ServiceError> {
7286        let deadline = Instant::now() + timeout;
7287        loop {
7288            if probe_absent()? {
7289                return Ok(true);
7290            }
7291            if Instant::now() >= deadline {
7292                return Ok(false);
7293            }
7294            std::thread::sleep(poll_interval);
7295        }
7296    }
7297
7298    // -- Task Scheduler ------------------------------------------------------
7299
7300    #[derive(Debug)]
7301    struct TaskControl;
7302
7303    fn task_error(operation: &'static str, name: &str, detail: String) -> ServiceError {
7304        if detail.to_ascii_lowercase().contains("access is denied") {
7305            return ServiceError::NeedsElevation {
7306                operation,
7307                name: name.to_string(),
7308                detail,
7309                remedy: ELEVATION_REMEDY,
7310            };
7311        }
7312        ServiceError::Control {
7313            operation,
7314            name: name.to_string(),
7315            manager: "Windows Task Scheduler",
7316            detail,
7317        }
7318    }
7319
7320    fn schtasks(
7321        operation: &'static str,
7322        name: &str,
7323        arguments: &[&OsStr],
7324    ) -> Result<(bool, String), ServiceError> {
7325        match run("schtasks.exe", arguments) {
7326            Ok((ok, stdout, stderr)) => Ok((ok, if ok { stdout } else { stderr })),
7327            Err(error) => Err(task_error(operation, name, error.to_string())),
7328        }
7329    }
7330
7331    /// Task Scheduler reads `/XML` files as UTF-16, so the document is written
7332    /// as UTF-16 little-endian with a byte-order mark rather than as UTF-8.
7333    fn write_utf16(path: &std::path::Path, text: &str) -> std::io::Result<()> {
7334        let mut bytes = vec![0xFF, 0xFE];
7335        for unit in text.encode_utf16() {
7336            bytes.extend_from_slice(&unit.to_le_bytes());
7337        }
7338        std::fs::write(path, bytes)
7339    }
7340
7341    impl ServiceControl for TaskControl {
7342        fn manager(&self) -> DefinitionKind {
7343            DefinitionKind::WindowsScheduledTask
7344        }
7345
7346        fn install(&self, plan: &InstallPlan) -> Result<ServiceDefinition, ServiceError> {
7347            let name = plan.identity().name().to_string();
7348            let principal = TaskPrincipal::current()?;
7349            let definition = ServiceDefinition::windows_scheduled_task(plan, &principal);
7350            let directory = tempfile::tempdir()
7351                .map_err(|error| task_error("install", &name, error.to_string()))?;
7352            let document = directory.path().join("task.xml");
7353            write_utf16(&document, definition.text())
7354                .map_err(|error| task_error("install", &name, error.to_string()))?;
7355            let (ok, message) = schtasks(
7356                "install",
7357                &name,
7358                &[
7359                    OsStr::new("/Create"),
7360                    OsStr::new("/TN"),
7361                    OsStr::new(&name),
7362                    OsStr::new("/XML"),
7363                    document.as_os_str(),
7364                ],
7365            )?;
7366            if !ok {
7367                return Err(task_error("install", &name, message));
7368            }
7369            Ok(definition)
7370        }
7371
7372        fn uninstall(&self, identity: &ServiceIdentity) -> Result<bool, ServiceError> {
7373            if self.query(identity)?.is_none() {
7374                return Ok(false);
7375            }
7376            let name = identity.name().to_string();
7377            let (ok, message) = schtasks(
7378                "uninstall",
7379                &name,
7380                &[
7381                    OsStr::new("/Delete"),
7382                    OsStr::new("/TN"),
7383                    OsStr::new(&name),
7384                    OsStr::new("/F"),
7385                ],
7386            )?;
7387            if !ok {
7388                return Err(task_error("uninstall", &name, message));
7389            }
7390            Ok(true)
7391        }
7392
7393        fn query(&self, identity: &ServiceIdentity) -> Result<Option<Registration>, ServiceError> {
7394            let name = identity.name().to_string();
7395            let (ok, document) = schtasks(
7396                "inspect",
7397                &name,
7398                &[
7399                    OsStr::new("/Query"),
7400                    OsStr::new("/TN"),
7401                    OsStr::new(&name),
7402                    OsStr::new("/XML"),
7403                    OsStr::new("ONE"),
7404                ],
7405            )?;
7406            if !ok {
7407                // `schtasks` reports a missing task and a broken Task Scheduler
7408                // the same way, with a non-zero exit; there is no distinct code.
7409                // Treating it as absence is the safe reading: the caller either
7410                // installs, which will then fail loudly, or reports "not
7411                // installed", which is what an operator with no task sees.
7412                return Ok(None);
7413            }
7414            let command = xml_value(&document, "Command").unwrap_or_default();
7415            let arguments = xml_value(&document, "Arguments").unwrap_or_default();
7416            let command_line = if arguments.is_empty() {
7417                super::quote_argument(&command)
7418            } else {
7419                format!("{} {arguments}", super::quote_argument(&command))
7420            };
7421            Ok(Some(Registration {
7422                manager: DefinitionKind::WindowsScheduledTask,
7423                start_mode: StartMode::Login,
7424                command_line,
7425                account: xml_value(&document, "UserId"),
7426                running: task_is_running(&name),
7427                starts_automatically: super::windows_login_task_starts_automatically(&document),
7428                restart_delay: xml_value(&document, "Interval")
7429                    .as_deref()
7430                    .and_then(parse_iso8601),
7431            }))
7432        }
7433
7434        fn start(&self, identity: &ServiceIdentity) -> Result<(), ServiceError> {
7435            let name = identity.name().to_string();
7436            let (ok, message) = schtasks(
7437                "start",
7438                &name,
7439                &[OsStr::new("/Run"), OsStr::new("/TN"), OsStr::new(&name)],
7440            )?;
7441            if ok {
7442                Ok(())
7443            } else {
7444                Err(task_error("start", &name, message))
7445            }
7446        }
7447
7448        fn stop(&self, identity: &ServiceIdentity) -> Result<bool, ServiceError> {
7449            let running = self
7450                .query(identity)?
7451                .is_some_and(|registration| registration.running);
7452            if !running {
7453                return Ok(false);
7454            }
7455            let name = identity.name().to_string();
7456            let (ok, message) = schtasks(
7457                "stop",
7458                &name,
7459                &[OsStr::new("/End"), OsStr::new("/TN"), OsStr::new(&name)],
7460            )?;
7461            if ok {
7462                Ok(true)
7463            } else {
7464                Err(task_error("stop", &name, message))
7465            }
7466        }
7467    }
7468
7469    /// Whether Task Scheduler reports the task as running.
7470    ///
7471    /// **This is the one value in this module read from localised output.**
7472    /// `schtasks /Query /FO CSV` prints a `Status` column in the machine's
7473    /// display language, and Task Scheduler exposes no locale-independent
7474    /// equivalent short of COM. On a non-English Windows this therefore reports
7475    /// `false` for a task that is in fact running.
7476    ///
7477    /// That is stated rather than hidden because of what does *not* depend on
7478    /// it: Journey 5's liveness question is answered by the last successful
7479    /// GitHub contact, which is a timestamp this product writes itself and no
7480    /// locale can move. Nothing in the Definition of Done rests on this
7481    /// boolean.
7482    fn task_is_running(name: &str) -> bool {
7483        let Ok((true, stdout, _)) = run(
7484            "schtasks.exe",
7485            &[
7486                OsStr::new("/Query"),
7487                OsStr::new("/TN"),
7488                OsStr::new(name),
7489                OsStr::new("/FO"),
7490                OsStr::new("CSV"),
7491                OsStr::new("/NH"),
7492            ],
7493        ) else {
7494            return false;
7495        };
7496        stdout
7497            .lines()
7498            .filter_map(|line| line.rsplit(',').next())
7499            .any(|status| {
7500                status
7501                    .trim()
7502                    .trim_matches('"')
7503                    .eq_ignore_ascii_case("running")
7504            })
7505    }
7506
7507    /// Parses the `PT<n>M` shape this module writes, and the `PT<n>S` shape
7508    /// Task Scheduler would accept if it took seconds. Anything richer -- days,
7509    /// hours, a combination -- is `None` rather than a guess.
7510    fn parse_iso8601(value: &str) -> Option<Duration> {
7511        let rest = value.strip_prefix("PT")?;
7512        if let Some(minutes) = rest.strip_suffix('M') {
7513            return minutes
7514                .parse::<u64>()
7515                .ok()
7516                .map(|minutes| Duration::from_secs(minutes * 60));
7517        }
7518        rest.strip_suffix('S')?
7519            .parse::<u64>()
7520            .ok()
7521            .map(Duration::from_secs)
7522    }
7523}
7524
7525// ---------------------------------------------------------------------------
7526// Unix: shared plumbing
7527// ---------------------------------------------------------------------------
7528
7529/// Writes a definition file, creating its directory, and says when the refusal
7530/// was a permissions one.
7531///
7532/// Both Unix backends write a file into a directory only `root` can write —
7533/// `/Library/LaunchDaemons` and `/etc/systemd/system` — and an operator who ran
7534/// `service install` without `sudo` deserves to be told that rather than handed
7535/// an `EACCES`.
7536#[cfg(unix)]
7537fn write_definition(
7538    operation: &'static str,
7539    name: &str,
7540    path: &Path,
7541    text: &str,
7542    remedy: &'static str,
7543) -> Result<(), ServiceError> {
7544    if let Some(parent) = path.parent()
7545        && let Err(error) = std::fs::create_dir_all(parent)
7546        && error.kind() != std::io::ErrorKind::AlreadyExists
7547    {
7548        return Err(definition_error(operation, name, error, remedy, parent));
7549    }
7550    std::fs::write(path, text)
7551        .map_err(|error| definition_error(operation, name, error, remedy, path))
7552}
7553
7554/// Replaces an installed definition with [`DefinitionDrift::rendered`].
7555///
7556/// Through a temporary file in the same directory and a rename, so a failure
7557/// at any point leaves the old definition in place rather than a truncated
7558/// one. The mode is set to `0644` before the rename, as `service install`
7559/// leaves it: launchd refuses a plist that others can write.
7560#[cfg(all(unix, any(target_os = "macos", test)))]
7561fn replace_definition(name: &str, drift: &DefinitionDrift) -> Result<(), ServiceError> {
7562    use std::io::Write as _;
7563
7564    let path = drift.path.as_path();
7565    let directory = path.parent().unwrap_or_else(|| Path::new("/"));
7566    let failed = |error: std::io::Error, at: &Path| {
7567        definition_error("rewrite", name, error, SUDO_REMEDY, at)
7568    };
7569    let mut staged =
7570        tempfile::NamedTempFile::new_in(directory).map_err(|error| failed(error, directory))?;
7571    staged
7572        .write_all(drift.rendered.as_bytes())
7573        .and_then(|()| staged.as_file().sync_all())
7574        .map_err(|error| failed(error, staged.path()))?;
7575    std::fs::set_permissions(
7576        staged.path(),
7577        <std::fs::Permissions as std::os::unix::fs::PermissionsExt>::from_mode(0o644),
7578    )
7579    .map_err(|error| failed(error, path))?;
7580    staged
7581        .persist(path)
7582        .map(drop)
7583        .map_err(|error| failed(error.error, path))
7584}
7585
7586#[cfg(unix)]
7587fn definition_error(
7588    operation: &'static str,
7589    name: &str,
7590    error: std::io::Error,
7591    remedy: &'static str,
7592    path: &Path,
7593) -> ServiceError {
7594    let detail = format!("{}: {error}", path.display());
7595    if error.kind() == std::io::ErrorKind::PermissionDenied {
7596        ServiceError::NeedsElevation {
7597            operation,
7598            name: name.to_string(),
7599            detail,
7600            remedy,
7601        }
7602    } else {
7603        ServiceError::Control {
7604            operation,
7605            name: name.to_string(),
7606            manager: "the local service manager",
7607            detail,
7608        }
7609    }
7610}
7611
7612/// `sudo` is the remedy on both Unixes, and the message says which command.
7613#[cfg(unix)]
7614const SUDO_REMEDY: &str = "A boot-start registration is machine-wide, so it needs root: run the same command with \
7615     `sudo`. `service install --start-at login` needs no elevation at all, at the cost of the \
7616     agent not running until you sign in.";
7617
7618// ---------------------------------------------------------------------------
7619// macOS
7620// ---------------------------------------------------------------------------
7621
7622#[cfg(target_os = "macos")]
7623mod sys {
7624    //! One manager, two domains. `--start-at boot` is a LaunchDaemon in
7625    //! `/Library/LaunchDaemons`, loaded into launchd's `system` domain;
7626    //! `--start-at login` is a LaunchAgent in the operator's own
7627    //! `~/Library/LaunchAgents`, loaded into `gui/<uid>`.
7628    //!
7629    //! `launchctl print` is the only inspection command here whose output is
7630    //! parsed, and it is English-only on every macOS release — launchd has no
7631    //! localisation for it. The registration's *existence* is decided by the
7632    //! plist file rather than by that output, so a launchd that changed its
7633    //! wording would cost this module the running/not-running line and nothing
7634    //! else.
7635
7636    use std::ffi::OsStr;
7637    use std::path::PathBuf;
7638    use std::time::Duration;
7639
7640    use runner_manager_domain::model::StartMode;
7641
7642    use super::{
7643        DefinitionKind, InstallPlan, LAUNCH_AGENTS_SUBDIR, LAUNCH_DAEMONS_DIR, Registration,
7644        SUDO_REMEDY, ServiceControl, ServiceDefinition, ServiceError, ServiceIdentity,
7645        enable_launchd_registration, home_directory, plist_string_value, quote_argument, run,
7646        write_definition, xml_value,
7647    };
7648
7649    pub(super) fn control(mode: StartMode) -> Result<Box<dyn ServiceControl>, ServiceError> {
7650        Ok(Box::new(LaunchdControl { mode }))
7651    }
7652
7653    #[derive(Debug)]
7654    struct LaunchdControl {
7655        mode: StartMode,
7656    }
7657
7658    impl LaunchdControl {
7659        /// launchd's domain target for this start mode.
7660        fn domain(&self) -> String {
7661            match self.mode {
7662                StartMode::Boot => "system".to_string(),
7663                // SAFETY: `getuid` reads the calling process's real user id and
7664                // cannot fail; it takes no arguments and touches no memory.
7665                StartMode::Login => format!("gui/{}", unsafe { libc::getuid() }),
7666            }
7667        }
7668
7669        fn service_target(&self, identity: &ServiceIdentity) -> String {
7670            format!("{}/{}", self.domain(), identity.launchd_label())
7671        }
7672
7673        /// Where this domain's plist lives, when the domain has a home to put
7674        /// it in.
7675        fn plist_path(&self, identity: &ServiceIdentity) -> Option<PathBuf> {
7676            let file = format!("{}.plist", identity.launchd_label());
7677            match self.mode {
7678                StartMode::Boot => Some(PathBuf::from(LAUNCH_DAEMONS_DIR).join(file)),
7679                StartMode::Login => {
7680                    home_directory().map(|home| home.join(LAUNCH_AGENTS_SUBDIR).join(file))
7681                }
7682            }
7683        }
7684
7685        fn failed(&self, operation: &'static str, name: &str, detail: String) -> ServiceError {
7686            ServiceError::Control {
7687                operation,
7688                name: name.to_string(),
7689                manager: "launchd",
7690                detail,
7691            }
7692        }
7693
7694        fn launchctl(&self, arguments: &[&OsStr]) -> (bool, String) {
7695            match run("launchctl", arguments) {
7696                Ok((ok, stdout, stderr)) => (ok, if ok { stdout } else { stderr }),
7697                Err(error) => (false, error.to_string()),
7698            }
7699        }
7700    }
7701
7702    impl ServiceControl for LaunchdControl {
7703        fn manager(&self) -> DefinitionKind {
7704            DefinitionKind::LaunchdPlist
7705        }
7706
7707        fn install(&self, plan: &InstallPlan) -> Result<ServiceDefinition, ServiceError> {
7708            let name = plan.identity().name().to_string();
7709            let definition = ServiceDefinition::launchd(plan, home_directory().as_deref());
7710            let Some(path) = definition.install_path().map(std::path::Path::to_path_buf) else {
7711                return Err(self.failed(
7712                    "install",
7713                    &name,
7714                    "this account has no home directory, so there is nowhere to put a \
7715                     LaunchAgent. Use --start-at boot, which installs a LaunchDaemon under \
7716                     /Library/LaunchDaemons."
7717                        .to_string(),
7718                ));
7719            };
7720            write_definition("install", &name, &path, definition.text(), SUDO_REMEDY)?;
7721            let target = self.domain();
7722            let (ok, message) = self.launchctl(&[
7723                OsStr::new("bootstrap"),
7724                OsStr::new(&target),
7725                path.as_os_str(),
7726            ]);
7727            if !ok {
7728                // Leave nothing behind: a plist launchd refused is a file that
7729                // would make the next `service status` claim a registration
7730                // that does not exist.
7731                let _ = std::fs::remove_file(&path);
7732                if message.to_ascii_lowercase().contains("permission denied") {
7733                    return Err(ServiceError::NeedsElevation {
7734                        operation: "install",
7735                        name,
7736                        detail: message,
7737                        remedy: SUDO_REMEDY,
7738                    });
7739                }
7740                return Err(self.failed("install", &name, message));
7741            }
7742            // A previously disabled label stays disabled through a bootstrap,
7743            // which is a service that is installed and will never start.
7744            let service_target = self.service_target(plan.identity());
7745            enable_launchd_registration(
7746                |arguments| self.launchctl(arguments),
7747                &target,
7748                &service_target,
7749                &path,
7750                &name,
7751                SUDO_REMEDY,
7752            )?;
7753            Ok(definition)
7754        }
7755
7756        fn uninstall(&self, identity: &ServiceIdentity) -> Result<bool, ServiceError> {
7757            let Some(path) = self.plist_path(identity) else {
7758                return Ok(false);
7759            };
7760            if !path.exists() {
7761                return Ok(false);
7762            }
7763            let target = self.service_target(identity);
7764            let (ok, message) = self.launchctl(&[OsStr::new("bootout"), OsStr::new(&target)]);
7765            if !ok
7766                && !message.to_ascii_lowercase().contains("no such process")
7767                && !message.contains("113")
7768            {
7769                if message.to_ascii_lowercase().contains("permission denied") {
7770                    return Err(ServiceError::NeedsElevation {
7771                        operation: "uninstall",
7772                        name: identity.name().to_string(),
7773                        detail: message,
7774                        remedy: SUDO_REMEDY,
7775                    });
7776                }
7777                return Err(self.failed("uninstall", identity.name(), message));
7778            }
7779            // Only the definition. `05-infrastructure.md` item 5: nothing else
7780            // on this host is touched here or anywhere below it.
7781            std::fs::remove_file(&path).map_err(|error| {
7782                super::definition_error(
7783                    "uninstall",
7784                    identity.name(),
7785                    error,
7786                    SUDO_REMEDY,
7787                    path.as_path(),
7788                )
7789            })?;
7790            Ok(true)
7791        }
7792
7793        fn query(&self, identity: &ServiceIdentity) -> Result<Option<Registration>, ServiceError> {
7794            let Some(path) = self.plist_path(identity) else {
7795                return Ok(None);
7796            };
7797            let Ok(document) = std::fs::read_to_string(&path) else {
7798                return Ok(None);
7799            };
7800            let target = self.service_target(identity);
7801            let (loaded, printed) = self.launchctl(&[OsStr::new("print"), OsStr::new(&target)]);
7802            Ok(Some(Registration {
7803                manager: DefinitionKind::LaunchdPlist,
7804                start_mode: self.mode,
7805                command_line: program_arguments(&document),
7806                account: plist_string_value(&document, "UserName")
7807                    .or_else(|| Some("the invoking user".to_string())),
7808                running: loaded && printed.contains("state = running"),
7809                starts_automatically: document.contains("<key>RunAtLoad</key>")
7810                    && super::plist_bool_value(&document, "RunAtLoad") == Some(true),
7811                restart_delay: xml_value(
7812                    super::plist_value_after_key(&document, "ThrottleInterval").unwrap_or(""),
7813                    "integer",
7814                )
7815                .and_then(|value| value.parse::<u64>().ok())
7816                .map(Duration::from_secs),
7817            }))
7818        }
7819
7820        fn start(&self, identity: &ServiceIdentity) -> Result<(), ServiceError> {
7821            let target = self.service_target(identity);
7822            let (ok, message) = self.launchctl(&[
7823                OsStr::new("kickstart"),
7824                OsStr::new("-k"),
7825                OsStr::new(&target),
7826            ]);
7827            if ok {
7828                Ok(())
7829            } else {
7830                Err(self.failed("start", identity.name(), message))
7831            }
7832        }
7833
7834        fn stop(&self, identity: &ServiceIdentity) -> Result<bool, ServiceError> {
7835            let running = self
7836                .query(identity)?
7837                .is_some_and(|registration| registration.running);
7838            if !running {
7839                return Ok(false);
7840            }
7841            let target = self.service_target(identity);
7842            let (ok, message) = self.launchctl(&[
7843                OsStr::new("kill"),
7844                OsStr::new("SIGTERM"),
7845                OsStr::new(&target),
7846            ]);
7847            if ok {
7848                Ok(true)
7849            } else {
7850                Err(self.failed("stop", identity.name(), message))
7851            }
7852        }
7853    }
7854
7855    /// Boots the job out and bootstraps it again from `plist`, from a shell
7856    /// that outlives this process.
7857    ///
7858    /// `setsid` puts the shell in a session and process group of its own. On a
7859    /// bootout launchd sends SIGTERM to the job's whole process group, the
7860    /// process that asked for the bootout included, and spares a child in
7861    /// another session (measured on macOS 27). Without `setsid` the helper
7862    /// survived that only by luck: one run in four left the job unloaded. The
7863    /// work runs in a background subshell, so the outer shell exits at once and
7864    /// is reaped here rather than left behind. `bootout` blocks until the daemon
7865    /// has exited; the bootstrap is retried for a minute because launchd
7866    /// refuses it while the old job is still being torn down. When the job is
7867    /// still loaded after a failed bootout, every bootstrap fails and the
7868    /// running daemon carries on as it was.
7869    pub(super) fn reload_detached(
7870        identity: &ServiceIdentity,
7871        mode: StartMode,
7872        plist: &std::path::Path,
7873    ) -> Result<(), ServiceError> {
7874        use std::os::unix::process::CommandExt;
7875
7876        const SCRIPT: &str = r#"(
7877  sleep 1
7878  /bin/launchctl bootout "$1"
7879  tries=0
7880  until /bin/launchctl bootstrap "$2" "$3"; do
7881    tries=$((tries + 1))
7882    if [ "$tries" -ge 60 ]; then
7883      echo "runner-manager: launchd did not load $3 again; run: $4" >&2
7884      exit 1
7885    fi
7886    sleep 1
7887  done
7888) </dev/null &"#;
7889
7890        let control = LaunchdControl { mode };
7891        // Once the job is booted out nothing holds the single-instance lock,
7892        // so a reinstall in the same start mode is the remedy.
7893        let reinstall = match mode {
7894            StartMode::Boot => "sudo runner-manager service install --start-at boot",
7895            StartMode::Login => "runner-manager service install --start-at login",
7896        };
7897        let mut command = std::process::Command::new("/bin/sh");
7898        command
7899            .args(["-c", SCRIPT, "runner-manager-reload"])
7900            .arg(control.service_target(identity))
7901            .arg(control.domain())
7902            .arg(plist)
7903            .arg(reinstall)
7904            .stdin(std::process::Stdio::null());
7905        // SAFETY: one async-signal-safe system call between `fork` and `exec`,
7906        // touching no memory, allocator or lock.
7907        unsafe {
7908            command.pre_exec(|| {
7909                if libc::setsid() == -1 {
7910                    return Err(std::io::Error::last_os_error());
7911                }
7912                Ok(())
7913            });
7914        }
7915        let status = command.status().map_err(|error| {
7916            control.failed("reload", identity.name(), format!("/bin/sh: {error}"))
7917        })?;
7918        if status.success() {
7919            Ok(())
7920        } else {
7921            Err(control.failed(
7922                "reload",
7923                identity.name(),
7924                format!("the reload helper exited with {status}"),
7925            ))
7926        }
7927    }
7928
7929    /// Rebuilds the command line from a plist's `ProgramArguments` array.
7930    fn program_arguments(document: &str) -> String {
7931        let Some(rest) = super::plist_value_after_key(document, "ProgramArguments") else {
7932            return String::new();
7933        };
7934        let Some(end) = rest.find("</array>") else {
7935            return String::new();
7936        };
7937        let mut out = Vec::new();
7938        let mut cursor = &rest[..end];
7939        while let Some(open) = cursor.find("<string>") {
7940            let after = &cursor[open + "<string>".len()..];
7941            let Some(close) = after.find("</string>") else {
7942                break;
7943            };
7944            out.push(quote_argument(&super::xml_unescape(&after[..close])));
7945            cursor = &after[close..];
7946        }
7947        out.join(" ")
7948    }
7949}
7950
7951// ---------------------------------------------------------------------------
7952// Linux and the other Unixes
7953// ---------------------------------------------------------------------------
7954
7955#[cfg(all(unix, not(target_os = "macos")))]
7956mod sys {
7957    //! One manager, two domains. `--start-at boot` is a system unit in
7958    //! `/etc/systemd/system`, wanted by `multi-user.target`; `--start-at login`
7959    //! is a user unit in `~/.config/systemd/user`, wanted by `default.target`.
7960    //!
7961    //! Every inspection here goes through `systemctl is-active` and
7962    //! `systemctl is-enabled`, whose output is a fixed machine-readable word
7963    //! rather than a sentence, so nothing in this module depends on the
7964    //! machine's display language.
7965
7966    use std::ffi::OsStr;
7967    use std::path::PathBuf;
7968    use std::time::Duration;
7969
7970    use runner_manager_domain::model::StartMode;
7971
7972    use super::{
7973        DefinitionKind, InstallPlan, Registration, SUDO_REMEDY, SYSTEMD_SYSTEM_DIR,
7974        SYSTEMD_USER_SUBDIR, ServiceControl, ServiceDefinition, ServiceError, ServiceIdentity,
7975        home_directory, ini_directives, run, write_definition,
7976    };
7977
7978    pub(super) fn control(mode: StartMode) -> Result<Box<dyn ServiceControl>, ServiceError> {
7979        Ok(Box::new(SystemdControl { mode }))
7980    }
7981
7982    #[derive(Debug)]
7983    struct SystemdControl {
7984        mode: StartMode,
7985    }
7986
7987    impl SystemdControl {
7988        fn unit_path(&self, identity: &ServiceIdentity) -> Option<PathBuf> {
7989            let file = identity.systemd_unit();
7990            match self.mode {
7991                StartMode::Boot => Some(PathBuf::from(SYSTEMD_SYSTEM_DIR).join(file)),
7992                StartMode::Login => {
7993                    home_directory().map(|home| home.join(SYSTEMD_USER_SUBDIR).join(file))
7994                }
7995            }
7996        }
7997
7998        /// `systemctl`, with `--user` for the login domain.
7999        fn systemctl(&self, arguments: &[&str]) -> (bool, String) {
8000            let mut all: Vec<&OsStr> = Vec::with_capacity(arguments.len() + 1);
8001            if self.mode == StartMode::Login {
8002                all.push(OsStr::new("--user"));
8003            }
8004            all.extend(arguments.iter().map(OsStr::new));
8005            match run("systemctl", &all) {
8006                Ok((ok, stdout, stderr)) => (
8007                    ok,
8008                    if stdout.trim().is_empty() {
8009                        stderr
8010                    } else {
8011                        stdout
8012                    },
8013                ),
8014                Err(error) => (false, error.to_string()),
8015            }
8016        }
8017
8018        fn failed(&self, operation: &'static str, name: &str, detail: String) -> ServiceError {
8019            ServiceError::Control {
8020                operation,
8021                name: name.to_string(),
8022                manager: "systemd",
8023                detail,
8024            }
8025        }
8026    }
8027
8028    impl ServiceControl for SystemdControl {
8029        fn manager(&self) -> DefinitionKind {
8030            DefinitionKind::SystemdUnit
8031        }
8032
8033        fn install(&self, plan: &InstallPlan) -> Result<ServiceDefinition, ServiceError> {
8034            let name = plan.identity().name().to_string();
8035            let definition = ServiceDefinition::systemd(plan, home_directory().as_deref());
8036            let Some(path) = definition.install_path().map(std::path::Path::to_path_buf) else {
8037                return Err(self.failed(
8038                    "install",
8039                    &name,
8040                    "this account has no home directory, so there is nowhere to put a systemd \
8041                     user unit. Use --start-at boot, which installs a system unit under \
8042                     /etc/systemd/system."
8043                        .to_string(),
8044                ));
8045            };
8046            write_definition("install", &name, &path, definition.text(), SUDO_REMEDY)?;
8047            let unit = plan.identity().systemd_unit();
8048            let (reloaded, message) = self.systemctl(&["daemon-reload"]);
8049            if !reloaded {
8050                let _ = std::fs::remove_file(&path);
8051                return Err(self.failed("install", &name, message));
8052            }
8053            let (enabled, message) = self.systemctl(&["enable", &unit]);
8054            if !enabled {
8055                let _ = std::fs::remove_file(&path);
8056                let _ = self.systemctl(&["daemon-reload"]);
8057                return Err(self.failed("install", &name, message));
8058            }
8059            Ok(definition)
8060        }
8061
8062        fn uninstall(&self, identity: &ServiceIdentity) -> Result<bool, ServiceError> {
8063            let Some(path) = self.unit_path(identity) else {
8064                return Ok(false);
8065            };
8066            if !path.exists() {
8067                return Ok(false);
8068            }
8069            let unit = identity.systemd_unit();
8070            // `--now` stops it as well as disabling it. A failure here is not
8071            // fatal: the unit file is about to go, and refusing to remove it
8072            // because `systemctl` disliked something would leave the host with
8073            // a unit nothing manages.
8074            let _ = self.systemctl(&["disable", "--now", &unit]);
8075            std::fs::remove_file(&path).map_err(|error| {
8076                super::definition_error(
8077                    "uninstall",
8078                    identity.name(),
8079                    error,
8080                    SUDO_REMEDY,
8081                    path.as_path(),
8082                )
8083            })?;
8084            let _ = self.systemctl(&["daemon-reload"]);
8085            Ok(true)
8086        }
8087
8088        fn query(&self, identity: &ServiceIdentity) -> Result<Option<Registration>, ServiceError> {
8089            let Some(path) = self.unit_path(identity) else {
8090                return Ok(None);
8091            };
8092            let Ok(document) = std::fs::read_to_string(&path) else {
8093                return Ok(None);
8094            };
8095            let unit = identity.systemd_unit();
8096            let directives = ini_directives(&document, "Service");
8097            let (_, active) = self.systemctl(&["is-active", &unit]);
8098            let (_, enabled) = self.systemctl(&["is-enabled", &unit]);
8099            Ok(Some(Registration {
8100                manager: DefinitionKind::SystemdUnit,
8101                start_mode: self.mode,
8102                command_line: directives.get("ExecStart").cloned().unwrap_or_default(),
8103                account: directives.get("User").cloned().or_else(|| {
8104                    Some(match self.mode {
8105                        StartMode::Boot => "root".to_string(),
8106                        StartMode::Login => "the invoking user".to_string(),
8107                    })
8108                }),
8109                running: active.trim() == "active",
8110                starts_automatically: enabled.trim() == "enabled",
8111                restart_delay: directives
8112                    .get("RestartSec")
8113                    .and_then(|value| value.trim().trim_end_matches('s').parse::<u64>().ok())
8114                    .map(Duration::from_secs),
8115            }))
8116        }
8117
8118        fn start(&self, identity: &ServiceIdentity) -> Result<(), ServiceError> {
8119            let unit = identity.systemd_unit();
8120            let (ok, message) = self.systemctl(&["start", &unit]);
8121            if ok {
8122                Ok(())
8123            } else {
8124                Err(self.failed("start", identity.name(), message))
8125            }
8126        }
8127
8128        fn stop(&self, identity: &ServiceIdentity) -> Result<bool, ServiceError> {
8129            let running = self
8130                .query(identity)?
8131                .is_some_and(|registration| registration.running);
8132            if !running {
8133                return Ok(false);
8134            }
8135            let unit = identity.systemd_unit();
8136            let (ok, message) = self.systemctl(&["stop", &unit]);
8137            if ok {
8138                Ok(true)
8139            } else {
8140                Err(self.failed("stop", identity.name(), message))
8141            }
8142        }
8143    }
8144}
8145
8146/// What the macOS tests that load real launch agents share.
8147#[cfg(all(test, target_os = "macos"))]
8148pub(crate) mod launchd_fixture {
8149    use std::path::{Path, PathBuf};
8150
8151    /// Opts a macOS host into the tests that load launch agents into the
8152    /// caller's GUI session. CI's macOS leg sets it, and
8153    /// `privileged_tests_are_wired_into_ci.rs` asserts that it still does.
8154    const ACCEPTANCE: &str = "RUNNER_MANAGER_LAUNCHD_ACCEPTANCE";
8155
8156    /// Whether this run may load launch agents. Says so when it may not, since
8157    /// the test then passes having measured nothing.
8158    pub(crate) fn enabled() -> bool {
8159        let enabled = std::env::var_os(ACCEPTANCE).is_some_and(|value| value == "1");
8160        if !enabled {
8161            eprintln!("skipped: set {ACCEPTANCE}=1 to load test launch agents");
8162        }
8163        enabled
8164    }
8165
8166    /// A copy of the running test binary in `directory`, for a launchd job to
8167    /// run as its program.
8168    ///
8169    /// A copy, because a launchd job whose program lives on an external volume
8170    /// waits on a privacy approval nobody gives before `main` runs (measured
8171    /// with a target directory under `/Volumes`), and the temporary directory
8172    /// is on the boot volume.
8173    pub(crate) fn job_binary(directory: &Path) -> PathBuf {
8174        let binary = directory.join("launch-agent-job");
8175        std::fs::copy(std::env::current_exe().expect("test binary"), &binary)
8176            .expect("test binary copied");
8177        binary
8178    }
8179
8180    /// A launch agent loaded into the caller's GUI domain, booted out on drop.
8181    pub(crate) struct LoadedAgent(String);
8182
8183    impl LoadedAgent {
8184        /// Bootstraps `plist`, whose `Label` is `label`, and fails the test when
8185        /// launchd refuses it.
8186        pub(crate) fn bootstrap(label: &str, plist: &Path) -> Self {
8187            // SAFETY: `getuid` reads the calling process's real user id and
8188            // cannot fail.
8189            let domain = format!("gui/{}", unsafe { libc::getuid() });
8190            // Armed before the bootstrap, so a half-loaded job is removed too.
8191            let loaded = Self(format!("{domain}/{label}"));
8192            let bootstrap = std::process::Command::new("launchctl")
8193                .arg("bootstrap")
8194                .arg(&domain)
8195                .arg(plist)
8196                .output()
8197                .expect("launchctl runs");
8198            assert!(
8199                bootstrap.status.success(),
8200                "launchctl bootstrap {domain} refused {label}, so nothing was measured: {}",
8201                String::from_utf8_lossy(&bootstrap.stderr)
8202            );
8203            loaded
8204        }
8205
8206        /// `gui/<uid>/<label>`.
8207        pub(crate) fn target(&self) -> &str {
8208            &self.0
8209        }
8210    }
8211
8212    impl Drop for LoadedAgent {
8213        fn drop(&mut self) {
8214            let _ = std::process::Command::new("launchctl")
8215                .args(["bootout", &self.0])
8216                .output();
8217        }
8218    }
8219}
8220
8221#[cfg(test)]
8222mod tests {
8223    use super::*;
8224
8225    use std::collections::BTreeMap;
8226
8227    #[test]
8228    fn task_scheduler_omitting_default_enabled_still_means_automatic() {
8229        assert!(windows_login_task_starts_automatically(
8230            "<Task><Triggers><LogonTrigger></LogonTrigger></Triggers></Task>"
8231        ));
8232        assert!(windows_login_task_starts_automatically(
8233            "<Task><LogonTrigger><Enabled>true</Enabled></LogonTrigger></Task>"
8234        ));
8235        assert!(!windows_login_task_starts_automatically(
8236            "<Task><LogonTrigger><Enabled>false</Enabled></LogonTrigger></Task>"
8237        ));
8238        assert!(!windows_login_task_starts_automatically(
8239            "<Task><BootTrigger></BootTrigger></Task>"
8240        ));
8241    }
8242
8243    // -----------------------------------------------------------------------
8244    // Fixtures
8245    // -----------------------------------------------------------------------
8246
8247    /// A plan against paths that exist on no platform, so that every renderer
8248    /// can be asserted on every leg of the CI matrix.
8249    fn linux_plan(mode: StartMode) -> InstallPlan {
8250        InstallPlan::unchecked(
8251            ServiceIdentity::product(),
8252            mode,
8253            "/opt/runner-manager/bin/runner-manager",
8254            ServiceDirectories {
8255                config: PathBuf::from("/var/lib/runner-manager/config"),
8256                state: PathBuf::from("/var/lib/runner-manager/state"),
8257                runtime: PathBuf::from("/var/lib/runner-manager/runtime"),
8258                logs: PathBuf::from("/var/log/runner-manager"),
8259            },
8260        )
8261        .with_secret_guard("/var/lib/runner-manager/secrets/user-access-token")
8262    }
8263
8264    fn windows_plan(mode: StartMode) -> InstallPlan {
8265        InstallPlan::unchecked(
8266            ServiceIdentity::product(),
8267            mode,
8268            "C:\\Program Files\\runner-manager\\runner-manager.exe",
8269            ServiceDirectories {
8270                config: PathBuf::from("C:\\Users\\op\\AppData\\Local\\rm\\config"),
8271                state: PathBuf::from("C:\\Users\\op\\AppData\\Local\\rm\\state"),
8272                runtime: PathBuf::from("C:\\Users\\op\\AppData\\Local\\rm\\runtime"),
8273                logs: PathBuf::from("C:\\Users\\op\\AppData\\Local\\rm\\logs"),
8274            },
8275        )
8276    }
8277
8278    /// Replaces one fragment of a rendered definition, and **fails the test if
8279    /// the fragment was not there**.
8280    ///
8281    /// Without the assertion a renderer change would quietly turn every
8282    /// "widened definition is rejected" test below into a test that reviews the
8283    /// unmodified definition and passes for the wrong reason. This is the
8284    /// single guard that keeps that class of vacuity out of this file.
8285    fn edited(text: &str, from: &str, to: &str) -> String {
8286        assert!(
8287            text.contains(from),
8288            "the rendered definition does not contain `{from}`, so this test would assert \
8289             nothing about a widened one"
8290        );
8291        text.replace(from, to)
8292    }
8293
8294    /// Every file under `root`, keyed by its path relative to `root`.
8295    fn snapshot(roots: &[&Path]) -> BTreeMap<PathBuf, Vec<u8>> {
8296        fn walk(directory: &Path, out: &mut BTreeMap<PathBuf, Vec<u8>>) {
8297            let Ok(entries) = std::fs::read_dir(directory) else {
8298                return;
8299            };
8300            for entry in entries.flatten() {
8301                let path = entry.path();
8302                if path.is_dir() {
8303                    walk(&path, out);
8304                } else if let Ok(bytes) = std::fs::read(&path) {
8305                    out.insert(path, bytes);
8306                }
8307            }
8308        }
8309        let mut out = BTreeMap::new();
8310        for root in roots {
8311            walk(root, &mut out);
8312        }
8313        out
8314    }
8315
8316    struct Host {
8317        _root: tempfile::TempDir,
8318        paths: AppPaths,
8319        binary: PathBuf,
8320        /// A runner root inside this host's own temporary tree.
8321        ///
8322        /// Without it every `install` in this module would create and
8323        /// re-permission the **real** `%SystemDrive%\rman` on the machine
8324        /// running the tests, and would fail outright on any machine where that
8325        /// directory already exists with the access control `C:\` gives it.
8326        /// A sibling of the four application-data directories rather than a
8327        /// child of one, so `b1`'s overlap check has nothing to object to.
8328        runner_root: LocalAbsolutePath,
8329        controls: RecordingControls,
8330    }
8331
8332    impl Host {
8333        fn new() -> Self {
8334            let root = tempfile::tempdir().expect("a temporary directory");
8335            let paths = AppPaths::rooted_at(root.path());
8336            paths.create_all().expect("the four directories");
8337            let binary = root.path().join(if cfg!(windows) {
8338                "runner-manager.exe"
8339            } else {
8340                "runner-manager"
8341            });
8342            std::fs::write(&binary, b"not a real binary").expect("a stand-in binary");
8343            let runner_root = LocalAbsolutePath::new(
8344                root.path()
8345                    .join("runner-root")
8346                    .to_str()
8347                    .expect("a unicode temporary path"),
8348            )
8349            .expect("a local absolute path");
8350            Self {
8351                _root: root,
8352                paths,
8353                binary,
8354                runner_root,
8355                controls: RecordingControls::new(),
8356            }
8357        }
8358
8359        fn operations(&self) -> ServiceOperations {
8360            ServiceOperations::with_controls(
8361                self.paths.clone(),
8362                ServiceIdentity::product(),
8363                std::sync::Arc::new(self.controls.clone()),
8364            )
8365            .with_runner_root(self.runner_root.clone())
8366        }
8367
8368        fn request(&self, mode: StartMode) -> InstallRequest {
8369            InstallRequest::new(mode).for_binary(&self.binary)
8370        }
8371    }
8372
8373    #[cfg(windows)]
8374    #[test]
8375    fn windows_uninstall_waits_through_the_marked_for_deletion_window() {
8376        let probes = std::cell::Cell::new(0);
8377        let absent =
8378            super::sys::wait_until_scm_absent(Duration::from_secs(1), Duration::ZERO, || {
8379                let next = probes.get() + 1;
8380                probes.set(next);
8381                Ok(next == 3)
8382            })
8383            .expect("the simulated SCM probe succeeds");
8384
8385        assert!(absent);
8386        assert_eq!(
8387            probes.get(),
8388            3,
8389            "uninstall must recheck after transient presence instead of treating it as a leak"
8390        );
8391    }
8392
8393    #[cfg(windows)]
8394    #[test]
8395    fn windows_recognises_an_already_pending_service_deletion() {
8396        let error = windows_service::Error::Winapi(std::io::Error::from_raw_os_error(1072));
8397
8398        assert!(super::sys::is_marked_for_delete(&error));
8399    }
8400
8401    #[test]
8402    fn launchd_enable_failure_is_returned_and_removes_the_bootstrapped_registration() {
8403        let root = tempfile::tempdir().expect("a temporary directory");
8404        let plist = root.path().join("fixture.plist");
8405        std::fs::write(&plist, b"fixture").expect("a plist fixture");
8406        let calls = std::cell::RefCell::new(Vec::new());
8407
8408        let error = enable_launchd_registration(
8409            |arguments| {
8410                let call = arguments
8411                    .iter()
8412                    .map(|argument| argument.to_string_lossy().into_owned())
8413                    .collect::<Vec<_>>();
8414                let operation = call[0].clone();
8415                calls.borrow_mut().push(call);
8416                if operation == "enable" {
8417                    (false, "label remains disabled".to_string())
8418                } else {
8419                    (true, String::new())
8420                }
8421            },
8422            "system",
8423            "system/com.openai.runner-manager-selftest",
8424            &plist,
8425            "runner-manager-selftest",
8426            "rerun with administrative rights",
8427        )
8428        .expect_err("enable failure must fail the install");
8429
8430        assert!(
8431            matches!(
8432                error,
8433                ServiceError::Control {
8434                    operation: "enable",
8435                    ..
8436                }
8437            ),
8438            "{error}"
8439        );
8440        assert_eq!(calls.borrow().len(), 2);
8441        assert_eq!(calls.borrow()[0][0], "enable");
8442        assert_eq!(calls.borrow()[1][0], "bootout");
8443        assert!(!plist.exists(), "rollback must remove the plist");
8444    }
8445
8446    // -----------------------------------------------------------------------
8447    // Quoting, and reading a command line back
8448    // -----------------------------------------------------------------------
8449
8450    #[test]
8451    fn a_path_with_spaces_survives_a_round_trip_through_a_command_line() {
8452        let plan = windows_plan(StartMode::Boot);
8453        let command_line = plan.command_line();
8454        assert!(
8455            command_line.starts_with('"'),
8456            "a path with a space must be quoted, got {command_line}"
8457        );
8458        assert_eq!(
8459            executable_from_command_line(&command_line).as_deref(),
8460            Some(plan.binary())
8461        );
8462    }
8463
8464    #[test]
8465    fn a_path_without_spaces_is_not_quoted_and_still_reads_back() {
8466        let plan = linux_plan(StartMode::Boot);
8467        let command_line = plan.command_line();
8468        assert!(!command_line.starts_with('"'), "got {command_line}");
8469        assert_eq!(
8470            executable_from_command_line(&command_line).as_deref(),
8471            Some(plan.binary())
8472        );
8473    }
8474
8475    #[test]
8476    fn a_quoted_path_containing_a_quote_reads_back_verbatim() {
8477        // Not a path anybody has, but it is the input that separates a real
8478        // implementation of Windows' quoting rules from a `split_whitespace`.
8479        let awkward = r#"C:\odd "name"\rm.exe"#;
8480        let quoted = quote_argument(awkward);
8481        assert_eq!(
8482            executable_from_command_line(&format!("{quoted} daemon run"))
8483                .as_deref()
8484                .map(Path::to_string_lossy)
8485                .as_deref(),
8486            Some(awkward)
8487        );
8488    }
8489
8490    #[test]
8491    fn an_empty_command_line_has_no_executable() {
8492        assert_eq!(executable_from_command_line("   "), None);
8493        assert_eq!(executable_from_command_line(""), None);
8494    }
8495
8496    #[test]
8497    fn xml_escaping_round_trips_the_characters_a_path_or_an_account_may_hold() {
8498        let awkward = r#"DOMAIN\R&D <team> "ops""#;
8499        assert_eq!(
8500            xml_escape(awkward),
8501            "DOMAIN\\R&amp;D &lt;team&gt; &quot;ops&quot;"
8502        );
8503        assert_eq!(xml_unescape(&xml_escape(awkward)), awkward);
8504    }
8505
8506    // -----------------------------------------------------------------------
8507    // The restart policy is bounded at both ends
8508    // -----------------------------------------------------------------------
8509
8510    #[test]
8511    fn a_restart_delay_under_the_floor_is_refused() {
8512        let error = RestartPolicy::new(Duration::from_millis(500), Duration::from_secs(60))
8513            .expect_err("half a second is under the one-second floor");
8514        assert!(
8515            matches!(error, ServiceError::RestartDelay { .. }),
8516            "{error}"
8517        );
8518    }
8519
8520    #[test]
8521    fn a_restart_delay_over_the_ceiling_is_refused() {
8522        let error = RestartPolicy::new(Duration::from_secs(3600), Duration::from_secs(7200))
8523            .expect_err("an hour is over the five-minute ceiling");
8524        assert!(
8525            matches!(error, ServiceError::RestartDelay { .. }),
8526            "{error}"
8527        );
8528    }
8529
8530    #[test]
8531    fn a_reset_window_no_longer_than_the_delay_is_refused() {
8532        let error = RestartPolicy::new(Duration::from_secs(15), Duration::from_secs(15))
8533            .expect_err("a window equal to the delay can never elapse between restarts");
8534        assert!(
8535            matches!(error, ServiceError::RestartResetWindow { .. }),
8536            "{error}"
8537        );
8538    }
8539
8540    #[test]
8541    fn a_delay_inside_the_bound_is_accepted() {
8542        let policy = RestartPolicy::new(Duration::from_secs(20), Duration::from_secs(300))
8543            .expect("twenty seconds is inside the bound");
8544        assert_eq!(policy.delay(), Duration::from_secs(20));
8545        assert_eq!(policy.reset_after(), Duration::from_secs(300));
8546    }
8547
8548    // -----------------------------------------------------------------------
8549    // Identity
8550    // -----------------------------------------------------------------------
8551
8552    #[test]
8553    fn a_fixture_identity_can_never_be_the_product_identity() {
8554        let fixture = ServiceIdentity::fixture("abc123");
8555        assert!(fixture.is_fixture());
8556        assert!(!ServiceIdentity::product().is_fixture());
8557        assert_ne!(fixture.name(), ServiceIdentity::product().name());
8558        assert_ne!(
8559            fixture.launchd_label(),
8560            ServiceIdentity::product().launchd_label()
8561        );
8562        assert_ne!(
8563            fixture.systemd_unit(),
8564            ServiceIdentity::product().systemd_unit()
8565        );
8566    }
8567
8568    #[test]
8569    fn a_fixture_tag_is_reduced_to_characters_every_manager_accepts() {
8570        let fixture = ServiceIdentity::fixture("A b/c:\\d");
8571        assert!(
8572            fixture
8573                .name()
8574                .chars()
8575                .all(|c| c.is_ascii_lowercase() || c.is_ascii_digit() || c == '-'),
8576            "got {}",
8577            fixture.name()
8578        );
8579    }
8580
8581    // -----------------------------------------------------------------------
8582    // The systemd unit
8583    // -----------------------------------------------------------------------
8584
8585    #[test]
8586    fn the_boot_unit_restarts_on_failure_after_the_bounded_delay() {
8587        let unit = systemd_unit(&linux_plan(StartMode::Boot));
8588        assert!(unit.contains("KillMode=process\n"), "{unit}");
8589        assert!(unit.contains("Restart=on-failure\n"), "{unit}");
8590        assert!(unit.contains("RestartSec=15\n"), "{unit}");
8591        assert!(unit.contains("StartLimitIntervalSec=600\n"), "{unit}");
8592        assert!(unit.contains("StartLimitBurst=5\n"), "{unit}");
8593        assert!(unit.contains("WantedBy=multi-user.target\n"), "{unit}");
8594    }
8595
8596    #[test]
8597    fn the_boot_unit_reads_the_live_store_instead_of_a_frozen_systemd_copy() {
8598        let unit = systemd_unit(&linux_plan(StartMode::Boot));
8599        assert!(
8600            !unit.contains("LoadCredential="),
8601            "a startup snapshot would shadow every rotated credential until restart:\n{unit}"
8602        );
8603    }
8604
8605    #[test]
8606    fn a_login_unit_carries_no_machine_credential_and_wants_the_session_target() {
8607        let unit = systemd_unit(&linux_plan(StartMode::Login));
8608        assert!(
8609            !unit.contains("LoadCredential="),
8610            "a user unit must not name a root-owned credential file, got:\n{unit}"
8611        );
8612        assert!(unit.contains("WantedBy=default.target\n"), "{unit}");
8613    }
8614
8615    #[test]
8616    fn the_unit_can_atomically_replace_the_credential_and_write_only_required_directories() {
8617        let plan = linux_plan(StartMode::Boot);
8618        let unit = systemd_unit(&plan);
8619        let directives = ini_directives(&unit, "Service");
8620        let listed = split_quoted(
8621            directives
8622                .get("ReadWritePaths")
8623                .expect("the unit names its writable paths"),
8624        );
8625        assert_eq!(listed.len(), 5, "{listed:?}");
8626        for path in plan.directories().all() {
8627            assert!(
8628                listed.iter().any(|entry| entry == &path.to_string_lossy()),
8629                "{} is missing from {listed:?}",
8630                path.display()
8631            );
8632        }
8633        assert!(
8634            listed
8635                .iter()
8636                .any(|entry| entry == "/var/lib/runner-manager/secrets"),
8637            "atomic credential replacement needs its parent directory in {listed:?}"
8638        );
8639    }
8640
8641    #[test]
8642    fn the_unit_records_the_absolute_binary_path() {
8643        let plan = linux_plan(StartMode::Boot);
8644        let unit = systemd_unit(&plan);
8645        assert!(
8646            unit.contains("ExecStart=/opt/runner-manager/bin/runner-manager daemon run\n"),
8647            "{unit}"
8648        );
8649    }
8650
8651    // -----------------------------------------------------------------------
8652    // The launchd property list
8653    // -----------------------------------------------------------------------
8654
8655    #[test]
8656    fn the_daemon_restarts_only_after_an_unsuccessful_exit() {
8657        let plist = launchd_plist(&linux_plan(StartMode::Boot));
8658        // A bare `KeepAlive` would also restart a job the operator stopped
8659        // deliberately, which turns `service stop` into a fight with launchd.
8660        assert!(
8661            plist.contains(
8662                "<key>KeepAlive</key>\n  <dict>\n    <key>SuccessfulExit</key>\n    <false/>\n"
8663            ),
8664            "{plist}"
8665        );
8666        assert!(
8667            plist.contains("<key>ThrottleInterval</key>\n  <integer>15</integer>"),
8668            "{plist}"
8669        );
8670    }
8671
8672    #[test]
8673    fn a_launch_daemon_names_root_and_a_launch_agent_names_nobody() {
8674        let daemon = launchd_plist(&linux_plan(StartMode::Boot));
8675        assert_eq!(
8676            plist_string_value(&daemon, "UserName").as_deref(),
8677            Some("root")
8678        );
8679        assert_eq!(plist_bool_value(&daemon, "SessionCreate"), Some(false));
8680
8681        let agent = launchd_plist(&linux_plan(StartMode::Login));
8682        assert_eq!(
8683            plist_string_value(&agent, "UserName"),
8684            None,
8685            "a LaunchAgent already runs as the operator:\n{agent}"
8686        );
8687    }
8688
8689    #[test]
8690    fn the_plist_runs_the_daemon_and_its_runners_as_an_interactive_job() {
8691        for mode in [StartMode::Boot, StartMode::Login] {
8692            let plan = linux_plan(mode);
8693            let plist = launchd_plist(&plan);
8694            assert_eq!(
8695                plist_string_value(&plist, "ProcessType").as_deref(),
8696                Some("Interactive"),
8697                "launchd applies ProcessType to every runner the daemon starts:\n{plist}"
8698            );
8699            let review = review_least_privilege(
8700                &ServiceDefinition::from_text(DefinitionKind::LaunchdPlist, plist),
8701                &plan,
8702            );
8703            assert!(
8704                review
8705                    .controls()
8706                    .iter()
8707                    .any(|control| control == "ProcessType=Interactive"),
8708                "{review}"
8709            );
8710            assert!(
8711                !review
8712                    .findings()
8713                    .iter()
8714                    .any(|finding| finding.subject == "ProcessType"),
8715                "{review}"
8716            );
8717        }
8718    }
8719
8720    /// The value 0.4.31 and earlier installed, and launchd's default, both
8721    /// throttle every runner. That is a shortfall the update repairs, not an
8722    /// excess: a scheduling class grants no authority.
8723    #[test]
8724    fn a_plist_that_throttles_its_runners_is_a_shortfall() {
8725        let plan = linux_plan(StartMode::Login);
8726        let rendered = launchd_plist(&plan);
8727        let interactive = "<key>ProcessType</key>\n  <string>Interactive</string>\n";
8728        for (label, throttled) in [
8729            (
8730                "Background",
8731                edited(
8732                    &rendered,
8733                    interactive,
8734                    "<key>ProcessType</key>\n  <string>Background</string>\n",
8735                ),
8736            ),
8737            ("absent", edited(&rendered, interactive, "")),
8738        ] {
8739            let review = review_least_privilege(
8740                &ServiceDefinition::from_text(DefinitionKind::LaunchdPlist, throttled),
8741                &plan,
8742            );
8743            assert!(review.is_least_privilege(), "{label}: {review}");
8744            assert!(
8745                review.findings().iter().any(|finding| {
8746                    finding.kind == FindingKind::Shortfall
8747                        && finding.subject == "ProcessType"
8748                        && finding.detail.contains(label)
8749                }),
8750                "{label}: {review}"
8751            );
8752        }
8753    }
8754
8755    #[test]
8756    fn a_plist_with_an_old_process_type_is_drift_and_an_operator_edit_is_not() {
8757        let host = Host::new();
8758        let identity = ServiceIdentity::product();
8759        let plan = InstallPlan::resolve(
8760            identity.clone(),
8761            &host.request(StartMode::Login),
8762            ServiceDirectories::of(&host.paths),
8763        )
8764        .expect("a resolvable plan");
8765        let installed = launchd_plist(&plan);
8766        let definition =
8767            ServiceDefinition::from_text(DefinitionKind::LaunchdPlist, installed.clone());
8768        let mut record = InstallRecord::of(&plan, &definition, Utc::now());
8769        record.installed_by_version = "0.4.31".into();
8770        let path = PathBuf::from("/Users/op/Library/LaunchAgents/runner-manager.plist");
8771
8772        assert_eq!(
8773            launchd_definition_drift(&identity, &record, path.clone(), &installed),
8774            None
8775        );
8776        let edited_by_operator = edited(
8777            &installed,
8778            "</dict>\n</plist>\n",
8779            "  <key>Nice</key>\n  <integer>0</integer>\n</dict>\n</plist>\n",
8780        );
8781        assert_eq!(
8782            launchd_definition_drift(&identity, &record, path.clone(), &edited_by_operator),
8783            None,
8784            "the daemon repairs this drift itself, so it must not key on edits it did not cause"
8785        );
8786
8787        let written_by_0_4_31 = edited(
8788            &installed,
8789            "<string>Interactive</string>",
8790            "<string>Background</string>",
8791        );
8792        let drift = launchd_definition_drift(&identity, &record, path, &written_by_0_4_31)
8793            .expect("a Background plist is drift");
8794        assert_eq!(drift.manager, DefinitionKind::LaunchdPlist);
8795        assert_eq!(drift.start_mode, StartMode::Login);
8796        assert_eq!(
8797            drift.rendered, installed,
8798            "the repair must write exactly what `service install` writes"
8799        );
8800
8801        let status = ServiceStatus::compose(identity, None, None, None, None, &host.paths)
8802            .with_definition_drift(Some(drift));
8803        let problem = status
8804            .problems()
8805            .iter()
8806            .find(|problem| problem.subject == "definition")
8807            .expect("the drift is reported");
8808        // The remedy is a restart: a running daemon holds the lock that refuses
8809        // `service install`.
8810        assert!(
8811            problem.detail.contains("0.4.31")
8812                && problem.detail.contains(
8813                    "launchctl kickstart -k gui/$(id -u)/io.github.IvanMurzak.runner-manager`"
8814                )
8815                && !problem.detail.contains("service install")
8816                && !problem.detail.contains("systemctl")
8817                && !problem.detail.contains("sudo"),
8818            "{}",
8819            problem.detail
8820        );
8821    }
8822
8823    #[cfg(unix)]
8824    #[test]
8825    fn a_replaced_definition_is_the_rendering_and_readable_by_launchd() {
8826        use std::os::unix::fs::PermissionsExt as _;
8827
8828        let root = tempfile::tempdir().expect("a temporary directory");
8829        let path = root
8830            .path()
8831            .join("io.github.IvanMurzak.runner-manager.plist");
8832        std::fs::write(&path, "<string>Background</string>").expect("old plist");
8833        std::fs::set_permissions(&path, std::fs::Permissions::from_mode(0o600)).unwrap();
8834        let drift = DefinitionDrift {
8835            path: path.clone(),
8836            installed_by_version: "0.4.31".into(),
8837            rendered: launchd_plist(&linux_plan(StartMode::Login)),
8838            start_mode: StartMode::Login,
8839            manager: DefinitionKind::LaunchdPlist,
8840        };
8841
8842        replace_definition("runner-manager", &drift).expect("replaced");
8843
8844        assert_eq!(std::fs::read_to_string(&path).unwrap(), drift.rendered);
8845        assert_eq!(
8846            std::fs::metadata(&path).unwrap().permissions().mode() & 0o777,
8847            0o644
8848        );
8849        assert_eq!(
8850            std::fs::read_dir(root.path()).unwrap().count(),
8851            1,
8852            "no staging file is left beside the plist"
8853        );
8854    }
8855
8856    /// The job of [`a_job_reloads_itself_from_its_rewritten_plist`], run by
8857    /// launchd and by nothing else.
8858    ///
8859    /// It does what the daemon does when it starts: when its own plist (kept
8860    /// in its working directory) does not carry [`LAUNCHD_PROCESS_TYPE`], it
8861    /// rewrites the plist and asks for the reload from inside the job that the
8862    /// reload boots out. Either way it then stays up, to be booted out by the
8863    /// helper or, once reloaded, by the test.
8864    #[cfg(target_os = "macos")]
8865    #[test]
8866    #[ignore = "run only by launchd, as the job of the detached-reload test"]
8867    fn detached_reload_job() {
8868        let label = std::env::var("XPC_SERVICE_NAME").unwrap_or_default();
8869        let tag = label
8870            .split_once("-selftest-")
8871            .map(|(_, tag)| tag.to_string())
8872            .unwrap_or_else(|| panic!("run only by launchd for the reload test, not {label:?}"));
8873        let identity = ServiceIdentity::fixture(&tag);
8874        assert_eq!(identity.launchd_label(), label);
8875        let path = std::env::current_dir()
8876            .expect("working directory")
8877            .join(format!("{label}.plist"));
8878        let installed = std::fs::read_to_string(&path).expect("own plist");
8879        if plist_string_value(&installed, "ProcessType").as_deref() != Some(LAUNCHD_PROCESS_TYPE) {
8880            let drift = DefinitionDrift {
8881                path: path.clone(),
8882                installed_by_version: "0.4.31".into(),
8883                rendered: installed.replace(
8884                    "<string>Background</string>",
8885                    &format!("<string>{LAUNCHD_PROCESS_TYPE}</string>"),
8886                ),
8887                start_mode: StartMode::Login,
8888                manager: DefinitionKind::LaunchdPlist,
8889            };
8890            replace_definition(identity.name(), &drift).expect("plist replaced");
8891            super::sys::reload_detached(&identity, StartMode::Login, &path)
8892                .expect("helper started");
8893        }
8894        std::thread::sleep(Duration::from_secs(120));
8895    }
8896
8897    /// A job that rewrites its own plist and asks for the reload comes back
8898    /// under the rewritten plist.
8899    ///
8900    /// The request comes from inside the job, as the daemon's does, so the
8901    /// helper has to outlive the bootout of the very job that started it.
8902    /// launchd sends SIGTERM to the job's whole process group, so a helper that
8903    /// stayed in it would sometimes die before its bootstrap and leave the job
8904    /// unloaded. Measured with `setsid` removed, that failed one run in four
8905    /// here: this test catches it, though not on every run.
8906    #[cfg(target_os = "macos")]
8907    #[test]
8908    fn a_job_reloads_itself_from_its_rewritten_plist() {
8909        if !super::launchd_fixture::enabled() {
8910            return;
8911        }
8912        let root = tempfile::tempdir().expect("a temporary directory");
8913        let identity = ServiceIdentity::fixture(&format!("reload-{}", std::process::id()));
8914        let label = identity.launchd_label();
8915        let path = root.path().join(format!("{label}.plist"));
8916        let binary = super::launchd_fixture::job_binary(root.path());
8917        let mut plist = String::from("<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n");
8918        plist.push_str("<plist version=\"1.0\">\n<dict>\n");
8919        plist.push_str(&plist_string("Label", &label));
8920        plist.push_str("  <key>ProgramArguments</key>\n  <array>\n");
8921        for argument in [
8922            binary.to_string_lossy().as_ref(),
8923            "--ignored",
8924            "--exact",
8925            "service::tests::detached_reload_job",
8926            "--test-threads=1",
8927        ] {
8928            plist.push_str(&format!("    <string>{}</string>\n", xml_escape(argument)));
8929        }
8930        plist.push_str("  </array>\n  <key>RunAtLoad</key>\n  <true/>\n");
8931        plist.push_str(&plist_string(
8932            "WorkingDirectory",
8933            &root.path().to_string_lossy(),
8934        ));
8935        plist.push_str(&plist_string("ProcessType", "Background"));
8936        plist.push_str("</dict>\n</plist>\n");
8937        std::fs::write(&path, plist).expect("plist written");
8938
8939        let loaded = super::launchd_fixture::LoadedAgent::bootstrap(&label, &path);
8940        let spawn_type = || {
8941            let printed = std::process::Command::new("launchctl")
8942                .args(["print", loaded.target()])
8943                .output()
8944                .expect("launchctl runs");
8945            String::from_utf8_lossy(&printed.stdout)
8946                .lines()
8947                .find_map(|line| {
8948                    line.trim()
8949                        .strip_prefix("spawn type = ")
8950                        .map(str::to_string)
8951                })
8952        };
8953        assert_eq!(spawn_type().as_deref(), Some("background (5)"));
8954
8955        let deadline = std::time::Instant::now() + Duration::from_secs(90);
8956        while spawn_type().as_deref() != Some("interactive (4)") {
8957            assert!(
8958                std::time::Instant::now() < deadline,
8959                "the job did not come back under its rewritten plist; launchd reports {:?}",
8960                spawn_type()
8961            );
8962            std::thread::sleep(Duration::from_millis(250));
8963        }
8964        let rewritten = std::fs::read_to_string(&path).expect("plist readable");
8965        assert_eq!(
8966            plist_string_value(&rewritten, "ProcessType").as_deref(),
8967            Some(LAUNCHD_PROCESS_TYPE)
8968        );
8969    }
8970
8971    #[test]
8972    fn the_plist_records_the_absolute_binary_path_and_the_daemon_arguments() {
8973        let plist = launchd_plist(&linux_plan(StartMode::Boot));
8974        assert!(
8975            plist.contains("<string>/opt/runner-manager/bin/runner-manager</string>"),
8976            "{plist}"
8977        );
8978        assert!(plist.contains("<string>daemon</string>"), "{plist}");
8979        assert!(plist.contains("<string>run</string>"), "{plist}");
8980    }
8981
8982    #[test]
8983    fn the_launchd_label_is_the_product_identity_in_reverse_domain_form() {
8984        assert_eq!(
8985            ServiceIdentity::product().launchd_label(),
8986            "io.github.IvanMurzak.runner-manager"
8987        );
8988    }
8989
8990    // -----------------------------------------------------------------------
8991    // The Task Scheduler document
8992    // -----------------------------------------------------------------------
8993
8994    #[test]
8995    fn the_task_runs_at_least_privilege_on_a_logon_trigger() {
8996        let xml = windows_scheduled_task_xml(
8997            &windows_plan(StartMode::Login),
8998            &TaskPrincipal::named("HOST\\operator"),
8999        );
9000        assert!(xml.contains("<LogonTrigger>"), "{xml}");
9001        assert!(xml.contains("<RunLevel>LeastPrivilege</RunLevel>"), "{xml}");
9002        assert!(
9003            xml.contains("<LogonType>InteractiveToken</LogonType>"),
9004            "{xml}"
9005        );
9006        assert!(xml.contains("<UserId>HOST\\operator</UserId>"), "{xml}");
9007        assert!(
9008            xml.contains("<Interval>PT1M</Interval>"),
9009            "Task Scheduler takes whole minutes only, and rejects the registration outright \
9010             for anything finer:\n{xml}"
9011        );
9012    }
9013
9014    #[test]
9015    fn task_schedulers_minute_granularity_only_ever_rounds_the_delay_up() {
9016        // Measured against the real Task Scheduler, which answers `PT15S` with
9017        // "The task XML contains a value which is incorrectly formatted or out
9018        // of range". Rounding *up* is what keeps "does not restart-loop faster
9019        // than that bound" true on this manager.
9020        for (asked, enforced) in [(1u64, 60u64), (15, 60), (60, 60), (61, 120), (300, 300)] {
9021            let policy =
9022                RestartPolicy::new(Duration::from_secs(asked), Duration::from_secs(asked + 600))
9023                    .expect("inside the supported range");
9024            assert_eq!(
9025                policy
9026                    .effective_delay(DefinitionKind::WindowsScheduledTask)
9027                    .as_secs(),
9028                enforced,
9029                "a {asked}s delay must be enforced as {enforced}s"
9030            );
9031            assert!(
9032                policy.effective_delay(DefinitionKind::WindowsScheduledTask) >= policy.delay(),
9033                "rounding must never shorten the bound"
9034            );
9035        }
9036    }
9037
9038    #[test]
9039    fn every_other_manager_enforces_the_delay_exactly_as_configured() {
9040        let policy = RestartPolicy::default();
9041        for kind in [
9042            DefinitionKind::WindowsService,
9043            DefinitionKind::LaunchdPlist,
9044            DefinitionKind::SystemdUnit,
9045        ] {
9046            assert_eq!(
9047                policy.effective_delay(kind),
9048                policy.delay(),
9049                "{kind:?} takes seconds and enforces exactly what it is given"
9050            );
9051        }
9052    }
9053
9054    #[test]
9055    fn a_task_whose_manager_reports_the_rounded_delay_is_not_a_fault() {
9056        let host = Host::new();
9057        let operations = host.operations();
9058        operations
9059            .install(&host.request(StartMode::Login))
9060            .expect("an install at login");
9061        // What a real Task Scheduler reports back for a 15-second policy.
9062        host.controls.edit("runner-manager", |registration| {
9063            registration.manager = DefinitionKind::WindowsScheduledTask;
9064            registration.restart_delay = Some(Duration::from_secs(60));
9065        });
9066
9067        let status = operations.status().expect("a status");
9068        assert!(
9069            status.is_healthy(),
9070            "minute granularity is the manager's, not a mis-registration: {status}"
9071        );
9072        assert!(
9073            status
9074                .notes()
9075                .iter()
9076                .any(|note| note.contains("whole minutes")),
9077            "but the operator must be told why 15 became 60: {status}"
9078        );
9079
9080        // The discriminator: a delay that is neither the configured one nor its
9081        // rounding is still a fault.
9082        host.controls.edit("runner-manager", |registration| {
9083            registration.restart_delay = Some(Duration::from_secs(1));
9084        });
9085        assert!(
9086            !operations.status().expect("a status").is_healthy(),
9087            "a one-second delay is not what any manager was asked for"
9088        );
9089    }
9090
9091    #[test]
9092    fn the_task_records_the_absolute_binary_path_and_the_daemon_arguments() {
9093        let plan = windows_plan(StartMode::Login);
9094        let xml = windows_scheduled_task_xml(&plan, &TaskPrincipal::named("HOST\\operator"));
9095        assert_eq!(
9096            xml_value(&xml, "Command").as_deref(),
9097            Some("C:\\Program Files\\runner-manager\\runner-manager.exe"),
9098            "{xml}"
9099        );
9100        assert_eq!(xml_value(&xml, "Arguments").as_deref(), Some("daemon run"));
9101    }
9102
9103    #[test]
9104    fn an_account_name_holding_xml_punctuation_is_escaped() {
9105        let xml = windows_scheduled_task_xml(
9106            &windows_plan(StartMode::Login),
9107            &TaskPrincipal::named("R&D\\ops"),
9108        );
9109        assert!(xml.contains("<UserId>R&amp;D\\ops</UserId>"), "{xml}");
9110        assert_eq!(xml_value(&xml, "UserId").as_deref(), Some("R&D\\ops"));
9111    }
9112
9113    // -----------------------------------------------------------------------
9114    // The Windows service descriptor
9115    // -----------------------------------------------------------------------
9116
9117    #[test]
9118    fn the_service_starts_automatically_under_the_account_the_store_admits() {
9119        let text = windows_service_descriptor(&windows_plan(StartMode::Boot));
9120        let directives = ini_directives(&text, "windows-service");
9121        assert_eq!(
9122            directives.get("StartType").map(String::as_str),
9123            Some("AutoStart")
9124        );
9125        assert_eq!(
9126            directives.get("Account").map(String::as_str),
9127            Some("NT AUTHORITY\\SYSTEM")
9128        );
9129        assert_eq!(
9130            directives.get("ServiceType").map(String::as_str),
9131            Some("OWN_PROCESS")
9132        );
9133        assert_eq!(
9134            directives
9135                .get("FailureActionRestartDelaySecs")
9136                .map(String::as_str),
9137            Some("15")
9138        );
9139        assert_eq!(
9140            directives
9141                .get("FailureActionsOnNonCrashFailures")
9142                .map(String::as_str),
9143            Some("true"),
9144            "without this flag a non-zero exit is not a failure the manager restarts"
9145        );
9146    }
9147
9148    #[test]
9149    fn the_service_spec_leaves_the_account_unnamed_so_the_api_means_local_system() {
9150        let spec = windows_service_spec(&windows_plan(StartMode::Boot));
9151        assert_eq!(spec.account, None);
9152        assert!(spec.automatic_start);
9153        assert!(
9154            spec.command_line.contains("daemon run"),
9155            "{}",
9156            spec.command_line
9157        );
9158    }
9159
9160    // -----------------------------------------------------------------------
9161    // Where each definition is installed
9162    // -----------------------------------------------------------------------
9163
9164    #[test]
9165    fn each_definition_goes_where_its_platform_expects_it() {
9166        let home = PathBuf::from("/home/op");
9167        assert_eq!(
9168            ServiceDefinition::launchd(&linux_plan(StartMode::Boot), Some(&home)).install_path(),
9169            Some(Path::new(
9170                "/Library/LaunchDaemons/io.github.IvanMurzak.runner-manager.plist"
9171            ))
9172        );
9173        assert_eq!(
9174            ServiceDefinition::launchd(&linux_plan(StartMode::Login), Some(&home)).install_path(),
9175            Some(Path::new(
9176                "/home/op/Library/LaunchAgents/io.github.IvanMurzak.runner-manager.plist"
9177            ))
9178        );
9179        assert_eq!(
9180            ServiceDefinition::systemd(&linux_plan(StartMode::Boot), Some(&home)).install_path(),
9181            Some(Path::new("/etc/systemd/system/runner-manager.service"))
9182        );
9183        assert_eq!(
9184            ServiceDefinition::systemd(&linux_plan(StartMode::Login), Some(&home)).install_path(),
9185            Some(Path::new(
9186                "/home/op/.config/systemd/user/runner-manager.service"
9187            ))
9188        );
9189        assert_eq!(
9190            ServiceDefinition::windows_service(&windows_plan(StartMode::Boot)).install_path(),
9191            None,
9192            "the Service Control Manager has no file"
9193        );
9194    }
9195
9196    #[test]
9197    fn a_login_definition_without_a_home_directory_has_nowhere_to_go() {
9198        assert_eq!(
9199            ServiceDefinition::systemd(&linux_plan(StartMode::Login), None).install_path(),
9200            None
9201        );
9202        assert_eq!(
9203            ServiceDefinition::launchd(&linux_plan(StartMode::Login), None).install_path(),
9204            None
9205        );
9206    }
9207
9208    // -----------------------------------------------------------------------
9209    // The least-privilege review: the passing case, then every failing one
9210    // -----------------------------------------------------------------------
9211
9212    #[test]
9213    fn the_rendered_definitions_are_all_least_privilege() {
9214        let linux = linux_plan(StartMode::Boot);
9215        let windows = windows_plan(StartMode::Boot);
9216        for (definition, plan) in [
9217            (ServiceDefinition::systemd(&linux, None), &linux),
9218            (ServiceDefinition::launchd(&linux, None), &linux),
9219            (ServiceDefinition::windows_service(&windows), &windows),
9220        ] {
9221            let review = review_least_privilege(&definition, plan);
9222            assert!(
9223                review.is_least_privilege(),
9224                "{:?} should be least privilege, got:\n{review}",
9225                definition.kind()
9226            );
9227            assert!(
9228                !review.controls().is_empty(),
9229                "a review that confirms nothing proves nothing: {:?}",
9230                definition.kind()
9231            );
9232        }
9233    }
9234
9235    #[test]
9236    fn the_rendered_task_is_least_privilege_and_says_what_it_checked() {
9237        let plan = windows_plan(StartMode::Login);
9238        let definition =
9239            ServiceDefinition::windows_scheduled_task(&plan, &TaskPrincipal::named("HOST\\op"));
9240        let review = review_least_privilege(&definition, &plan);
9241        assert!(review.is_least_privilege(), "{review}");
9242        assert!(
9243            review
9244                .controls()
9245                .iter()
9246                .any(|control| control.contains("LeastPrivilege")),
9247            "{review}"
9248        );
9249    }
9250
9251    #[test]
9252    fn a_unit_that_makes_one_more_directory_writable_is_not_least_privilege() {
9253        let plan = linux_plan(StartMode::Boot);
9254        let rendered = systemd_unit(&plan);
9255        let widened = edited(
9256            &rendered,
9257            "ReadWritePaths=/var/lib/runner-manager/config",
9258            "ReadWritePaths=/etc /var/lib/runner-manager/config",
9259        );
9260        let review = review_least_privilege(
9261            &ServiceDefinition::from_text(DefinitionKind::SystemdUnit, widened),
9262            &plan,
9263        );
9264        assert!(!review.is_least_privilege(), "{review}");
9265        assert!(
9266            review
9267                .excesses()
9268                .iter()
9269                .any(|finding| finding.detail.contains("/etc")),
9270            "the review must name the directory it objects to: {review}"
9271        );
9272    }
9273
9274    #[test]
9275    fn a_unit_that_drops_a_hardening_directive_is_not_least_privilege() {
9276        let plan = linux_plan(StartMode::Boot);
9277        let rendered = systemd_unit(&plan);
9278        let weakened = edited(&rendered, "NoNewPrivileges=yes\n", "");
9279        let review = review_least_privilege(
9280            &ServiceDefinition::from_text(DefinitionKind::SystemdUnit, weakened),
9281            &plan,
9282        );
9283        assert!(!review.is_least_privilege(), "{review}");
9284        assert!(
9285            review
9286                .excesses()
9287                .iter()
9288                .any(|finding| finding.subject == "NoNewPrivileges"),
9289            "{review}"
9290        );
9291    }
9292
9293    #[test]
9294    fn a_unit_that_keeps_capabilities_is_not_least_privilege() {
9295        let plan = linux_plan(StartMode::Boot);
9296        let rendered = systemd_unit(&plan);
9297        let widened = edited(
9298            &rendered,
9299            "CapabilityBoundingSet=\n",
9300            "CapabilityBoundingSet=CAP_NET_ADMIN CAP_SYS_ADMIN\n",
9301        );
9302        let review = review_least_privilege(
9303            &ServiceDefinition::from_text(DefinitionKind::SystemdUnit, widened),
9304            &plan,
9305        );
9306        assert!(!review.is_least_privilege(), "{review}");
9307        assert!(
9308            review
9309                .excesses()
9310                .iter()
9311                .any(|finding| finding.subject == "CapabilityBoundingSet"),
9312            "{review}"
9313        );
9314    }
9315
9316    #[test]
9317    fn a_unit_that_opens_a_listening_socket_is_not_least_privilege() {
9318        let plan = linux_plan(StartMode::Boot);
9319        let rendered = systemd_unit(&plan);
9320        let widened = edited(
9321            &rendered,
9322            "[Install]",
9323            "ListenStream=127.0.0.1:9000\n\n[Install]",
9324        );
9325        let review = review_least_privilege(
9326            &ServiceDefinition::from_text(DefinitionKind::SystemdUnit, widened),
9327            &plan,
9328        );
9329        assert!(
9330            !review.is_least_privilege(),
9331            "07-security.md rule 2 forbids any inbound surface: {review}"
9332        );
9333    }
9334
9335    #[test]
9336    fn a_unit_that_makes_a_directory_unwritable_is_a_shortfall_not_an_excess() {
9337        let plan = linux_plan(StartMode::Boot);
9338        let rendered = systemd_unit(&plan);
9339        let narrowed = edited(&rendered, " /var/lib/runner-manager/runtime", "");
9340        let review = review_least_privilege(
9341            &ServiceDefinition::from_text(DefinitionKind::SystemdUnit, narrowed),
9342            &plan,
9343        );
9344        assert!(
9345            review.is_least_privilege(),
9346            "too little authority is not an excess: {review}"
9347        );
9348        assert!(
9349            review
9350                .findings()
9351                .iter()
9352                .any(|finding| finding.kind == FindingKind::Shortfall
9353                    && finding.detail.contains("runtime")),
9354            "{review}"
9355        );
9356    }
9357
9358    #[test]
9359    fn a_launch_agent_that_names_an_account_is_not_least_privilege() {
9360        let plan = linux_plan(StartMode::Login);
9361        let rendered = launchd_plist(&plan);
9362        let widened = edited(
9363            &rendered,
9364            "<key>ProcessType</key>",
9365            "<key>UserName</key>\n  <string>root</string>\n  <key>ProcessType</key>",
9366        );
9367        let review = review_least_privilege(
9368            &ServiceDefinition::from_text(DefinitionKind::LaunchdPlist, widened),
9369            &plan,
9370        );
9371        assert!(!review.is_least_privilege(), "{review}");
9372        assert!(
9373            review
9374                .excesses()
9375                .iter()
9376                .any(|finding| finding.subject == "UserName"),
9377            "{review}"
9378        );
9379    }
9380
9381    #[test]
9382    fn a_launch_daemon_that_asks_for_a_session_is_not_least_privilege() {
9383        let plan = linux_plan(StartMode::Boot);
9384        let rendered = launchd_plist(&plan);
9385        let widened = edited(
9386            &rendered,
9387            "<key>SessionCreate</key>\n  <false/>",
9388            "<key>SessionCreate</key>\n  <true/>",
9389        );
9390        let review = review_least_privilege(
9391            &ServiceDefinition::from_text(DefinitionKind::LaunchdPlist, widened),
9392            &plan,
9393        );
9394        assert!(!review.is_least_privilege(), "{review}");
9395        assert!(
9396            review
9397                .excesses()
9398                .iter()
9399                .any(|finding| finding.subject == "SessionCreate"),
9400            "{review}"
9401        );
9402    }
9403
9404    #[test]
9405    fn a_launchd_job_that_publishes_a_mach_service_is_not_least_privilege() {
9406        let plan = linux_plan(StartMode::Boot);
9407        let rendered = launchd_plist(&plan);
9408        let widened = edited(
9409            &rendered,
9410            "<key>ProcessType</key>",
9411            "<key>MachServices</key>\n  <dict/>\n  <key>ProcessType</key>",
9412        );
9413        let review = review_least_privilege(
9414            &ServiceDefinition::from_text(DefinitionKind::LaunchdPlist, widened),
9415            &plan,
9416        );
9417        assert!(!review.is_least_privilege(), "{review}");
9418    }
9419
9420    #[test]
9421    fn a_task_asking_for_the_highest_available_token_is_not_least_privilege() {
9422        let plan = windows_plan(StartMode::Login);
9423        let rendered = windows_scheduled_task_xml(&plan, &TaskPrincipal::named("HOST\\op"));
9424        let widened = edited(
9425            &rendered,
9426            "<RunLevel>LeastPrivilege</RunLevel>",
9427            "<RunLevel>HighestAvailable</RunLevel>",
9428        );
9429        let review = review_least_privilege(
9430            &ServiceDefinition::from_text(DefinitionKind::WindowsScheduledTask, widened),
9431            &plan,
9432        );
9433        assert!(!review.is_least_privilege(), "{review}");
9434        assert!(
9435            review
9436                .excesses()
9437                .iter()
9438                .any(|finding| finding.subject == "RunLevel"),
9439            "{review}"
9440        );
9441    }
9442
9443    #[test]
9444    fn a_task_that_would_store_a_password_is_not_least_privilege() {
9445        let plan = windows_plan(StartMode::Login);
9446        let rendered = windows_scheduled_task_xml(&plan, &TaskPrincipal::named("HOST\\op"));
9447        let widened = edited(
9448            &rendered,
9449            "<LogonType>InteractiveToken</LogonType>",
9450            "<LogonType>Password</LogonType>",
9451        );
9452        let review = review_least_privilege(
9453            &ServiceDefinition::from_text(DefinitionKind::WindowsScheduledTask, widened),
9454            &plan,
9455        );
9456        assert!(!review.is_least_privilege(), "{review}");
9457    }
9458
9459    #[test]
9460    fn an_interactive_windows_service_is_not_least_privilege() {
9461        let plan = windows_plan(StartMode::Boot);
9462        let rendered = windows_service_descriptor(&plan);
9463        let widened = edited(
9464            &rendered,
9465            "ServiceType=OWN_PROCESS",
9466            "ServiceType=OWN_PROCESS|INTERACTIVE_PROCESS",
9467        );
9468        let review = review_least_privilege(
9469            &ServiceDefinition::from_text(DefinitionKind::WindowsService, widened),
9470            &plan,
9471        );
9472        assert!(!review.is_least_privilege(), "{review}");
9473        assert!(
9474            review
9475                .excesses()
9476                .iter()
9477                .any(|finding| finding.subject == "ServiceType"),
9478            "{review}"
9479        );
9480    }
9481
9482    #[test]
9483    fn a_windows_service_under_an_account_the_store_dacl_does_not_name_is_reported() {
9484        let plan = windows_plan(StartMode::Boot);
9485        let rendered = windows_service_descriptor(&plan);
9486        let changed = edited(
9487            &rendered,
9488            "Account=NT AUTHORITY\\SYSTEM",
9489            "Account=NT AUTHORITY\\LocalService",
9490        );
9491        let review = review_least_privilege(
9492            &ServiceDefinition::from_text(DefinitionKind::WindowsService, changed),
9493            &plan,
9494        );
9495        // LocalService is *less* privileged, so this is not an excess. It is
9496        // still wrong, because `d2`'s DACL names SY, BA and OW and nothing else:
9497        // the daemon would start and find no credential. Reporting it as a
9498        // shortfall rather than as an excess is the difference between a check
9499        // that understands the requirement and one that counts adjectives.
9500        assert!(review.is_least_privilege(), "{review}");
9501        assert!(
9502            review.findings().iter().any(|finding| {
9503                finding.kind == FindingKind::Shortfall && finding.subject == "Account"
9504            }),
9505            "{review}"
9506        );
9507    }
9508
9509    // -----------------------------------------------------------------------
9510    // The recorded absolute path, and what became of it
9511    // -----------------------------------------------------------------------
9512
9513    #[test]
9514    fn a_recorded_path_that_is_still_there_is_current() {
9515        let host = Host::new();
9516        let state = inspect_binary(&host.binary, Some(&host.binary));
9517        assert!(!state.is_error(), "{state}");
9518        assert!(matches!(state, BinaryPath::Current { .. }), "{state}");
9519    }
9520
9521    #[test]
9522    fn the_npm_upgrade_case_reports_a_stale_path_as_an_error() {
9523        let host = Host::new();
9524        // An `npm i -g` binary lives under the active Node installation's
9525        // global prefix. Switching Node versions moves the prefix, and the
9526        // recorded path stops existing while the registration survives.
9527        let recorded = host.binary.clone();
9528        let healthy = inspect_binary(&recorded, Some(&recorded));
9529        assert!(
9530            !healthy.is_error(),
9531            "the discriminator: before the binary moves, this must be healthy"
9532        );
9533
9534        std::fs::remove_file(&recorded).expect("the binary moves out from under the record");
9535
9536        let state = inspect_binary(&recorded, Some(&recorded));
9537        assert!(state.is_error(), "{state}");
9538        assert!(matches!(state, BinaryPath::Missing { .. }), "{state}");
9539        assert!(
9540            state.to_string().contains("npm"),
9541            "the message must name the cause an operator will not otherwise connect: {state}"
9542        );
9543    }
9544
9545    #[test]
9546    fn a_directory_at_the_recorded_path_is_not_something_the_manager_can_start() {
9547        let root = tempfile::tempdir().expect("a temporary directory");
9548        let state = inspect_binary(root.path(), None);
9549        assert!(state.is_error(), "{state}");
9550        assert!(matches!(state, BinaryPath::NotExecutable { .. }), "{state}");
9551    }
9552
9553    #[test]
9554    fn a_registration_naming_a_different_binary_is_a_divergence() {
9555        let host = Host::new();
9556        let other = host.binary.with_file_name("something-else");
9557        let state = inspect_binary(&host.binary, Some(&other));
9558        assert!(state.is_error(), "{state}");
9559        assert!(matches!(state, BinaryPath::Diverged { .. }), "{state}");
9560    }
9561
9562    #[test]
9563    fn absence_is_reported_before_divergence() {
9564        let host = Host::new();
9565        let recorded = host.binary.clone();
9566        std::fs::remove_file(&recorded).expect("removable");
9567        let other = recorded.with_file_name("something-else");
9568        // Both faults are true at once. The operator needs to hear about the
9569        // missing file, not about a disagreement between two paths of which one
9570        // does not exist.
9571        assert!(matches!(
9572            inspect_binary(&recorded, Some(&other)),
9573            BinaryPath::Missing { .. }
9574        ));
9575    }
9576
9577    // -----------------------------------------------------------------------
9578    // The record
9579    // -----------------------------------------------------------------------
9580
9581    #[test]
9582    fn the_record_round_trips_through_toml() {
9583        let host = Host::new();
9584        let plan = InstallPlan::resolve(
9585            ServiceIdentity::product(),
9586            &host.request(StartMode::Boot),
9587            ServiceDirectories::of(&host.paths),
9588        )
9589        .expect("a resolvable plan");
9590        let definition = ServiceDefinition::from_text(DefinitionKind::SystemdUnit, "[Service]\n");
9591        let record = InstallRecord::of(&plan, &definition, Utc::now());
9592        record.write(&host.paths).expect("a writable record");
9593        let read = InstallRecord::read(&host.paths)
9594            .expect("a readable record")
9595            .expect("a record is there");
9596        assert_eq!(read, record);
9597        assert_eq!(read.binary, host.binary);
9598        assert!(read.binary.is_absolute());
9599    }
9600
9601    #[test]
9602    fn a_unit_written_by_an_older_build_is_reported_and_a_current_one_is_not() {
9603        let host = Host::new();
9604        let identity = ServiceIdentity::product();
9605        let plan = InstallPlan::resolve(
9606            identity.clone(),
9607            &host.request(StartMode::Boot),
9608            ServiceDirectories::of(&host.paths),
9609        )
9610        .expect("a resolvable plan");
9611        let installed = systemd_unit(&plan);
9612        let definition =
9613            ServiceDefinition::from_text(DefinitionKind::SystemdUnit, installed.clone());
9614        let mut record = InstallRecord::of(&plan, &definition, Utc::now());
9615        let path = PathBuf::from("/etc/systemd/system/runner-manager.service");
9616
9617        assert_eq!(
9618            systemd_definition_drift(&identity, &record, path.clone(), &installed),
9619            None,
9620            "the record must rebuild exactly the unit `service install` wrote"
9621        );
9622
9623        record.installed_by_version = "0.4.7".into();
9624        let frozen = installed.replace(
9625            "[Service]\n",
9626            "[Service]\nLoadCredential=runner-manager.user-access-token:/var/lib/runner-manager/secrets/user-access-token\n",
9627        );
9628        let drift = systemd_definition_drift(&identity, &record, path, &frozen)
9629            .expect("an outdated unit is drift");
9630        assert_eq!(drift.installed_by_version, "0.4.7");
9631        assert_eq!(drift.rendered, installed);
9632
9633        let status = ServiceStatus::compose(identity, None, None, None, None, &host.paths)
9634            .with_definition_drift(Some(drift));
9635        assert!(!status.is_healthy());
9636        assert!(
9637            status
9638                .problems()
9639                .iter()
9640                .any(|problem| problem.subject == "definition" && problem.detail.contains("0.4.7")),
9641            "{status}"
9642        );
9643    }
9644
9645    #[test]
9646    fn a_rejected_credential_keeps_its_first_moment_and_makes_the_service_unhealthy() {
9647        let host = Host::new();
9648        assert_eq!(github_credential_rejected_since(&host.paths).unwrap(), None);
9649        let first = Utc::now() - chrono::Duration::hours(9);
9650        record_github_credential_rejected(&host.paths, first).unwrap();
9651        record_github_credential_rejected(&host.paths, Utc::now()).unwrap();
9652        let since = github_credential_rejected_since(&host.paths).unwrap();
9653        assert_eq!(since, Some(first));
9654
9655        let status = ServiceStatus::compose(
9656            ServiceIdentity::product(),
9657            None,
9658            None,
9659            None,
9660            None,
9661            &host.paths,
9662        )
9663        .with_credential_rejection(since);
9664        assert!(!status.is_healthy());
9665        assert_eq!(status.credential_rejected_since(), Some(first));
9666        assert!(status.to_string().contains("auth login"), "{status}");
9667
9668        clear_github_credential_rejection(&host.paths).unwrap();
9669        clear_github_credential_rejection(&host.paths).unwrap();
9670        assert_eq!(github_credential_rejected_since(&host.paths).unwrap(), None);
9671    }
9672
9673    /// A guest that started nothing for 13 days while this said `healthy`.
9674    #[test]
9675    fn blocked_launches_make_the_service_unhealthy_with_their_remedy() {
9676        use crate::launch_health::{
9677            BlockCause, Viewpoint, clear_launches_blocked, launches_blocked,
9678            record_launches_blocked,
9679        };
9680
9681        let host = Host::new();
9682        let compose = || {
9683            ServiceStatus::compose(
9684                ServiceIdentity::product(),
9685                None,
9686                None,
9687                None,
9688                None,
9689                &host.paths,
9690            )
9691            .with_launches_blocked(launches_blocked(&host.paths, Utc::now()))
9692        };
9693        assert!(
9694            !compose()
9695                .problems()
9696                .iter()
9697                .any(|problem| problem.subject == LAUNCHES_BLOCKED_SUBJECT)
9698        );
9699        let blocked = BlockCause::FenceHeld {
9700            directory: PathBuf::from("launch-fence"),
9701            owner: None,
9702        }
9703        .blocked_since(Utc::now(), Viewpoint::Guest);
9704        record_launches_blocked(&host.paths, &blocked, Utc::now()).unwrap();
9705
9706        let status = compose();
9707        assert!(!status.is_healthy());
9708        assert_eq!(status.launches_blocked(), Some(&blocked));
9709        let problem = status
9710            .problems()
9711            .iter()
9712            .find(|problem| problem.subject == LAUNCHES_BLOCKED_SUBJECT)
9713            .expect("reported under its own subject");
9714        assert!(problem.detail.contains("systemctl restart"), "{status}");
9715
9716        clear_launches_blocked(&host.paths).unwrap();
9717        assert_eq!(compose().launches_blocked(), None);
9718    }
9719
9720    /// The daemon's own "I cannot read my credential", which used to exist
9721    /// only in its log while `service status` said `healthy`.
9722    #[test]
9723    fn an_unreadable_credential_keeps_its_first_moment_and_names_the_exact_command() {
9724        let host = Host::new();
9725        assert_eq!(credential_unreadable_since(&host.paths).unwrap(), None);
9726        let first = Utc::now() - chrono::Duration::minutes(40);
9727        record_credential_unreadable(&host.paths, first).unwrap();
9728        record_credential_unreadable(&host.paths, Utc::now()).unwrap();
9729        let since = credential_unreadable_since(&host.paths).unwrap();
9730        assert_eq!(since, Some(first), "a restart loop must not move the start");
9731
9732        let status = ServiceStatus::compose(
9733            ServiceIdentity::product(),
9734            None,
9735            None,
9736            None,
9737            None,
9738            &host.paths,
9739        )
9740        .with_credential_unreadable(since);
9741        assert!(!status.is_healthy());
9742        let problem = status
9743            .problems()
9744            .iter()
9745            .find(|problem| problem.subject == CREDENTIAL_UNREADABLE_SUBJECT)
9746            .expect("reported under its own subject");
9747        // No record, so the default start mode, whose exact command is named.
9748        assert!(
9749            problem
9750                .detail
9751                .contains(&sign_in_instruction(StartMode::default())),
9752            "{status}"
9753        );
9754
9755        clear_credential_unreadable(&host.paths).unwrap();
9756        clear_credential_unreadable(&host.paths).unwrap();
9757        assert_eq!(credential_unreadable_since(&host.paths).unwrap(), None);
9758        let healthy = ServiceStatus::compose(
9759            ServiceIdentity::product(),
9760            None,
9761            None,
9762            None,
9763            None,
9764            &host.paths,
9765        )
9766        .with_credential_unreadable(None);
9767        assert!(
9768            !healthy
9769                .problems()
9770                .iter()
9771                .any(|problem| problem.subject == CREDENTIAL_UNREADABLE_SUBJECT),
9772            "{healthy}"
9773        );
9774    }
9775
9776    #[test]
9777    fn the_sign_in_instruction_names_the_start_mode_it_writes() {
9778        assert!(sign_in_instruction(StartMode::Login).contains("auth login --start-at login`"));
9779        assert!(sign_in_instruction(StartMode::Boot).contains("auth login --start-at boot`"));
9780        assert_eq!(
9781            sign_in_instruction(StartMode::Boot).starts_with("`sudo "),
9782            cfg!(unix),
9783            "a machine-scoped store needs root on Unix"
9784        );
9785        assert_eq!(
9786            sign_in_instruction(StartMode::Login).contains("SSH"),
9787            cfg!(target_os = "macos"),
9788            "only the macOS login keychain is out of an SSH session's reach"
9789        );
9790    }
9791
9792    /// The record is readable by the account whose directory it is in.
9793    ///
9794    /// A boot-mode `service install` runs under `sudo`, and `NamedTempFile`
9795    /// creates at `0600`, so the record landed `root`-owned and unreadable by
9796    /// the operator — `service status` then failed on their own host with
9797    /// `Permission denied`. Ownership cannot be reproduced in an unprivileged
9798    /// test, but the mode that made ownership fatal can, and it is the half
9799    /// this code controls.
9800    #[cfg(unix)]
9801    #[test]
9802    fn the_record_is_not_written_readable_only_by_whoever_installed_it() {
9803        use std::os::unix::fs::PermissionsExt as _;
9804
9805        let host = Host::new();
9806        let plan = InstallPlan::resolve(
9807            ServiceIdentity::product(),
9808            &host.request(StartMode::Boot),
9809            ServiceDirectories::of(&host.paths),
9810        )
9811        .expect("a resolvable plan");
9812        let definition = ServiceDefinition::from_text(DefinitionKind::SystemdUnit, "[Service]\n");
9813        InstallRecord::of(&plan, &definition, Utc::now())
9814            .write(&host.paths)
9815            .expect("a writable record");
9816
9817        let mode = std::fs::metadata(InstallRecord::path(&host.paths))
9818            .expect("the record is there")
9819            .permissions()
9820            .mode()
9821            & 0o777;
9822        assert_eq!(
9823            mode, 0o644,
9824            "the record is mode {mode:04o}; at 0600 an operator cannot read a record `sudo \
9825             service install` wrote, and `service status` fails on their own host. It holds no \
9826             credential and sits in a 0700 directory, so 0644 discloses nothing"
9827        );
9828    }
9829
9830    /// A record written before the service ran a copy of its own must still
9831    /// load. It reads as `None`, which is the truth about it: that registration
9832    /// names the package manager's file and cannot be upgraded under itself.
9833    #[test]
9834    fn a_record_without_a_source_binary_still_reads_and_says_it_has_none() {
9835        let host = Host::new();
9836        let path = InstallRecord::path(&host.paths);
9837        std::fs::write(
9838            &path,
9839            format!(
9840                "schema_version = {RECORD_SCHEMA_VERSION}
9841service_name = \"runner-manager\"
9842                 manager = \"systemd\"
9843start_mode = \"boot\"
9844account = \"root\"
9845                 binary = \"/x\"
9846arguments = []
9847restart_delay_secs = 15
9848                 restart_reset_secs = 600
9849log_file = \"/x\"
9850                 installed_at = \"2026-01-01T00:00:00Z\"
9851installed_by_version = \"0.1.0\"
9852                 [directories]
9853config = \"/a\"
9854state = \"/b\"
9855runtime = \"/c\"
9856logs = \"/d\"
9857"
9858            ),
9859        )
9860        .expect("a writable record");
9861        let read = InstallRecord::read(&host.paths)
9862            .expect("a record missing an optional field is still readable")
9863            .expect("a record is there");
9864        assert_eq!(
9865            read.source_binary, None,
9866            "the legacy layout has no source, and must not invent one"
9867        );
9868    }
9869
9870    /// The field that makes an upgrade possible survives the write.
9871    #[test]
9872    fn a_registration_remembers_the_file_it_was_copied_from() {
9873        let host = Host::new();
9874        let source = host.binary.with_file_name("npm-installed-runner-manager");
9875        std::fs::copy(&host.binary, &source).expect("a second file to stand in for the package");
9876        let plan = InstallPlan::resolve(
9877            ServiceIdentity::product(),
9878            &host.request(StartMode::Boot).copied_from(&source),
9879            ServiceDirectories::of(&host.paths),
9880        )
9881        .expect("a resolvable plan");
9882        let definition = ServiceDefinition::from_text(
9883            DefinitionKind::SystemdUnit,
9884            "[Service]
9885",
9886        );
9887        let record = InstallRecord::of(&plan, &definition, Utc::now());
9888        record.write(&host.paths).expect("a writable record");
9889
9890        let read = InstallRecord::read(&host.paths)
9891            .expect("a readable record")
9892            .expect("a record is there");
9893        assert_eq!(read.source_binary.as_deref(), Some(source.as_path()));
9894        assert_ne!(
9895            read.source_binary.as_deref(),
9896            Some(read.binary.as_path()),
9897            "the whole point is that the two are different files: one the service holds open,              one the package manager is free to replace"
9898        );
9899    }
9900
9901    #[test]
9902    fn a_record_from_a_schema_this_build_cannot_read_is_refused_with_a_remedy() {
9903        let host = Host::new();
9904        let path = InstallRecord::path(&host.paths);
9905        std::fs::write(
9906            &path,
9907            format!(
9908                "schema_version = {}\nservice_name = \"runner-manager\"\nmanager = \"systemd\"\n\
9909                 start_mode = \"boot\"\naccount = \"root\"\nbinary = \"/x\"\narguments = []\n\
9910                 restart_delay_secs = 15\nrestart_reset_secs = 600\nlog_file = \"/x\"\n\
9911                 installed_at = \"2026-01-01T00:00:00Z\"\ninstalled_by_version = \"0.1.0\"\n\
9912                 [directories]\nconfig = \"/a\"\nstate = \"/b\"\nruntime = \"/c\"\nlogs = \"/d\"\n",
9913                RECORD_SCHEMA_VERSION + 1
9914            ),
9915        )
9916        .expect("a writable record");
9917        let error = InstallRecord::read(&host.paths).expect_err("a future schema is refused");
9918        assert!(
9919            matches!(error, ServiceError::RecordUnreadable { .. }),
9920            "{error}"
9921        );
9922        assert!(
9923            error.to_string().contains("service uninstall"),
9924            "the message must say how to recover: {error}"
9925        );
9926    }
9927
9928    #[test]
9929    fn no_record_is_not_an_error() {
9930        let host = Host::new();
9931        assert_eq!(InstallRecord::read(&host.paths).expect("no record"), None);
9932        assert!(!InstallRecord::remove(&host.paths).expect("nothing to remove"));
9933    }
9934
9935    // -----------------------------------------------------------------------
9936    // The last successful GitHub contact
9937    // -----------------------------------------------------------------------
9938
9939    #[test]
9940    fn no_heartbeat_reads_as_never_rather_than_as_the_epoch() {
9941        let host = Host::new();
9942        assert_eq!(last_github_contact(&host.paths).expect("readable"), None);
9943    }
9944
9945    #[test]
9946    fn the_heartbeat_round_trips_to_the_second() {
9947        let host = Host::new();
9948        let at = DateTime::parse_from_rfc3339("2026-08-22T10:11:12Z")
9949            .expect("a valid timestamp")
9950            .with_timezone(&Utc);
9951        record_github_contact(&host.paths, at).expect("a writable heartbeat");
9952        assert_eq!(
9953            last_github_contact(&host.paths).expect("readable"),
9954            Some(at)
9955        );
9956    }
9957
9958    #[test]
9959    fn a_malformed_heartbeat_is_an_error_and_not_silently_never() {
9960        let host = Host::new();
9961        std::fs::write(contact_path(&host.paths), b"this is not toml \x00").expect("writable");
9962        let error = last_github_contact(&host.paths)
9963            .expect_err("a heartbeat that cannot be parsed is not the same as no heartbeat");
9964        assert!(matches!(error, ServiceError::Record { .. }), "{error}");
9965    }
9966
9967    /// The surface the three-hour outage did not have.
9968    ///
9969    /// A root that refuses a launch does so before any attempt row exists, and
9970    /// the daemon's log scrubs the paths out of the sentence, so this file is
9971    /// the only place the directory and the remediation can reach the operator.
9972    #[test]
9973    fn a_runner_root_refusal_round_trips_for_service_status() {
9974        let host = Host::new();
9975        let at = DateTime::from_timestamp(1_760_000_000, 0).expect("a valid instant");
9976
9977        assert!(
9978            runner_root_refusals(&host.paths)
9979                .expect("readable")
9980                .is_empty(),
9981            "no record means every policy is placing runners"
9982        );
9983
9984        record_runner_root_refusal(
9985            &host.paths,
9986            "policy-a",
9987            at,
9988            "denied_by_privacy_policy",
9989            "/Volumes/NVME/runners",
9990            "the runner root /Volumes/NVME/runners cannot be used: ... Grant Full Disk Access",
9991        )
9992        .expect("a writable record");
9993
9994        let refusals = runner_root_refusals(&host.paths).expect("readable");
9995        assert_eq!(refusals.len(), 1);
9996        assert_eq!(refusals[0].policy, "policy-a");
9997        assert_eq!(refusals[0].at, at);
9998        assert_eq!(refusals[0].kind, "denied_by_privacy_policy");
9999        assert_eq!(refusals[0].root, "/Volumes/NVME/runners");
10000        assert!(
10001            refusals[0].detail.contains("/Volumes/NVME/runners")
10002                && refusals[0].detail.contains("Full Disk Access"),
10003            "the path and the remediation are the whole point of this file: {refusals:?}"
10004        );
10005    }
10006
10007    /// The regression that would put the outage straight back.
10008    ///
10009    /// A host runs several policies and they do not share a fate. When the
10010    /// record was one host-wide slot, the policy that placed a runner deleted
10011    /// the record of the policy that could not — on the same reconcile pass —
10012    /// and `service status` reported a healthy host that had started zero
10013    /// runners for an entire target.
10014    #[test]
10015    fn one_policy_placing_a_runner_does_not_clear_another_policys_refusal() {
10016        let host = Host::new();
10017        let at = DateTime::from_timestamp(1_760_000_000, 0).expect("a valid instant");
10018        record_runner_root_refusal(
10019            &host.paths,
10020            "broken",
10021            at,
10022            "denied_by_privacy_policy",
10023            "/Volumes/NVME/runners",
10024            "detail",
10025        )
10026        .expect("a writable record");
10027        record_runner_root_refusal(
10028            &host.paths,
10029            "also-broken",
10030            at,
10031            "not_writable",
10032            "/srv/other",
10033            "detail",
10034        )
10035        .expect("a writable record");
10036
10037        // The policy with its own working root succeeds and clears only itself.
10038        clear_runner_root_refusal(&host.paths, "healthy")
10039            .expect("clearing an absent policy is fine");
10040        clear_runner_root_refusal(&host.paths, "also-broken").expect("that policy recovered");
10041
10042        let refusals = runner_root_refusals(&host.paths).expect("readable");
10043        assert_eq!(
10044            refusals
10045                .iter()
10046                .map(|r| r.policy.as_str())
10047                .collect::<Vec<_>>(),
10048            vec!["broken"],
10049            "the policy that is still refused must keep its record"
10050        );
10051    }
10052
10053    /// Clearing is what keeps a fixed host from reporting a stale fault, and the
10054    /// file goes when the last entry does.
10055    #[test]
10056    fn clearing_the_last_refusal_removes_the_file_and_is_idempotent() {
10057        let host = Host::new();
10058        record_runner_root_refusal(
10059            &host.paths,
10060            "p",
10061            Utc::now(),
10062            "not_writable",
10063            "/srv/x",
10064            "detail",
10065        )
10066        .expect("a writable record");
10067
10068        clear_runner_root_refusal(&host.paths, "p").expect("the record is removed");
10069        assert!(
10070            !root_refusal_path(&host.paths).exists(),
10071            "a host with nothing refused leaves nothing behind"
10072        );
10073        clear_runner_root_refusal(&host.paths, "p").expect("removing what is gone is not an error");
10074    }
10075
10076    /// Same rule as the heartbeat: absence means "every policy is placing
10077    /// runners", so a record that cannot be parsed must not be reported as one.
10078    #[test]
10079    fn a_malformed_refusal_is_an_error_and_not_silently_none() {
10080        let host = Host::new();
10081        std::fs::write(root_refusal_path(&host.paths), b"not toml \x00").expect("writable");
10082        let error = runner_root_refusals(&host.paths)
10083            .expect_err("an unparseable record is not the same as no record");
10084        assert!(matches!(error, ServiceError::Record { .. }), "{error}");
10085    }
10086
10087    /// The half of this feature that turns the file into a diagnosis, and the
10088    /// half that had no test: `service status` must actually read it.
10089    ///
10090    /// A **note** and not a problem, deliberately. `service status` runs as
10091    /// whoever typed it, must stay readable on a host with nothing installed,
10092    /// and drives an exit code -- and nothing an operator can type clears this
10093    /// record, so a problem would fail that exit code on a host whose root was
10094    /// fixed but which has had no queued job since, with a printed remedy
10095    /// (`service uninstall && service install`) that does not touch `state/`.
10096    #[test]
10097    fn service_status_reports_a_refusal_as_a_note_and_stays_healthy() {
10098        let host = Host::new();
10099        record_runner_root_refusal(
10100            &host.paths,
10101            "policy-a",
10102            DateTime::from_timestamp(1_760_000_000, 0).expect("a valid instant"),
10103            "denied_by_privacy_policy",
10104            "/Volumes/NVME/runners",
10105            "Grant Full Disk Access to the program that runs the service",
10106        )
10107        .expect("a writable record");
10108
10109        let status = host.operations().status().expect("a readable status");
10110        let notes = status.notes().join("\n");
10111
10112        assert!(
10113            notes.contains("/Volumes/NVME/runners") && notes.contains("Full Disk Access"),
10114            "the directory and the remediation the log had to scrub must appear here: {notes}"
10115        );
10116        assert!(
10117            notes.contains("policy-a"),
10118            "the operator has to know which target placed no runner: {notes}"
10119        );
10120        assert!(
10121            !status
10122                .problems()
10123                .iter()
10124                .any(|problem| problem.subject == "runner root"),
10125            "a record nothing an operator types can clear must not drive the exit code"
10126        );
10127    }
10128
10129    // -----------------------------------------------------------------------
10130    // -----------------------------------------------------------------------
10131    // Install
10132    // -----------------------------------------------------------------------
10133
10134    #[test]
10135    fn install_records_the_absolute_binary_path_and_the_four_directories() {
10136        let host = Host::new();
10137        let installed = host
10138            .operations()
10139            .install(&host.request(StartMode::Boot))
10140            .expect("an install against the recording controls");
10141
10142        assert_eq!(installed.record.binary, host.binary);
10143        assert!(installed.record.binary.is_absolute());
10144        assert_eq!(installed.record.start_mode, StartMode::Boot);
10145        assert_eq!(installed.record.arguments, vec!["daemon", "run"]);
10146        assert_eq!(
10147            installed.record.directories,
10148            ServiceDirectories::of(&host.paths)
10149        );
10150        assert_eq!(
10151            installed.record.log_file,
10152            host.paths.logs_dir().join(LOG_FILE_STEM)
10153        );
10154        assert_eq!(installed.record.restart_delay_secs, 15);
10155
10156        let registrations = host.controls.registrations();
10157        assert_eq!(registrations.len(), 1);
10158        assert_eq!(registrations[0].0, StartMode::Boot);
10159        assert_eq!(registrations[0].1, "runner-manager");
10160    }
10161
10162    #[test]
10163    fn install_is_refused_while_the_single_instance_lock_is_held() {
10164        let host = Host::new();
10165        // The discriminator: with the lock free, the same call succeeds.
10166        {
10167            let operations = host.operations();
10168            operations
10169                .install(&host.request(StartMode::Boot))
10170                .expect("an install with the lock free");
10171            operations.uninstall().expect("a clean slate");
10172        }
10173
10174        let _held = HostLock::try_acquire(&host.paths, LockKind::SingleInstance)
10175            .expect("this process takes the lock first");
10176
10177        let error = host
10178            .operations()
10179            .install(&host.request(StartMode::Boot))
10180            .expect_err("a second agent must not be registered while one is running");
10181        assert!(matches!(error, ServiceError::LockHeld { .. }), "{error}");
10182        assert!(
10183            error.to_string().contains("already running"),
10184            "the message must be actionable: {error}"
10185        );
10186        assert!(
10187            host.controls.registrations().is_empty(),
10188            "a refused install must register nothing"
10189        );
10190        assert_eq!(
10191            InstallRecord::read(&host.paths).expect("readable"),
10192            None,
10193            "a refused install must write no record"
10194        );
10195    }
10196
10197    /// Installing over the same start mode replaces the registration.
10198    ///
10199    /// It used to be refused, and the refusal is what broke a real host: the
10200    /// caller has already swapped the binary the service runs by the time this
10201    /// is reached — that is the documented way to upgrade by hand — so refusing
10202    /// left a daemon running an unregistered binary and, on macOS, one the
10203    /// stored credential's keychain grant did not name.
10204    #[test]
10205    fn installing_over_the_same_start_mode_replaces_the_registration() {
10206        let host = Host::new();
10207        let operations = host.operations();
10208        operations
10209            .install(&host.request(StartMode::Boot))
10210            .expect("the first install");
10211
10212        let again = operations
10213            .install(&host.request(StartMode::Boot))
10214            .expect("an install over the same mode replaces rather than refusing");
10215
10216        assert!(
10217            again.replaced_existing,
10218            "the operator is told this replaced something rather than made it"
10219        );
10220        assert_eq!(
10221            host.controls.registrations().len(),
10222            1,
10223            "replacing must not leave two registrations behind"
10224        );
10225        assert_eq!(
10226            InstallRecord::read(&host.paths)
10227                .expect("readable")
10228                .expect("a record")
10229                .start_mode,
10230            StartMode::Boot
10231        );
10232    }
10233
10234    /// The *other* mode is still refused, because changing it moves the
10235    /// registration between two service managers and changes the account and
10236    /// the secret store with it. `set_start_mode` is what does that.
10237    #[test]
10238    fn installing_over_the_other_start_mode_is_refused() {
10239        let host = Host::new();
10240        let operations = host.operations();
10241        operations
10242            .install(&host.request(StartMode::Boot))
10243            .expect("the first install");
10244
10245        let error = operations
10246            .install(&host.request(StartMode::Login))
10247            .expect_err("a mode change is not an install");
10248        assert!(
10249            matches!(
10250                error,
10251                ServiceError::AlreadyInstalled {
10252                    existing: StartMode::Boot,
10253                    requested: StartMode::Login,
10254                    ..
10255                }
10256            ),
10257            "{error}"
10258        );
10259        assert!(
10260            !error
10261                .to_string()
10262                .contains("switch the start mode in place,"),
10263            "the old remedy named a capability no command offers; the terminal UI is where \
10264             the start mode moves: {error}"
10265        );
10266        assert_eq!(host.controls.registrations().len(), 1);
10267    }
10268
10269    #[test]
10270    fn install_rolls_back_the_registration_when_record_persistence_fails() {
10271        let host = Host::new();
10272        let record_path = InstallRecord::path(&host.paths);
10273        std::fs::create_dir(&record_path).expect("a directory blocks the record file");
10274
10275        let error = host
10276            .operations()
10277            .install(&host.request(StartMode::Boot))
10278            .expect_err("record persistence must fail");
10279
10280        assert!(matches!(error, ServiceError::Record { .. }), "{error}");
10281        assert!(
10282            host.controls.registrations().is_empty(),
10283            "a failed install must not leave a live unrecorded registration"
10284        );
10285        assert!(
10286            host.controls
10287                .calls()
10288                .iter()
10289                .any(|call| call == "uninstall runner-manager (boot)"),
10290            "the registration must be explicitly rolled back: {:?}",
10291            host.controls.calls()
10292        );
10293        assert!(
10294            !host.runner_root.as_path().exists(),
10295            "the rollback must take the runner root this install created with it; a directory \
10296             prepared for a registration that does not exist is litter, and on Windows it is \
10297             litter with a security descriptor"
10298        );
10299    }
10300
10301    // -----------------------------------------------------------------------
10302    // The runner root (b2)
10303    // -----------------------------------------------------------------------
10304
10305    #[test]
10306    fn the_runner_root_a_boot_registration_needs_admits_only_the_service() {
10307        use crate::runner_root_access::{RootAdmission, default_root_sddl, grants_broad_write};
10308
10309        // The mapping `prepare_runner_root` reads, asserted against the same
10310        // function the registration's own principal comes from — which is the
10311        // point of deriving it there rather than restating it.
10312        assert_eq!(
10313            ServiceAccount::for_definition(DefinitionKind::WindowsService, StartMode::Boot),
10314            ServiceAccount::LocalSystem
10315        );
10316        assert_eq!(
10317            ServiceAccount::for_definition(DefinitionKind::WindowsScheduledTask, StartMode::Login),
10318            ServiceAccount::InvokingUser
10319        );
10320
10321        let boot = default_root_sddl(&RootAdmission::LocalSystem);
10322        assert!(!grants_broad_write(&boot), "{boot}");
10323        assert!(
10324            !boot.contains("S-1-5-21"),
10325            "a boot registration runs as LocalSystem, so its root names no operator: {boot}"
10326        );
10327
10328        // A login task runs under a *filtered* token — `RunLevel` is
10329        // `LeastPrivilege`, see `windows_scheduled_task_xml` — in which
10330        // Administrators is deny-only. Without an ace of its own the account
10331        // the task runs as would be admitted by nothing at all.
10332        let login = default_root_sddl(&RootAdmission::Account("S-1-5-21-1-2-3-1001".to_owned()));
10333        assert!(login.contains("S-1-5-21-1-2-3-1001"), "{login}");
10334        assert!(!grants_broad_write(&login), "{login}");
10335    }
10336
10337    #[test]
10338    fn an_install_reports_the_runner_root_it_prepared() {
10339        let host = Host::new();
10340        let installed = host
10341            .operations()
10342            .install(&host.request(StartMode::Boot))
10343            .expect("an install");
10344        let rendered = installed.runner_root.to_string();
10345        assert!(
10346            !rendered.contains("S-1-5-21"),
10347            "the report must add no identity to the output: {rendered}"
10348        );
10349        if cfg!(windows) {
10350            assert_eq!(
10351                installed.runner_root.path(),
10352                Some(host.runner_root.as_path())
10353            );
10354            assert!(
10355                host.runner_root.as_path().is_dir(),
10356                "the directory jobs would run in has to exist once the service is registered"
10357            );
10358        } else {
10359            assert_eq!(
10360                installed.runner_root,
10361                crate::runner_root_access::RootAccessSummary::NotApplicable,
10362                "macOS and Linux keep the runtime directory they have always used"
10363            );
10364        }
10365    }
10366
10367    #[test]
10368    fn switching_start_mode_reconciles_the_runner_root_for_the_new_account() {
10369        let host = Host::new();
10370        let operations = host.operations();
10371        operations
10372            .install(&host.request(StartMode::Boot))
10373            .expect("an install at boot");
10374
10375        let change = operations
10376            .set_start_mode(StartMode::Login)
10377            .expect("a switch to login");
10378
10379        assert!(change.changed);
10380        if cfg!(windows) {
10381            assert_eq!(change.runner_root.path(), Some(host.runner_root.as_path()));
10382            assert!(
10383                host.runner_root.as_path().is_dir(),
10384                "the switch must not remove the directory it reconciled"
10385            );
10386        }
10387        // The operator is told, because the account that may write there has
10388        // moved with the mode.
10389        assert!(
10390            change.to_string().contains("runner root"),
10391            "{}",
10392            change.to_string()
10393        );
10394    }
10395
10396    #[test]
10397    fn switching_to_the_mode_already_in_force_touches_no_runner_root() {
10398        let host = Host::new();
10399        let operations = host.operations();
10400        operations
10401            .install(&host.request(StartMode::Boot))
10402            .expect("an install at boot");
10403
10404        let change = operations
10405            .set_start_mode(StartMode::Boot)
10406            .expect("a switch to the mode already in force");
10407
10408        assert!(!change.changed);
10409        assert_eq!(
10410            change.runner_root,
10411            crate::runner_root_access::RootAccessSummary::NotApplicable,
10412            "nothing moves, so nothing about the root's access control has to; reconciling here \
10413             would turn a no-op command into one that can fail on a permission it does not need"
10414        );
10415    }
10416
10417    #[cfg(windows)]
10418    #[test]
10419    fn a_registration_the_manager_refuses_leaves_no_runner_root_behind() {
10420        let host = Host::new();
10421        host.controls
10422            .fail_next_install(StartMode::Boot, "injected registration failure");
10423
10424        let error = host
10425            .operations()
10426            .install(&host.request(StartMode::Boot))
10427            .expect_err("the manager refuses the registration");
10428
10429        assert!(matches!(error, ServiceError::Control { .. }), "{error}");
10430        assert!(
10431            !host.runner_root.as_path().exists(),
10432            "the directory was created for a registration that does not exist"
10433        );
10434    }
10435
10436    #[cfg(windows)]
10437    #[test]
10438    fn an_existing_broad_runner_root_refuses_the_install_before_anything_is_registered() {
10439        let host = Host::new();
10440        // What a directory created below `C:\` with inheritance left on looks
10441        // like, built deliberately because a per-account `%TEMP%` never
10442        // produces one by accident.
10443        crate::runner_root_access::create_with_descriptor_for_tests(
10444            host.runner_root.as_path(),
10445            "D:P(A;OICI;FA;;;SY)(A;OICI;FA;;;WD)",
10446        )
10447        .expect("a deliberately open runner root");
10448        let before = crate::runner_root_access::report(host.runner_root.as_path());
10449
10450        let error = host
10451            .operations()
10452            .install(&host.request(StartMode::Boot))
10453            .expect_err("an open runner root is refused");
10454
10455        assert!(matches!(error, ServiceError::RunnerRoot { .. }), "{error}");
10456        assert!(
10457            error.to_string().contains("nothing was registered"),
10458            "{error}"
10459        );
10460        assert!(
10461            host.controls.registrations().is_empty(),
10462            "the refusal has to come before the platform is asked to register anything: {:?}",
10463            host.controls.calls()
10464        );
10465        assert_eq!(
10466            crate::runner_root_access::report(host.runner_root.as_path()),
10467            before,
10468            "an open directory is refused rather than tightened: its contents cannot be trusted, \
10469             so adopting it would be worse than declining it"
10470        );
10471    }
10472
10473    #[cfg(windows)]
10474    #[test]
10475    fn uninstall_leaves_the_runner_root_exactly_where_it_is() {
10476        let host = Host::new();
10477        let operations = host.operations();
10478        operations
10479            .install(&host.request(StartMode::Boot))
10480            .expect("an install");
10481        assert!(host.runner_root.as_path().is_dir());
10482
10483        operations.uninstall().expect("an uninstall");
10484
10485        assert!(
10486            host.runner_root.as_path().is_dir(),
10487            "`05-infrastructure.md` item 5: uninstall deregisters and deletes nothing else. A \
10488             runner root may hold an operator's retained workspaces."
10489        );
10490    }
10491
10492    #[test]
10493    fn install_reviews_what_it_registered() {
10494        let host = Host::new();
10495        let installed = host
10496            .operations()
10497            .install(&host.request(StartMode::Boot))
10498            .expect("an install");
10499        assert!(
10500            installed.review.is_least_privilege(),
10501            "{}",
10502            installed.review
10503        );
10504        assert!(
10505            !installed.review.controls().is_empty(),
10506            "a review that confirms nothing proves nothing: {}",
10507            installed.review
10508        );
10509        assert_eq!(
10510            installed.review.kind(),
10511            host_definition_kind(StartMode::Boot),
10512            "the review must be of the definition this host's manager was given"
10513        );
10514        assert!(
10515            !installed.review.account().justification().is_empty(),
10516            "a privileged account with no stated reason is an unreviewed one"
10517        );
10518    }
10519
10520    // -----------------------------------------------------------------------
10521    // Uninstall deletes nothing else
10522    // -----------------------------------------------------------------------
10523
10524    #[test]
10525    fn uninstall_leaves_configuration_sqlite_secrets_and_cache_exactly_as_they_were() {
10526        let host = Host::new();
10527        let operations = host.operations();
10528        operations
10529            .install(&host.request(StartMode::Boot))
10530            .expect("an install");
10531
10532        // The files `05-infrastructure.md` says must survive. Written *after*
10533        // the install so that the snapshot below is of a host in the state an
10534        // operator's would be in.
10535        let config = host.paths.config_dir();
10536        std::fs::write(config.join("runner-manager.db"), b"sqlite fixture").expect("writable");
10537        std::fs::write(config.join("config.toml"), b"host_capacity = 2").expect("writable");
10538        std::fs::create_dir_all(host.paths.state_dir().join("packages/2.330.0")).expect("writable");
10539        std::fs::write(
10540            host.paths
10541                .state_dir()
10542                .join("packages/2.330.0/runner.tar.gz"),
10543            b"cached package",
10544        )
10545        .expect("writable");
10546        std::fs::create_dir_all(host.paths.state_dir().join("secrets")).expect("writable");
10547        std::fs::write(
10548            host.paths.state_dir().join("secrets/user-access-token"),
10549            b"a stand-in for the stored credential",
10550        )
10551        .expect("writable");
10552        std::fs::write(
10553            host.paths.logs_dir().join("runner-manager.log.2026-08-22"),
10554            b"diagnostics",
10555        )
10556        .expect("writable");
10557
10558        let roots: Vec<PathBuf> = host
10559            .paths
10560            .all()
10561            .iter()
10562            .map(|(_, path)| (*path).to_path_buf())
10563            .collect();
10564        let roots: Vec<&Path> = roots.iter().map(PathBuf::as_path).collect();
10565        let before = snapshot(&roots);
10566
10567        // Non-vacuity: a comparison of two empty maps would pass whatever
10568        // `uninstall` did.
10569        assert!(
10570            before.len() >= 6,
10571            "the fixture must actually contain the files this test is about, got {before:#?}"
10572        );
10573        let record_path = InstallRecord::path(&host.paths);
10574        assert!(
10575            before.contains_key(&record_path),
10576            "the install record must be present before uninstall"
10577        );
10578
10579        let uninstalled = operations.uninstall().expect("an uninstall");
10580        assert!(uninstalled.removed_registration);
10581        assert!(uninstalled.removed_record);
10582
10583        let after = snapshot(&roots);
10584
10585        // Exactly one thing changed, and it is the registration's own record.
10586        let mut expected = before.clone();
10587        expected.remove(&record_path);
10588        assert_eq!(
10589            after, expected,
10590            "uninstall must remove its own record and nothing else"
10591        );
10592        assert!(
10593            !record_path.exists(),
10594            "the record itself must go, or `uninstall` did nothing at all"
10595        );
10596        assert!(
10597            uninstalled
10598                .preserved
10599                .iter()
10600                .all(|path| roots.contains(&path.as_path())),
10601            "the preserved list must name the four directories: {uninstalled}"
10602        );
10603    }
10604
10605    #[test]
10606    fn uninstall_on_a_host_with_no_registration_is_not_a_failure() {
10607        let host = Host::new();
10608        let uninstalled = host.operations().uninstall().expect("a no-op uninstall");
10609        assert!(!uninstalled.removed_registration);
10610        assert!(!uninstalled.removed_record);
10611    }
10612
10613    #[test]
10614    fn uninstall_removes_a_registration_even_when_the_record_is_gone() {
10615        let host = Host::new();
10616        let operations = host.operations();
10617        operations
10618            .install(&host.request(StartMode::Boot))
10619            .expect("an install");
10620        std::fs::remove_file(InstallRecord::path(&host.paths)).expect("the record is lost");
10621
10622        let uninstalled = operations.uninstall().expect("an uninstall");
10623        assert!(
10624            uninstalled.removed_registration,
10625            "a lost record must not strand a registration"
10626        );
10627        assert!(host.controls.registrations().is_empty());
10628    }
10629
10630    // -----------------------------------------------------------------------
10631    // Switching start mode
10632    // -----------------------------------------------------------------------
10633
10634    #[test]
10635    fn switching_start_mode_reuses_the_recorded_path_and_re_resolves_nothing() {
10636        let host = Host::new();
10637        let operations = host.operations();
10638        operations
10639            .install(&host.request(StartMode::Boot))
10640            .expect("an install at boot");
10641
10642        // The discriminator. If the switch re-resolved the binary the way
10643        // `install` does, it would either fail here or silently record the test
10644        // binary instead. Removing the file makes the difference visible.
10645        std::fs::remove_file(&host.binary).expect("the installed binary goes away");
10646
10647        let change = operations
10648            .set_start_mode(StartMode::Login)
10649            .expect("a switch that does not reinstall the product");
10650        assert!(change.changed);
10651        assert_eq!(change.from, StartMode::Boot);
10652        assert_eq!(change.to, StartMode::Login);
10653        assert_eq!(change.store_scope, crate::secrets::SecretScope::User);
10654
10655        let record = InstallRecord::read(&host.paths)
10656            .expect("readable")
10657            .expect("a record");
10658        assert_eq!(record.start_mode, StartMode::Login);
10659        assert_eq!(
10660            record.binary, host.binary,
10661            "the recorded path must survive the switch untouched"
10662        );
10663
10664        let registrations = host.controls.registrations();
10665        assert_eq!(registrations.len(), 1, "{registrations:?}");
10666        assert_eq!(registrations[0].0, StartMode::Login);
10667        assert!(
10668            registrations[0]
10669                .2
10670                .command_line
10671                .contains(&host.binary.to_string_lossy().into_owned()),
10672            "{:?}",
10673            registrations[0].2
10674        );
10675    }
10676
10677    #[test]
10678    fn switching_start_mode_keeps_the_live_registration_when_target_install_fails() {
10679        let host = Host::new();
10680        let operations = host.operations();
10681        operations
10682            .install(&host.request(StartMode::Boot))
10683            .expect("an install at boot");
10684        let record_before = std::fs::read(InstallRecord::path(&host.paths)).expect("the record");
10685        host.controls
10686            .fail_next_install(StartMode::Login, "injected target failure");
10687
10688        let error = operations
10689            .set_start_mode(StartMode::Login)
10690            .expect_err("the target manager refuses the install");
10691
10692        assert!(matches!(error, ServiceError::Control { .. }), "{error}");
10693        assert_eq!(
10694            std::fs::read(InstallRecord::path(&host.paths)).expect("the old record survives"),
10695            record_before
10696        );
10697        let registrations = host.controls.registrations();
10698        assert_eq!(registrations.len(), 1, "{registrations:?}");
10699        assert_eq!(registrations[0].0, StartMode::Boot);
10700    }
10701
10702    #[test]
10703    fn switching_start_mode_rolls_back_target_when_record_persistence_fails() {
10704        let host = Host::new();
10705        let operations = host.operations();
10706        operations
10707            .install(&host.request(StartMode::Boot))
10708            .expect("an install at boot");
10709        let record_before = std::fs::read(InstallRecord::path(&host.paths)).expect("the record");
10710        let config = host.paths.config_dir().to_path_buf();
10711        let hidden = config.with_file_name("config-hidden-by-fault");
10712        host.controls.hide_directory_after_install(
10713            StartMode::Login,
10714            config.clone(),
10715            hidden.clone(),
10716        );
10717
10718        let error = operations
10719            .set_start_mode(StartMode::Login)
10720            .expect_err("the injected filesystem fault prevents persistence");
10721
10722        std::fs::remove_file(&config).expect("remove the injected blocker");
10723        std::fs::rename(&hidden, &config).expect("restore the record directory");
10724        assert!(matches!(error, ServiceError::Record { .. }), "{error}");
10725        assert_eq!(
10726            std::fs::read(InstallRecord::path(&host.paths)).expect("the old record survives"),
10727            record_before
10728        );
10729        let registrations = host.controls.registrations();
10730        assert_eq!(registrations.len(), 1, "{registrations:?}");
10731        assert_eq!(registrations[0].0, StartMode::Boot);
10732        assert!(
10733            host.controls
10734                .calls()
10735                .iter()
10736                .any(|call| call == "uninstall runner-manager (login)"),
10737            "the target must be rolled back: {:?}",
10738            host.controls.calls()
10739        );
10740    }
10741
10742    #[test]
10743    fn switching_to_the_mode_already_in_force_registers_nothing_again() {
10744        let host = Host::new();
10745        let operations = host.operations();
10746        operations
10747            .install(&host.request(StartMode::Boot))
10748            .expect("an install");
10749        let before = host.controls.calls().len();
10750
10751        let change = operations
10752            .set_start_mode(StartMode::Boot)
10753            .expect("a no-op switch");
10754        assert!(!change.changed);
10755        assert_eq!(
10756            host.controls.calls().len(),
10757            before,
10758            "a no-op switch must not touch the service manager"
10759        );
10760    }
10761
10762    #[test]
10763    fn switching_start_mode_on_a_host_with_no_registration_is_refused() {
10764        let host = Host::new();
10765        let error = host
10766            .operations()
10767            .set_start_mode(StartMode::Login)
10768            .expect_err("there is nothing to switch");
10769        assert!(
10770            matches!(error, ServiceError::NotInstalled { .. }),
10771            "{error}"
10772        );
10773    }
10774
10775    // -----------------------------------------------------------------------
10776    // Status
10777    // -----------------------------------------------------------------------
10778
10779    #[test]
10780    fn status_reports_the_four_facts_journey_five_asks_for() {
10781        let host = Host::new();
10782        let operations = host.operations();
10783        operations
10784            .install(&host.request(StartMode::Boot))
10785            .expect("an install");
10786        let at = DateTime::parse_from_rfc3339("2026-08-22T09:00:00Z")
10787            .expect("a valid timestamp")
10788            .with_timezone(&Utc);
10789        record_github_contact(&host.paths, at).expect("a heartbeat");
10790
10791        let status = operations.status().expect("a status");
10792        assert_eq!(status.start_mode(), Some(StartMode::Boot));
10793        assert_eq!(
10794            status.binary().map(BinaryPath::recorded),
10795            Some(host.binary.as_path())
10796        );
10797        assert_eq!(status.log_file(), host.paths.logs_dir().join(LOG_FILE_STEM));
10798        assert_eq!(status.last_github_contact(), Some(at));
10799        assert!(status.is_installed());
10800        assert!(status.is_healthy(), "{status}");
10801
10802        let printed = status.to_string();
10803        for fragment in [
10804            "start mode",
10805            "diagnostic log",
10806            "last GitHub contact",
10807            "binary",
10808        ] {
10809            assert!(printed.contains(fragment), "{printed}");
10810        }
10811    }
10812
10813    /// A record this account may not read is reported, and `status` keeps
10814    /// going.
10815    ///
10816    /// It used to end the command with `Permission denied`, which is exactly
10817    /// what an operator met on a boot-mode host: `sudo service install` wrote
10818    /// the record `0600` and `root`-owned, and `service status` — a command
10819    /// that asks no privilege of anybody — then printed nothing at all about a
10820    /// registration the service manager could describe perfectly well.
10821    ///
10822    /// Ownership cannot be reproduced without privileges; the mode that made it
10823    /// fatal can.
10824    #[cfg(unix)]
10825    #[test]
10826    fn status_reports_a_record_it_may_not_read_and_still_reports_the_registration() {
10827        use std::os::unix::fs::PermissionsExt as _;
10828
10829        // `root` is not subject to the mode bits and would read the record
10830        // regardless, leaving the assertions below testing nothing.
10831        // SAFETY: `geteuid` takes no argument and cannot fail.
10832        if unsafe { libc::geteuid() } == 0 {
10833            return;
10834        }
10835
10836        let host = Host::new();
10837        let operations = host.operations();
10838        operations
10839            .install(&host.request(StartMode::Boot))
10840            .expect("an install");
10841        assert!(
10842            operations.status().expect("a status").is_healthy(),
10843            "the discriminator: healthy before the record is made unreadable"
10844        );
10845
10846        let record = InstallRecord::path(&host.paths);
10847        std::fs::set_permissions(&record, std::fs::Permissions::from_mode(0o000))
10848            .expect("the mode is applied");
10849
10850        let status = operations
10851            .status()
10852            .expect("a record this account may not read is reported, not thrown");
10853        assert!(status.is_installed(), "{status}");
10854        assert!(!status.is_healthy(), "{status}");
10855
10856        let printed = status.to_string();
10857        assert!(
10858            printed.contains("this account may not read it"),
10859            "the operator is told which of the two states this is: {printed}"
10860        );
10861        assert!(
10862            !printed.contains("there is no install record"),
10863            "a record that is there and unreadable is not a record that is missing, and the \
10864             missing one's remedy starts with `service uninstall`: {printed}"
10865        );
10866
10867        // Left readable so the fixture's own cleanup is not the thing that
10868        // fails on a host where TempDir removal walks the tree.
10869        std::fs::set_permissions(&record, std::fs::Permissions::from_mode(0o644))
10870            .expect("the mode is restored");
10871    }
10872
10873    #[test]
10874    fn status_reports_a_stale_binary_as_an_error_rather_than_appearing_healthy() {
10875        let host = Host::new();
10876        let operations = host.operations();
10877        operations
10878            .install(&host.request(StartMode::Boot))
10879            .expect("an install");
10880
10881        // The discriminator: healthy first, so a `is_healthy` that always
10882        // returned false could not pass this test.
10883        assert!(
10884            operations.status().expect("a status").is_healthy(),
10885            "the freshly installed host must be healthy"
10886        );
10887
10888        std::fs::remove_file(&host.binary).expect("the binary moves out from under the record");
10889
10890        let status = operations.status().expect("a status");
10891        assert!(!status.is_healthy(), "{status}");
10892        assert!(
10893            status
10894                .problems()
10895                .iter()
10896                .any(|problem| problem.subject == "binary"),
10897            "{status}"
10898        );
10899        assert!(status.to_string().contains("STALE"), "{status}");
10900    }
10901
10902    #[test]
10903    fn status_reports_a_registration_that_would_not_start_at_boot() {
10904        let host = Host::new();
10905        let operations = host.operations();
10906        operations
10907            .install(&host.request(StartMode::Boot))
10908            .expect("an install");
10909        assert!(operations.status().expect("a status").is_healthy());
10910
10911        host.controls.edit("runner-manager", |registration| {
10912            registration.starts_automatically = false;
10913        });
10914
10915        let status = operations.status().expect("a status");
10916        assert!(!status.is_healthy(), "{status}");
10917        assert!(
10918            status
10919                .problems()
10920                .iter()
10921                .any(|problem| problem.detail.contains("after a reboot")),
10922            "{status}"
10923        );
10924    }
10925
10926    #[test]
10927    fn install_starts_the_registration_and_a_later_stop_is_unhealthy() {
10928        let host = Host::new();
10929        let operations = host.operations();
10930        operations
10931            .install(&host.request(StartMode::Login))
10932            .expect("install and immediate start");
10933
10934        assert!(
10935            operations.status().expect("running status").is_running(),
10936            "install must not wait for the next login trigger"
10937        );
10938        operations.stop().expect("stop the registration");
10939        let status = operations.status().expect("stopped status");
10940        assert!(!status.is_healthy(), "{status}");
10941        assert!(
10942            status
10943                .problems()
10944                .iter()
10945                .any(|problem| problem.subject == "runtime"),
10946            "{status}"
10947        );
10948        let rendered = status.to_string();
10949        assert!(
10950            rendered.contains("runner-manager service install --start-at login"),
10951            "{rendered}"
10952        );
10953        assert!(
10954            !rendered.contains("runner-manager service start"),
10955            "{rendered}"
10956        );
10957    }
10958
10959    #[test]
10960    fn status_reports_a_restart_policy_something_else_edited() {
10961        let host = Host::new();
10962        let operations = host.operations();
10963        operations
10964            .install(&host.request(StartMode::Boot))
10965            .expect("an install");
10966        assert!(operations.status().expect("a status").is_healthy());
10967
10968        host.controls.edit("runner-manager", |registration| {
10969            registration.restart_delay = Some(Duration::from_secs(1));
10970        });
10971
10972        let status = operations.status().expect("a status");
10973        assert!(!status.is_healthy(), "{status}");
10974        assert!(
10975            status
10976                .problems()
10977                .iter()
10978                .any(|problem| problem.subject == "restart policy"),
10979            "{status}"
10980        );
10981    }
10982
10983    #[test]
10984    fn status_reports_a_registration_naming_a_binary_the_record_does_not() {
10985        let host = Host::new();
10986        let operations = host.operations();
10987        operations
10988            .install(&host.request(StartMode::Boot))
10989            .expect("an install");
10990        let other = host.binary.with_file_name("someone-elses.exe");
10991        std::fs::write(&other, b"x").expect("writable");
10992
10993        host.controls.edit("runner-manager", |registration| {
10994            registration.command_line = quote_argument(&other.to_string_lossy());
10995        });
10996
10997        let status = operations.status().expect("a status");
10998        assert!(!status.is_healthy(), "{status}");
10999        assert!(
11000            matches!(status.binary(), Some(BinaryPath::Diverged { .. })),
11001            "{status}"
11002        );
11003    }
11004
11005    #[test]
11006    fn status_reports_a_record_no_service_manager_knows_about() {
11007        let host = Host::new();
11008        let operations = host.operations();
11009        operations
11010            .install(&host.request(StartMode::Boot))
11011            .expect("an install");
11012        // Something removed the registration behind this product's back.
11013        for mode in [StartMode::Boot, StartMode::Login] {
11014            host.controls
11015                .control(mode)
11016                .expect("a control")
11017                .uninstall(&ServiceIdentity::product())
11018                .expect("removed");
11019        }
11020
11021        let status = operations.status().expect("a status");
11022        assert!(!status.is_healthy(), "{status}");
11023        assert!(
11024            status
11025                .problems()
11026                .iter()
11027                .any(|problem| problem.subject == "registration"),
11028            "{status}"
11029        );
11030    }
11031
11032    #[test]
11033    fn status_on_a_host_with_nothing_installed_is_neither_healthy_nor_broken() {
11034        let host = Host::new();
11035        let status = host.operations().status().expect("a status");
11036        assert!(!status.is_installed());
11037        assert!(
11038            status.is_healthy(),
11039            "a host that never installed the service has no fault to report: {status}"
11040        );
11041        assert!(status.to_string().contains("installed"), "{status}");
11042    }
11043
11044    #[test]
11045    fn status_says_a_login_registration_does_not_resume_after_an_unattended_reboot() {
11046        let host = Host::new();
11047        let operations = host.operations();
11048        operations
11049            .install(&host.request(StartMode::Login))
11050            .expect("an install at login");
11051        let status = operations.status().expect("a status");
11052        assert!(
11053            status
11054                .notes()
11055                .iter()
11056                .any(|note| note.contains("does not run until the operator signs in")),
11057            "05-infrastructure.md requires `service status` to say so: {status}"
11058        );
11059    }
11060
11061    /// A Mac that signs its runner account in automatically does resume, and
11062    /// `service status` used to tell it otherwise.
11063    #[test]
11064    fn a_mac_with_automatic_login_as_the_service_account_resumes_after_a_reboot() {
11065        use crate::unattended_login::{AutoLogin, UnattendedLogin};
11066        let automatic = UnattendedLogin {
11067            auto_login: AutoLogin::As("ivan".into()),
11068            filevault_on: Some(false),
11069        };
11070        assert_eq!(
11071            unattended_gap(StartMode::Login, "ivan", Some(&automatic)),
11072            None
11073        );
11074
11075        let off = UnattendedLogin {
11076            auto_login: AutoLogin::Off,
11077            filevault_on: Some(false),
11078        };
11079        let gap = unattended_gap(StartMode::Login, "ivan", Some(&off)).expect("a gap");
11080        assert!(gap.reason.contains("automatic login is off"), "{gap:?}");
11081        assert!(gap.reason.contains("Automatically log in as"), "{gap:?}");
11082
11083        let locked = UnattendedLogin {
11084            auto_login: AutoLogin::As("ivan".into()),
11085            filevault_on: Some(true),
11086        };
11087        let gap = unattended_gap(StartMode::Boot, "root", Some(&locked)).expect("a gap");
11088        assert!(gap.reason.contains("FileVault is on"), "{gap:?}");
11089
11090        // An unanswered probe is no gap: `host doctor` says what it could not
11091        // read.
11092        let unread = UnattendedLogin {
11093            auto_login: AutoLogin::As("ivan".into()),
11094            filevault_on: None,
11095        };
11096        assert_eq!(
11097            unattended_gap(StartMode::Login, "ivan", Some(&unread)),
11098            None
11099        );
11100
11101        // Where nothing can be probed, a login registration still says so.
11102        let gap = unattended_gap(StartMode::Login, "ivan", None).expect("a gap");
11103        assert!(
11104            gap.reason
11105                .contains("does not run until the operator signs in")
11106        );
11107        assert_eq!(unattended_gap(StartMode::Boot, "root", None), None);
11108    }
11109
11110    /// A daemon blocked in a file system call is still "running" to the
11111    /// service manager; its heartbeat is what says it is stuck.
11112    #[test]
11113    fn a_daemon_that_stopped_beating_makes_the_service_unhealthy() {
11114        let host = Host::new();
11115        let operations = host.operations();
11116        operations
11117            .install(&host.request(StartMode::Login))
11118            .expect("an install at login");
11119        let stalled_problem = |operations: &ServiceOperations| {
11120            operations
11121                .status()
11122                .expect("a status")
11123                .problems()
11124                .iter()
11125                .find(|problem| problem.subject == DAEMON_STALLED_SUBJECT)
11126                .map(|problem| problem.detail.clone())
11127        };
11128        crate::daemon_heartbeat::beat(&operations.paths, Utc::now()).unwrap();
11129        assert_eq!(stalled_problem(&operations), None, "beating");
11130
11131        crate::daemon_heartbeat::beat(
11132            &operations.paths,
11133            Utc::now() - chrono::Duration::minutes(20),
11134        )
11135        .unwrap();
11136        let detail = stalled_problem(&operations).expect("a stall is a problem");
11137        assert!(detail.contains("has made no progress since"), "{detail}");
11138        // The restart is the service manager's command for this login-mode
11139        // registration: the product has no `service stop` or `service start`.
11140        let restart = ServiceIdentity::product().restart_command(StartMode::Login);
11141        assert!(detail.contains(&restart), "{detail}");
11142        assert!(!detail.contains("runner-manager service stop"), "{detail}");
11143        assert!(!detail.contains("runner-manager service start"), "{detail}");
11144    }
11145
11146    /// Each platform's restart is its service manager's own command for the
11147    /// registration's start mode, never a `runner-manager service` subcommand
11148    /// that does not exist.
11149    #[test]
11150    fn the_restart_command_is_the_service_managers_own() {
11151        let identity = ServiceIdentity::product();
11152        let boot = identity.restart_command(StartMode::Boot);
11153        let login = identity.restart_command(StartMode::Login);
11154        for command in [&boot, &login] {
11155            assert!(!command.contains("runner-manager service"), "{command}");
11156            assert!(command.contains(identity.name()), "{command}");
11157        }
11158        assert_ne!(boot, login);
11159        if cfg!(windows) {
11160            assert!(
11161                boot.starts_with("Restart-Service -Name runner-manager"),
11162                "{boot}"
11163            );
11164            assert!(
11165                login.contains("schtasks /Run /TN runner-manager"),
11166                "{login}"
11167            );
11168        } else if cfg!(target_os = "macos") {
11169            assert!(
11170                boot.starts_with("sudo launchctl kickstart -k system/"),
11171                "{boot}"
11172            );
11173            assert!(login.starts_with("launchctl kickstart -k gui/"), "{login}");
11174        } else {
11175            assert_eq!(boot, "sudo systemctl restart runner-manager.service");
11176            assert_eq!(login, "systemctl --user restart runner-manager.service");
11177        }
11178    }
11179
11180    /// A daemon refusing every runner is not a healthy service.
11181    #[test]
11182    fn a_daemon_that_finds_the_host_unfit_makes_the_service_unhealthy() {
11183        let host = Host::new();
11184        let operations = host.operations();
11185        operations
11186            .install(&host.request(StartMode::Login))
11187            .expect("an install at login");
11188        let paths = operations.paths.clone();
11189        crate::host_fitness::record_host_unfit(
11190            &paths,
11191            &[crate::host_fitness::UnfitFinding {
11192                id: "host.runner_root_responsive".into(),
11193                detail: "macOS has not let the service list /Volumes/NVME/runners".into(),
11194                remedy: Some("click Allow".into()),
11195            }],
11196            Utc::now(),
11197        )
11198        .unwrap();
11199        let status = operations.status().expect("a status");
11200        assert!(!status.is_healthy(), "{status}");
11201        let problem = status
11202            .problems()
11203            .iter()
11204            .find(|problem| problem.subject == HOST_UNFIT_SUBJECT)
11205            .expect("the refusal is a problem");
11206        assert!(
11207            problem.detail.contains("/Volumes/NVME/runners"),
11208            "{problem:?}"
11209        );
11210        assert!(problem.detail.contains("Fix: click Allow"), "{problem:?}");
11211
11212        // A record nobody has re-stamped for a while is a stopped daemon's,
11213        // not a refusal in force.
11214        crate::host_fitness::clear_host_unfit(&paths).unwrap();
11215        crate::host_fitness::record_host_unfit(
11216            &paths,
11217            &[],
11218            Utc::now() - chrono::Duration::hours(2),
11219        )
11220        .unwrap();
11221        let status = operations.status().expect("a status");
11222        assert!(
11223            !status
11224                .problems()
11225                .iter()
11226                .any(|problem| problem.subject == HOST_UNFIT_SUBJECT),
11227            "{status}"
11228        );
11229    }
11230
11231    // -----------------------------------------------------------------------
11232    // Start and stop
11233    // -----------------------------------------------------------------------
11234
11235    #[test]
11236    fn start_and_stop_reach_the_domain_that_holds_the_registration() {
11237        let host = Host::new();
11238        let operations = host.operations();
11239        operations
11240            .install(&host.request(StartMode::Login))
11241            .expect("an install at login");
11242        operations.start().expect("a start");
11243        assert!(operations.status().expect("a status").is_running());
11244        assert!(operations.stop().expect("a stop"));
11245        assert!(!operations.status().expect("a status").is_running());
11246    }
11247
11248    #[test]
11249    fn starting_a_host_with_no_registration_is_refused() {
11250        let host = Host::new();
11251        let error = host.operations().start().expect_err("nothing to start");
11252        assert!(
11253            matches!(error, ServiceError::NotInstalled { .. }),
11254            "{error}"
11255        );
11256    }
11257
11258    // -----------------------------------------------------------------------
11259    // The seam with `d2`
11260    // -----------------------------------------------------------------------
11261
11262    /// Requirement 2 is *"an account that can **read the secret store**"*, and
11263    /// on Windows what an account may read is decided by a DACL in a file this
11264    /// task does not own. Asserting the account name alone would be asserting
11265    /// this module against itself; this reads `d2`'s protection back off a real
11266    /// store and checks that it admits the account this installer registers.
11267    ///
11268    /// No privileges are needed. `d2` documents that a rooted machine-scoped
11269    /// store *"is protected and encrypted exactly as the standard one is"* —
11270    /// the Windows backend picks its DACL from the scope, not from the site —
11271    /// so a store in a temporary directory carries the access control the real
11272    /// one does.
11273    #[cfg(windows)]
11274    #[test]
11275    fn the_account_this_installer_registers_is_one_the_stores_own_dacl_admits() {
11276        use crate::secrets::{PlatformSecretStore, SecretScope, SecretStore as _};
11277
11278        let root = tempfile::tempdir().expect("a temporary directory");
11279        let store = PlatformSecretStore::rooted_at(SecretScope::Machine, root.path())
11280            .expect("a rooted machine-scoped store");
11281        store
11282            .store(&secrecy::SecretString::from("a stand-in for the token"))
11283            .expect("the store accepts a value");
11284        let protection = store.protection().expect("the DACL can be read back");
11285
11286        assert!(
11287            protection.description().contains(";;;SY)"),
11288            "the machine-scoped store must admit LocalSystem, or a boot-start service cannot \
11289             read the token. `d2` writes this DACL and it is not this task's to widen. Got: {}",
11290            protection.description()
11291        );
11292        assert_eq!(
11293            ServiceAccount::for_definition(DefinitionKind::WindowsService, StartMode::Boot),
11294            ServiceAccount::LocalSystem,
11295            "and that is the account this installer registers, which is why SY is what matters"
11296        );
11297        assert!(
11298            !protection.readable_by_other_local_users(),
11299            "the same DACL must still exclude ordinary local users: {}",
11300            protection.description()
11301        );
11302
11303        // The other half of the analysis in `docs/service-account.md`, and the
11304        // reason LocalService and NetworkService are not used: the DACL names
11305        // three trustees and neither of them is among them.
11306        for rejected in [";;;LS)", ";;;NS)"] {
11307            assert!(
11308                !protection.description().contains(rejected),
11309                "if the store ever admitted {rejected}, the least-privilege analysis in \
11310                 docs/service-account.md would need redoing: {}",
11311                protection.description()
11312            );
11313        }
11314    }
11315
11316    // -- another installation's service ---------------------------------------
11317
11318    /// A request that records `host`'s own four directories on the command
11319    /// line, the way `service install` registers one.
11320    fn request_with_directories(host: &Host, mode: StartMode) -> InstallRequest {
11321        let mut arguments: Vec<OsString> = DAEMON_ARGUMENTS.iter().map(OsString::from).collect();
11322        arguments.extend(ServiceDirectories::of(&host.paths).arguments());
11323        host.request(mode).with_arguments(arguments)
11324    }
11325
11326    /// The operations a second account on the same machine gets: its own
11327    /// directories, and the machine's one service manager.
11328    fn second_account(account: &Host, machine: &Host) -> ServiceOperations {
11329        ServiceOperations::with_controls(
11330            account.paths.clone(),
11331            ServiceIdentity::product(),
11332            std::sync::Arc::new(machine.controls.clone()),
11333        )
11334        .with_runner_root(account.runner_root.clone())
11335    }
11336
11337    /// The owner's boot service, and a second account holding the record of
11338    /// an earlier login install of its own -- the layout an owner reported on
11339    /// a Windows host with two local accounts.
11340    fn a_machine_with_a_stale_second_account() -> (Host, Host) {
11341        let owner = Host::new();
11342        owner
11343            .operations()
11344            .install(&request_with_directories(&owner, StartMode::Boot))
11345            .expect("the owner's boot install");
11346        let earlier = Host::new();
11347        earlier
11348            .operations()
11349            .install(&request_with_directories(&earlier, StartMode::Login))
11350            .expect("the second account's earlier login install");
11351        assert!(
11352            InstallRecord::read(&earlier.paths).unwrap().is_some(),
11353            "the earlier install left its record behind"
11354        );
11355        (owner, earlier)
11356    }
11357
11358    /// The advice that, followed from the second account, breaks or replaces
11359    /// the owner's healthy service.
11360    const DESTRUCTIVE_ADVICE: [&str; 4] = [
11361        "service uninstall",
11362        "service install`",
11363        "service install --",
11364        "auth login",
11365    ];
11366
11367    #[test]
11368    fn a_service_installed_from_another_data_root_is_reported_as_that_installations() {
11369        let (owner, earlier) = a_machine_with_a_stale_second_account();
11370
11371        let status = second_account(&earlier, &owner).status().expect("a status");
11372
11373        let foreign = status.foreign().expect("the service is the owner's");
11374        assert_eq!(foreign.directories, ServiceDirectories::of(&owner.paths));
11375        assert_eq!(foreign.root, ServiceDirectories::of(&owner.paths).root());
11376        assert!(foreign.unreadable.is_none(), "{status}");
11377        assert!(
11378            status.is_healthy(),
11379            "the owner's service is healthy, and a stale record elsewhere is not its fault: \
11380             {status}"
11381        );
11382        for subject in [
11383            "binary",
11384            "start mode",
11385            "secret store",
11386            "record",
11387            "registration",
11388        ] {
11389            assert!(
11390                !status
11391                    .problems()
11392                    .iter()
11393                    .any(|problem| problem.subject == subject),
11394                "{subject} compared the second account's record against the owner's \
11395                 registration: {status}"
11396            );
11397        }
11398        let rendered = status.to_string();
11399        for advice in DESTRUCTIVE_ADVICE {
11400            assert!(!rendered.contains(advice), "{advice}: {rendered}");
11401        }
11402        assert!(rendered.contains("another installation's"), "{rendered}");
11403        assert!(
11404            status
11405                .notes()
11406                .iter()
11407                .any(|note| note.contains("unused local configuration")
11408                    && note.contains("host forget-local")),
11409            "{status}"
11410        );
11411        // What the owner recorded, not what the second account did.
11412        assert_eq!(status.start_mode(), Some(StartMode::Boot));
11413    }
11414
11415    #[test]
11416    fn a_foreign_service_whose_records_cannot_be_read_reports_only_what_the_manager_says() {
11417        let (owner, earlier) = a_machine_with_a_stale_second_account();
11418        // Stands in for a profile this account may not open: there is nothing
11419        // to read where the registration points.
11420        InstallRecord::remove(&owner.paths).expect("the owner's record removed");
11421
11422        let operations = second_account(&earlier, &owner);
11423        let status = operations.status().expect("a status");
11424        let foreign = status.foreign().expect("the service is the owner's");
11425        assert!(foreign.unreadable.is_some(), "{status}");
11426        assert!(
11427            status.is_healthy(),
11428            "running, and nothing it could judge: {status}"
11429        );
11430        let rendered = status.to_string();
11431        for advice in DESTRUCTIVE_ADVICE {
11432            assert!(!rendered.contains(advice), "{advice}: {rendered}");
11433        }
11434        assert!(rendered.contains("running; installed from"), "{rendered}");
11435        assert!(!rendered.contains("last GitHub contact"), "{rendered}");
11436
11437        // Stopped, it is a problem -- with a place to look, not a reinstall.
11438        owner.operations().stop().expect("stopped");
11439        let stopped = operations.status().expect("a status");
11440        assert!(!stopped.is_healthy(), "{stopped}");
11441        let rendered = stopped.to_string();
11442        for advice in DESTRUCTIVE_ADVICE {
11443            assert!(!rendered.contains(advice), "{advice}: {rendered}");
11444        }
11445    }
11446
11447    #[test]
11448    fn this_accounts_own_registration_still_reports_every_mismatch() {
11449        let host = Host::new();
11450        let operations = host.operations();
11451        operations
11452            .install(&request_with_directories(&host, StartMode::Boot))
11453            .expect("an install");
11454        let other = host.binary.with_file_name("someone-elses");
11455        std::fs::write(&other, b"x").expect("writable");
11456        // Something edited the registration, keeping its directories.
11457        host.controls.edit("runner-manager", |registration| {
11458            let mut arguments = split_command_line(&registration.command_line);
11459            arguments[0] = other.to_string_lossy().into_owned();
11460            registration.command_line = arguments
11461                .iter()
11462                .map(|argument| quote_argument(argument))
11463                .collect::<Vec<_>>()
11464                .join(" ");
11465            registration.start_mode = StartMode::Login;
11466        });
11467
11468        let status = operations.status().expect("a status");
11469        assert!(status.foreign().is_none(), "{status}");
11470        assert!(
11471            matches!(status.binary(), Some(BinaryPath::Diverged { .. })),
11472            "{status}"
11473        );
11474        assert!(
11475            status
11476                .problems()
11477                .iter()
11478                .any(|problem| problem.subject == "start mode"),
11479            "{status}"
11480        );
11481        assert!(status.to_string().contains("service uninstall"), "{status}");
11482    }
11483
11484    #[test]
11485    fn directories_read_back_from_a_command_line_with_spaces() {
11486        let root = Path::new("/Users/someone/Library/Application Support/runner manager");
11487        let directories = ServiceDirectories {
11488            config: root.join("config"),
11489            state: root.join("data").join("state"),
11490            runtime: root.join("data").join("runtime"),
11491            logs: root.join("data").join("logs"),
11492        };
11493        let mut command_line = quote_argument("/usr/local/bin/runner-manager");
11494        for argument in ["daemon", "run"]
11495            .map(OsString::from)
11496            .into_iter()
11497            .chain(directories.arguments())
11498        {
11499            command_line.push(' ');
11500            command_line.push_str(&quote_argument(&argument.to_string_lossy()));
11501        }
11502        assert_eq!(
11503            ServiceDirectories::from_command_line(&command_line),
11504            Some(directories.clone())
11505        );
11506        assert_eq!(directories.root(), root);
11507        assert_eq!(
11508            ServiceDirectories::from_command_line("runner-manager daemon run"),
11509            None,
11510            "a registration that names no directories cannot be judged foreign"
11511        );
11512    }
11513
11514    #[test]
11515    fn the_owner_of_a_data_root_is_read_from_its_path() {
11516        let users = Path::new(if cfg!(windows) { r"C:\Users" } else { "/home" });
11517        let home = users.join("ivand");
11518        assert_eq!(
11519            data_root_owner(&users.join("yuriv").join("data"), Some(&home)).as_deref(),
11520            Some("yuriv")
11521        );
11522        assert_eq!(
11523            data_root_owner(
11524                Path::new(if cfg!(windows) { r"D:\rm" } else { "/opt/rm" }),
11525                Some(&home)
11526            ),
11527            None
11528        );
11529        if cfg!(windows) {
11530            assert_eq!(
11531                data_root_owner(
11532                    Path::new(r"C:\Windows\System32\config\systemprofile\AppData"),
11533                    Some(&home)
11534                )
11535                .as_deref(),
11536                Some("LocalSystem")
11537            );
11538        } else {
11539            assert_eq!(
11540                data_root_owner(Path::new("/root/.config/runner-manager"), Some(&home)).as_deref(),
11541                Some("root")
11542            );
11543        }
11544    }
11545
11546    #[test]
11547    fn forget_local_archives_the_unused_record_and_never_touches_the_service_root() {
11548        let (owner, earlier) = a_machine_with_a_stale_second_account();
11549        let owner_before = snapshot(&[owner._root.path()]);
11550        let config = earlier.paths.config_dir().to_path_buf();
11551        let record = std::fs::read(InstallRecord::path(&earlier.paths)).unwrap();
11552
11553        let forgotten = second_account(&earlier, &owner)
11554            .forget_local(Utc::now())
11555            .expect("archived");
11556
11557        assert_eq!(
11558            snapshot(&[owner._root.path()]),
11559            owner_before,
11560            "the owner's directories must not change by one byte"
11561        );
11562        let archive = forgotten.archive.expect("something was archived");
11563        assert!(archive.starts_with(&config), "{}", archive.display());
11564        assert_eq!(
11565            std::fs::read(archive.join(RECORD_FILE)).unwrap(),
11566            record,
11567            "moved, not rewritten or deleted"
11568        );
11569        assert!(InstallRecord::read(&earlier.paths).unwrap().is_none());
11570        assert_eq!(
11571            forgotten.service_root,
11572            Some(ServiceDirectories::of(&owner.paths).root())
11573        );
11574        // Afterwards the second account's status has no unused record to note.
11575        let status = second_account(&earlier, &owner).status().expect("status");
11576        assert!(
11577            !status
11578                .notes()
11579                .iter()
11580                .any(|note| note.contains("unused local configuration")),
11581            "{status}"
11582        );
11583    }
11584
11585    #[test]
11586    fn forget_local_refuses_a_configuration_a_service_runs_against() {
11587        let (owner, earlier) = a_machine_with_a_stale_second_account();
11588        let before = snapshot(&[owner._root.path()]);
11589        let refused = owner.operations().forget_local(Utc::now());
11590        assert!(
11591            matches!(refused, Err(ServiceError::LocalConfigurationInUse { .. })),
11592            "{refused:?}"
11593        );
11594        assert_eq!(snapshot(&[owner._root.path()]), before);
11595
11596        // Nor while an agent holds the second account's own lock.
11597        let _agent = HostLock::try_acquire(&earlier.paths, LockKind::SingleInstance)
11598            .expect("the agent's lock");
11599        let refused = second_account(&earlier, &owner).forget_local(Utc::now());
11600        assert!(
11601            matches!(refused, Err(ServiceError::LocalConfigurationInUse { .. })),
11602            "{refused:?}"
11603        );
11604        assert!(InstallRecord::read(&earlier.paths).unwrap().is_some());
11605    }
11606
11607    #[test]
11608    fn a_readable_foreign_service_keeps_its_diagnoses_and_loses_its_remedies() {
11609        let (owner, earlier) = a_machine_with_a_stale_second_account();
11610        // The owner's service has every problem whose remedy acts on the
11611        // directories of whoever runs it.
11612        owner.operations().stop().expect("stopped");
11613        record_github_credential_rejected(&owner.paths, Utc::now()).expect("rejected");
11614        record_credential_unreadable(&owner.paths, Utc::now()).expect("unreadable");
11615
11616        let status = second_account(&earlier, &owner).status().expect("a status");
11617        assert!(status.foreign().is_some_and(|foreign| !foreign.is_unread()));
11618        let subjects: Vec<_> = status.problems().iter().map(|p| p.subject).collect();
11619        for subject in [
11620            "runtime",
11621            "github credential",
11622            CREDENTIAL_UNREADABLE_SUBJECT,
11623        ] {
11624            assert!(subjects.contains(&subject), "{subject}: {status}");
11625        }
11626        let rendered = status.to_string();
11627        for advice in DESTRUCTIVE_ADVICE {
11628            assert!(!rendered.contains(advice), "{advice}: {rendered}");
11629        }
11630        for problem in status.problems() {
11631            assert!(problem.detail.contains("Act on it"), "{problem}");
11632        }
11633    }
11634
11635    #[test]
11636    fn no_command_of_this_account_changes_another_installations_service() {
11637        let (owner, earlier) = a_machine_with_a_stale_second_account();
11638        let operations = second_account(&earlier, &owner);
11639        let foreign = |result: Result<(), ServiceError>| {
11640            assert!(
11641                matches!(result, Err(ServiceError::ForeignInstallation { .. })),
11642                "{result:?}"
11643            );
11644        };
11645        foreign(
11646            operations
11647                .install(&request_with_directories(&earlier, StartMode::Boot))
11648                .map(drop),
11649        );
11650        foreign(operations.uninstall().map(drop));
11651        foreign(operations.set_start_mode(StartMode::Boot).map(drop));
11652        foreign(operations.start());
11653        foreign(operations.stop().map(drop));
11654
11655        let registration = owner
11656            .controls
11657            .control(StartMode::Boot)
11658            .unwrap()
11659            .query(&ServiceIdentity::product())
11660            .unwrap()
11661            .expect("the owner's registration is still there");
11662        assert_eq!(
11663            ServiceDirectories::from_command_line(&registration.command_line),
11664            Some(ServiceDirectories::of(&owner.paths)),
11665            "and still runs against the owner's directories"
11666        );
11667        assert!(registration.running, "and was never stopped");
11668        assert!(
11669            InstallRecord::read(&earlier.paths).unwrap().is_some(),
11670            "the refused uninstall kept this account's record"
11671        );
11672    }
11673
11674    #[test]
11675    fn forget_local_leaves_agent_directories_nested_in_the_configuration_alone() {
11676        // macOS puts state, runtime and logs inside the configuration
11677        // directory: `config_local_dir` and `data_local_dir` are one folder.
11678        let root = tempfile::tempdir().unwrap();
11679        let config = root.path().join("Application Support");
11680        let paths = AppPaths::from_directories(
11681            &config,
11682            config.join("state"),
11683            config.join("runtime"),
11684            config.join("logs"),
11685        );
11686        paths.create_all().unwrap();
11687        std::fs::write(config.join(RECORD_FILE), b"an earlier install").unwrap();
11688        std::fs::write(paths.logs_dir().join("runner-manager.log"), b"log").unwrap();
11689        let operations = ServiceOperations::with_controls(
11690            paths.clone(),
11691            ServiceIdentity::product(),
11692            std::sync::Arc::new(RecordingControls::new()),
11693        );
11694
11695        let forgotten = operations.forget_local(Utc::now()).expect("archived");
11696
11697        assert_eq!(forgotten.moved, vec![config.join(RECORD_FILE)]);
11698        for kept in [paths.state_dir(), paths.runtime_dir(), paths.logs_dir()] {
11699            assert!(kept.is_dir(), "{} was moved", kept.display());
11700        }
11701        assert!(paths.logs_dir().join("runner-manager.log").is_file());
11702    }
11703
11704    #[test]
11705    fn a_remedy_is_cut_from_every_detail_a_builder_writes() {
11706        for (detail, diagnosis) in [
11707            (
11708                "x holds an automatic registration, but the daemon is stopped. Run `runner-manager \
11709                 service install --start-at boot`; if the service manager denies replacement",
11710                "x holds an automatic registration, but the daemon is stopped",
11711            ),
11712            (
11713                "store -- MISMATCH. Run `auth login` again so the token is stored",
11714                "store -- MISMATCH",
11715            ),
11716            (
11717                "unit differs, so the daemon runs under outdated settings. Re-register it with \
11718                 `sudo runner-manager service install --start-at boot`",
11719                "unit differs, so the daemon runs under outdated settings",
11720            ),
11721            (
11722                "it starts no runner; the log has it. Sign in again: `runner-manager auth login`",
11723                "it starts no runner; the log has it",
11724            ),
11725            (
11726                "C:\\a\\runner-manager.exe -- STALE: moved. Run `service uninstall` and `service \
11727                 install`",
11728                "C:\\a\\runner-manager.exe -- STALE: moved",
11729            ),
11730            (
11731                "a new builder: run `runner-manager service install` here, or not",
11732                "the installation's own records report this",
11733            ),
11734        ] {
11735            assert_eq!(diagnosis_only(detail), diagnosis, "{detail}");
11736        }
11737    }
11738}