Skip to main content

runner_manager_platform/
service.rs

1// owner: d3-service-installers
2
3//! Registering `daemon run` with the operating system, so that a home host
4//! resumes work by itself after a reboot with nobody logged in.
5//!
6//! Journey 5 is the whole of the requirement: *"the machine reboots with nobody
7//! logged in; the boot-start service starts the agent, which reads the user
8//! access token from the machine-scoped secret store"*. Everything in this file
9//! exists to make that sentence true on Windows, macOS, and Linux, and to make
10//! `service status` say so — or say precisely why not.
11//!
12//! # The shape: render, then apply
13//!
14//! A service manager is the one dependency a `cargo test` run cannot have. So
15//! this module is split in two, and the split is the reason most of it is
16//! testable on a developer's laptop whatever OS that laptop runs:
17//!
18//! * **Rendering** is pure. [`ServiceDefinition`] turns an [`InstallPlan`] into
19//!   the exact text the platform consumes — a systemd unit, a launchd property
20//!   list, a Task Scheduler XML document, or a canonical descriptor of the
21//!   Windows service parameters. No `cfg`, no privileges, no I/O. A Windows
22//!   developer renders and asserts the systemd unit; a Linux CI leg renders and
23//!   asserts the launchd plist.
24//! * **Applying** is behind [`ServiceControl`], one trait with a backend per
25//!   platform and a public in-memory double ([`RecordingControls`]) that `f3`
26//!   and this crate's own tests drive without touching the host.
27//!
28//! [`ServiceOperations`] is the layer above both, and it is where the logic
29//! that is *not* platform-specific lives: refusing an install while the
30//! single-instance lock is held, recording the resolved absolute binary path,
31//! detecting a stale one, switching start mode without reinstalling, and
32//! uninstalling without deleting a byte of configuration, secrets, or cache.
33//!
34//! # Two domains, not two products
35//!
36//! `--start-at boot` and `--start-at login` are the same daemon registered in
37//! two different *domains*:
38//!
39//! | | boot | login |
40//! |---|---|---|
41//! | Windows | a service in the Service Control Manager, `LocalSystem` | a Task Scheduler task with a logon trigger, running at `LeastPrivilege` |
42//! | macOS | a LaunchDaemon in `/Library/LaunchDaemons` | a LaunchAgent in `~/Library/LaunchAgents` |
43//! | Linux | a system unit in `/etc/systemd/system` | a user unit in `~/.config/systemd/user` |
44//!
45//! The Windows row is the one that is not symmetric, and it is worth saying why
46//! rather than leaving a reader to wonder. **Windows services cannot start at
47//! logon.** Service trigger-start covers domain join, an IP address becoming
48//! available, a device arriving, a firewall port event and a group-policy
49//! change; there is no logon trigger, and there is no user-session service
50//! type this product could use instead. Task Scheduler is the mechanism Windows
51//! actually provides for "run this when the operator signs in", so that is what
52//! `--start-at login` uses there. It is registered, inspected, and removed
53//! through the same [`ServiceControl`] trait as everything else, so nothing
54//! above this module has to know.
55//!
56//! # What the account can reach, and why it is not more
57//!
58//! `05-infrastructure.md` requires *"a least-privilege account that can read the
59//! machine-scoped secret store and write its configured cache and runtime
60//! directories"*. Those two clauses pull in opposite directions on every
61//! platform, and the resolution is recorded per platform in
62//! `docs/service-account.md` and checked by [`review_least_privilege`], which
63//! reads the rendered definition back and reports anything it grants beyond the
64//! requirement.
65//!
66//! The Windows resolution is the one that surprises people. `d2` protects the
67//! machine-scoped store with `D:P(A;;FA;;;SY)(A;;FA;;;BA)(A;;FA;;;OW)` and
68//! documents that the DACL *is* the access control, because a machine-scope
69//! DPAPI blob is decryptable by any process on the host. `NT AUTHORITY\
70//! LocalService` and `NT AUTHORITY\NetworkService` — the two accounts a
71//! "least privilege service" reflex reaches for — are named by none of those
72//! three ACEs and therefore **cannot read the token at all**. Widening the DACL
73//! to reach them would grant every service on the host read access to the one
74//! credential this product holds, which is strictly worse than running as
75//! `LocalSystem`. `secrets.rs` is not this task's file and is not widened; the
76//! service runs as `LocalSystem`, and [`review_least_privilege`] records the
77//! account together with the reason it is the minimum that satisfies the
78//! requirement rather than pretending it is small.
79//!
80//! # What is deliberately not claimed
81//!
82//! **A reboot is not something a test suite can have.** Every assertion here is
83//! about configuration a boot-time start depends on — the start type, the
84//! account, the recorded absolute path, the restart policy the service manager
85//! reports back — and about a store that is readable outside any login session,
86//! which `d2` proves separately. That the machine actually comes back up and
87//! the agent actually resumes is human gate 3 in `06-migration-rollout.md`, and
88//! nothing in this file is evidence for it.
89
90use std::collections::BTreeMap;
91use std::ffi::OsString;
92use std::fmt;
93use std::path::{Path, PathBuf};
94use std::time::Duration;
95
96use chrono::{DateTime, Utc};
97use runner_manager_domain::model::StartMode;
98use runner_manager_domain::path::LocalAbsolutePath;
99use serde::{Deserialize, Serialize};
100
101use crate::lock::{HostLock, LockError, LockKind};
102use crate::paths::AppPaths;
103#[cfg(windows)]
104use crate::runner_root_access::RootAdmission;
105use crate::runner_root_access::{
106    Reversal, RootAccessChange, RootAccessError, RootAccessReport, RootAccessSummary,
107};
108
109// ---------------------------------------------------------------------------
110// Names
111// ---------------------------------------------------------------------------
112
113/// The product's own service name, on every platform that wants a short one.
114pub const SERVICE_NAME: &str = "runner-manager";
115/// Hidden CLI marker carried only by Windows boot-service registrations.
116///
117/// Application-data arguments are shared by every service manager and cannot
118/// distinguish SCM from Task Scheduler. This marker is the durable routing
119/// contract between the Windows installer and the shipping binary.
120pub const WINDOWS_SCM_HOST_ARGUMENT: &str = "--windows-service-host";
121
122/// What an operator sees in `services.msc`, `launchctl list`, or
123/// `systemctl status`.
124pub const DISPLAY_NAME: &str = "GitHub Actions Runner Manager";
125
126/// One line of explanation, for the same three places.
127pub const DESCRIPTION: &str =
128    "Starts ephemeral GitHub Actions self-hosted runners on this machine on demand.";
129
130/// The arguments the installed command line carries.
131///
132/// `05-infrastructure.md`: *"`service install` registers `daemon run` for the
133/// current host"*. Spelled once, here, so the installer and `f3`'s command
134/// surface cannot disagree about what was registered.
135pub const DAEMON_ARGUMENTS: [&str; 2] = ["daemon", "run"];
136
137/// The file `install` writes and `status` reads, inside `config/`.
138///
139/// `config/` and not `state/`: `05-infrastructure.md` gives `config/` to
140/// *"non-secret TOML"*, and this is exactly that — a record of what was
141/// registered, holding no credential. It is also the one file `uninstall`
142/// removes, which is what keeps "uninstall deletes no configuration" honest:
143/// the record is not configuration, it is the registration's own footprint.
144pub const RECORD_FILE: &str = "service.toml";
145
146/// The file the daemon touches after every successful GitHub call, inside
147/// `state/`.
148///
149/// See [`record_github_contact`] for the contract; `service status` reads it
150/// and Journey 5 step 4 requires it.
151pub const CONTACT_FILE: &str = "github-contact.toml";
152
153/// Present while GitHub is rejecting the daemon's credential, inside `state/`.
154///
155/// See [`record_github_credential_rejected`] for the contract.
156pub const CREDENTIAL_REJECTION_FILE: &str = "github-credential-rejected.toml";
157
158/// Present while the daemon cannot read its own stored credential, inside
159/// `state/`.
160///
161/// See [`record_credential_unreadable`] for the contract.
162pub const CREDENTIAL_UNREADABLE_FILE: &str = "credential-unreadable.toml";
163
164/// The `service status` subject a [`CREDENTIAL_UNREADABLE_FILE`] is reported
165/// under, which the TUI matches to offer the same command.
166pub const CREDENTIAL_UNREADABLE_SUBJECT: &str = "stored credential";
167
168/// The `service status` subject blocked runner launches are reported under.
169pub const LAUNCHES_BLOCKED_SUBJECT: &str = "runner launches";
170
171/// The [`StatusProblem::subject`] of a daemon that has stopped making progress.
172pub const DAEMON_STALLED_SUBJECT: &str = "daemon progress";
173
174/// The [`StatusProblem::subject`] of a daemon that refuses to start runners
175/// because a required host check fails from its own session.
176pub const HOST_UNFIT_SUBJECT: &str = "host fitness";
177
178/// The agent's last runner-root refusal, for `service status` to report.
179///
180/// See [`record_runner_root_refusal`] for the contract.
181pub const ROOT_REFUSAL_FILE: &str = "runner-root-refusal.toml";
182
183/// The rotating diagnostic log the **daemon** writes, inside `logs/`.
184///
185/// The appender adds its own date suffix, so this is the stem rather than a
186/// file that exists. `service status` reports the directory and the stem,
187/// because that is what an operator needs in order to find today's file and
188/// yesterday's.
189///
190/// It is [`crate::logging::SERVICE_LOG_STEM`] and not the operator's, because a
191/// boot-mode daemon runs under a different account and writes a different file.
192/// [`crate::logging::LogRole`] says why the two were separated.
193pub const LOG_FILE_STEM: &str = crate::logging::SERVICE_LOG_STEM;
194
195// ---------------------------------------------------------------------------
196// Identity
197// ---------------------------------------------------------------------------
198
199/// What the operating system calls this registration.
200///
201/// A type rather than three constants, for one reason that is not tidiness: the
202/// privileged installer tests register a **real** service on a **real** machine,
203/// and they must not be able to collide with — or remove — an operator's
204/// installation. [`ServiceIdentity::fixture`] produces a name that is
205/// unmistakably a test artefact, and every backend takes its name from here, so
206/// there is no path by which a test reaches the product's own registration.
207#[derive(Debug, Clone, PartialEq, Eq)]
208pub struct ServiceIdentity {
209    name: String,
210    display_name: String,
211    description: String,
212}
213
214impl ServiceIdentity {
215    /// The registration an operator installs.
216    #[must_use]
217    pub fn product() -> Self {
218        Self {
219            name: SERVICE_NAME.to_string(),
220            display_name: DISPLAY_NAME.to_string(),
221            description: DESCRIPTION.to_string(),
222        }
223    }
224
225    /// A disposable registration for a privileged test, named so that it cannot
226    /// be mistaken for — or collide with — [`ServiceIdentity::product`].
227    ///
228    /// `tag` distinguishes concurrent runs; it is reduced to ASCII alphanumerics
229    /// and `-` so that it is a legal service name, launchd label, systemd unit
230    /// name, and Task Scheduler task name at once.
231    #[must_use]
232    pub fn fixture(tag: &str) -> Self {
233        let tag: String = tag
234            .chars()
235            .map(|c| {
236                if c.is_ascii_alphanumeric() {
237                    c.to_ascii_lowercase()
238                } else {
239                    '-'
240                }
241            })
242            .collect();
243        let name = format!("{SERVICE_NAME}-selftest-{tag}");
244        Self {
245            display_name: format!("{DISPLAY_NAME} (self-test fixture {tag})"),
246            description: "Disposable fixture created by runner-manager's own installer tests. \
247                          Safe to remove."
248                .to_string(),
249            name,
250        }
251    }
252
253    /// Whether this identity is a disposable test fixture.
254    ///
255    /// The privileged tests assert this before they delete anything, which is
256    /// the mechanical half of *"never touch a service you did not create"*.
257    #[must_use]
258    pub fn is_fixture(&self) -> bool {
259        self.name.starts_with(&format!("{SERVICE_NAME}-selftest-"))
260    }
261
262    /// The Windows service name, the systemd unit stem, and the Task Scheduler
263    /// task name.
264    #[must_use]
265    pub fn name(&self) -> &str {
266        &self.name
267    }
268
269    /// The human-readable name.
270    #[must_use]
271    pub fn display_name(&self) -> &str {
272        &self.display_name
273    }
274
275    /// One line of explanation.
276    #[must_use]
277    pub fn description(&self) -> &str {
278        &self.description
279    }
280
281    /// The launchd label — reverse-domain, as launchd expects.
282    ///
283    /// Built from the same three product segments [`crate::paths`] resolves the
284    /// application-data directories with, so a label and a directory cannot
285    /// drift apart.
286    #[must_use]
287    pub fn launchd_label(&self) -> String {
288        format!(
289            "{}.{}.{}",
290            crate::paths::QUALIFIER,
291            crate::paths::ORGANIZATION,
292            self.name
293        )
294    }
295
296    /// The systemd unit file name.
297    #[must_use]
298    pub fn systemd_unit(&self) -> String {
299        format!("{}.service", self.name)
300    }
301}
302
303impl fmt::Display for ServiceIdentity {
304    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
305        f.write_str(&self.name)
306    }
307}
308
309// ---------------------------------------------------------------------------
310// Restart policy
311// ---------------------------------------------------------------------------
312
313/// The restart-on-failure policy, and the bound on how fast it may retry.
314///
315/// `05-infrastructure.md` item 3: *"set a restart-on-failure policy with bounded
316/// delay"*. Both halves matter and they are different requirements. A service
317/// that does not come back after a crash defeats Journey 5; a service that comes
318/// back instantly, forever, turns one bad configuration into a fork bomb against
319/// GitHub's rate limit. So the delay has a floor as well as a ceiling, and the
320/// floor is what `does not restart-loop faster than that bound` is measured
321/// against.
322#[derive(Debug, Clone, Copy, PartialEq, Eq)]
323pub struct RestartPolicy {
324    delay: Duration,
325    reset_after: Duration,
326}
327
328impl RestartPolicy {
329    /// The shortest delay this will accept.
330    ///
331    /// One second is not a tuning preference. Below it, launchd's own throttle
332    /// and systemd's start-limit logic both take over and the configured number
333    /// stops being the number in force, so a policy under this floor would be a
334    /// value `service status` reported and the platform ignored.
335    pub const MIN_DELAY: Duration = Duration::from_secs(1);
336
337    /// The longest delay this will accept. Past five minutes an operator
338    /// watching a restart would reasonably conclude the service is simply gone.
339    pub const MAX_DELAY: Duration = Duration::from_secs(300);
340
341    /// What `service install` uses when nothing is said.
342    pub const DEFAULT_DELAY: Duration = Duration::from_secs(15);
343
344    /// How long the service must stay up before its failure count is forgotten.
345    pub const DEFAULT_RESET_AFTER: Duration = Duration::from_secs(600);
346
347    /// # Errors
348    ///
349    /// [`ServiceError::RestartDelay`] when `delay` is outside
350    /// [`RestartPolicy::MIN_DELAY`]..=[`RestartPolicy::MAX_DELAY`], or when
351    /// `reset_after` is not longer than `delay` — a reset window shorter than
352    /// the delay can never elapse between two restarts, so the failure count
353    /// would reset on every attempt and no start-limit could ever trip.
354    pub fn new(delay: Duration, reset_after: Duration) -> Result<Self, ServiceError> {
355        if delay < Self::MIN_DELAY || delay > Self::MAX_DELAY {
356            return Err(ServiceError::RestartDelay {
357                requested_secs: delay.as_secs(),
358                min_secs: Self::MIN_DELAY.as_secs(),
359                max_secs: Self::MAX_DELAY.as_secs(),
360            });
361        }
362        if reset_after <= delay {
363            return Err(ServiceError::RestartResetWindow {
364                reset_secs: reset_after.as_secs(),
365                delay_secs: delay.as_secs(),
366            });
367        }
368        Ok(Self { delay, reset_after })
369    }
370
371    /// How long the service manager waits before restarting a failed service.
372    #[must_use]
373    pub const fn delay(&self) -> Duration {
374        self.delay
375    }
376
377    /// How long the service must run before its failure count resets.
378    #[must_use]
379    pub const fn reset_after(&self) -> Duration {
380        self.reset_after
381    }
382
383    /// The delay a given manager can actually express.
384    ///
385    /// Three of the four take seconds and enforce exactly what they are given.
386    /// **Windows Task Scheduler does not**: `RestartOnFailure/Interval` is
387    /// expressed in whole minutes with a one-minute floor, and it *rejects* the
388    /// registration outright rather than rounding — `PT15S` comes back as
389    /// "The task XML contains a value which is incorrectly formatted or out of
390    /// range", which is how this was found.
391    ///
392    /// So the interval is rounded **up**, never down, and never below one
393    /// minute. The direction is the point: the requirement is that the service
394    /// *"does not restart-loop faster than that bound"*, and a delay longer
395    /// than the configured one still satisfies it, while a shorter one would
396    /// not. `service status` reports the difference as a note so that an
397    /// operator reading `15s` in the record and `60s` from the manager is told
398    /// why rather than left to wonder.
399    #[must_use]
400    pub const fn effective_delay(&self, kind: DefinitionKind) -> Duration {
401        match kind {
402            DefinitionKind::WindowsScheduledTask => {
403                let seconds = self.delay.as_secs();
404                let minutes = seconds.div_ceil(60);
405                Duration::from_secs(if minutes == 0 { 60 } else { minutes * 60 })
406            }
407            _ => self.delay,
408        }
409    }
410}
411
412impl Default for RestartPolicy {
413    fn default() -> Self {
414        Self {
415            delay: Self::DEFAULT_DELAY,
416            reset_after: Self::DEFAULT_RESET_AFTER,
417        }
418    }
419}
420
421impl fmt::Display for RestartPolicy {
422    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
423        write!(
424            f,
425            "restart after {}s, failure count resets after {}s",
426            self.delay.as_secs(),
427            self.reset_after.as_secs()
428        )
429    }
430}
431
432// ---------------------------------------------------------------------------
433// The account
434// ---------------------------------------------------------------------------
435
436/// The account a registration runs under.
437///
438/// Not an operator choice. It is a function of the start mode and the platform,
439/// because the start mode decides which secret store the daemon must read and
440/// the store's own access control decides which accounts can read it. See
441/// `docs/service-account.md`, and [`review_least_privilege`] for the check.
442#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
443#[serde(rename_all = "snake_case")]
444pub enum ServiceAccount {
445    /// `NT AUTHORITY\SYSTEM`. The only stock Windows account named by the
446    /// machine-scoped store's DACL.
447    LocalSystem,
448    /// `root`. What a LaunchDaemon and a systemd system unit run as, and what
449    /// the macOS System Keychain's root-only master key requires.
450    Root,
451    /// The operator's own account, for a login-mode registration.
452    InvokingUser,
453}
454
455impl ServiceAccount {
456    /// The account a given kind of definition obliges.
457    ///
458    /// Keyed on the definition rather than on `cfg!(windows)`, and the
459    /// difference is not academic: a Windows developer rendering the launchd
460    /// property list would otherwise write `UserName = NT AUTHORITY\SYSTEM`
461    /// into it, which is not an account macOS has. A definition's account is a
462    /// property of the definition, and the whole point of rendering being pure
463    /// is that any host can render any platform's and get the right answer.
464    #[must_use]
465    pub const fn for_definition(kind: DefinitionKind, mode: StartMode) -> Self {
466        match (kind, mode) {
467            (DefinitionKind::WindowsService, _) => Self::LocalSystem,
468            (DefinitionKind::WindowsScheduledTask, _) | (_, StartMode::Login) => Self::InvokingUser,
469            (_, StartMode::Boot) => Self::Root,
470        }
471    }
472
473    /// The account the given start mode obliges on the platform this binary was
474    /// built for.
475    #[must_use]
476    pub const fn for_start_mode(mode: StartMode) -> Self {
477        Self::for_definition(host_definition_kind(mode), mode)
478    }
479
480    /// How the platform spells it.
481    #[must_use]
482    pub const fn as_str(&self) -> &'static str {
483        match self {
484            Self::LocalSystem => "NT AUTHORITY\\SYSTEM",
485            Self::Root => "root",
486            Self::InvokingUser => "the invoking user",
487        }
488    }
489
490    /// Why this is the *minimum* account that can do the job, not merely the
491    /// convenient one.
492    ///
493    /// Printed by `service status` and by [`review_least_privilege`], because a
494    /// privileged account with no stated reason is indistinguishable from a
495    /// privileged account nobody thought about.
496    #[must_use]
497    pub const fn justification(&self) -> &'static str {
498        match self {
499            Self::LocalSystem => {
500                "the machine-scoped store's DACL names SY, BA and OW only; LocalService and \
501                 NetworkService cannot read it, and widening the DACL to reach them would grant \
502                 every service on this host read access to the one credential this product holds"
503            }
504            Self::Root => {
505                "a boot-time registration runs outside every login session: on macOS the System \
506                 Keychain is unlocked by /var/db/SystemKey, which is root-only, and on Linux the \
507                 machine-scoped store is a 0600 file under /var/lib that only root can open \
508                 before a session exists"
509            }
510            Self::InvokingUser => {
511                "a login-mode registration reads the user-scoped store, which is deliberately \
512                 readable by exactly one account and needs no elevation at all"
513            }
514        }
515    }
516
517    /// Whether registering under this account needs administrative rights.
518    #[must_use]
519    pub const fn needs_elevation(&self) -> bool {
520        matches!(self, Self::LocalSystem | Self::Root)
521    }
522}
523
524impl fmt::Display for ServiceAccount {
525    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
526        f.write_str(self.as_str())
527    }
528}
529
530// ---------------------------------------------------------------------------
531// Errors
532// ---------------------------------------------------------------------------
533
534/// Something went wrong installing, inspecting, or removing the registration.
535///
536/// Every variant is formatted straight into an operator-facing message, so each
537/// one says what to do next rather than only what failed. No variant carries a
538/// secret: this module never reads the token, only the *location* `d2` publishes
539/// for exactly this purpose.
540#[derive(Debug, thiserror::Error)]
541pub enum ServiceError {
542    /// The single-instance lock is held, so a second agent must not be
543    /// registered.
544    ///
545    /// `05-infrastructure.md` item 1. The message is `d1`'s, verbatim, because
546    /// `d1` already worked out what to tell an operator who lost that race.
547    #[error("cannot install the service while an agent is already running on this host: {source}")]
548    LockHeld {
549        /// What `d1` said, including who holds it.
550        #[source]
551        source: Box<LockError>,
552    },
553
554    /// The lock could not be inspected at all, which is not the same as it being
555    /// free.
556    #[error(
557        "cannot tell whether an agent is already running on this host, so the service was not \
558         installed: {source}. Fix the reported problem with the state directory and try again."
559    )]
560    LockUnreadable {
561        /// The underlying failure.
562        #[source]
563        source: Box<LockError>,
564    },
565
566    /// The runner root this registration would run jobs under could not be
567    /// created, inspected, or given the access the registration's account needs.
568    ///
569    /// Reported *before* anything is registered. A registration whose workspaces
570    /// would land in a directory ordinary local users can write is a
571    /// registration that should not exist, and `04-security-recovery.md` asks
572    /// for exactly that refusal rather than a warning.
573    #[error(
574        "this registration would run jobs under the default runner root, and that root could \
575         not be prepared, so nothing was registered: {source}"
576    )]
577    RunnerRoot {
578        /// What [`crate::runner_root_access`] reported, including the remedy.
579        #[source]
580        source: Box<RootAccessError>,
581    },
582
583    /// The path of the running binary could not be resolved.
584    #[error(
585        "cannot resolve the absolute path of this executable, so there is nothing to register: \
586         {detail}. Run the installer from the installed binary rather than through a shell \
587         function or a wrapper that replaces argv[0]."
588    )]
589    BinaryPath {
590        /// What the platform reported.
591        detail: String,
592    },
593
594    /// The binary named for the registration is not there.
595    #[error(
596        "{} is not a file, so registering it would create a service that cannot start. Install \
597         the product first, then run `service install` from the installed binary.",
598        path.display()
599    )]
600    BinaryMissing {
601        /// The path that was to be registered.
602        path: PathBuf,
603    },
604
605    /// The requested restart delay is outside the bound.
606    #[error(
607        "a restart delay of {requested_secs}s is outside the supported range \
608         {min_secs}s-{max_secs}s. Below the floor the platform's own throttle overrides the \
609         value, so `service status` would report a delay that is not the one in force."
610    )]
611    RestartDelay {
612        /// What was asked for.
613        requested_secs: u64,
614        /// The floor.
615        min_secs: u64,
616        /// The ceiling.
617        max_secs: u64,
618    },
619
620    /// The failure-count reset window is not longer than the restart delay.
621    #[error(
622        "a failure-count reset window of {reset_secs}s is not longer than the {delay_secs}s \
623         restart delay, so the count would reset between every pair of restarts and no \
624         start limit could ever apply."
625    )]
626    RestartResetWindow {
627        /// What was asked for.
628        reset_secs: u64,
629        /// The delay it has to exceed.
630        delay_secs: u64,
631    },
632
633    /// A registration is already there, in the **other** start mode.
634    ///
635    /// Only the other one. A registration in the mode being asked for is
636    /// replaced rather than refused — see [`Operations::install`], and the
637    /// outage that taught us why.
638    #[error(
639        "{name} is already registered to start at {existing}, and this asks for {requested}. \
640         Changing the start mode moves the registration between two different service \
641         managers, so it is not something an install does by itself: run `service uninstall` \
642         first, or change the start mode in the terminal UI (`runner-manager tui`), which \
643         switches it in place and keeps the registration running throughout."
644    )]
645    AlreadyInstalled {
646        /// Which registration.
647        name: String,
648        /// The mode it currently carries.
649        existing: StartMode,
650        /// The mode this install asked for.
651        requested: StartMode,
652    },
653
654    /// No registration is there.
655    #[error("{name} is not registered on this host, so there is nothing to {operation}.")]
656    NotInstalled {
657        /// Which registration.
658        name: String,
659        /// What was being attempted.
660        operation: &'static str,
661    },
662
663    /// The record `install` wrote could not be read or written.
664    #[error("cannot {operation} the service record {}: {detail}", path.display())]
665    Record {
666        /// Read, write, or remove.
667        operation: &'static str,
668        /// The record file.
669        path: PathBuf,
670        /// The underlying failure.
671        detail: String,
672    },
673
674    /// The record is there but is not one this version wrote.
675    #[error(
676        "the service record {} was not written by this product, or was written by a version \
677         this one cannot read: {detail}. Run `service uninstall` and `service install` again; \
678         neither touches configuration, secrets, or the cache.",
679        path.display()
680    )]
681    RecordUnreadable {
682        /// The record file.
683        path: PathBuf,
684        /// What was wrong with it.
685        detail: String,
686    },
687
688    /// The record is there and **this account** may not read it.
689    ///
690    /// Its own variant rather than a [`ServiceError::Record`] with a
691    /// `Permission denied` in the detail, because it is the one record failure
692    /// that is not a fault in the record: `service status` reports it and
693    /// carries on, where every other read failure ends the command. See
694    /// [`Operations::status`].
695    ///
696    /// A version before this one wrote the record at `0600` through a temporary
697    /// file, so a boot-mode `sudo service install` left it owned by `root` and
698    /// unreadable by the operator whose profile it is in. `service status` then
699    /// failed on their own host, with a remedy naming the command that had just
700    /// failed.
701    #[error(
702        "the service record {} is there and this account may not read it: {detail}. It was \
703         written by whichever account installed the service -- on a boot-mode host, `sudo \
704         service install`. Running `service install` again rewrites it readable; until then \
705         only what the service manager itself reports is available.",
706        path.display()
707    )]
708    RecordNotPermitted {
709        /// The record file.
710        path: PathBuf,
711        /// What the operating system said.
712        detail: String,
713    },
714
715    /// The application-data directories could not be resolved or created.
716    #[error("cannot prepare this host's application-data directories: {source}")]
717    Paths {
718        /// The underlying failure.
719        #[source]
720        source: Box<crate::paths::PathsError>,
721    },
722
723    /// The platform's service manager refused, or could not be reached.
724    #[error("cannot {operation} {name} through {manager}: {detail}")]
725    Control {
726        /// What was being attempted.
727        operation: &'static str,
728        /// Which registration.
729        name: String,
730        /// Which service manager.
731        manager: &'static str,
732        /// What it said.
733        detail: String,
734    },
735
736    /// An operation failed and its compensating action failed too.
737    #[error(
738        "cannot {operation} {name}: {cause}. The attempted rollback also failed: {rollback}. \
739         Inspect `service status` before retrying."
740    )]
741    Rollback {
742        /// The transaction that could not be completed.
743        operation: &'static str,
744        /// Which registration was involved.
745        name: String,
746        /// The original failure.
747        cause: String,
748        /// The failure while restoring the previous state.
749        rollback: String,
750    },
751
752    /// The machine's registration is another installation's (see
753    /// [`ForeignService`]), and changing it from this account would act on it
754    /// with this account's directories.
755    #[error(
756        "this machine's {name} service was installed from {installed_by}, not from this \
757         account's data root, so this account does not {operation} it: doing so would \
758         re-register or remove that installation's service with this account's directories. \
759         Do it {act_as}. If that account no longer exists, remove the registration with the \
760         service manager itself first."
761    )]
762    ForeignInstallation {
763        /// Which registration.
764        name: String,
765        /// What was refused.
766        operation: &'static str,
767        /// [`ForeignService::installed_by`].
768        installed_by: String,
769        /// [`ForeignService::act_as`].
770        act_as: String,
771    },
772
773    /// `host forget-local` found this account's configuration in use, and
774    /// moved nothing.
775    #[error("this account's local configuration is in use, so nothing was archived: {detail}.")]
776    LocalConfigurationInUse {
777        /// What uses it.
778        detail: String,
779    },
780
781    /// The operation needs administrative rights it does not have.
782    #[error("{operation} {name} needs administrative rights: {detail}. {remedy}")]
783    NeedsElevation {
784        /// What was being attempted.
785        operation: &'static str,
786        /// Which registration.
787        name: String,
788        /// What the platform said.
789        detail: String,
790        /// How to get them, in this platform's own terms.
791        remedy: &'static str,
792    },
793}
794
795// ---------------------------------------------------------------------------
796// Directories
797// ---------------------------------------------------------------------------
798
799/// The four directories the registration was installed against.
800///
801/// Recorded rather than re-derived, because `05-infrastructure.md` item 2 speaks
802/// of the *configured* cache and runtime directories: the account the service
803/// runs under and the account that ran `service install` do not always resolve
804/// the same platform-standard locations, and a claim about what the service can
805/// write is only checkable against a specific set of paths.
806#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
807pub struct ServiceDirectories {
808    /// Non-secret TOML and the SQLite database.
809    pub config: PathBuf,
810    /// The agent lock, the attempt journal, and the runner package cache.
811    pub state: PathBuf,
812    /// Per-attempt disposable runner workspaces.
813    pub runtime: PathBuf,
814    /// Rotating redacted diagnostics.
815    pub logs: PathBuf,
816}
817
818impl ServiceDirectories {
819    /// Snapshots the four directories a resolved [`AppPaths`] names.
820    #[must_use]
821    pub fn of(paths: &AppPaths) -> Self {
822        Self {
823            config: paths.config_dir().to_path_buf(),
824            state: paths.state_dir().to_path_buf(),
825            runtime: paths.runtime_dir().to_path_buf(),
826            logs: paths.logs_dir().to_path_buf(),
827        }
828    }
829
830    /// The four, in the order `05-infrastructure.md` lists them.
831    #[must_use]
832    pub fn all(&self) -> [&Path; 4] {
833        [
834            self.config.as_path(),
835            self.state.as_path(),
836            self.runtime.as_path(),
837            self.logs.as_path(),
838        ]
839    }
840
841    /// The rotating diagnostic log's stem, which is what `service status`
842    /// reports and what `05-infrastructure.md` item 4 requires be preserved.
843    #[must_use]
844    pub fn log_file(&self) -> PathBuf {
845        self.logs.join(LOG_FILE_STEM)
846    }
847
848    /// The four as the hidden `daemon run` options a registration passes them
849    /// in, [`SERVICE_DIRECTORY_OPTIONS`] order.
850    #[must_use]
851    pub fn arguments(&self) -> [OsString; 8] {
852        let [config, state, runtime, logs] = SERVICE_DIRECTORY_OPTIONS.map(OsString::from);
853        [
854            config,
855            self.config.clone().into_os_string(),
856            state,
857            self.state.clone().into_os_string(),
858            runtime,
859            self.runtime.clone().into_os_string(),
860            logs,
861            self.logs.clone().into_os_string(),
862        ]
863    }
864
865    /// Reads the four back out of a registration's command line.
866    ///
867    /// `None` unless all four options are there: a registration made before
868    /// they existed starts the daemon against whatever its account's
869    /// platform-standard directories are, and which those are cannot be told
870    /// from here.
871    #[must_use]
872    pub fn from_command_line(command_line: &str) -> Option<Self> {
873        let arguments = split_command_line(command_line);
874        let value = |option: &str| {
875            let at = arguments.iter().position(|argument| argument == option)?;
876            arguments.get(at + 1).map(PathBuf::from)
877        };
878        let [config, state, runtime, logs] = SERVICE_DIRECTORY_OPTIONS;
879        Some(Self {
880            config: value(config)?,
881            state: value(state)?,
882            runtime: value(runtime)?,
883            logs: value(logs)?,
884        })
885    }
886
887    /// The deepest directory all four sit under: what an operator would call
888    /// the installation's data root. The configuration directory when they
889    /// share nothing.
890    #[must_use]
891    pub fn root(&self) -> PathBuf {
892        let mut shared: Vec<std::path::Component<'_>> = self.config.components().collect();
893        for other in [&self.state, &self.runtime, &self.logs] {
894            let common = shared
895                .iter()
896                .zip(other.components())
897                .take_while(|(left, right)| **left == *right)
898                .count();
899            shared.truncate(common);
900        }
901        if shared.is_empty() {
902            self.config.clone()
903        } else {
904            shared.iter().collect()
905        }
906    }
907
908    /// The four as an [`AppPaths`], to read the records a daemon running
909    /// against them writes.
910    #[must_use]
911    pub fn paths(&self) -> AppPaths {
912        AppPaths::from_directories(&self.config, &self.state, &self.runtime, &self.logs)
913    }
914}
915
916/// The hidden `daemon run` options that carry the installing account's four
917/// application-data directories across to the service's account, in
918/// [`ServiceDirectories`] order. Written by `service install`, and read back
919/// by [`ServiceDirectories::from_command_line`] to tell whose data root a
920/// registration runs against.
921pub const SERVICE_DIRECTORY_OPTIONS: [&str; 4] = [
922    "--service-config-dir",
923    "--service-state-dir",
924    "--service-runtime-dir",
925    "--service-logs-dir",
926];
927
928// ---------------------------------------------------------------------------
929// The install plan
930// ---------------------------------------------------------------------------
931
932/// What `service install` was asked for, before anything has been resolved.
933///
934/// Separate from [`InstallPlan`] because a request is the operator's words and
935/// a plan is what the host actually supports: the plan carries a *resolved
936/// absolute* binary path, the account the start mode obliges, and the four
937/// directories the registration is installed against, none of which the caller
938/// supplies.
939#[derive(Debug, Clone, PartialEq, Eq)]
940pub struct InstallRequest {
941    start_mode: StartMode,
942    binary: Option<PathBuf>,
943    source_binary: Option<PathBuf>,
944    arguments: Vec<OsString>,
945    restart: RestartPolicy,
946    on_demand: bool,
947}
948
949impl InstallRequest {
950    /// Register the binary that is running this call.
951    #[must_use]
952    pub fn new(start_mode: StartMode) -> Self {
953        Self {
954            start_mode,
955            binary: None,
956            source_binary: None,
957            arguments: DAEMON_ARGUMENTS.iter().map(OsString::from).collect(),
958            restart: RestartPolicy::default(),
959            on_demand: false,
960        }
961    }
962
963    /// Registers the service but does not ask the manager to start it by
964    /// itself.
965    ///
966    /// Production never uses this: a boot-mode registration that does not start
967    /// at boot is the failure `service status` reports. **The privileged
968    /// installer tests use it for every fixture they create**, so that a
969    /// registration which somehow escaped its cleanup guard cannot start with
970    /// the owner's machine on the next reboot. A leaked service is bad; a
971    /// leaked service that runs is worse, and the difference costs one flag.
972    #[must_use]
973    pub const fn started_on_demand(mut self) -> Self {
974        self.on_demand = true;
975        self
976    }
977
978    /// Register a named binary instead of the running one.
979    ///
980    /// The privileged installer tests use this to register a fixture service
981    /// host. Production does not: `05-infrastructure.md` item 6 is specifically
982    /// about the path of *the running binary*, and letting an operator name a
983    /// different one would defeat the stale-path detection it asks for.
984    #[must_use]
985    pub fn for_binary(mut self, binary: impl Into<PathBuf>) -> Self {
986        self.binary = Some(binary.into());
987        self
988    }
989
990    /// Where the registered binary was copied from.
991    ///
992    /// # Why a registration has a source at all
993    ///
994    /// A service registered directly against a package manager's own file can
995    /// never be upgraded on Windows, because the running service holds that
996    /// file open and `npm i -g` cannot replace it. What it does instead is
997    /// worse than failing: it rewrites the package metadata, reports success,
998    /// and leaves the old executable in place — so the operator is told the new
999    /// version is installed while the old one keeps running. That was observed,
1000    /// twice in a row, before this existed.
1001    ///
1002    /// So the service runs a copy the product owns, and this records where that
1003    /// copy came from. The source is what a package manager updates, what
1004    /// [`inspect_binary`] watches for item 6's stale-path case, and what the
1005    /// daemon compares its own version against to know an upgrade is waiting.
1006    #[must_use]
1007    pub fn copied_from(mut self, source: impl Into<PathBuf>) -> Self {
1008        self.source_binary = Some(source.into());
1009        self
1010    }
1011
1012    /// Replace the registered arguments. Defaults to [`DAEMON_ARGUMENTS`].
1013    #[must_use]
1014    pub fn with_arguments<I, S>(mut self, arguments: I) -> Self
1015    where
1016        I: IntoIterator<Item = S>,
1017        S: Into<OsString>,
1018    {
1019        self.arguments = arguments.into_iter().map(Into::into).collect();
1020        self
1021    }
1022
1023    /// Replace the restart-on-failure policy. Defaults to
1024    /// [`RestartPolicy::default`].
1025    #[must_use]
1026    pub const fn with_restart(mut self, restart: RestartPolicy) -> Self {
1027        self.restart = restart;
1028        self
1029    }
1030
1031    /// The start mode asked for.
1032    #[must_use]
1033    pub const fn start_mode(&self) -> StartMode {
1034        self.start_mode
1035    }
1036}
1037
1038/// Everything a backend needs, resolved and validated.
1039///
1040/// Constructing one is where `05-infrastructure.md` item 6 is actually
1041/// satisfied: the binary path is made absolute and is confirmed to be a file
1042/// *before* anything is registered, so a registration that would fail at boot
1043/// is refused at install time instead.
1044#[derive(Debug, Clone, PartialEq, Eq)]
1045pub struct InstallPlan {
1046    identity: ServiceIdentity,
1047    start_mode: StartMode,
1048    binary: PathBuf,
1049    source_binary: Option<PathBuf>,
1050    arguments: Vec<OsString>,
1051    account: ServiceAccount,
1052    restart: RestartPolicy,
1053    directories: ServiceDirectories,
1054    secret_guard: Option<PathBuf>,
1055    on_demand: bool,
1056}
1057
1058impl InstallPlan {
1059    /// Resolves a request against this host.
1060    ///
1061    /// # Errors
1062    ///
1063    /// [`ServiceError::BinaryPath`] when the running executable cannot be
1064    /// located, and [`ServiceError::BinaryMissing`] when the path that would be
1065    /// registered is not a file.
1066    pub fn resolve(
1067        identity: ServiceIdentity,
1068        request: &InstallRequest,
1069        directories: ServiceDirectories,
1070    ) -> Result<Self, ServiceError> {
1071        let binary = match &request.binary {
1072            Some(named) => absolute(named)?,
1073            None => running_executable()?,
1074        };
1075        if !binary.is_file() {
1076            return Err(ServiceError::BinaryMissing { path: binary });
1077        }
1078        // `d2` publishes the guard file the store lives in so the systemd
1079        // sandbox can admit only its parent for atomic token rotation. A store
1080        // that cannot be resolved is not fatal here; it merely cannot receive
1081        // that narrow writable-path exception.
1082        let secret_guard = crate::secrets::PlatformSecretStore::for_start_mode(request.start_mode)
1083            .ok()
1084            .map(|store| store.guard());
1085        Ok(Self {
1086            identity,
1087            start_mode: request.start_mode,
1088            binary,
1089            source_binary: request.source_binary.clone(),
1090            arguments: request.arguments.clone(),
1091            account: ServiceAccount::for_start_mode(request.start_mode),
1092            restart: request.restart,
1093            directories,
1094            secret_guard,
1095            on_demand: request.on_demand,
1096        })
1097    }
1098
1099    /// Builds a plan from values a caller already holds, without touching the
1100    /// filesystem.
1101    ///
1102    /// This is what makes the renderers testable on a host that has none of the
1103    /// paths involved: a Windows developer can render the systemd unit for
1104    /// `/opt/runner-manager/bin/runner-manager` and assert every line of it.
1105    #[must_use]
1106    pub fn unchecked(
1107        identity: ServiceIdentity,
1108        start_mode: StartMode,
1109        binary: impl Into<PathBuf>,
1110        directories: ServiceDirectories,
1111    ) -> Self {
1112        Self {
1113            identity,
1114            start_mode,
1115            binary: binary.into(),
1116            source_binary: None,
1117            arguments: DAEMON_ARGUMENTS.iter().map(OsString::from).collect(),
1118            account: ServiceAccount::for_start_mode(start_mode),
1119            restart: RestartPolicy::default(),
1120            directories,
1121            secret_guard: None,
1122            on_demand: false,
1123        }
1124    }
1125
1126    /// Registers without arming the manager's automatic start. See
1127    /// [`InstallRequest::started_on_demand`].
1128    #[must_use]
1129    pub const fn started_on_demand(mut self) -> Self {
1130        self.on_demand = true;
1131        self
1132    }
1133
1134    /// Whether the manager was asked not to start this by itself.
1135    #[must_use]
1136    pub const fn is_on_demand(&self) -> bool {
1137        self.on_demand
1138    }
1139
1140    /// Names the file the secret store lives in, so systemd can permit atomic
1141    /// replacement of it under `ProtectSystem=strict`.
1142    ///
1143    /// [`InstallPlan::resolve`] fills this from `d2`; a test names it directly
1144    /// so that a Windows or macOS CI leg can assert the Linux unit's writable
1145    /// path for a store that platform does not have.
1146    #[must_use]
1147    pub fn with_secret_guard(mut self, guard: impl Into<PathBuf>) -> Self {
1148        self.secret_guard = Some(guard.into());
1149        self
1150    }
1151
1152    /// The file the machine-scoped secret store lives in, when one resolved.
1153    #[must_use]
1154    pub fn secret_guard(&self) -> Option<&Path> {
1155        self.secret_guard.as_deref()
1156    }
1157
1158    /// Overrides the restart policy on an already-built plan.
1159    #[must_use]
1160    pub const fn with_restart(mut self, restart: RestartPolicy) -> Self {
1161        self.restart = restart;
1162        self
1163    }
1164
1165    /// Overrides the registered arguments on an already-built plan.
1166    #[must_use]
1167    pub fn with_arguments<I, S>(mut self, arguments: I) -> Self
1168    where
1169        I: IntoIterator<Item = S>,
1170        S: Into<OsString>,
1171    {
1172        self.arguments = arguments.into_iter().map(Into::into).collect();
1173        self
1174    }
1175
1176    /// What the operating system calls this registration.
1177    #[must_use]
1178    pub const fn identity(&self) -> &ServiceIdentity {
1179        &self.identity
1180    }
1181
1182    /// Boot or login.
1183    #[must_use]
1184    pub const fn start_mode(&self) -> StartMode {
1185        self.start_mode
1186    }
1187
1188    /// The resolved absolute path of the binary to register.
1189    #[must_use]
1190    pub fn binary(&self) -> &Path {
1191        &self.binary
1192    }
1193
1194    /// Where [`Self::binary`] was copied from, when it is a copy.
1195    ///
1196    /// `None` is the legacy layout: the registration names the file a package
1197    /// manager owns, and cannot be upgraded while it runs. See
1198    /// [`InstallRequest::copied_from`].
1199    #[must_use]
1200    pub fn source_binary(&self) -> Option<&Path> {
1201        self.source_binary.as_deref()
1202    }
1203
1204    /// The arguments it is registered with.
1205    #[must_use]
1206    pub fn arguments(&self) -> &[OsString] {
1207        &self.arguments
1208    }
1209
1210    /// The account it runs under.
1211    #[must_use]
1212    pub const fn account(&self) -> &ServiceAccount {
1213        &self.account
1214    }
1215
1216    /// The restart-on-failure policy.
1217    #[must_use]
1218    pub const fn restart(&self) -> RestartPolicy {
1219        self.restart
1220    }
1221
1222    /// The four directories it was installed against.
1223    #[must_use]
1224    pub const fn directories(&self) -> &ServiceDirectories {
1225        &self.directories
1226    }
1227
1228    /// The command line as one string, quoted the way each platform expects.
1229    #[must_use]
1230    pub fn command_line(&self) -> String {
1231        let mut out = quote_argument(&self.binary.to_string_lossy());
1232        for argument in &self.arguments {
1233            out.push(' ');
1234            out.push_str(&quote_argument(&argument.to_string_lossy()));
1235        }
1236        out
1237    }
1238}
1239
1240/// Undoes a runner-root change, and says what could not be undone.
1241///
1242/// `None` when the rollback was complete — either because there was nothing to
1243/// undo, or because the directory this operation created was removed again and
1244/// the descriptor it replaced was written back. `Some` is the "report any
1245/// non-reversible existing directory state explicitly" half of the requirement,
1246/// and the caller folds it into the failure it was already reporting.
1247fn retained_runner_root(change: &RootAccessChange) -> Option<String> {
1248    let reversal = change.revert();
1249    matches!(reversal, Reversal::Retained { .. }).then(|| reversal.to_string())
1250}
1251
1252/// The error a failed step returns once both of its rollbacks have been
1253/// attempted.
1254///
1255/// `retained` is [`retained_runner_root`]'s answer and `rollback` is what
1256/// undoing the *registration* reported — whatever it reports on success, which
1257/// a failure has no use for. Both arrive already evaluated, because each may be
1258/// attempted exactly once and the order is the caller's to choose.
1259///
1260/// A complete rollback leaves `cause` exactly as it was: the operator's problem
1261/// is what failed, not the tidying up afterwards. An incomplete one is promoted
1262/// to [`ServiceError::Rollback`], which is the variant that exists to say "this
1263/// failed *and* something is left behind" — and it names everything that did.
1264fn rolled_back<T>(
1265    retained: Option<String>,
1266    rollback: Result<T, ServiceError>,
1267    operation: &'static str,
1268    identity: &ServiceIdentity,
1269    cause: ServiceError,
1270) -> ServiceError {
1271    let left_behind = match (rollback.err(), retained) {
1272        (Some(rollback), Some(retained)) => Some(format!("{rollback}; {retained}")),
1273        (Some(rollback), None) => Some(rollback.to_string()),
1274        (None, retained) => retained,
1275    };
1276    match left_behind {
1277        Some(rollback) => ServiceError::Rollback {
1278            operation,
1279            name: identity.name().to_string(),
1280            cause: cause.to_string(),
1281            rollback,
1282        },
1283        None => cause,
1284    }
1285}
1286
1287/// [`rolled_back`] for a failure with nothing registered yet, where the runner
1288/// root is the only thing there is to undo.
1289fn undo_runner_root(
1290    change: &RootAccessChange,
1291    operation: &'static str,
1292    identity: &ServiceIdentity,
1293    cause: ServiceError,
1294) -> ServiceError {
1295    rolled_back(
1296        retained_runner_root(change),
1297        Ok(()),
1298        operation,
1299        identity,
1300        cause,
1301    )
1302}
1303
1304/// Makes a path absolute without resolving symlinks.
1305///
1306/// Symbolic links are deliberately left alone. A Homebrew installation puts a
1307/// stable link at `/opt/homebrew/bin/runner-manager` and moves the file it
1308/// points at on every upgrade; recording the link's target would make an
1309/// ordinary `brew upgrade` look exactly like the npm failure this module exists
1310/// to detect, and recording the link records the thing the operator installed.
1311///
1312/// Linux is the one platform where this is not fully in the caller's hands:
1313/// `std::env::current_exe` reads `/proc/self/exe`, which the kernel has already
1314/// resolved. There the recorded path is the real file rather than the shim —
1315/// which is still correct for the npm case, because a real file under a Node
1316/// prefix disappears with the prefix.
1317fn absolute(path: &Path) -> Result<PathBuf, ServiceError> {
1318    std::path::absolute(path).map_err(|error| ServiceError::BinaryPath {
1319        detail: format!("{} could not be made absolute: {error}", path.display()),
1320    })
1321}
1322
1323/// The absolute path of the executable running this call.
1324fn running_executable() -> Result<PathBuf, ServiceError> {
1325    let raw = std::env::current_exe().map_err(|error| ServiceError::BinaryPath {
1326        detail: error.to_string(),
1327    })?;
1328    absolute(&raw)
1329}
1330
1331/// Quotes one command-line argument when it needs it.
1332///
1333/// The rule is Windows', because Windows is the platform where a command line is
1334/// a single string the callee re-splits, and because `windows-service` applies
1335/// exactly this rule when it builds `lpBinaryPathName`. The Unix backends embed
1336/// arguments in a plist array and in a systemd `ExecStart=`, both of which
1337/// accept the same quoting, so one rule serves all three rather than three
1338/// nearly-identical ones.
1339pub(crate) fn quote_argument(argument: &str) -> String {
1340    if !argument.is_empty() && !argument.contains([' ', '"', '\t', '\n']) {
1341        return argument.to_string();
1342    }
1343    let mut out = String::with_capacity(argument.len() + 2);
1344    out.push('"');
1345    let mut backslashes = 0usize;
1346    for c in argument.chars() {
1347        match c {
1348            '\\' => {
1349                backslashes += 1;
1350                out.push('\\');
1351            }
1352            '"' => {
1353                // Every backslash immediately before a quote must be doubled,
1354                // and the quote itself escaped.
1355                for _ in 0..=backslashes {
1356                    out.push('\\');
1357                }
1358                out.push('"');
1359                backslashes = 0;
1360            }
1361            other => {
1362                backslashes = 0;
1363                out.push(other);
1364            }
1365        }
1366    }
1367    // Trailing backslashes would otherwise escape the closing quote.
1368    for _ in 0..backslashes {
1369        out.push('\\');
1370    }
1371    out.push('"');
1372    out
1373}
1374
1375/// Reads the executable back out of a command line quoted by
1376/// [`quote_argument`].
1377///
1378/// `service status` needs this because the Windows Service Control Manager
1379/// stores the binary and its arguments as one string and hands the whole thing
1380/// back from `QueryServiceConfigW`. Comparing a record against a registration
1381/// means splitting that string the same way Windows itself does.
1382///
1383/// Returns `None` for an empty or whitespace-only command line, which is the
1384/// only input with no first argument to find.
1385#[must_use]
1386pub fn executable_from_command_line(command_line: &str) -> Option<PathBuf> {
1387    split_command_line(command_line)
1388        .into_iter()
1389        .next()
1390        .filter(|executable| !executable.is_empty())
1391        .map(PathBuf::from)
1392}
1393
1394/// Splits a command line quoted by [`quote_argument`] back into its
1395/// arguments, by the rule Windows itself applies: whitespace separates
1396/// arguments outside quotes, `2n` backslashes before a quote are `n`
1397/// backslashes and a quote that opens or closes, `2n+1` are `n` backslashes
1398/// and a literal quote, and every other backslash is literal.
1399#[must_use]
1400pub fn split_command_line(command_line: &str) -> Vec<String> {
1401    let mut arguments = Vec::new();
1402    let mut current = String::new();
1403    let mut started = false;
1404    let mut quoted = false;
1405    let mut backslashes = 0usize;
1406    for c in command_line.chars() {
1407        match c {
1408            '\\' => {
1409                backslashes += 1;
1410                started = true;
1411            }
1412            '"' => {
1413                current.extend(std::iter::repeat_n('\\', backslashes / 2));
1414                if backslashes.is_multiple_of(2) {
1415                    quoted = !quoted;
1416                } else {
1417                    current.push('"');
1418                }
1419                backslashes = 0;
1420                started = true;
1421            }
1422            ' ' | '\t' if !quoted => {
1423                current.extend(std::iter::repeat_n('\\', backslashes));
1424                backslashes = 0;
1425                if started {
1426                    arguments.push(std::mem::take(&mut current));
1427                    started = false;
1428                }
1429            }
1430            other => {
1431                current.extend(std::iter::repeat_n('\\', backslashes));
1432                backslashes = 0;
1433                current.push(other);
1434                started = true;
1435            }
1436        }
1437    }
1438    current.extend(std::iter::repeat_n('\\', backslashes));
1439    if started {
1440        arguments.push(current);
1441    }
1442    arguments
1443}
1444
1445// ---------------------------------------------------------------------------
1446// Rendered definitions
1447// ---------------------------------------------------------------------------
1448
1449/// Which of the four things a platform reads.
1450#[derive(Debug, Clone, Copy, PartialEq, Eq)]
1451pub enum DefinitionKind {
1452    /// The parameters `CreateServiceW` is called with, rendered as a canonical
1453    /// descriptor so that they can be reviewed and asserted like the other
1454    /// three. The Service Control Manager has no file.
1455    WindowsService,
1456    /// A Task Scheduler XML document, for `--start-at login` on Windows.
1457    WindowsScheduledTask,
1458    /// A launchd property list — a LaunchDaemon at boot, a LaunchAgent at login.
1459    LaunchdPlist,
1460    /// A systemd unit — a system unit at boot, a user unit at login.
1461    SystemdUnit,
1462}
1463
1464impl DefinitionKind {
1465    /// What to call the thing that reads it, in an operator-facing message.
1466    #[must_use]
1467    pub const fn manager(self) -> &'static str {
1468        match self {
1469            Self::WindowsService => "the Windows Service Control Manager",
1470            Self::WindowsScheduledTask => "Windows Task Scheduler",
1471            Self::LaunchdPlist => "launchd",
1472            Self::SystemdUnit => "systemd",
1473        }
1474    }
1475}
1476
1477impl fmt::Display for DefinitionKind {
1478    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
1479        f.write_str(self.manager())
1480    }
1481}
1482
1483/// One platform's definition of the registration, as text.
1484///
1485/// Producing this is pure: no `cfg`, no privileges, no filesystem. That is what
1486/// lets every leg of the CI matrix assert every platform's definition, instead
1487/// of each leg asserting only its own and the other two being reviewed by
1488/// reading.
1489#[derive(Debug, Clone, PartialEq, Eq)]
1490pub struct ServiceDefinition {
1491    kind: DefinitionKind,
1492    text: String,
1493    install_path: Option<PathBuf>,
1494}
1495
1496impl ServiceDefinition {
1497    /// Which platform this is for.
1498    #[must_use]
1499    pub const fn kind(&self) -> DefinitionKind {
1500        self.kind
1501    }
1502
1503    /// The definition itself.
1504    #[must_use]
1505    pub fn text(&self) -> &str {
1506        &self.text
1507    }
1508
1509    /// Where the platform expects the file, when the platform reads a file.
1510    ///
1511    /// `None` for [`DefinitionKind::WindowsService`], whose "definition" is a
1512    /// set of arguments to `CreateServiceW`, and for a Task Scheduler document,
1513    /// which is handed to `schtasks` from a temporary file and is not stored
1514    /// where it was written.
1515    #[must_use]
1516    pub fn install_path(&self) -> Option<&Path> {
1517        self.install_path.as_deref()
1518    }
1519
1520    /// The definition this host's own service manager would be given.
1521    ///
1522    /// The value-level twin of [`host_definition_kind`]. Rendering needs no
1523    /// privileges and no service manager, so anything that wants to *see* what
1524    /// an install would register — `f3`'s `service install --dry-run`, a
1525    /// support bundle, [`RecordingControls`] — can have it without registering
1526    /// anything.
1527    ///
1528    /// # Errors
1529    ///
1530    /// [`ServiceError::Control`] when a Windows logon-triggered registration is
1531    /// asked for and the session reports no account to register it for.
1532    pub fn for_host(plan: &InstallPlan) -> Result<Self, ServiceError> {
1533        Ok(match host_definition_kind(plan.start_mode()) {
1534            DefinitionKind::WindowsService => Self::windows_service(plan),
1535            DefinitionKind::WindowsScheduledTask => {
1536                Self::windows_scheduled_task(plan, &TaskPrincipal::current()?)
1537            }
1538            DefinitionKind::LaunchdPlist => Self::launchd(plan, host_home().as_deref()),
1539            DefinitionKind::SystemdUnit => Self::systemd(plan, host_home().as_deref()),
1540        })
1541    }
1542
1543    /// The Windows service parameters, as a canonical descriptor.
1544    #[must_use]
1545    pub fn windows_service(plan: &InstallPlan) -> Self {
1546        Self {
1547            kind: DefinitionKind::WindowsService,
1548            text: windows_service_descriptor(plan),
1549            install_path: None,
1550        }
1551    }
1552
1553    /// A Task Scheduler document for `--start-at login`.
1554    #[must_use]
1555    pub fn windows_scheduled_task(plan: &InstallPlan, principal: &TaskPrincipal) -> Self {
1556        Self {
1557            kind: DefinitionKind::WindowsScheduledTask,
1558            text: windows_scheduled_task_xml(plan, principal),
1559            install_path: None,
1560        }
1561    }
1562
1563    /// A launchd property list.
1564    ///
1565    /// `home` is only consulted for `--start-at login`, where the file belongs
1566    /// in the operator's own `~/Library/LaunchAgents`.
1567    #[must_use]
1568    pub fn launchd(plan: &InstallPlan, home: Option<&Path>) -> Self {
1569        let file = format!("{}.plist", plan.identity().launchd_label());
1570        let install_path = match plan.start_mode() {
1571            StartMode::Boot => Some(PathBuf::from(LAUNCH_DAEMONS_DIR).join(file)),
1572            StartMode::Login => home.map(|home| home.join(LAUNCH_AGENTS_SUBDIR).join(file)),
1573        };
1574        Self {
1575            kind: DefinitionKind::LaunchdPlist,
1576            text: launchd_plist(plan),
1577            install_path,
1578        }
1579    }
1580
1581    /// Wraps text this module did not render.
1582    ///
1583    /// Two callers need it and both matter. A test builds a deliberately
1584    /// widened definition and watches [`review_least_privilege`] reject it —
1585    /// without which the review would be a function nobody had ever seen fail.
1586    /// And `service status` can review the definition **actually on disk**,
1587    /// which on Linux and macOS an operator is free to edit after installation.
1588    #[must_use]
1589    pub fn from_text(kind: DefinitionKind, text: impl Into<String>) -> Self {
1590        Self {
1591            kind,
1592            text: text.into(),
1593            install_path: None,
1594        }
1595    }
1596
1597    /// A systemd unit.
1598    ///
1599    /// `home` is only consulted for `--start-at login`, where the unit belongs
1600    /// in the operator's own `~/.config/systemd/user`.
1601    #[must_use]
1602    pub fn systemd(plan: &InstallPlan, home: Option<&Path>) -> Self {
1603        let file = plan.identity().systemd_unit();
1604        let install_path = match plan.start_mode() {
1605            StartMode::Boot => Some(PathBuf::from(SYSTEMD_SYSTEM_DIR).join(file)),
1606            StartMode::Login => home.map(|home| home.join(SYSTEMD_USER_SUBDIR).join(file)),
1607        };
1608        Self {
1609            kind: DefinitionKind::SystemdUnit,
1610            text: systemd_unit(plan),
1611            install_path,
1612        }
1613    }
1614}
1615
1616/// Where a LaunchDaemon lives.
1617pub const LAUNCH_DAEMONS_DIR: &str = "/Library/LaunchDaemons";
1618/// Where a LaunchAgent lives, under the operator's home directory.
1619pub const LAUNCH_AGENTS_SUBDIR: &str = "Library/LaunchAgents";
1620/// Where a systemd system unit lives.
1621pub const SYSTEMD_SYSTEM_DIR: &str = "/etc/systemd/system";
1622/// Where a systemd user unit lives, under the operator's home directory.
1623pub const SYSTEMD_USER_SUBDIR: &str = ".config/systemd/user";
1624
1625/// The product's own page, cited by every definition so an operator who finds
1626/// one on a host can find out what put it there.
1627const DOCUMENTATION: &str = "https://github.com/IvanMurzak/GitHub-Runner-Scaler-UI";
1628
1629/// How many consecutive failures the platform tolerates before it stops
1630/// retrying.
1631///
1632/// The companion to [`RestartPolicy::reset_after`]: the delay bounds how *fast*
1633/// a restart may come, and this bounds how *many* come before the host gives
1634/// up and leaves the failure visible instead of hiding it behind an endless
1635/// retry.
1636pub const START_LIMIT_BURST: u32 = 5;
1637
1638// -- systemd -----------------------------------------------------------------
1639
1640/// Renders the systemd unit for this plan.
1641///
1642/// # Why the sandbox is this tight, and what it costs
1643///
1644/// `05-infrastructure.md` item 2 ends *"and write its configured cache and
1645/// runtime directories — and no more"*. `ProtectSystem=strict` plus an explicit
1646/// `ReadWritePaths` is what "and no more" means on Linux: everything outside the
1647/// recorded application-data directories and the credential store directory
1648/// is read-only to this unit. The latter must be writable because refresh-token
1649/// rotation atomically replaces the credential file.
1650///
1651/// **The runner inherits it.** The agent spawns the GitHub Actions runner as a
1652/// child, so a workflow running on this host also runs inside this sandbox: it
1653/// cannot write outside those directories and its private `/tmp`, and
1654/// `NoNewPrivileges=yes` means it cannot `sudo`. `07-security.md` assumes a
1655/// hostile workflow may run here, so that is the intended direction — but it is
1656/// a real behavioural limit and `docs/service-account.md` states it where an
1657/// operator will find it.
1658#[must_use]
1659pub fn systemd_unit(plan: &InstallPlan) -> String {
1660    let identity = plan.identity();
1661    let restart = plan.restart();
1662    let directories = plan.directories();
1663
1664    let mut out = String::new();
1665    out.push_str("[Unit]\n");
1666    out.push_str(&format!("Description={}\n", identity.display_name()));
1667    out.push_str(&format!("Documentation={DOCUMENTATION}\n"));
1668    out.push_str("After=network-online.target\n");
1669    out.push_str("Wants=network-online.target\n");
1670    // In `[Unit]` rather than `[Service]`: systemd moved these in v229 and only
1671    // accepts them here without a deprecation warning.
1672    out.push_str(&format!(
1673        "StartLimitIntervalSec={}\n",
1674        restart.reset_after().as_secs()
1675    ));
1676    out.push_str(&format!("StartLimitBurst={START_LIMIT_BURST}\n"));
1677
1678    out.push_str("\n[Service]\n");
1679    out.push_str("Type=simple\n");
1680    // A runner is a child of the daemon. In particular, systemd must not
1681    // signal it merely because a cooperative daemon handover replaces the
1682    // manager process; startup recovery supervises any survivor instead.
1683    out.push_str("KillMode=process\n");
1684    out.push_str(&format!("ExecStart={}\n", plan.command_line()));
1685    out.push_str(&format!(
1686        "WorkingDirectory={}\n",
1687        directories.state.display()
1688    ));
1689    out.push_str(&format!("SyslogIdentifier={identity}\n"));
1690    out.push_str("Restart=on-failure\n");
1691    out.push_str(&format!("RestartSec={}\n", restart.delay().as_secs()));
1692
1693    out.push_str("\n# Least privilege. See docs/service-account.md.\n");
1694    for directive in SYSTEMD_HARDENING {
1695        out.push_str(directive);
1696        out.push('\n');
1697    }
1698    let mut writable = directories.all().to_vec();
1699    if let Some(secret_directory) = plan.secret_guard().and_then(Path::parent) {
1700        writable.push(secret_directory);
1701    }
1702    out.push_str(&format!(
1703        "ReadWritePaths={}\n",
1704        writable
1705            .iter()
1706            .map(|path| quote_argument(&path.to_string_lossy()))
1707            .collect::<Vec<_>>()
1708            .join(" ")
1709    ));
1710
1711    out.push_str("\n[Install]\n");
1712    out.push_str(match plan.start_mode() {
1713        StartMode::Boot => "WantedBy=multi-user.target\n",
1714        StartMode::Login => "WantedBy=default.target\n",
1715    });
1716    out
1717}
1718
1719/// The hardening directives every unit carries, in the order they are rendered.
1720///
1721/// A constant rather than a sequence of `push_str` calls so that
1722/// [`review_least_privilege`] can be written against the same list the renderer
1723/// emits: a directive dropped from the unit is a directive the review then
1724/// reports as missing, rather than one that silently stops being checked.
1725pub const SYSTEMD_HARDENING: [&str; 13] = [
1726    "NoNewPrivileges=yes",
1727    "CapabilityBoundingSet=",
1728    "AmbientCapabilities=",
1729    "PrivateTmp=yes",
1730    "PrivateDevices=yes",
1731    "ProtectSystem=strict",
1732    "ProtectKernelTunables=yes",
1733    "ProtectKernelModules=yes",
1734    "ProtectControlGroups=yes",
1735    "RestrictNamespaces=yes",
1736    "RestrictRealtime=yes",
1737    "RestrictSUIDSGID=yes",
1738    "RestrictAddressFamilies=AF_INET AF_INET6 AF_UNIX",
1739];
1740
1741// -- launchd -----------------------------------------------------------------
1742
1743/// The `ProcessType` every rendered plist carries.
1744///
1745/// # Why not `Background`
1746///
1747/// launchd applies a job's `ProcessType` to the whole process tree it starts,
1748/// not only to the daemon. Under `Background` every runner the daemon launched
1749/// ran at base priority 4, on efficiency cores with throttled I/O, so each job
1750/// took 4 to 20 times as long as on a classic runner.
1751///
1752/// A child cannot leave it. This is the process type launchd chose at spawn
1753/// (`spawn type = background (5)` in `launchctl print`), not the
1754/// `PRIO_DARWIN_BG` flag `setpriority` controls: measured on macOS 27, a child
1755/// of a `Background` agent reads `getpriority(PRIO_DARWIN_PROCESS) == 0`, and
1756/// `setpriority(PRIO_DARWIN_PROCESS, 0, 0)` returns success and leaves it at
1757/// 4, before `exec` and after it alike. 0.4.31 tried exactly that.
1758///
1759/// The measured alternatives, as the base priority of a runner the agent
1760/// starts: `Background` 4, `Adaptive` 4, `Standard` (the default when the key
1761/// is absent) 20, `Interactive` 31 (the same as a classic runner, and the value
1762/// GitHub's own runner service template uses). The daemon itself mostly waits,
1763/// so it costs nothing to run it at that priority too.
1764pub const LAUNCHD_PROCESS_TYPE: &str = "Interactive";
1765
1766/// Renders the launchd property list for this plan.
1767///
1768/// `KeepAlive` is a dictionary rather than `<true/>` on purpose: the
1769/// requirement is *restart on failure*, and a bare `KeepAlive` also restarts a
1770/// job that exited cleanly, which would turn a deliberate `service stop` into a
1771/// fight with launchd.
1772#[must_use]
1773pub fn launchd_plist(plan: &InstallPlan) -> String {
1774    let identity = plan.identity();
1775    let directories = plan.directories();
1776    let mut out = String::new();
1777    out.push_str("<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n");
1778    out.push_str(
1779        "<!DOCTYPE plist PUBLIC \"-//Apple//DTD PLIST 1.0//EN\" \
1780         \"http://www.apple.com/DTDs/PropertyList-1.0.dtd\">\n",
1781    );
1782    out.push_str("<plist version=\"1.0\">\n<dict>\n");
1783    out.push_str(&plist_string("Label", &identity.launchd_label()));
1784
1785    out.push_str("  <key>ProgramArguments</key>\n  <array>\n");
1786    out.push_str(&format!(
1787        "    <string>{}</string>\n",
1788        xml_escape(&plan.binary().to_string_lossy())
1789    ));
1790    for argument in plan.arguments() {
1791        out.push_str(&format!(
1792            "    <string>{}</string>\n",
1793            xml_escape(&argument.to_string_lossy())
1794        ));
1795    }
1796    out.push_str("  </array>\n");
1797
1798    out.push_str("  <key>RunAtLoad</key>\n  <true/>\n");
1799    out.push_str("  <key>KeepAlive</key>\n  <dict>\n");
1800    out.push_str("    <key>SuccessfulExit</key>\n    <false/>\n");
1801    out.push_str("  </dict>\n");
1802    out.push_str(&format!(
1803        "  <key>ThrottleInterval</key>\n  <integer>{}</integer>\n",
1804        plan.restart().delay().as_secs()
1805    ));
1806    out.push_str(&plist_string("ProcessType", LAUNCHD_PROCESS_TYPE));
1807    out.push_str(&plist_string(
1808        "WorkingDirectory",
1809        &directories.state.to_string_lossy(),
1810    ));
1811    out.push_str(&plist_string(
1812        "StandardOutPath",
1813        &directories
1814            .logs
1815            .join("runner-manager.launchd.out.log")
1816            .to_string_lossy(),
1817    ));
1818    out.push_str(&plist_string(
1819        "StandardErrorPath",
1820        &directories
1821            .logs
1822            .join("runner-manager.launchd.err.log")
1823            .to_string_lossy(),
1824    ));
1825
1826    match plan.start_mode() {
1827        StartMode::Boot => {
1828            // A LaunchDaemon runs as root unless told otherwise, and here it
1829            // must: the System Keychain's master key is root-only.
1830            out.push_str(&plist_string(
1831                "UserName",
1832                ServiceAccount::for_definition(DefinitionKind::LaunchdPlist, StartMode::Boot)
1833                    .as_str(),
1834            ));
1835            // A daemon has no user session and must not be given one.
1836            out.push_str("  <key>SessionCreate</key>\n  <false/>\n");
1837        }
1838        StartMode::Login => {
1839            // A LaunchAgent already runs as the operator. Naming a `UserName`
1840            // here would be asking launchd for an account switch a login-mode
1841            // registration has no reason to want.
1842        }
1843    }
1844
1845    out.push_str("</dict>\n</plist>\n");
1846    out
1847}
1848
1849/// One `<key>`/`<string>` pair, indented as the rest of the plist is.
1850fn plist_string(key: &str, value: &str) -> String {
1851    format!(
1852        "  <key>{}</key>\n  <string>{}</string>\n",
1853        xml_escape(key),
1854        xml_escape(value)
1855    )
1856}
1857
1858// -- Windows Task Scheduler --------------------------------------------------
1859
1860/// The account a Task Scheduler task runs as.
1861///
1862/// Task Scheduler, unlike the other three managers, needs the principal spelled
1863/// out in the document. Isolating the one impure step keeps
1864/// [`windows_scheduled_task_xml`] a pure function that every CI leg can assert.
1865#[derive(Debug, Clone, PartialEq, Eq)]
1866pub struct TaskPrincipal {
1867    user_id: String,
1868}
1869
1870impl TaskPrincipal {
1871    /// The account running this call, as `DOMAIN\user`.
1872    ///
1873    /// # Errors
1874    ///
1875    /// [`ServiceError::BinaryPath`] is not the right shape here, so this
1876    /// reports [`ServiceError::Control`] naming what the environment failed to
1877    /// say.
1878    pub fn current() -> Result<Self, ServiceError> {
1879        let user = std::env::var("USERNAME")
1880            .ok()
1881            .filter(|value| !value.trim().is_empty());
1882        let Some(user) = user else {
1883            return Err(ServiceError::Control {
1884                operation: "identify the account for",
1885                name: SERVICE_NAME.to_string(),
1886                manager: "Windows Task Scheduler",
1887                detail: "this session reports no %USERNAME%, so there is no principal to \
1888                         register a logon-triggered task for"
1889                    .to_string(),
1890            });
1891        };
1892        let domain = std::env::var("USERDOMAIN")
1893            .ok()
1894            .filter(|value| !value.trim().is_empty());
1895        Ok(Self {
1896            user_id: match domain {
1897                Some(domain) => format!("{domain}\\{user}"),
1898                None => user,
1899            },
1900        })
1901    }
1902
1903    /// A named principal, for tests and for a caller that already knows.
1904    #[must_use]
1905    pub fn named(user_id: impl Into<String>) -> Self {
1906        Self {
1907            user_id: user_id.into(),
1908        }
1909    }
1910
1911    /// `DOMAIN\user`.
1912    #[must_use]
1913    pub fn user_id(&self) -> &str {
1914        &self.user_id
1915    }
1916}
1917
1918/// Renders the Task Scheduler document for a `--start-at login` registration.
1919///
1920/// `RunLevel` is `LeastPrivilege`, which is the whole of Windows' answer to
1921/// item 2 in this domain: the task runs with the operator's ordinary filtered
1922/// token and never with an elevated one, whatever the operator's group
1923/// membership.
1924#[must_use]
1925pub fn windows_scheduled_task_xml(plan: &InstallPlan, principal: &TaskPrincipal) -> String {
1926    let identity = plan.identity();
1927    let user = xml_escape(principal.user_id());
1928    let arguments = plan
1929        .arguments()
1930        .iter()
1931        .map(|argument| quote_argument(&argument.to_string_lossy()))
1932        .collect::<Vec<_>>()
1933        .join(" ");
1934    let mut out = String::new();
1935    out.push_str("<?xml version=\"1.0\" encoding=\"UTF-16\"?>\n");
1936    out.push_str(
1937        "<Task version=\"1.4\" \
1938         xmlns=\"http://schemas.microsoft.com/windows/2004/02/mit/task\">\n",
1939    );
1940    out.push_str("  <RegistrationInfo>\n");
1941    out.push_str(&format!(
1942        "    <Description>{}</Description>\n",
1943        xml_escape(identity.description())
1944    ));
1945    out.push_str(&format!(
1946        "    <URI>\\{}</URI>\n",
1947        xml_escape(identity.name())
1948    ));
1949    out.push_str("  </RegistrationInfo>\n");
1950
1951    out.push_str("  <Triggers>\n    <LogonTrigger>\n");
1952    out.push_str("      <Enabled>true</Enabled>\n");
1953    out.push_str(&format!("      <UserId>{user}</UserId>\n"));
1954    out.push_str("    </LogonTrigger>\n  </Triggers>\n");
1955
1956    out.push_str("  <Principals>\n    <Principal id=\"Author\">\n");
1957    out.push_str(&format!("      <UserId>{user}</UserId>\n"));
1958    out.push_str("      <LogonType>InteractiveToken</LogonType>\n");
1959    out.push_str("      <RunLevel>LeastPrivilege</RunLevel>\n");
1960    out.push_str("    </Principal>\n  </Principals>\n");
1961
1962    out.push_str("  <Settings>\n");
1963    // One agent per host is `d1`'s lock; saying so here means Task Scheduler
1964    // refuses the second start rather than starting a process that then loses
1965    // the race and exits.
1966    out.push_str("    <MultipleInstancesPolicy>IgnoreNew</MultipleInstancesPolicy>\n");
1967    out.push_str("    <DisallowStartIfOnBatteries>false</DisallowStartIfOnBatteries>\n");
1968    out.push_str("    <StopIfGoingOnBatteries>false</StopIfGoingOnBatteries>\n");
1969    out.push_str("    <AllowHardTerminate>true</AllowHardTerminate>\n");
1970    out.push_str("    <StartWhenAvailable>true</StartWhenAvailable>\n");
1971    out.push_str("    <RunOnlyIfNetworkAvailable>false</RunOnlyIfNetworkAvailable>\n");
1972    out.push_str("    <IdleSettings>\n");
1973    out.push_str("      <StopOnIdleEnd>false</StopOnIdleEnd>\n");
1974    out.push_str("      <RestartOnIdle>false</RestartOnIdle>\n");
1975    out.push_str("    </IdleSettings>\n");
1976    out.push_str("    <AllowStartOnDemand>true</AllowStartOnDemand>\n");
1977    out.push_str("    <Enabled>true</Enabled>\n");
1978    out.push_str("    <Hidden>false</Hidden>\n");
1979    out.push_str("    <RunOnlyIfIdle>false</RunOnlyIfIdle>\n");
1980    out.push_str("    <WakeToRun>false</WakeToRun>\n");
1981    // A daemon has no natural end, so any limit here would be a scheduled kill.
1982    out.push_str("    <ExecutionTimeLimit>PT0S</ExecutionTimeLimit>\n");
1983    out.push_str("    <Priority>7</Priority>\n");
1984    out.push_str("    <RestartOnFailure>\n");
1985    out.push_str(&format!(
1986        "      <Interval>{}</Interval>\n",
1987        iso8601_minutes(
1988            plan.restart()
1989                .effective_delay(DefinitionKind::WindowsScheduledTask)
1990        )
1991    ));
1992    out.push_str(&format!("      <Count>{START_LIMIT_BURST}</Count>\n"));
1993    out.push_str("    </RestartOnFailure>\n");
1994    out.push_str("  </Settings>\n");
1995
1996    out.push_str("  <Actions Context=\"Author\">\n    <Exec>\n");
1997    out.push_str(&format!(
1998        "      <Command>{}</Command>\n",
1999        xml_escape(&plan.binary().to_string_lossy())
2000    ));
2001    if !arguments.is_empty() {
2002        out.push_str(&format!(
2003            "      <Arguments>{}</Arguments>\n",
2004            xml_escape(&arguments)
2005        ));
2006    }
2007    out.push_str(&format!(
2008        "      <WorkingDirectory>{}</WorkingDirectory>\n",
2009        xml_escape(&plan.directories().state.to_string_lossy())
2010    ));
2011    out.push_str("    </Exec>\n  </Actions>\n");
2012    out.push_str("</Task>\n");
2013    out
2014}
2015
2016/// `PT<n>M`, which is the only shape Task Scheduler accepts for a restart
2017/// interval. See [`RestartPolicy::effective_delay`].
2018fn iso8601_minutes(duration: Duration) -> String {
2019    format!("PT{}M", duration.as_secs() / 60)
2020}
2021
2022/// Escapes the five XML entities. Applied to every value that reaches a plist
2023/// or a task document, because a Windows account name may legitimately contain
2024/// `&` and a path may contain `<`.
2025///
2026/// `pub(crate)` rather than private because [`crate::wsl::task`] renders a
2027/// second kind of Task Scheduler document and must escape it the same way; two
2028/// escapers in one crate is exactly how one of them ends up subtly different.
2029pub(crate) fn xml_escape(value: &str) -> String {
2030    let mut out = String::with_capacity(value.len());
2031    for c in value.chars() {
2032        match c {
2033            '&' => out.push_str("&amp;"),
2034            '<' => out.push_str("&lt;"),
2035            '>' => out.push_str("&gt;"),
2036            '"' => out.push_str("&quot;"),
2037            '\'' => out.push_str("&apos;"),
2038            other => out.push(other),
2039        }
2040    }
2041    out
2042}
2043
2044/// The inverse of [`xml_escape`], for reading a definition back off disk.
2045///
2046/// `&amp;` is replaced last, which is the whole subtlety: replacing it first
2047/// would turn a literal `&amp;amp;` into `&` in two passes instead of `&amp;`
2048/// in one.
2049fn xml_unescape(value: &str) -> String {
2050    value
2051        .replace("&lt;", "<")
2052        .replace("&gt;", ">")
2053        .replace("&quot;", "\"")
2054        .replace("&apos;", "'")
2055        .replace("&amp;", "&")
2056}
2057
2058// -- Windows Service Control Manager -----------------------------------------
2059
2060/// How `CreateServiceW` should be called, in a form that can be reviewed,
2061/// asserted, and printed on any platform.
2062///
2063/// The Windows backend builds its `ServiceInfo` from **this** value rather than
2064/// from the plan, so the descriptor a Linux CI leg asserts and the parameters a
2065/// Windows host registers cannot drift apart.
2066#[derive(Debug, Clone, PartialEq, Eq)]
2067pub struct WindowsServiceSpec {
2068    /// The service name.
2069    pub name: String,
2070    /// What `services.msc` shows.
2071    pub display_name: String,
2072    /// The one-line description.
2073    pub description: String,
2074    /// `true` for `SERVICE_AUTO_START`, `false` for `SERVICE_DEMAND_START`.
2075    pub automatic_start: bool,
2076    /// The account, or `None` for `LocalSystem`, which is what
2077    /// `CreateServiceW` takes a null `lpServiceStartName` to mean.
2078    pub account: Option<String>,
2079    /// The full launch command, quoted.
2080    pub command_line: String,
2081    /// The restart-on-failure policy.
2082    pub restart: RestartPolicy,
2083}
2084
2085/// Derives the Windows service parameters from a plan.
2086#[must_use]
2087pub fn windows_service_spec(plan: &InstallPlan) -> WindowsServiceSpec {
2088    WindowsServiceSpec {
2089        name: plan.identity().name().to_string(),
2090        display_name: plan.identity().display_name().to_string(),
2091        description: plan.identity().description().to_string(),
2092        // Only a boot-mode registration is a service at all; a login-mode one
2093        // is a scheduled task. So an automatic start is the only kind here, and
2094        // the field exists because the privileged tests register an on-demand
2095        // fixture rather than one that starts with the test machine.
2096        automatic_start: plan.start_mode() == StartMode::Boot && !plan.is_on_demand(),
2097        account: match ServiceAccount::for_definition(
2098            DefinitionKind::WindowsService,
2099            plan.start_mode(),
2100        ) {
2101            // `None` means LocalSystem to `CreateServiceW`, and naming it
2102            // explicitly would be one more string to spell right.
2103            ServiceAccount::LocalSystem => None,
2104            other => Some(other.as_str().to_string()),
2105        },
2106        command_line: plan.command_line(),
2107        restart: plan.restart(),
2108    }
2109}
2110
2111/// Renders the descriptor [`review_least_privilege`] reads and `service status`
2112/// can print.
2113#[must_use]
2114fn windows_service_descriptor(plan: &InstallPlan) -> String {
2115    let spec = windows_service_spec(plan);
2116    let mut out = String::new();
2117    out.push_str("[windows-service]\n");
2118    out.push_str(&format!("Name={}\n", spec.name));
2119    out.push_str(&format!("DisplayName={}\n", spec.display_name));
2120    out.push_str(&format!("Description={}\n", spec.description));
2121    // OWN_PROCESS and never INTERACTIVE_PROCESS: an interactive service would
2122    // put a process this product controls on the operator's desktop, which is
2123    // both deprecated by Windows and more than the requirement asks for.
2124    out.push_str("ServiceType=OWN_PROCESS\n");
2125    out.push_str(&format!(
2126        "StartType={}\n",
2127        if spec.automatic_start {
2128            "AutoStart"
2129        } else {
2130            "OnDemand"
2131        }
2132    ));
2133    out.push_str("ErrorControl=Normal\n");
2134    out.push_str(&format!(
2135        "Account={}\n",
2136        spec.account
2137            .as_deref()
2138            .unwrap_or(ServiceAccount::LocalSystem.as_str())
2139    ));
2140    out.push_str(&format!("CommandLine={}\n", spec.command_line));
2141    out.push_str(&format!(
2142        "FailureActionRestartDelaySecs={}\n",
2143        spec.restart.delay().as_secs()
2144    ));
2145    out.push_str(&format!(
2146        "FailureActionsResetPeriodSecs={}\n",
2147        spec.restart.reset_after().as_secs()
2148    ));
2149    // Without this flag the Service Control Manager applies failure actions
2150    // only to a crash, and a daemon that exits non-zero after failing to reach
2151    // GitHub is not a crash.
2152    out.push_str("FailureActionsOnNonCrashFailures=true\n");
2153    out.push_str(&format!(
2154        "ReadWritePaths={}\n",
2155        plan.directories()
2156            .all()
2157            .iter()
2158            .map(|path| quote_argument(&path.to_string_lossy()))
2159            .collect::<Vec<_>>()
2160            .join(" ")
2161    ));
2162    out
2163}
2164
2165// ---------------------------------------------------------------------------
2166// The least-privilege review
2167// ---------------------------------------------------------------------------
2168
2169/// Whether a finding is about too much authority or too little.
2170#[derive(Debug, Clone, Copy, PartialEq, Eq)]
2171pub enum FindingKind {
2172    /// The definition grants more than `05-infrastructure.md` item 2 asks for.
2173    /// This is what makes a review fail.
2174    Excess,
2175    /// The definition does not grant something the daemon needs, or does not
2176    /// state something the review cannot verify without.
2177    Shortfall,
2178}
2179
2180impl fmt::Display for FindingKind {
2181    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
2182        f.write_str(match self {
2183            Self::Excess => "excess",
2184            Self::Shortfall => "shortfall",
2185        })
2186    }
2187}
2188
2189/// One thing the review has to say about a definition.
2190#[derive(Debug, Clone, PartialEq, Eq)]
2191pub struct PrivilegeFinding {
2192    /// Excess or shortfall.
2193    pub kind: FindingKind,
2194    /// The directive, key, or element it is about.
2195    pub subject: String,
2196    /// What is wrong with it, in terms an operator can act on.
2197    pub detail: String,
2198}
2199
2200impl fmt::Display for PrivilegeFinding {
2201    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
2202        write!(f, "{}: {} -- {}", self.kind, self.subject, self.detail)
2203    }
2204}
2205
2206/// What a definition actually grants, measured against the requirement.
2207///
2208/// `05-infrastructure.md` item 2 and `07-security.md`'s release gate
2209/// (*"Service account permissions are documented and verified least
2210/// privilege"*) are the same requirement stated twice, and this is the
2211/// verification half. The documentation half is `docs/service-account.md`.
2212///
2213/// The review reads the **rendered text**, not the plan it came from. That is
2214/// deliberate and it is the only version of this check worth having: a review
2215/// that re-derived its expectations from the same values the renderer used
2216/// would agree with the renderer by construction and could never fail. Reading
2217/// the text means a hand-edited unit on a real host is reviewed as it is, and
2218/// means a test can widen a definition and watch this say so.
2219#[derive(Debug, Clone, PartialEq, Eq)]
2220pub struct PrivilegeReview {
2221    kind: DefinitionKind,
2222    account: ServiceAccount,
2223    controls: Vec<String>,
2224    findings: Vec<PrivilegeFinding>,
2225}
2226
2227impl PrivilegeReview {
2228    /// Whether the definition grants nothing beyond the requirement.
2229    #[must_use]
2230    pub fn is_least_privilege(&self) -> bool {
2231        !self
2232            .findings
2233            .iter()
2234            .any(|finding| finding.kind == FindingKind::Excess)
2235    }
2236
2237    /// Everything the review found, excesses and shortfalls together.
2238    #[must_use]
2239    pub fn findings(&self) -> &[PrivilegeFinding] {
2240        &self.findings
2241    }
2242
2243    /// Only the excesses — the findings that make [`Self::is_least_privilege`]
2244    /// false.
2245    #[must_use]
2246    pub fn excesses(&self) -> Vec<&PrivilegeFinding> {
2247        self.findings
2248            .iter()
2249            .filter(|finding| finding.kind == FindingKind::Excess)
2250            .collect()
2251    }
2252
2253    /// The controls the definition was confirmed to carry.
2254    ///
2255    /// Present so that a passing review says *what it checked* rather than only
2256    /// that it passed. A check that reports nothing when it succeeds is
2257    /// indistinguishable from a check that did not run.
2258    #[must_use]
2259    pub fn controls(&self) -> &[String] {
2260        &self.controls
2261    }
2262
2263    /// The account the registration runs under, and why it is the minimum.
2264    #[must_use]
2265    pub const fn account(&self) -> &ServiceAccount {
2266        &self.account
2267    }
2268
2269    /// Which platform's definition was reviewed.
2270    #[must_use]
2271    pub const fn kind(&self) -> DefinitionKind {
2272        self.kind
2273    }
2274}
2275
2276impl fmt::Display for PrivilegeReview {
2277    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
2278        writeln!(
2279            f,
2280            "{} runs as {} ({})",
2281            self.kind,
2282            self.account,
2283            self.account.justification()
2284        )?;
2285        for control in &self.controls {
2286            writeln!(f, "  confirmed  {control}")?;
2287        }
2288        for finding in &self.findings {
2289            writeln!(f, "  {finding}")?;
2290        }
2291        if self.is_least_privilege() {
2292            write!(f, "  verdict    least privilege")
2293        } else {
2294            write!(
2295                f,
2296                "  verdict    NOT least privilege: {} excess(es)",
2297                self.excesses().len()
2298            )
2299        }
2300    }
2301}
2302
2303/// Reads a definition back and reports what it grants.
2304///
2305/// `plan` supplies only the two things the text cannot: the four directories
2306/// the registration is *allowed* to write, and the start mode, which decides
2307/// which account is the minimum.
2308#[must_use]
2309pub fn review_least_privilege(
2310    definition: &ServiceDefinition,
2311    plan: &InstallPlan,
2312) -> PrivilegeReview {
2313    let mut controls = Vec::new();
2314    let mut findings = Vec::new();
2315    match definition.kind() {
2316        DefinitionKind::SystemdUnit => {
2317            review_systemd(definition.text(), plan, &mut controls, &mut findings);
2318        }
2319        DefinitionKind::LaunchdPlist => {
2320            review_launchd(definition.text(), plan, &mut controls, &mut findings);
2321        }
2322        DefinitionKind::WindowsScheduledTask => {
2323            review_scheduled_task(definition.text(), &mut controls, &mut findings);
2324        }
2325        DefinitionKind::WindowsService => {
2326            review_windows_service(definition.text(), plan, &mut controls, &mut findings);
2327        }
2328    }
2329    PrivilegeReview {
2330        kind: definition.kind(),
2331        // The definition's account, not the plan's: a Linux CI leg reviewing the
2332        // Windows descriptor must report `LocalSystem`, which is the account
2333        // that descriptor registers, and not `root`, which is the account this
2334        // leg's own host would use.
2335        account: ServiceAccount::for_definition(definition.kind(), plan.start_mode()),
2336        controls,
2337        findings,
2338    }
2339}
2340
2341/// Every writable path a definition may name, as the strings it names them by.
2342fn permitted_paths(kind: DefinitionKind, plan: &InstallPlan) -> Vec<String> {
2343    let mut permitted = plan
2344        .directories()
2345        .all()
2346        .iter()
2347        .map(|path| path.to_string_lossy().into_owned())
2348        .collect::<Vec<_>>();
2349    if kind == DefinitionKind::SystemdUnit
2350        && let Some(secret_directory) = plan.secret_guard().and_then(Path::parent)
2351    {
2352        permitted.push(secret_directory.to_string_lossy().into_owned());
2353    }
2354    permitted
2355}
2356
2357/// Whether two path spellings name the same place on **this host's**
2358/// filesystem. Case-insensitive on Windows, exact elsewhere.
2359///
2360/// For a rendered definition use [`same_path_for`] instead: which comparison is
2361/// right there follows the platform the definition is *for*, and every leg of
2362/// the CI matrix reviews all four.
2363fn same_path_text(left: &str, right: &str) -> bool {
2364    if cfg!(windows) {
2365        left.eq_ignore_ascii_case(right)
2366    } else {
2367        left == right
2368    }
2369}
2370
2371/// Whether two path spellings name the same place on the platform a given
2372/// definition targets.
2373fn same_path_for(kind: DefinitionKind, left: &str, right: &str) -> bool {
2374    match kind {
2375        DefinitionKind::WindowsService | DefinitionKind::WindowsScheduledTask => {
2376            left.eq_ignore_ascii_case(right)
2377        }
2378        DefinitionKind::LaunchdPlist | DefinitionKind::SystemdUnit => left == right,
2379    }
2380}
2381
2382/// Checks a `ReadWritePaths`-style list against the permitted directories.
2383fn review_writable_paths(
2384    kind: DefinitionKind,
2385    subject: &str,
2386    listed: &[String],
2387    plan: &InstallPlan,
2388    controls: &mut Vec<String>,
2389    findings: &mut Vec<PrivilegeFinding>,
2390) {
2391    let permitted = permitted_paths(kind, plan);
2392    for entry in listed {
2393        if !permitted
2394            .iter()
2395            .any(|allowed| same_path_for(kind, allowed, entry))
2396        {
2397            findings.push(PrivilegeFinding {
2398                kind: FindingKind::Excess,
2399                subject: subject.to_string(),
2400                detail: format!(
2401                    "{entry} is writable but is not one of this registration's required paths"
2402                ),
2403            });
2404        }
2405    }
2406    for allowed in &permitted {
2407        if !listed
2408            .iter()
2409            .any(|entry| same_path_for(kind, allowed, entry))
2410        {
2411            findings.push(PrivilegeFinding {
2412                kind: FindingKind::Shortfall,
2413                subject: subject.to_string(),
2414                detail: format!(
2415                    "{allowed} is one of this registration's directories but is not writable, \
2416                     so the daemon cannot use it"
2417                ),
2418            });
2419        }
2420    }
2421    if listed.len() == permitted.len() && findings.iter().all(|f| f.subject != subject) {
2422        controls.push(format!(
2423            "{subject} names exactly the required application-data and credential paths"
2424        ));
2425    }
2426}
2427
2428/// The one inbound-surface rule, applied to whichever platform names it.
2429///
2430/// `07-security.md` handling rule 2: *"The product exposes no inbound HTTP,
2431/// socket, or RPC surface anywhere, in any command."* A definition that asks the
2432/// service manager to open a socket or publish a Mach service on the daemon's
2433/// behalf would create exactly that surface without a line of product code
2434/// changing, which is why it is checked here and not only in review.
2435fn review_inbound_surface(
2436    text: &str,
2437    markers: &[(&str, &str)],
2438    controls: &mut Vec<String>,
2439    findings: &mut Vec<PrivilegeFinding>,
2440) {
2441    let mut clean = true;
2442    for (marker, detail) in markers {
2443        if text.contains(marker) {
2444            clean = false;
2445            findings.push(PrivilegeFinding {
2446                kind: FindingKind::Excess,
2447                subject: (*marker).to_string(),
2448                detail: (*detail).to_string(),
2449            });
2450        }
2451    }
2452    if clean {
2453        controls.push(
2454            "no socket, listener, or Mach service is published on the daemon's behalf".to_string(),
2455        );
2456    }
2457}
2458
2459fn review_systemd(
2460    text: &str,
2461    plan: &InstallPlan,
2462    controls: &mut Vec<String>,
2463    findings: &mut Vec<PrivilegeFinding>,
2464) {
2465    let directives = ini_directives(text, "Service");
2466    for expected in SYSTEMD_HARDENING {
2467        let (key, value) = expected
2468            .split_once('=')
2469            .expect("every hardening directive is written as key=value");
2470        match directives.get(key) {
2471            Some(actual) if actual == value => controls.push((*expected).to_string()),
2472            Some(actual) => findings.push(PrivilegeFinding {
2473                kind: FindingKind::Excess,
2474                subject: key.to_string(),
2475                detail: format!(
2476                    "is `{actual}`, not `{value}`, so the unit keeps authority the \
2477                                 requirement does not ask for"
2478                ),
2479            }),
2480            None => findings.push(PrivilegeFinding {
2481                kind: FindingKind::Excess,
2482                subject: key.to_string(),
2483                detail: format!(
2484                    "is absent, so the unit inherits systemd's default rather than `{value}`"
2485                ),
2486            }),
2487        }
2488    }
2489
2490    match directives.get("ReadWritePaths") {
2491        Some(value) => {
2492            let listed = split_quoted(value);
2493            review_writable_paths(
2494                DefinitionKind::SystemdUnit,
2495                "ReadWritePaths",
2496                &listed,
2497                plan,
2498                controls,
2499                findings,
2500            );
2501        }
2502        None => findings.push(PrivilegeFinding {
2503            kind: FindingKind::Shortfall,
2504            subject: "ReadWritePaths".to_string(),
2505            detail: "is absent, so `ProtectSystem=strict` leaves the daemon nowhere to write"
2506                .to_string(),
2507        }),
2508    }
2509
2510    // `User=` on a system unit is an escalation only in the other direction, so
2511    // what is checked is the pair systemd actually treats as privileged.
2512    if directives.contains_key("PrivateUsers")
2513        && directives.get("PrivateUsers") == Some(&"no".to_string())
2514    {
2515        findings.push(PrivilegeFinding {
2516            kind: FindingKind::Excess,
2517            subject: "PrivateUsers".to_string(),
2518            detail: "is explicitly disabled, which is broader than leaving it at systemd's default"
2519                .to_string(),
2520        });
2521    }
2522
2523    review_inbound_surface(
2524        text,
2525        &[
2526            (
2527                "ListenStream=",
2528                "asks systemd to open a listening socket for this service, which \
2529                 07-security.md rule 2 forbids the product to have",
2530            ),
2531            (
2532                "ListenDatagram=",
2533                "asks systemd to open a listening socket for this service, which \
2534                 07-security.md rule 2 forbids the product to have",
2535            ),
2536        ],
2537        controls,
2538        findings,
2539    );
2540}
2541
2542fn review_launchd(
2543    text: &str,
2544    plan: &InstallPlan,
2545    controls: &mut Vec<String>,
2546    findings: &mut Vec<PrivilegeFinding>,
2547) {
2548    // A scheduling class is not an authority, so no value here is an excess.
2549    // Anything but `Interactive` is a shortfall: launchd applies it to every
2550    // runner the daemon starts. See `LAUNCHD_PROCESS_TYPE`.
2551    match plist_string_value(text, "ProcessType") {
2552        Some(value) if value == LAUNCHD_PROCESS_TYPE => {
2553            controls.push(format!("ProcessType={LAUNCHD_PROCESS_TYPE}"));
2554        }
2555        found => findings.push(PrivilegeFinding {
2556            kind: FindingKind::Shortfall,
2557            subject: "ProcessType".to_string(),
2558            detail: format!(
2559                "is {}, and launchd applies it to every runner the daemon starts, so their jobs \
2560                 run below normal priority. The daemon rewrites it to `{LAUNCHD_PROCESS_TYPE}` \
2561                 when it next starts with no runner; `service status` says how to restart it",
2562                found.map_or_else(
2563                    || "absent (launchd's `Standard`)".to_string(),
2564                    |value| format!("`{value}`")
2565                )
2566            ),
2567        }),
2568    }
2569
2570    match plan.start_mode() {
2571        StartMode::Boot => {
2572            if plist_bool_value(text, "SessionCreate") == Some(true) {
2573                findings.push(PrivilegeFinding {
2574                    kind: FindingKind::Excess,
2575                    subject: "SessionCreate".to_string(),
2576                    detail: "asks launchd to create a security session for a job that runs \
2577                             outside every login session and has no use for one"
2578                        .to_string(),
2579                });
2580            } else {
2581                controls.push("SessionCreate is not requested".to_string());
2582            }
2583            match plist_string_value(text, "UserName").as_deref() {
2584                Some("root") => {
2585                    controls.push("UserName=root, stated rather than inherited".to_string())
2586                }
2587                Some(other) => findings.push(PrivilegeFinding {
2588                    kind: FindingKind::Shortfall,
2589                    subject: "UserName".to_string(),
2590                    detail: format!(
2591                        "is `{other}`, which cannot unlock the System Keychain: \
2592                         /var/db/SystemKey is root-only, so the daemon would start and then \
2593                         find no credential"
2594                    ),
2595                }),
2596                None => findings.push(PrivilegeFinding {
2597                    kind: FindingKind::Shortfall,
2598                    subject: "UserName".to_string(),
2599                    detail: "is absent, so the account is launchd's implicit default and this \
2600                             review cannot confirm it"
2601                        .to_string(),
2602                }),
2603            }
2604        }
2605        StartMode::Login => {
2606            if let Some(named) = plist_string_value(text, "UserName") {
2607                findings.push(PrivilegeFinding {
2608                    kind: FindingKind::Excess,
2609                    subject: "UserName".to_string(),
2610                    detail: format!(
2611                        "names `{named}` in a LaunchAgent, which already runs as the operator; \
2612                         naming an account here asks launchd for a switch a login-mode \
2613                         registration has no reason to want"
2614                    ),
2615                });
2616            } else {
2617                controls
2618                    .push("no UserName: the agent runs as the operator and no other".to_string());
2619            }
2620        }
2621    }
2622
2623    review_inbound_surface(
2624        text,
2625        &[
2626            (
2627                "<key>Sockets</key>",
2628                "asks launchd to open a socket for this job, which 07-security.md rule 2 \
2629                 forbids the product to have",
2630            ),
2631            (
2632                "<key>MachServices</key>",
2633                "publishes a Mach service, which is the RPC surface 07-security.md rule 2 \
2634                 forbids the product to have",
2635            ),
2636        ],
2637        controls,
2638        findings,
2639    );
2640}
2641
2642fn review_scheduled_task(
2643    text: &str,
2644    controls: &mut Vec<String>,
2645    findings: &mut Vec<PrivilegeFinding>,
2646) {
2647    match xml_value(text, "RunLevel").as_deref() {
2648        Some("LeastPrivilege") => controls.push("RunLevel=LeastPrivilege".to_string()),
2649        Some(other) => findings.push(PrivilegeFinding {
2650            kind: FindingKind::Excess,
2651            subject: "RunLevel".to_string(),
2652            detail: format!(
2653                "is `{other}`, so the task runs with an elevated token whenever the operator is \
2654                 an administrator"
2655            ),
2656        }),
2657        None => findings.push(PrivilegeFinding {
2658            kind: FindingKind::Excess,
2659            subject: "RunLevel".to_string(),
2660            detail: "is absent, so Task Scheduler decides the token rather than the definition"
2661                .to_string(),
2662        }),
2663    }
2664
2665    match xml_value(text, "LogonType").as_deref() {
2666        Some("InteractiveToken") => controls.push("LogonType=InteractiveToken".to_string()),
2667        Some(other) => findings.push(PrivilegeFinding {
2668            kind: FindingKind::Excess,
2669            subject: "LogonType".to_string(),
2670            detail: format!(
2671                "is `{other}`, which asks Windows to store or synthesise a credential for this \
2672                 task; an interactive token needs neither"
2673            ),
2674        }),
2675        None => findings.push(PrivilegeFinding {
2676            kind: FindingKind::Shortfall,
2677            subject: "LogonType".to_string(),
2678            detail: "is absent, so this review cannot confirm that no credential is stored"
2679                .to_string(),
2680        }),
2681    }
2682}
2683
2684fn review_windows_service(
2685    text: &str,
2686    plan: &InstallPlan,
2687    controls: &mut Vec<String>,
2688    findings: &mut Vec<PrivilegeFinding>,
2689) {
2690    let directives = ini_directives(text, "windows-service");
2691
2692    match directives.get("ServiceType").map(String::as_str) {
2693        Some("OWN_PROCESS") => controls.push("ServiceType=OWN_PROCESS".to_string()),
2694        Some(other) => findings.push(PrivilegeFinding {
2695            kind: FindingKind::Excess,
2696            subject: "ServiceType".to_string(),
2697            detail: format!(
2698                "is `{other}`; an interactive or shared-process service reaches further than a \
2699                 daemon that only talks to GitHub over HTTPS"
2700            ),
2701        }),
2702        None => findings.push(PrivilegeFinding {
2703            kind: FindingKind::Shortfall,
2704            subject: "ServiceType".to_string(),
2705            detail: "is absent, so this review cannot confirm the service is not interactive"
2706                .to_string(),
2707        }),
2708    }
2709
2710    // The account is not a free choice on Windows: `d2`'s DACL decides it. What
2711    // the review can check is that the definition names the one account that
2712    // DACL admits, and no broader one.
2713    match directives.get("Account").map(String::as_str) {
2714        Some(account) if account == ServiceAccount::LocalSystem.as_str() => {
2715            controls.push(format!(
2716                "Account={account}: the only stock account the machine-scoped store's DACL \
2717                 (SY, BA, OW) admits"
2718            ));
2719        }
2720        Some(other) => findings.push(PrivilegeFinding {
2721            kind: FindingKind::Shortfall,
2722            subject: "Account".to_string(),
2723            detail: format!(
2724                "is `{other}`, which the machine-scoped store's DACL does not name, so the \
2725                 daemon would start and then find no credential. Widening that DACL is not this \
2726                 registration's to do: an ACE reaching `{other}` would also reach every other \
2727                 service running under it"
2728            ),
2729        }),
2730        None => findings.push(PrivilegeFinding {
2731            kind: FindingKind::Shortfall,
2732            subject: "Account".to_string(),
2733            detail: "is absent, so this review cannot confirm which account was registered"
2734                .to_string(),
2735        }),
2736    }
2737
2738    match directives.get("ReadWritePaths") {
2739        Some(value) => {
2740            let listed = split_quoted(value);
2741            review_writable_paths(
2742                DefinitionKind::WindowsService,
2743                "ReadWritePaths",
2744                &listed,
2745                plan,
2746                controls,
2747                findings,
2748            );
2749        }
2750        None => findings.push(PrivilegeFinding {
2751            kind: FindingKind::Shortfall,
2752            subject: "ReadWritePaths".to_string(),
2753            detail: "is absent, so the directories the service was installed against are not \
2754                     recorded"
2755                .to_string(),
2756        }),
2757    }
2758}
2759
2760// -- parsing helpers ---------------------------------------------------------
2761
2762/// Reads `key=value` lines out of one `[section]` of an INI-shaped document.
2763///
2764/// systemd's unit format and this module's Windows descriptor are both this
2765/// shape. A repeated key takes its last value, which is systemd's own rule for
2766/// every directive here.
2767fn ini_directives(text: &str, section: &str) -> BTreeMap<String, String> {
2768    let mut out = BTreeMap::new();
2769    let mut inside = false;
2770    for line in text.lines() {
2771        let line = line.trim();
2772        if line.starts_with('[') && line.ends_with(']') {
2773            inside = &line[1..line.len() - 1] == section;
2774            continue;
2775        }
2776        if !inside || line.is_empty() || line.starts_with('#') || line.starts_with(';') {
2777            continue;
2778        }
2779        if let Some((key, value)) = line.split_once('=') {
2780            out.insert(key.trim().to_string(), value.trim().to_string());
2781        }
2782    }
2783    out
2784}
2785
2786/// Splits a whitespace-separated list that may contain arguments quoted by
2787/// [`quote_argument`].
2788fn split_quoted(value: &str) -> Vec<String> {
2789    let mut out = Vec::new();
2790    let mut rest = value.trim();
2791    while !rest.is_empty() {
2792        if rest.starts_with('"') {
2793            // `executable_from_command_line` already implements Windows' quoting
2794            // rules; reuse it rather than write a second, subtly different one.
2795            if let Some(parsed) = executable_from_command_line(rest) {
2796                out.push(parsed.to_string_lossy().into_owned());
2797            }
2798            // Advance past the closing quote.
2799            let mut depth = 0usize;
2800            let mut end = rest.len();
2801            for (index, c) in rest.char_indices() {
2802                match c {
2803                    '\\' => depth += 1,
2804                    '"' => {
2805                        if depth.is_multiple_of(2) && index > 0 {
2806                            end = index + 1;
2807                            break;
2808                        }
2809                        depth = 0;
2810                    }
2811                    _ => depth = 0,
2812                }
2813            }
2814            rest = rest[end..].trim_start();
2815        } else {
2816            let end = rest.find(char::is_whitespace).unwrap_or(rest.len());
2817            out.push(rest[..end].to_string());
2818            rest = rest[end..].trim_start();
2819        }
2820    }
2821    out
2822}
2823
2824/// The text between the **first** `<tag>` and its `</tag>`, unescaped, for the
2825/// small, known documents this module renders. Not a general XML parser and
2826/// does not pretend to be one.
2827///
2828/// "First" matters for a document that repeats a tag: a Task Scheduler task has
2829/// an `<Enabled>` in its trigger *and* one in its settings, so a caller after
2830/// the second one passes the slice that starts at `<Settings>`.
2831///
2832/// `pub(crate)` rather than private because [`crate::wsl::task`] reads back a
2833/// second kind of Task Scheduler document; one reader for both is the same
2834/// argument [`xml_escape`] makes for one escaper.
2835pub(crate) fn xml_value(text: &str, tag: &str) -> Option<String> {
2836    let open = format!("<{tag}>");
2837    let close = format!("</{tag}>");
2838    let start = text.find(&open)? + open.len();
2839    let end = text[start..].find(&close)? + start;
2840    Some(xml_unescape(text[start..end].trim()))
2841}
2842
2843/// Whether Task Scheduler will launch a login task automatically.
2844///
2845/// Its `/Query /XML` output normalises away values that equal the schema
2846/// default. In particular, an enabled task commonly has no `<Enabled>` node at
2847/// all, so only an explicit `false` means disabled.
2848#[cfg(any(windows, test))]
2849fn windows_login_task_starts_automatically(document: &str) -> bool {
2850    document.contains("<LogonTrigger>")
2851        && xml_value(document, "Enabled").as_deref() != Some("false")
2852}
2853
2854/// The `<string>` that follows `<key>key</key>` in a property list.
2855fn plist_value_after_key<'a>(text: &'a str, key: &str) -> Option<&'a str> {
2856    let marker = format!("<key>{key}</key>");
2857    let start = text.find(&marker)? + marker.len();
2858    Some(text[start..].trim_start())
2859}
2860
2861pub fn plist_string_value(text: &str, key: &str) -> Option<String> {
2862    let rest = plist_value_after_key(text, key)?;
2863    if !rest.starts_with("<string>") {
2864        return None;
2865    }
2866    xml_value(rest, "string")
2867}
2868
2869fn plist_bool_value(text: &str, key: &str) -> Option<bool> {
2870    let rest = plist_value_after_key(text, key)?;
2871    if rest.starts_with("<true/>") {
2872        Some(true)
2873    } else if rest.starts_with("<false/>") {
2874        Some(false)
2875    } else {
2876        None
2877    }
2878}
2879
2880// ---------------------------------------------------------------------------
2881// The install record
2882// ---------------------------------------------------------------------------
2883
2884/// The version of [`InstallRecord`] on disk.
2885///
2886/// Bumped when a record written by an older version can no longer be read.
2887/// `service status` reports an unreadable record as a problem with a remedy
2888/// rather than failing, because the remedy — uninstall and install again —
2889/// touches no configuration, no secret and no cache.
2890pub const RECORD_SCHEMA_VERSION: u32 = 1;
2891
2892/// What `service install` wrote down, and what `service status` reads back.
2893///
2894/// This is the *record* half of `05-infrastructure.md` item 6. The service
2895/// manager also knows the binary path, and the two are compared: a registration
2896/// whose command line no longer matches the record is a registration something
2897/// else has edited, and saying so is more useful than picking one of them to
2898/// believe.
2899///
2900/// It holds no credential. The nearest thing is the *location* of the secret
2901/// store, which `d2` publishes for `host show` to print, and even that is
2902/// derived rather than stored.
2903#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
2904pub struct InstallRecord {
2905    /// [`RECORD_SCHEMA_VERSION`] at the time of writing.
2906    pub schema_version: u32,
2907    /// What the operating system calls the registration.
2908    pub service_name: String,
2909    /// Which service manager holds it.
2910    pub manager: String,
2911    /// Boot or login.
2912    pub start_mode: StartMode,
2913    /// The account it runs under.
2914    pub account: ServiceAccount,
2915    /// **The resolved absolute path of the binary at install time.** Item 6.
2916    pub binary: PathBuf,
2917    /// Where [`Self::binary`] was copied from, when it is a copy.
2918    ///
2919    /// `#[serde(default)]` rather than a schema bump: a record written before
2920    /// this existed is still readable, and reads as `None`. That is not a
2921    /// silent downgrade — `None` means the registration names a package
2922    /// manager's own file, which is the layout that cannot be upgraded while
2923    /// the service runs, and `service status` says so by name.
2924    #[serde(default)]
2925    pub source_binary: Option<PathBuf>,
2926    /// The arguments it was registered with.
2927    pub arguments: Vec<String>,
2928    /// The restart-on-failure delay.
2929    pub restart_delay_secs: u64,
2930    /// How long it must run before the failure count resets.
2931    pub restart_reset_secs: u64,
2932    /// The rotating diagnostic log's stem. Item 4.
2933    pub log_file: PathBuf,
2934    /// Whether the manager was asked not to start this by itself. Always
2935    /// `false` for a registration an operator made; see
2936    /// [`InstallRequest::started_on_demand`].
2937    ///
2938    /// `#[serde(default)]` so that a record missing the field is rejected by
2939    /// the schema check below, with its remedy, rather than by a parse error
2940    /// that names a field an operator has never heard of.
2941    #[serde(default)]
2942    pub starts_on_demand: bool,
2943    /// Where the platform's definition file was written, when there is one.
2944    pub definition_path: Option<PathBuf>,
2945    /// When the registration was made.
2946    pub installed_at: DateTime<Utc>,
2947    /// Which build of the product made it.
2948    pub installed_by_version: String,
2949    // A TOML table has to follow every scalar at its level, so this field is
2950    // last by necessity rather than by taste.
2951    /// The four directories the registration was installed against. Item 2's
2952    /// *"configured cache and runtime directories"*.
2953    pub directories: ServiceDirectories,
2954}
2955
2956impl InstallRecord {
2957    /// Where the record lives, given this host's directories.
2958    #[must_use]
2959    pub fn path(paths: &AppPaths) -> PathBuf {
2960        paths.config_dir().join(RECORD_FILE)
2961    }
2962
2963    /// Builds the record for a plan that has just been applied.
2964    #[must_use]
2965    pub fn of(plan: &InstallPlan, definition: &ServiceDefinition, at: DateTime<Utc>) -> Self {
2966        Self {
2967            schema_version: RECORD_SCHEMA_VERSION,
2968            service_name: plan.identity().name().to_string(),
2969            manager: definition.kind().manager().to_string(),
2970            start_mode: plan.start_mode(),
2971            account: plan.account().clone(),
2972            binary: plan.binary().to_path_buf(),
2973            arguments: plan
2974                .arguments()
2975                .iter()
2976                .map(|argument| argument.to_string_lossy().into_owned())
2977                .collect(),
2978            restart_delay_secs: plan.restart().delay().as_secs(),
2979            restart_reset_secs: plan.restart().reset_after().as_secs(),
2980            starts_on_demand: plan.is_on_demand(),
2981            source_binary: plan.source_binary().map(Path::to_path_buf),
2982            log_file: plan.directories().log_file(),
2983            definition_path: definition.install_path().map(Path::to_path_buf),
2984            installed_at: at,
2985            installed_by_version: env!("CARGO_PKG_VERSION").to_string(),
2986            directories: plan.directories().clone(),
2987        }
2988    }
2989
2990    /// The plan this record describes, as this build would register it.
2991    ///
2992    /// The directories, binary, arguments and restart policy are the recorded
2993    /// ones; everything else -- hardening, the secret store's guard -- is what
2994    /// this build renders. That difference is the point: it is how a
2995    /// registration written by an older build is compared against, and
2996    /// repaired to, the current definition.
2997    #[must_use]
2998    pub fn plan(&self, identity: ServiceIdentity) -> InstallPlan {
2999        let plan = InstallPlan::unchecked(
3000            identity,
3001            self.start_mode,
3002            self.binary.clone(),
3003            self.directories.clone(),
3004        )
3005        .with_arguments(self.arguments.clone())
3006        .with_restart(self.restart());
3007        let plan = if self.starts_on_demand {
3008            plan.started_on_demand()
3009        } else {
3010            plan
3011        };
3012        match crate::secrets::PlatformSecretStore::for_start_mode(self.start_mode) {
3013            Ok(store) => plan.with_secret_guard(store.guard()),
3014            Err(_) => plan,
3015        }
3016    }
3017
3018    /// The restart policy this record describes, or [`RestartPolicy::default`]
3019    /// when the recorded numbers are outside the supported range — which can
3020    /// only happen to a record something else has edited.
3021    #[must_use]
3022    pub fn restart(&self) -> RestartPolicy {
3023        RestartPolicy::new(
3024            Duration::from_secs(self.restart_delay_secs),
3025            Duration::from_secs(self.restart_reset_secs),
3026        )
3027        .unwrap_or_default()
3028    }
3029
3030    /// Reads the record, or reports that there is none.
3031    ///
3032    /// `Ok(None)` means no registration was made by this product on this host.
3033    ///
3034    /// # Errors
3035    ///
3036    /// [`ServiceError::Record`] when the file exists and cannot be read,
3037    /// [`ServiceError::RecordNotPermitted`] when it exists and this account may
3038    /// not read it, and [`ServiceError::RecordUnreadable`] when it is not a
3039    /// record this version understands.
3040    pub fn read(paths: &AppPaths) -> Result<Option<Self>, ServiceError> {
3041        let path = Self::path(paths);
3042        let text = match std::fs::read_to_string(&path) {
3043            Ok(text) => text,
3044            Err(error) if error.kind() == std::io::ErrorKind::NotFound => return Ok(None),
3045            // Told apart from every other read failure because `status` carries
3046            // on through this one alone. See `ServiceError::RecordNotPermitted`.
3047            Err(error) if error.kind() == std::io::ErrorKind::PermissionDenied => {
3048                return Err(ServiceError::RecordNotPermitted {
3049                    path,
3050                    detail: error.to_string(),
3051                });
3052            }
3053            Err(error) => {
3054                return Err(ServiceError::Record {
3055                    operation: "read",
3056                    path,
3057                    detail: error.to_string(),
3058                });
3059            }
3060        };
3061        let record: Self =
3062            toml::from_str(&text).map_err(|error| ServiceError::RecordUnreadable {
3063                path: path.clone(),
3064                detail: error.to_string(),
3065            })?;
3066        if record.schema_version != RECORD_SCHEMA_VERSION {
3067            return Err(ServiceError::RecordUnreadable {
3068                path,
3069                detail: format!(
3070                    "it declares schema version {} and this build reads version {}",
3071                    record.schema_version, RECORD_SCHEMA_VERSION
3072                ),
3073            });
3074        }
3075        Ok(Some(record))
3076    }
3077
3078    /// Writes the record, replacing whatever was there.
3079    ///
3080    /// # Errors
3081    ///
3082    /// [`ServiceError::Record`].
3083    pub fn write(&self, paths: &AppPaths) -> Result<(), ServiceError> {
3084        use std::io::Write as _;
3085
3086        let path = Self::path(paths);
3087        let text = toml::to_string_pretty(self).map_err(|error| ServiceError::Record {
3088            operation: "encode",
3089            path: path.clone(),
3090            detail: error.to_string(),
3091        })?;
3092        if let Some(parent) = path.parent() {
3093            std::fs::create_dir_all(parent).map_err(|error| ServiceError::Record {
3094                operation: "write",
3095                path: path.clone(),
3096                detail: error.to_string(),
3097            })?;
3098        }
3099        let parent = path.parent().ok_or_else(|| ServiceError::Record {
3100            operation: "write",
3101            path: path.clone(),
3102            detail: "the record path has no parent directory".to_string(),
3103        })?;
3104        let mut temporary =
3105            tempfile::NamedTempFile::new_in(parent).map_err(|error| ServiceError::Record {
3106                operation: "write",
3107                path: path.clone(),
3108                detail: error.to_string(),
3109            })?;
3110        temporary
3111            .write_all(text.as_bytes())
3112            .and_then(|()| temporary.as_file().sync_all())
3113            .map_err(|error| ServiceError::Record {
3114                operation: "write",
3115                path: path.clone(),
3116                detail: error.to_string(),
3117            })?;
3118        // ------------------------------------------------------------------
3119        // THE RECORD IS READABLE BY THE ACCOUNT WHOSE DIRECTORY IT IS IN.
3120        // ------------------------------------------------------------------
3121        // `NamedTempFile` creates at `0600`, which is right for a temporary
3122        // file and wrong for this one. A boot-mode `service install` runs under
3123        // `sudo`, so `0600` made the record `root`-owned and unreadable by the
3124        // operator — and `service status`, which asks no privilege of anybody,
3125        // failed on their own host with `Permission denied` and a remedy that
3126        // named the command that had just failed.
3127        //
3128        // `0644` does not widen anything: `RECORD_FILE`'s own documentation is
3129        // that this is *"non-secret TOML"* holding no credential, and
3130        // `AppPaths::create_all` puts it in a `0700` directory, so no other
3131        // local account can reach it whatever its own mode says. It is the same
3132        // arrangement the rotating diagnostics already have.
3133        #[cfg(unix)]
3134        {
3135            use std::os::unix::fs::PermissionsExt as _;
3136
3137            temporary
3138                .as_file()
3139                .set_permissions(std::fs::Permissions::from_mode(0o644))
3140                .map_err(|error| ServiceError::Record {
3141                    operation: "write",
3142                    path: path.clone(),
3143                    detail: error.to_string(),
3144                })?;
3145        }
3146        temporary
3147            .persist(&path)
3148            .map(|_| ())
3149            .map_err(|error| ServiceError::Record {
3150                operation: "write",
3151                path,
3152                detail: error.error.to_string(),
3153            })
3154    }
3155
3156    /// Removes the record. Returns whether there was one.
3157    ///
3158    /// # Errors
3159    ///
3160    /// [`ServiceError::Record`].
3161    pub fn remove(paths: &AppPaths) -> Result<bool, ServiceError> {
3162        let path = Self::path(paths);
3163        match std::fs::remove_file(&path) {
3164            Ok(()) => Ok(true),
3165            Err(error) if error.kind() == std::io::ErrorKind::NotFound => Ok(false),
3166            Err(error) => Err(ServiceError::Record {
3167                operation: "remove",
3168                path,
3169                detail: error.to_string(),
3170            }),
3171        }
3172    }
3173}
3174
3175// ---------------------------------------------------------------------------
3176// The last successful GitHub contact
3177// ---------------------------------------------------------------------------
3178
3179/// The heartbeat file's own schema version.
3180const CONTACT_SCHEMA_VERSION: u32 = 1;
3181
3182#[derive(Debug, Clone, Serialize, Deserialize)]
3183struct ContactRecord {
3184    schema_version: u32,
3185    last_success: DateTime<Utc>,
3186}
3187
3188/// Records that GitHub was reached successfully, for `service status` to
3189/// report.
3190///
3191/// # The contract, for `f3` and for the agent
3192///
3193/// Journey 5 step 4 requires `service status` to report *"the last successful
3194/// GitHub contact"*, and `service status` runs in the operator's terminal while
3195/// the daemon runs in a service. They share no memory, so the fact has to be on
3196/// disk, and this is the file. **The daemon calls this after each successful
3197/// GitHub call; nothing else writes it.**
3198///
3199/// It is a single timestamp and deliberately not a log: an operator asking this
3200/// question wants to know whether the agent is alive and reaching GitHub *now*,
3201/// and a value that is minutes old answers it. Writing it is a whole-file
3202/// replace through a temporary in the same directory, so a status command can
3203/// never read a half-written timestamp.
3204///
3205/// # Errors
3206///
3207/// [`ServiceError::Record`] when `state/` cannot be written.
3208pub fn record_github_contact(paths: &AppPaths, at: DateTime<Utc>) -> Result<(), ServiceError> {
3209    write_state_record(
3210        &contact_path(paths),
3211        &ContactRecord {
3212            schema_version: CONTACT_SCHEMA_VERSION,
3213            last_success: at,
3214        },
3215    )
3216}
3217
3218/// Reads the last successful GitHub contact, or reports that none was recorded.
3219///
3220/// `Ok(None)` is the honest answer on a host whose agent has never run, and it
3221/// is what `service status` prints rather than a zero timestamp.
3222///
3223/// # Errors
3224///
3225/// [`ServiceError::Record`] when the file exists and cannot be read or parsed.
3226/// A malformed heartbeat is reported rather than treated as absence: absence
3227/// means *"the agent has never reached GitHub"*, and reporting a parse failure
3228/// as that would be a wrong answer to the question Journey 5 asks.
3229pub fn last_github_contact(paths: &AppPaths) -> Result<Option<DateTime<Utc>>, ServiceError> {
3230    Ok(read_state_record::<ContactRecord>(&contact_path(paths))?.map(|record| record.last_success))
3231}
3232
3233/// Where the heartbeat lives.
3234#[must_use]
3235pub fn contact_path(paths: &AppPaths) -> PathBuf {
3236    paths.state_dir().join(CONTACT_FILE)
3237}
3238
3239/// Writes a small TOML record under `state/`, through a temporary in the same
3240/// directory and a rename, so a status command never reads half of one.
3241pub(crate) fn write_state_record(path: &Path, record: &impl Serialize) -> Result<(), ServiceError> {
3242    let failed = |detail: String| ServiceError::Record {
3243        operation: "write",
3244        path: path.to_path_buf(),
3245        detail,
3246    };
3247    let text = toml::to_string_pretty(record).map_err(|error| failed(error.to_string()))?;
3248    let directory = path.parent().unwrap_or_else(|| Path::new("."));
3249    std::fs::create_dir_all(directory).map_err(|error| failed(error.to_string()))?;
3250    let temporary = path.with_extension("toml.new");
3251    std::fs::write(&temporary, text).map_err(|error| failed(error.to_string()))?;
3252    std::fs::rename(&temporary, path).map_err(|error| failed(error.to_string()))
3253}
3254
3255/// Reads a record [`write_state_record`] wrote. `Ok(None)` when there is none;
3256/// one that exists and does not parse is an error, never absence.
3257pub(crate) fn read_state_record<T: serde::de::DeserializeOwned>(
3258    path: &Path,
3259) -> Result<Option<T>, ServiceError> {
3260    let failed = |detail: String| ServiceError::Record {
3261        operation: "read",
3262        path: path.to_path_buf(),
3263        detail,
3264    };
3265    let text = match std::fs::read_to_string(path) {
3266        Ok(text) => text,
3267        Err(error) if error.kind() == std::io::ErrorKind::NotFound => return Ok(None),
3268        Err(error) => return Err(failed(error.to_string())),
3269    };
3270    toml::from_str(&text)
3271        .map(Some)
3272        .map_err(|error| failed(error.to_string()))
3273}
3274
3275/// Removes a record; one that is not there is already removed.
3276pub(crate) fn remove_state_record(path: &Path) -> Result<(), ServiceError> {
3277    match std::fs::remove_file(path) {
3278        Ok(()) => Ok(()),
3279        Err(error) if error.kind() == std::io::ErrorKind::NotFound => Ok(()),
3280        Err(error) => Err(ServiceError::Record {
3281            operation: "remove",
3282            path: path.to_path_buf(),
3283            detail: error.to_string(),
3284        }),
3285    }
3286}
3287
3288// ---------------------------------------------------------------------------
3289// The credential GitHub rejected
3290// ---------------------------------------------------------------------------
3291
3292#[derive(Debug, Serialize, Deserialize)]
3293#[serde(deny_unknown_fields)]
3294struct CredentialRejectionRecord {
3295    schema_version: u32,
3296    rejected_since: DateTime<Utc>,
3297}
3298
3299/// Records that GitHub rejected the daemon's credential, keeping the first
3300/// moment of an ongoing rejection.
3301///
3302/// # Why a file of its own
3303///
3304/// A rejected credential used to be visible only in the daemon's log: the
3305/// contact heartbeat simply stopped moving, `service status` still printed
3306/// `healthy`, and the TUI -- which authenticates with its own, perhaps freshly
3307/// renewed, copy -- showed nothing at all while the service started no runner
3308/// for hours. Separate from [`CONTACT_FILE`] so that an older build reading
3309/// that file is not handed a field it refuses.
3310///
3311/// # Errors
3312///
3313/// [`ServiceError::Record`] when `state/` cannot be written.
3314pub fn record_github_credential_rejected(
3315    paths: &AppPaths,
3316    at: DateTime<Utc>,
3317) -> Result<(), ServiceError> {
3318    if github_credential_rejected_since(paths)
3319        .ok()
3320        .flatten()
3321        .is_some()
3322    {
3323        return Ok(());
3324    }
3325    write_state_record(
3326        &credential_rejection_path(paths),
3327        &CredentialRejectionRecord {
3328            schema_version: CONTACT_SCHEMA_VERSION,
3329            rejected_since: at,
3330        },
3331    )
3332}
3333
3334/// Clears a recorded rejection once GitHub accepts the credential again.
3335///
3336/// # Errors
3337///
3338/// [`ServiceError::Record`] when the record exists and cannot be removed.
3339pub fn clear_github_credential_rejection(paths: &AppPaths) -> Result<(), ServiceError> {
3340    remove_state_record(&credential_rejection_path(paths))
3341}
3342
3343/// Since when GitHub has been rejecting the daemon's credential, if it is.
3344///
3345/// # Errors
3346///
3347/// [`ServiceError::Record`] when the file exists and cannot be read or parsed.
3348pub fn github_credential_rejected_since(
3349    paths: &AppPaths,
3350) -> Result<Option<DateTime<Utc>>, ServiceError> {
3351    Ok(
3352        read_state_record::<CredentialRejectionRecord>(&credential_rejection_path(paths))?
3353            .map(|record| record.rejected_since),
3354    )
3355}
3356
3357/// Where the rejection record lives.
3358#[must_use]
3359pub fn credential_rejection_path(paths: &AppPaths) -> PathBuf {
3360    paths.state_dir().join(CREDENTIAL_REJECTION_FILE)
3361}
3362
3363#[derive(Debug, Serialize, Deserialize)]
3364#[serde(deny_unknown_fields)]
3365struct CredentialUnreadableRecord {
3366    schema_version: u32,
3367    unreadable_since: DateTime<Utc>,
3368}
3369
3370/// Records that the daemon could not read its stored credential, keeping the
3371/// first moment of an ongoing failure.
3372///
3373/// # Why a file of its own
3374///
3375/// A daemon that cannot read its credential exits, and the service manager
3376/// starts it again a few seconds later, for ever. `service status` reads the
3377/// registration, which is fine, and the heartbeat, which merely stops moving --
3378/// so it said `healthy` while a macOS host that had just been upgraded was
3379/// locked out of its own keychain item and started no runner. The reason was
3380/// only in the daemon's log. The daemon is the one process whose view of the
3381/// store matters, and it shares no memory with `service status`, so it says
3382/// so on disk; [`clear_credential_unreadable`] removes it as soon as a read
3383/// succeeds.
3384///
3385/// # Errors
3386///
3387/// [`ServiceError::Record`] when `state/` cannot be written.
3388pub fn record_credential_unreadable(
3389    paths: &AppPaths,
3390    at: DateTime<Utc>,
3391) -> Result<(), ServiceError> {
3392    if credential_unreadable_since(paths).ok().flatten().is_some() {
3393        return Ok(());
3394    }
3395    write_state_record(
3396        &credential_unreadable_path(paths),
3397        &CredentialUnreadableRecord {
3398            schema_version: CONTACT_SCHEMA_VERSION,
3399            unreadable_since: at,
3400        },
3401    )
3402}
3403
3404/// Clears a recorded failure once the daemon reads its credential again.
3405///
3406/// # Errors
3407///
3408/// [`ServiceError::Record`] when the record exists and cannot be removed.
3409pub fn clear_credential_unreadable(paths: &AppPaths) -> Result<(), ServiceError> {
3410    remove_state_record(&credential_unreadable_path(paths))
3411}
3412
3413/// Since when the daemon has been unable to read its stored credential, if it
3414/// has.
3415///
3416/// # Errors
3417///
3418/// [`ServiceError::Record`] when the file exists and cannot be read or parsed.
3419pub fn credential_unreadable_since(
3420    paths: &AppPaths,
3421) -> Result<Option<DateTime<Utc>>, ServiceError> {
3422    Ok(
3423        read_state_record::<CredentialUnreadableRecord>(&credential_unreadable_path(paths))?
3424            .map(|record| record.unreadable_since),
3425    )
3426}
3427
3428/// Where the unreadable-credential record lives.
3429#[must_use]
3430pub fn credential_unreadable_path(paths: &AppPaths) -> PathBuf {
3431    paths.state_dir().join(CREDENTIAL_UNREADABLE_FILE)
3432}
3433
3434/// How to sign this host in again for `mode`: the exact command, in
3435/// backticks, and where to run it when that matters.
3436///
3437/// A machine-scoped store needs privilege: `sudo` on Unix, an elevated
3438/// terminal on Windows. A login-mode store on macOS is the login keychain,
3439/// which is unlocked for the desktop session only, so an SSH session cannot
3440/// write it.
3441#[must_use]
3442pub fn sign_in_instruction(mode: StartMode) -> String {
3443    let (command, place) = match mode {
3444        StartMode::Boot if cfg!(unix) => ("sudo runner-manager auth login --start-at boot", ""),
3445        StartMode::Boot => (
3446            "runner-manager auth login --start-at boot",
3447            " from an elevated terminal",
3448        ),
3449        StartMode::Login => (
3450            "runner-manager auth login --start-at login",
3451            MACOS_LOGIN_PLACE,
3452        ),
3453    };
3454    format!("`{command}`{place}")
3455}
3456
3457/// Where a login-mode sign-in has to happen on macOS, as a phrase to follow
3458/// the command; nothing elsewhere.
3459pub const MACOS_LOGIN_PLACE: &str = if cfg!(target_os = "macos") {
3460    " in a Terminal in this Mac's desktop session (an SSH session cannot unlock the login \
3461     keychain)"
3462} else {
3463    ""
3464};
3465
3466// ---------------------------------------------------------------------------
3467// A definition written by an older build
3468// ---------------------------------------------------------------------------
3469
3470/// An installed service definition that differs from what this build renders
3471/// for the same recorded installation.
3472///
3473/// # Why this exists
3474///
3475/// An upgrade replaces the binary a registration runs and nothing else. A
3476/// systemd unit written by 0.4.7 still carried `LoadCredential=`, which hands
3477/// the daemon a copy of the credential frozen at service start, and lacked
3478/// write access to the credential store. Every release after the fix kept
3479/// running under that unit: the daemon could neither renew the token nor see
3480/// one renewed by anybody else, and stopped starting runners the moment the
3481/// frozen access token expired.
3482#[derive(Debug, Clone, PartialEq, Eq)]
3483pub struct DefinitionDrift {
3484    /// The definition file on disk.
3485    pub path: PathBuf,
3486    /// The build that wrote it, as its install record says.
3487    pub installed_by_version: String,
3488    /// What this build renders in its place.
3489    pub rendered: String,
3490    /// The start mode it was installed for.
3491    pub start_mode: StartMode,
3492    /// Which manager reads the definition.
3493    pub manager: DefinitionKind,
3494}
3495
3496/// Compares the installed definition against this build's rendering.
3497///
3498/// A systemd unit is compared byte for byte. A launchd plist is compared on
3499/// `ProcessType` alone (see [`launchd_definition_drift`]). `None` for every
3500/// other manager, for a definition that cannot be read, and for one that
3501/// already matches. The file is read through `controls`, so a double never
3502/// compares against a real host's definition.
3503fn definition_drift(
3504    controls: &dyn ControlFactory,
3505    identity: &ServiceIdentity,
3506    record: &InstallRecord,
3507) -> Option<DefinitionDrift> {
3508    let compare = match host_definition_kind(record.start_mode) {
3509        DefinitionKind::SystemdUnit => systemd_definition_drift,
3510        DefinitionKind::LaunchdPlist => launchd_definition_drift,
3511        DefinitionKind::WindowsService | DefinitionKind::WindowsScheduledTask => return None,
3512    };
3513    let path = record.definition_path.clone()?;
3514    let installed = controls.installed_definition(&path)?;
3515    compare(identity, record, path, &installed)
3516}
3517
3518/// A plist whose `ProcessType` is not [`LAUNCHD_PROCESS_TYPE`], with the plist
3519/// this build renders in its place.
3520///
3521/// Only that one key, unlike the systemd comparison. The daemon repairs this
3522/// drift by itself (`daemon.rs`, `converge_launchd_definition`), so a
3523/// whole-file comparison would make it rewrite and reload over every edit an
3524/// operator made to the plist. A plist that does carry the wrong value is
3525/// replaced whole, as `service install` would replace it.
3526fn launchd_definition_drift(
3527    identity: &ServiceIdentity,
3528    record: &InstallRecord,
3529    path: PathBuf,
3530    installed: &str,
3531) -> Option<DefinitionDrift> {
3532    (plist_string_value(installed, "ProcessType").as_deref() != Some(LAUNCHD_PROCESS_TYPE)).then(
3533        || DefinitionDrift {
3534            path,
3535            installed_by_version: record.installed_by_version.clone(),
3536            rendered: launchd_plist(&record.plan(identity.clone())),
3537            start_mode: record.start_mode,
3538            manager: DefinitionKind::LaunchdPlist,
3539        },
3540    )
3541}
3542
3543fn systemd_definition_drift(
3544    identity: &ServiceIdentity,
3545    record: &InstallRecord,
3546    path: PathBuf,
3547    installed: &str,
3548) -> Option<DefinitionDrift> {
3549    let rendered = systemd_unit(&record.plan(identity.clone()));
3550    (installed != rendered).then(|| DefinitionDrift {
3551        path,
3552        installed_by_version: record.installed_by_version.clone(),
3553        rendered,
3554        start_mode: record.start_mode,
3555        manager: DefinitionKind::SystemdUnit,
3556    })
3557}
3558
3559// ---------------------------------------------------------------------------
3560// The last runner-root refusal
3561// ---------------------------------------------------------------------------
3562
3563const ROOT_REFUSAL_SCHEMA_VERSION: u32 = 1;
3564
3565#[derive(Debug, Clone, Serialize, Deserialize)]
3566struct RootRefusalFile {
3567    schema_version: u32,
3568    /// Keyed by policy, because the fact is per-policy and not per-host.
3569    #[serde(default)]
3570    refusals: BTreeMap<String, RootRefusalEntry>,
3571}
3572
3573#[derive(Debug, Clone, Serialize, Deserialize)]
3574struct RootRefusalEntry {
3575    at: DateTime<Utc>,
3576    kind: String,
3577    root: String,
3578    detail: String,
3579}
3580
3581/// One policy's last runner-root refusal, as `service status` reports it.
3582#[derive(Debug, Clone, PartialEq, Eq)]
3583pub struct RunnerRootRefusal {
3584    /// The policy that placed no runner.
3585    pub policy: String,
3586    /// When it was last refused.
3587    pub at: DateTime<Utc>,
3588    /// The closed-vocabulary cause, as `RunnerRootError::kind` names it.
3589    pub kind: String,
3590    /// The root it was refused.
3591    pub root: String,
3592    /// The refusal in full, including the remediation. Not redacted: this file
3593    /// is read by a command in the operator's terminal, not by the log sink.
3594    pub detail: String,
3595}
3596
3597/// Records that one policy could not use its runner root.
3598///
3599/// # The contract, and why a file
3600///
3601/// A launch the runner root refuses fails before any attempt row exists, so the
3602/// journal cannot carry it and no screen can show it. The daemon's own log
3603/// cannot either: it redacts every field it does not allow-list and scrubs
3604/// anything path-shaped out of the ones it does, which is most of a
3605/// `RunnerRootError`. So the sentence an operator needs — the directory and the
3606/// remediation, verbatim — has nowhere to go, and the failure reads
3607/// `runner_start_failed reason=other` once per poll for as long as it lasts.
3608/// That is not hypothetical: it cost three hours.
3609///
3610/// **Keyed by policy, and that is load-bearing.** A host runs several policies
3611/// and they do not share a fate: `allocate_persistent_slot` tolerates an
3612/// unresolvable host default, so a repository with its own persistent root
3613/// places runners happily while an ephemeral policy on a withheld volume places
3614/// none. A single host-wide slot would have the working policy clear the broken
3615/// one's record on the same reconcile pass, and `service status` would report a
3616/// healthy host that had started zero runners for an entire target.
3617///
3618/// **The daemon writes this when a root refuses a launch and clears the policy's
3619/// entry when one succeeds; nothing else writes it.** It keeps the last refusal
3620/// per policy rather than a history, for the reason [`record_github_contact`] is
3621/// a single timestamp: the operator is asking whether runners can be placed
3622/// *now*.
3623///
3624/// Written whole through a temporary in the same directory, so a status command
3625/// cannot read half a record.
3626///
3627/// # Errors
3628///
3629/// [`ServiceError::Record`] when `state/` cannot be read or written.
3630pub fn record_runner_root_refusal(
3631    paths: &AppPaths,
3632    policy: &str,
3633    at: DateTime<Utc>,
3634    kind: &str,
3635    root: &str,
3636    detail: &str,
3637) -> Result<(), ServiceError> {
3638    let mut file = read_refusal_file(paths)?.unwrap_or(RootRefusalFile {
3639        schema_version: ROOT_REFUSAL_SCHEMA_VERSION,
3640        refusals: BTreeMap::new(),
3641    });
3642    file.schema_version = ROOT_REFUSAL_SCHEMA_VERSION;
3643    file.refusals.insert(
3644        policy.to_owned(),
3645        RootRefusalEntry {
3646            at,
3647            kind: kind.to_owned(),
3648            root: root.to_owned(),
3649            detail: detail.to_owned(),
3650        },
3651    );
3652    write_refusal_file(paths, &file)
3653}
3654
3655/// Clears one policy's entry, because that policy placed a runner.
3656///
3657/// Removes the file once the last entry goes, so a host that is working leaves
3658/// nothing behind for a later `service status` to report. A policy with no entry
3659/// is not an error: this is called on every successful placement, and almost
3660/// every one of those follows another success.
3661///
3662/// # Errors
3663///
3664/// [`ServiceError::Record`] when `state/` cannot be read or written.
3665pub fn clear_runner_root_refusal(paths: &AppPaths, policy: &str) -> Result<(), ServiceError> {
3666    let Some(mut file) = read_refusal_file(paths)? else {
3667        return Ok(());
3668    };
3669    if file.refusals.remove(policy).is_none() {
3670        return Ok(());
3671    }
3672    if file.refusals.is_empty() {
3673        let path = root_refusal_path(paths);
3674        return match std::fs::remove_file(&path) {
3675            Ok(()) => Ok(()),
3676            Err(error) if error.kind() == std::io::ErrorKind::NotFound => Ok(()),
3677            Err(error) => Err(ServiceError::Record {
3678                operation: "remove",
3679                path,
3680                detail: error.to_string(),
3681            }),
3682        };
3683    }
3684    write_refusal_file(paths, &file)
3685}
3686
3687/// Every policy whose runner root refused it and has not since succeeded.
3688///
3689/// Ordered by policy, so two runs of `service status` on an unchanged host
3690/// print the same thing in the same order.
3691///
3692/// # Errors
3693///
3694/// [`ServiceError::Record`] when the file exists and cannot be read or parsed.
3695/// Reported rather than treated as absence, for the reason
3696/// [`last_github_contact`] gives: absence means "every policy is placing
3697/// runners", and answering a parse failure with that would be a wrong answer.
3698pub fn runner_root_refusals(paths: &AppPaths) -> Result<Vec<RunnerRootRefusal>, ServiceError> {
3699    Ok(read_refusal_file(paths)?
3700        .map(|file| {
3701            file.refusals
3702                .into_iter()
3703                .map(|(policy, entry)| RunnerRootRefusal {
3704                    policy,
3705                    at: entry.at,
3706                    kind: entry.kind,
3707                    root: entry.root,
3708                    detail: entry.detail,
3709                })
3710                .collect()
3711        })
3712        .unwrap_or_default())
3713}
3714
3715fn read_refusal_file(paths: &AppPaths) -> Result<Option<RootRefusalFile>, ServiceError> {
3716    read_state_record(&root_refusal_path(paths))
3717}
3718
3719fn write_refusal_file(paths: &AppPaths, file: &RootRefusalFile) -> Result<(), ServiceError> {
3720    write_state_record(&root_refusal_path(paths), file)
3721}
3722
3723/// Where the refusals live.
3724#[must_use]
3725pub fn root_refusal_path(paths: &AppPaths) -> PathBuf {
3726    paths.state_dir().join(ROOT_REFUSAL_FILE)
3727}
3728
3729// ---------------------------------------------------------------------------
3730// The recorded binary path
3731// ---------------------------------------------------------------------------
3732
3733/// What became of the absolute path `install` recorded.
3734///
3735/// `05-infrastructure.md` item 6 in one type. Three of the four variants are
3736/// errors, and `service status` reports them as errors rather than as health —
3737/// which is the whole point, because the npm case produces a service that looks
3738/// installed, is registered, and cannot start.
3739#[derive(Debug, Clone, PartialEq, Eq)]
3740pub enum BinaryPath {
3741    /// The recorded path is a file, and the service manager still starts it.
3742    Current {
3743        /// The path.
3744        path: PathBuf,
3745    },
3746    /// **Nothing is at the recorded path.**
3747    ///
3748    /// This is the npm upgrade: `npm i -g @ivan-murzak/runner-manager` puts the binary under
3749    /// the active Node installation's global prefix, and switching Node versions
3750    /// with `nvm`, `fnm`, or `volta` moves that prefix. The service is still
3751    /// registered, still set to start at boot, and starts nothing.
3752    Missing {
3753        /// The path the record names.
3754        recorded: PathBuf,
3755    },
3756    /// Something is at the recorded path, but it is not a file the service
3757    /// manager could start.
3758    NotExecutable {
3759        /// The path the record names.
3760        recorded: PathBuf,
3761        /// What is there instead.
3762        detail: String,
3763    },
3764    /// The record and the service manager name different binaries.
3765    Diverged {
3766        /// What the record says.
3767        recorded: PathBuf,
3768        /// What the service manager is registered to start.
3769        registered: PathBuf,
3770    },
3771}
3772
3773impl BinaryPath {
3774    /// Whether this is a state `service status` must report as an error.
3775    #[must_use]
3776    pub const fn is_error(&self) -> bool {
3777        !matches!(self, Self::Current { .. })
3778    }
3779
3780    /// The path the record names, whatever state it is in.
3781    #[must_use]
3782    pub fn recorded(&self) -> &Path {
3783        match self {
3784            Self::Current { path } => path,
3785            Self::Missing { recorded }
3786            | Self::NotExecutable { recorded, .. }
3787            | Self::Diverged { recorded, .. } => recorded,
3788        }
3789    }
3790}
3791
3792impl fmt::Display for BinaryPath {
3793    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
3794        match self {
3795            Self::Current { path } => write!(f, "{}", path.display()),
3796            Self::Missing { recorded } => write!(
3797                f,
3798                "{} -- STALE: nothing is at the recorded path, so the service cannot start. A \
3799                 package manager that moved the binary is the usual cause; an `npm i -g` \
3800                 installation moves with the active Node version. Run `service install` again \
3801                 from the binary that is now installed.",
3802                recorded.display()
3803            ),
3804            Self::NotExecutable { recorded, detail } => write!(
3805                f,
3806                "{} -- STALE: {detail}, so the service cannot start. Run `service install` again \
3807                 from the installed binary.",
3808                recorded.display()
3809            ),
3810            Self::Diverged {
3811                recorded,
3812                registered,
3813            } => write!(
3814                f,
3815                "{} -- STALE: the service manager is registered to start {} instead. Something \
3816                 has edited the registration since it was installed. Run `service uninstall` and \
3817                 `service install`; neither touches configuration, secrets, or the cache.",
3818                recorded.display(),
3819                registered.display()
3820            ),
3821        }
3822    }
3823}
3824
3825/// Decides what became of a recorded path.
3826///
3827/// `registered` is what the service manager says it will start, when the
3828/// platform can be asked. Order matters and is deliberate: **absence is checked
3829/// first**, because a missing binary is the failure item 6 exists for and an
3830/// operator hearing "the registration disagrees with the record" about a file
3831/// that is not there would be sent to the wrong problem.
3832#[must_use]
3833pub fn inspect_binary(recorded: &Path, registered: Option<&Path>) -> BinaryPath {
3834    match std::fs::metadata(recorded) {
3835        Err(error) if error.kind() == std::io::ErrorKind::NotFound => {
3836            return BinaryPath::Missing {
3837                recorded: recorded.to_path_buf(),
3838            };
3839        }
3840        Err(error) => {
3841            return BinaryPath::NotExecutable {
3842                recorded: recorded.to_path_buf(),
3843                detail: format!("it cannot be inspected ({error})"),
3844            };
3845        }
3846        Ok(metadata) if !metadata.is_file() => {
3847            return BinaryPath::NotExecutable {
3848                recorded: recorded.to_path_buf(),
3849                detail: "what is there is not a file".to_string(),
3850            };
3851        }
3852        Ok(_) => {}
3853    }
3854    if let Some(registered) = registered
3855        && !same_path_text(&recorded.to_string_lossy(), &registered.to_string_lossy())
3856    {
3857        return BinaryPath::Diverged {
3858            recorded: recorded.to_path_buf(),
3859            registered: registered.to_path_buf(),
3860        };
3861    }
3862    BinaryPath::Current {
3863        path: recorded.to_path_buf(),
3864    }
3865}
3866
3867// ---------------------------------------------------------------------------
3868// The control seam
3869// ---------------------------------------------------------------------------
3870
3871/// What a service manager says about a registration it holds.
3872#[derive(Debug, Clone, PartialEq, Eq)]
3873pub struct Registration {
3874    /// Which manager holds it, which is also what decides the start mode:
3875    /// a Windows service is a boot registration and a Task Scheduler task is a
3876    /// login one; a LaunchDaemon and a system unit are boot, a LaunchAgent and
3877    /// a user unit are login.
3878    pub manager: DefinitionKind,
3879    /// The start mode the *domain* it was found in implies.
3880    pub start_mode: StartMode,
3881    /// The full command line, as the manager stores it.
3882    pub command_line: String,
3883    /// The account it runs under, when the manager reports one.
3884    pub account: Option<String>,
3885    /// Whether it is running now.
3886    pub running: bool,
3887    /// Whether it will start by itself.
3888    ///
3889    /// Distinct from [`Registration::start_mode`]: a Windows service can be
3890    /// registered in the boot domain and still be set to `demand` start, which
3891    /// is a service that exists, looks installed, and does not come back after
3892    /// a reboot.
3893    pub starts_automatically: bool,
3894    /// The restart-on-failure delay the manager reports, when it reports one.
3895    ///
3896    /// The *delay* rather than the whole [`RestartPolicy`], because that is the
3897    /// half every one of the four managers can be asked for. The failure-count
3898    /// reset window has no representation at all in Task Scheduler or launchd,
3899    /// so a `RestartPolicy` read back from them would carry one number the
3900    /// manager reported and one this module invented — and a comparison against
3901    /// an invented number is a comparison that cannot fail.
3902    pub restart_delay: Option<Duration>,
3903}
3904
3905impl Registration {
3906    /// The executable the manager will start, parsed out of the command line.
3907    #[must_use]
3908    pub fn binary(&self) -> Option<PathBuf> {
3909        executable_from_command_line(&self.command_line)
3910    }
3911}
3912
3913/// One platform's service manager, for one start-mode domain.
3914///
3915/// Every method takes the identity rather than storing it, so one control can
3916/// answer about the product's registration and about a test fixture's without
3917/// either being able to reach the other by accident.
3918pub trait ServiceControl: fmt::Debug {
3919    /// Which manager this is.
3920    fn manager(&self) -> DefinitionKind;
3921
3922    /// Registers the plan, returning the definition that was applied.
3923    ///
3924    /// # Errors
3925    ///
3926    /// [`ServiceError::Control`] or [`ServiceError::NeedsElevation`].
3927    fn install(&self, plan: &InstallPlan) -> Result<ServiceDefinition, ServiceError>;
3928
3929    /// Deregisters. Returns whether there was a registration to remove.
3930    ///
3931    /// **Removes the registration and nothing else.** No implementation of this
3932    /// method may delete a directory, a database, a secret, or a cache;
3933    /// `05-infrastructure.md` item 5 is a property of every backend, not a
3934    /// check somewhere above them.
3935    ///
3936    /// # Errors
3937    ///
3938    /// [`ServiceError::Control`] or [`ServiceError::NeedsElevation`].
3939    fn uninstall(&self, identity: &ServiceIdentity) -> Result<bool, ServiceError>;
3940
3941    /// What the manager knows about this registration, if anything.
3942    ///
3943    /// # Errors
3944    ///
3945    /// [`ServiceError::Control`].
3946    fn query(&self, identity: &ServiceIdentity) -> Result<Option<Registration>, ServiceError>;
3947
3948    /// Starts it now, without waiting for the next boot or logon.
3949    ///
3950    /// # Errors
3951    ///
3952    /// [`ServiceError::Control`], [`ServiceError::NeedsElevation`], or
3953    /// [`ServiceError::NotInstalled`].
3954    fn start(&self, identity: &ServiceIdentity) -> Result<(), ServiceError>;
3955
3956    /// Stops it. Returns whether it was running.
3957    ///
3958    /// # Errors
3959    ///
3960    /// As [`ServiceControl::start`].
3961    fn stop(&self, identity: &ServiceIdentity) -> Result<bool, ServiceError>;
3962}
3963
3964/// Chooses the control for a start mode.
3965///
3966/// The indirection is what makes [`ServiceOperations`] testable: production
3967/// hands it [`HostControls`], every test in this file hands it
3968/// [`RecordingControls`], and neither knows the difference.
3969pub trait ControlFactory: fmt::Debug + Send + Sync {
3970    /// The control for one start-mode domain.
3971    ///
3972    /// # Errors
3973    ///
3974    /// [`ServiceError::Control`] when this host has no manager for that domain.
3975    fn control(&self, mode: StartMode) -> Result<Box<dyn ServiceControl>, ServiceError>;
3976
3977    /// The text of an installed definition file, for drift detection. `None`
3978    /// when there is none to read. The default reads nothing, so a double
3979    /// never reports the real host's unit as its own.
3980    fn installed_definition(&self, _path: &Path) -> Option<String> {
3981        None
3982    }
3983
3984    /// Whether this Mac signs somebody in after a restart, and whether
3985    /// FileVault holds it at the unlock screen. `None` where the platform has
3986    /// no such probe; the default probes nothing, for the same reason as
3987    /// [`Self::installed_definition`].
3988    fn unattended_login(&self) -> Option<crate::unattended_login::UnattendedLogin> {
3989        None
3990    }
3991}
3992
3993/// The real service managers of the host this binary was built for.
3994#[derive(Debug, Clone, Copy, Default)]
3995pub struct HostControls;
3996
3997// ---------------------------------------------------------------------------
3998// The operations
3999// ---------------------------------------------------------------------------
4000
4001/// What `install` did.
4002#[derive(Debug, Clone)]
4003pub struct Installed {
4004    /// The plan that was applied, including the resolved absolute binary path.
4005    pub plan: InstallPlan,
4006    /// The definition the platform was given.
4007    pub definition: ServiceDefinition,
4008    /// The record that was written.
4009    pub record: InstallRecord,
4010    /// What the definition grants, measured against the requirement.
4011    pub review: PrivilegeReview,
4012    /// What was done to the runner root the registration will run jobs under.
4013    ///
4014    /// Named trustees rather than SIDs, so printing it adds no identity to the
4015    /// output. See [`crate::runner_root_access`].
4016    pub runner_root: RootAccessSummary,
4017    /// Whether this replaced a registration that was already there, rather than
4018    /// making a new one.
4019    ///
4020    /// Reported rather than glossed over: "Service installed" on a host that
4021    /// had one already reads as though nothing had been taken away and put
4022    /// back, and what was taken away and put back is the thing an operator
4023    /// watching a running agent wants to know about.
4024    pub replaced_existing: bool,
4025}
4026
4027/// What `uninstall` did — and, as importantly, what it did not.
4028#[derive(Debug, Clone, PartialEq, Eq)]
4029pub struct Uninstalled {
4030    /// Whether a registration was removed.
4031    pub removed_registration: bool,
4032    /// Whether the install record was removed.
4033    pub removed_record: bool,
4034    /// Whether a definition file was removed, and which.
4035    pub removed_definition: Option<PathBuf>,
4036    /// The directories that were **left exactly as they were**.
4037    ///
4038    /// `05-infrastructure.md` item 5 stated as a value rather than as a
4039    /// promise: `service uninstall` prints this list, so an operator can see
4040    /// that the configuration, the SQLite database, the stored token and the
4041    /// runner package cache are all still there.
4042    pub preserved: Vec<PathBuf>,
4043}
4044
4045impl fmt::Display for Uninstalled {
4046    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
4047        if self.removed_registration {
4048            writeln!(f, "The service registration was removed.")?;
4049        } else {
4050            writeln!(f, "There was no service registration to remove.")?;
4051        }
4052        writeln!(f, "Nothing else was deleted. These are untouched:")?;
4053        for path in &self.preserved {
4054            writeln!(f, "  {}", path.display())?;
4055        }
4056        write!(
4057            f,
4058            "The stored GitHub token is untouched too; `auth logout` is what purges it."
4059        )
4060    }
4061}
4062
4063/// What `set_start_mode` did.
4064#[derive(Debug, Clone, PartialEq, Eq)]
4065pub struct StartModeChange {
4066    /// What it was.
4067    pub from: StartMode,
4068    /// What it is now.
4069    pub to: StartMode,
4070    /// `false` when the registration was already in the requested mode.
4071    pub changed: bool,
4072    /// The secret store the new mode obliges, so a caller can tell an operator
4073    /// whether the token has to move too.
4074    pub store_scope: crate::secrets::SecretScope,
4075    /// What the mode change did to the runner root's access control.
4076    ///
4077    /// A mode change moves the account the daemon runs as, and the runner root
4078    /// admits that account by name — so the two move together or the new
4079    /// registration cannot write its own workspaces.
4080    pub runner_root: RootAccessSummary,
4081}
4082
4083impl fmt::Display for StartModeChange {
4084    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
4085        if !self.changed {
4086            return write!(f, "The service already starts at {}.", self.to);
4087        }
4088        write!(
4089            f,
4090            "The service now starts at {} instead of {}. It reads the {}-scoped secret store; \
4091             if the token was stored under the other scope, run `auth login` again. {}",
4092            self.to, self.from, self.store_scope, self.runner_root
4093        )
4094    }
4095}
4096
4097/// Install, uninstall, inspect, and switch start mode.
4098///
4099/// This is the whole of the library contract `f3` builds `service install`,
4100/// `service uninstall` and `service status` on. Nothing here prints; every
4101/// operation returns a value with a [`fmt::Display`] a command can write, so the
4102/// same facts are available to the TUI and to a JSON document without being
4103/// re-derived from text.
4104#[derive(Debug, Clone)]
4105pub struct ServiceOperations {
4106    paths: AppPaths,
4107    identity: ServiceIdentity,
4108    controls: std::sync::Arc<dyn ControlFactory>,
4109    runner_root: Option<LocalAbsolutePath>,
4110}
4111
4112impl ServiceOperations {
4113    /// Operates on this host's real service managers.
4114    #[must_use]
4115    pub fn on_this_host(paths: AppPaths) -> Self {
4116        Self::with_controls(
4117            paths,
4118            ServiceIdentity::product(),
4119            std::sync::Arc::new(HostControls),
4120        )
4121    }
4122
4123    /// Operates on the controls a caller supplies.
4124    ///
4125    /// Both other arguments are explicit for the same reason: the privileged
4126    /// installer tests register a real service under a
4127    /// [`ServiceIdentity::fixture`] name, against a disposable
4128    /// [`AppPaths::rooted_at`] tree, and must not be able to touch an operator's
4129    /// installation even by mistake.
4130    #[must_use]
4131    pub fn with_controls(
4132        paths: AppPaths,
4133        identity: ServiceIdentity,
4134        controls: std::sync::Arc<dyn ControlFactory>,
4135    ) -> Self {
4136        Self {
4137            paths,
4138            identity,
4139            controls,
4140            runner_root: None,
4141        }
4142    }
4143
4144    /// Points runner-root preparation at a directory the caller owns, for the
4145    /// **current account**.
4146    ///
4147    /// **Only a [`ServiceIdentity::fixture`] registration is allowed to move
4148    /// it, and the guard is here rather than at the call site.** A product
4149    /// registration ignores the override entirely and resolves
4150    /// [`crate::runner_root::default_runner_root`] itself, which is what keeps
4151    /// "custom roots are never re-ACLed" true no matter what a caller passes.
4152    ///
4153    /// It exists because the directory the product uses is
4154    /// `%SystemDrive%\rman`, and a smoke test that created and re-permissioned
4155    /// *that* would be editing the machine it runs on from outside its own
4156    /// fixture. So a fixture aims this at a temporary directory it created and
4157    /// will delete — and "and will delete" is why the override is always
4158    /// reconciled for the calling account rather than for the account the start
4159    /// mode obliges. A boot-mode root admits `SY` and `BA` only, which an
4160    /// ordinary filtered token is neither, so a test that supplied one could
4161    /// not then inspect or remove its own temporary directory.
4162    #[must_use]
4163    pub fn with_runner_root(mut self, root: LocalAbsolutePath) -> Self {
4164        self.runner_root = Some(root);
4165        self
4166    }
4167
4168    /// The directories this operates against.
4169    #[must_use]
4170    pub const fn paths(&self) -> &AppPaths {
4171        &self.paths
4172    }
4173
4174    /// The registration this operates on.
4175    #[must_use]
4176    pub const fn identity(&self) -> &ServiceIdentity {
4177        &self.identity
4178    }
4179
4180    /// Registers `daemon run` with the operating system.
4181    ///
4182    /// The order is the contract, and each step is a requirement:
4183    ///
4184    /// 1. the four directories are created, because item 2 is about writing them;
4185    /// 2. **the single-instance lock is taken** — item 1. It is held for the
4186    ///    whole of the install rather than probed and released, so a daemon that
4187    ///    starts halfway through cannot end up racing a registration;
4188    /// 3. neither start-mode domain already holds a registration — item 6's
4189    ///    record would otherwise describe one of two;
4190    /// 4. the binary path is resolved and confirmed to be a file — item 6;
4191    /// 5. the platform registers it;
4192    /// 6. the record is written.
4193    ///
4194    /// # Errors
4195    ///
4196    /// [`ServiceError::LockHeld`] when an agent is already running,
4197    /// [`ServiceError::AlreadyInstalled`], [`ServiceError::BinaryMissing`], and
4198    /// whatever the platform reports.
4199    pub fn install(&self, request: &InstallRequest) -> Result<Installed, ServiceError> {
4200        self.paths
4201            .create_all()
4202            .map_err(|source| ServiceError::Paths {
4203                source: Box::new(source),
4204            })?;
4205
4206        // Item 1. The guard lives until the end of this function; dropping it is
4207        // the release, so an early return releases it too.
4208        let _guard = self.refuse_while_an_agent_runs()?;
4209        // Replacing applies to this account's own registration only: over
4210        // another installation's it would re-register that service against
4211        // this account's directories.
4212        self.refuse_foreign("replace")?;
4213
4214        // --------------------------------------------------------------------
4215        // AN INSTALL OVER THE SAME START MODE REPLACES; IT DOES NOT REFUSE.
4216        // --------------------------------------------------------------------
4217        // This used to refuse any existing registration, and that was wrong in
4218        // a way that took a real host down.
4219        //
4220        // `service install` is *also* how an operator moves to a new version by
4221        // hand — `install_owned_copy` says so, and the `.old` rename it does
4222        // exists precisely so that reinstalling over a **running** service
4223        // works. So the caller replaces the copy the service runs and *then*
4224        // asks to register it. Refusing at that point does not undo the swap:
4225        // the daemon is left running a binary nobody registered, the command
4226        // reports failure, and on macOS the new binary is a program the stored
4227        // credential's keychain grant does not name. Watched on 0.1.17: the
4228        // copy under `state/bin` was replaced at 00:16, `service install` said
4229        // "already registered", and the daemon crash-looped on `-25293` from
4230        // then on.
4231        //
4232        // Registering the same service twice is not what an operator is asking
4233        // for here and is not what they get: the platform is asked to drop the
4234        // registration and take it again, which is a deregister followed by a
4235        // register because that is the only thing launchd, systemd and the SCM
4236        // all support — `launchctl bootstrap` refuses a label already loaded.
4237        //
4238        // The *other* mode is still refused. Moving between boot and login
4239        // moves the registration between two service managers and changes the
4240        // account and the secret store with it; `set_start_mode` does that,
4241        // carefully, and an install is not the place for it.
4242        let (replacing, previous_was_running) = match self.find_registration()? {
4243            Some((existing, registration)) if existing == request.start_mode() => {
4244                (true, registration.running)
4245            }
4246            Some((existing, _)) => {
4247                return Err(ServiceError::AlreadyInstalled {
4248                    name: self.identity.name().to_string(),
4249                    existing,
4250                    requested: request.start_mode(),
4251                });
4252            }
4253            None => (false, false),
4254        };
4255
4256        let plan = InstallPlan::resolve(
4257            self.identity.clone(),
4258            request,
4259            ServiceDirectories::of(&self.paths),
4260        )?;
4261
4262        // Resolved before the runner root, and used after it: every step from
4263        // here on has to be able to undo the directory, and a `?` between the
4264        // preparation and the first fallible use would leave one behind with
4265        // nothing to say so.
4266        let control = self.controls.control(plan.start_mode())?;
4267
4268        // Item 8, and the one step that can refuse an otherwise valid install.
4269        // Before the platform is asked to register anything, because a
4270        // registration whose workspaces would land in a directory ordinary local
4271        // users can write is one that should never have existed — and because
4272        // undoing a directory is cheaper than undoing a service.
4273        let root = self.prepare_runner_root(plan.start_mode())?;
4274
4275        // Read before the registration is dropped, so that a replace which then
4276        // fails to take can put back what was there. A record this account
4277        // cannot read leaves nothing to reinstate, which is reported rather
4278        // than pretended: see `reinstate`.
4279        let previous = if replacing {
4280            InstallRecord::read(&self.paths).ok().flatten()
4281        } else {
4282            None
4283        };
4284        if replacing && let Err(cause) = control.uninstall(&self.identity) {
4285            return Err(undo_runner_root(&root, "install", &self.identity, cause));
4286        }
4287
4288        let definition = match control.install(&plan) {
4289            Ok(definition) => definition,
4290            Err(cause) => {
4291                // Nothing was deregistered on a first install, so there is
4292                // nothing to put back and `reinstate` says so with `Ok(())`.
4293                let restored =
4294                    self.reinstate(control.as_ref(), previous.as_ref(), previous_was_running);
4295                return Err(rolled_back(
4296                    retained_runner_root(&root),
4297                    restored,
4298                    "install",
4299                    &self.identity,
4300                    cause,
4301                ));
4302            }
4303        };
4304        let review = review_least_privilege(&definition, &plan);
4305        let record = InstallRecord::of(&plan, &definition, Utc::now());
4306        if let Err(cause) = record.write(&self.paths) {
4307            let rollback = control
4308                .uninstall(&self.identity)
4309                .map(|_| ())
4310                .and_then(|()| {
4311                    self.reinstate(control.as_ref(), previous.as_ref(), previous_was_running)
4312                });
4313            return Err(rolled_back(
4314                retained_runner_root(&root),
4315                rollback,
4316                "install",
4317                &self.identity,
4318                cause,
4319            ));
4320        }
4321
4322        // Registration is not the postcondition of an install: a service the
4323        // current session still has to start by hand is only half-installed.
4324        // This matters most for a logon-triggered Windows task because its
4325        // trigger has already passed by the time an operator reinstalls it.
4326        // Start through the same control that registered the definition and
4327        // roll the whole replacement back if the manager refuses it.
4328        if let Err(cause) = control.start(&self.identity) {
4329            let remove_new = control.uninstall(&self.identity).map(|_| ());
4330            let restore_registration = remove_new.and_then(|()| {
4331                self.reinstate(control.as_ref(), previous.as_ref(), previous_was_running)
4332            });
4333            let restore_record = restore_registration.and_then(|()| match &previous {
4334                Some(previous) => previous.write(&self.paths),
4335                None => InstallRecord::remove(&self.paths).map(|_| ()),
4336            });
4337            return Err(rolled_back(
4338                retained_runner_root(&root),
4339                restore_record,
4340                "install and start",
4341                &self.identity,
4342                cause,
4343            ));
4344        }
4345        Ok(Installed {
4346            plan,
4347            definition,
4348            record,
4349            review,
4350            runner_root: root.summary().clone(),
4351            replaced_existing: replacing,
4352        })
4353    }
4354
4355    /// Puts back a registration this call deregistered in order to replace it.
4356    ///
4357    /// `Ok(())` when there was nothing to put back, which is the ordinary case:
4358    /// a first install deregisters nothing. When there *was* a record, the plan
4359    /// is rebuilt from it rather than from the request, because the request is
4360    /// the one that just failed.
4361    ///
4362    /// Best effort by construction — the caller passes the result to
4363    /// [`rolled_back`], which reports a failed rollback alongside the original
4364    /// cause rather than replacing it.
4365    fn reinstate(
4366        &self,
4367        control: &dyn ServiceControl,
4368        previous: Option<&InstallRecord>,
4369        was_running: bool,
4370    ) -> Result<(), ServiceError> {
4371        let Some(record) = previous else {
4372            return Ok(());
4373        };
4374        control.install(&record.plan(self.identity.clone()))?;
4375        if was_running {
4376            control.start(&self.identity)?;
4377        }
4378        Ok(())
4379    }
4380
4381    /// Deregisters, and deletes nothing else.
4382    ///
4383    /// # Errors
4384    ///
4385    /// Whatever the platform reports. A missing registration is **not** an
4386    /// error: `service uninstall` on a host that has none should say so and
4387    /// exit cleanly, because an operator running it twice has not made a
4388    /// mistake.
4389    pub fn uninstall(&self) -> Result<Uninstalled, ServiceError> {
4390        let record = InstallRecord::read(&self.paths).ok().flatten();
4391        let removed_definition = record
4392            .as_ref()
4393            .and_then(|record| record.definition_path.clone());
4394
4395        let mut removed_registration = false;
4396        // Both domains, not only the recorded one. A record can be lost while a
4397        // registration survives, and leaving a registration behind because a
4398        // TOML file was deleted is exactly the state `uninstall` exists to end.
4399        //
4400        // Except another installation's: it is skipped, and when it is all
4401        // there is, the uninstall is refused and this account's record kept.
4402        let mut foreign = None;
4403        for mode in [StartMode::Boot, StartMode::Login] {
4404            let control = self.controls.control(mode)?;
4405            if let Some(directories) = control
4406                .query(&self.identity)
4407                .ok()
4408                .flatten()
4409                .and_then(|registration| self.foreign_directories(&registration))
4410            {
4411                foreign = Some(directories);
4412                continue;
4413            }
4414            if control.uninstall(&self.identity)? {
4415                removed_registration = true;
4416            }
4417        }
4418        if !removed_registration && let Some(directories) = foreign {
4419            return Err(self.foreign_refusal("uninstall", directories));
4420        }
4421        let removed_record = InstallRecord::remove(&self.paths)?;
4422        Ok(Uninstalled {
4423            removed_registration,
4424            removed_record,
4425            removed_definition: removed_definition.filter(|_| removed_registration),
4426            preserved: self
4427                .paths
4428                .all()
4429                .iter()
4430                .map(|(_, path)| (*path).to_path_buf())
4431                .collect(),
4432        })
4433    }
4434
4435    /// Switches between `boot` and `login` **without re-resolving anything**.
4436    ///
4437    /// `05-infrastructure.md` item 7. Everything the new registration carries —
4438    /// the absolute binary path, the arguments, the restart policy, the four
4439    /// directories — comes from the existing record, so the operator is not
4440    /// asked to reinstall the product and a binary that has since moved is not
4441    /// silently swapped for whichever one happens to be running this command.
4442    ///
4443    /// On Windows the registration necessarily moves between two different
4444    /// Windows facilities, because Windows has no service that starts at logon;
4445    /// see this module's documentation. That is still not a reinstall: no file
4446    /// is downloaded, replaced, or re-resolved.
4447    ///
4448    /// The caller must persist the new mode onto this host's `Host` record so
4449    /// that `host show` reports it — [`StartModeChange::to`] is the value, and
4450    /// `f1` already emits `host.service_start_mode` from that field.
4451    ///
4452    /// # Errors
4453    ///
4454    /// [`ServiceError::NotInstalled`] when there is no record to switch, and
4455    /// whatever the platform reports.
4456    pub fn set_start_mode(&self, to: StartMode) -> Result<StartModeChange, ServiceError> {
4457        let Some(record) = InstallRecord::read(&self.paths)? else {
4458            return Err(ServiceError::NotInstalled {
4459                name: self.identity.name().to_string(),
4460                operation: "switch the start mode of",
4461            });
4462        };
4463        let from = record.start_mode;
4464        if from == to {
4465            // Nothing moves, so nothing about the root's access control has to.
4466            // Reconciling it here would turn a no-op command into one that can
4467            // fail on a permission it does not need.
4468            return Ok(StartModeChange {
4469                from,
4470                to,
4471                changed: false,
4472                store_scope: crate::secrets::SecretScope::for_start_mode(to),
4473                runner_root: RootAccessSummary::NotApplicable,
4474            });
4475        }
4476
4477        // A real switch only: a no-op touches no service manager at all. Over
4478        // another installation's registration it would move that service.
4479        self.refuse_foreign("switch the start mode of")?;
4480
4481        #[cfg(windows)]
4482        let arguments = {
4483            let mut arguments = record.arguments.clone();
4484            arguments.retain(|argument| argument != WINDOWS_SCM_HOST_ARGUMENT);
4485            if to == StartMode::Boot {
4486                arguments.push(WINDOWS_SCM_HOST_ARGUMENT.to_string());
4487            }
4488            arguments
4489        };
4490        #[cfg(not(windows))]
4491        let arguments = record.arguments.clone();
4492        let plan = InstallPlan::unchecked(
4493            self.identity.clone(),
4494            to,
4495            record.binary.clone(),
4496            record.directories.clone(),
4497        )
4498        .with_arguments(arguments)
4499        .with_restart(record.restart());
4500        let plan = if record.starts_on_demand {
4501            plan.started_on_demand()
4502        } else {
4503            plan
4504        };
4505        let plan = match crate::secrets::PlatformSecretStore::for_start_mode(to) {
4506            Ok(store) => plan.with_secret_guard(store.guard()),
4507            Err(_) => plan,
4508        };
4509
4510        // Install the target domain before touching the live one. This makes a
4511        // failed target install a no-op from the operator's point of view and,
4512        // unlike uninstall-first ordering, never trades a working service for
4513        // an error message. Resolved before the runner root and used after it,
4514        // so that no `?` sits between preparing the directory and the first
4515        // step that knows how to undo it.
4516        let target = self.controls.control(to)?;
4517        // The domain being left, resolved here rather than where it is used for
4518        // the same reason: the only step that removes it runs after the runner
4519        // root has been prepared, and a `?` there would abandon the target
4520        // registration, the record and the directory without a word.
4521        let previous = self.controls.control(from)?;
4522
4523        // The account changes with the mode, and so must the account the runner
4524        // root admits: `04-security-recovery.md` requires the selected identity
4525        // to be *reconciled* when service mode changes, which means adding the
4526        // operator's on the way to login and dropping it again on the way back.
4527        // Before the target install, for the same reason `install` does it
4528        // first — a root that cannot be made safe must not produce a working
4529        // registration.
4530        let root = self.prepare_runner_root(to)?;
4531
4532        let definition = match target.install(&plan) {
4533            Ok(definition) => definition,
4534            Err(cause) => {
4535                return Err(undo_runner_root(
4536                    &root,
4537                    "switch start mode",
4538                    &self.identity,
4539                    cause,
4540                ));
4541            }
4542        };
4543        let next_record = InstallRecord::of(&plan, &definition, record.installed_at);
4544        if let Err(cause) = next_record.write(&self.paths) {
4545            return Err(rolled_back(
4546                retained_runner_root(&root),
4547                target.uninstall(&self.identity),
4548                "switch start mode",
4549                &self.identity,
4550                cause,
4551            ));
4552        }
4553
4554        // Only after the target registration and its durable record exist is
4555        // it safe to remove the old domain. If that last step fails, remove the
4556        // target and put the old record back so status and reality agree.
4557        if let Err(cause) = previous.uninstall(&self.identity) {
4558            let target_rollback = target.uninstall(&self.identity);
4559            let record_rollback = record.write(&self.paths);
4560            return Err(rolled_back(
4561                retained_runner_root(&root),
4562                target_rollback.and(record_rollback),
4563                "switch start mode",
4564                &self.identity,
4565                cause,
4566            ));
4567        }
4568        Ok(StartModeChange {
4569            from,
4570            to,
4571            changed: true,
4572            store_scope: crate::secrets::SecretScope::for_start_mode(to),
4573            runner_root: root.summary().clone(),
4574        })
4575    }
4576
4577    /// Starts the registration now.
4578    ///
4579    /// # Errors
4580    ///
4581    /// [`ServiceError::NotInstalled`], or whatever the platform reports.
4582    pub fn start(&self) -> Result<(), ServiceError> {
4583        let mode = self.own_registration_mode("start")?;
4584        self.controls.control(mode)?.start(&self.identity)
4585    }
4586
4587    /// The domain holding this account's own registration, for `operation`.
4588    ///
4589    /// # Errors
4590    ///
4591    /// [`ServiceError::NotInstalled`] when there is none, and
4592    /// [`ServiceError::ForeignInstallation`] when the only one is another
4593    /// installation's.
4594    fn own_registration_mode(&self, operation: &'static str) -> Result<StartMode, ServiceError> {
4595        match self.registration_for_status()? {
4596            None => Err(ServiceError::NotInstalled {
4597                name: self.identity.name().to_string(),
4598                operation,
4599            }),
4600            Some((_, Some(directories))) => Err(self.foreign_refusal(operation, directories)),
4601            Some((registration, None)) => Ok(registration.start_mode),
4602        }
4603    }
4604
4605    /// Stops the registration. Returns whether it was running.
4606    ///
4607    /// # Errors
4608    ///
4609    /// As [`ServiceOperations::start`].
4610    pub fn stop(&self) -> Result<bool, ServiceError> {
4611        let mode = self.own_registration_mode("stop")?;
4612        self.controls.control(mode)?.stop(&self.identity)
4613    }
4614
4615    /// Everything Journey 5 step 4 asks `service status` to report.
4616    ///
4617    /// # Errors
4618    ///
4619    /// [`ServiceError::Record`] when local state cannot be read, and whatever
4620    /// the platform reports. A *stale binary path* is deliberately **not** an
4621    /// error here: it is a reported state, because a status command that
4622    /// refused to print anything else would hide the very facts an operator
4623    /// needs in order to fix it.
4624    ///
4625    /// A record **this account may not read** is reported for the same reason
4626    /// and is the second exception. On a boot-mode host the record was written
4627    /// by `sudo service install`, and a version before this one wrote it `0600`
4628    /// — so the operator's own `service status` ended with `Permission denied`
4629    /// and printed nothing at all, on a host whose registration the service
4630    /// manager would have described perfectly well. What launchd, systemd or
4631    /// the SCM says is a separate fact from the record, and it is still worth
4632    /// having.
4633    ///
4634    /// # A registration made from another data root
4635    ///
4636    /// The service manager holds one registration per machine under one name,
4637    /// and every account on the machine sees it. When its command line names
4638    /// application-data directories that are not this account's, it is
4639    /// **another installation's service**, and comparing this account's
4640    /// install record against it would report a stale binary, a mismatched
4641    /// start mode and a secret store in the wrong scope, each with a remedy
4642    /// (`service uninstall`, `service install`, `auth login`) that, followed
4643    /// from this account, breaks a healthy service. So it is reported as what
4644    /// it is: see [`ForeignService`].
4645    pub fn status(&self) -> Result<ServiceStatus, ServiceError> {
4646        match self.registration_for_status()? {
4647            Some((registration, Some(directories))) => {
4648                Ok(self.foreign_status(registration, directories))
4649            }
4650            found => {
4651                let (record, record_refused) = match InstallRecord::read(&self.paths) {
4652                    Ok(record) => (record, None),
4653                    Err(refusal @ ServiceError::RecordNotPermitted { .. }) => (None, Some(refusal)),
4654                    Err(error) => return Err(error),
4655                };
4656                self.status_at(
4657                    &self.paths,
4658                    record,
4659                    record_refused.as_ref(),
4660                    found.map(|(registration, _)| registration),
4661                )
4662            }
4663        }
4664    }
4665
4666    /// [`Self::status`] for the installation whose directories are `paths`,
4667    /// from the record already read there.
4668    fn status_at(
4669        &self,
4670        paths: &AppPaths,
4671        record: Option<InstallRecord>,
4672        record_refused: Option<&ServiceError>,
4673        found: Option<Registration>,
4674    ) -> Result<ServiceStatus, ServiceError> {
4675        let last_github_contact = last_github_contact(paths)?;
4676        let drift = record
4677            .as_ref()
4678            .and_then(|record| definition_drift(self.controls.as_ref(), &self.identity, record));
4679        let credential_rejected_since = github_credential_rejected_since(paths).ok().flatten();
4680        let credential_unreadable_since = credential_unreadable_since(paths).ok().flatten();
4681        let probed = self.controls.unattended_login();
4682        let unattended = record.as_ref().and_then(|record| {
4683            // The account is only read where something was probed to judge
4684            // it against.
4685            let account = probed
4686                .as_ref()
4687                .map_or_else(String::new, |_| service_account_name(&record.account));
4688            unattended_gap(record.start_mode, &account, probed.as_ref())
4689        });
4690        let now = Utc::now();
4691        let installed = record.is_some() || found.is_some();
4692        Ok(ServiceStatus::compose(
4693            self.identity.clone(),
4694            record,
4695            record_refused,
4696            found,
4697            last_github_contact,
4698            paths,
4699        )
4700        .with_definition_drift(drift)
4701        .with_credential_rejection(credential_rejected_since)
4702        .with_credential_unreadable(credential_unreadable_since)
4703        .with_launches_blocked(crate::launch_health::launches_blocked(paths, now))
4704        .with_unattended_gap(unattended)
4705        .with_host_unfit(crate::host_fitness::host_unfit_in_force(paths, now).filter(|_| installed))
4706        .with_stall(crate::daemon_heartbeat::liveness(paths, now).stall(last_github_contact)))
4707    }
4708
4709    /// The installed definition, when it is not what this build renders for
4710    /// the recorded installation. See [`definition_drift`].
4711    ///
4712    /// # Errors
4713    ///
4714    /// Whatever reading the install record reports.
4715    pub fn definition_drift(&self) -> Result<Option<DefinitionDrift>, ServiceError> {
4716        Ok(InstallRecord::read(&self.paths)?
4717            .and_then(|record| definition_drift(self.controls.as_ref(), &self.identity, &record)))
4718    }
4719
4720    /// Writes the plist this build renders over an outdated one, then has a
4721    /// detached helper boot the job out and bootstrap it again from the new
4722    /// file.
4723    ///
4724    /// launchd reads a plist only when the job is bootstrapped. Rewriting the
4725    /// file changes nothing for a job that is already loaded, and a `KeepAlive`
4726    /// restart reuses the loaded copy. So the job has to be booted out, and
4727    /// that terminates the daemon that asked for it. The helper runs in a
4728    /// session of its own, which launchd does not kill with the job, and
4729    /// retries the bootstrap: launchd refuses it for a few seconds while the
4730    /// old job is still exiting (`5: Input/output error`, measured).
4731    ///
4732    /// Call this only from the daemon the job runs, and only while it holds no
4733    /// runner. It is an operation on the registration the daemon itself runs
4734    /// under, and it never changes the start mode.
4735    ///
4736    /// # Errors
4737    ///
4738    /// [`ServiceError::Control`] or [`ServiceError::NeedsElevation`] when the
4739    /// plist cannot be written or the helper cannot be started. The loaded job
4740    /// is untouched in both cases.
4741    #[cfg(target_os = "macos")]
4742    pub fn reload_launchd_definition(&self, drift: &DefinitionDrift) -> Result<(), ServiceError> {
4743        // The record names the start mode; the account this process runs as has
4744        // to agree, or the record describes a registration other than the one
4745        // running, and root would end up rewriting an operator's LaunchAgent.
4746        // SAFETY: `getuid` reads the calling process's real user id and cannot
4747        // fail.
4748        let root = unsafe { libc::getuid() } == 0;
4749        if root != (drift.start_mode == StartMode::Boot) {
4750            return Err(ServiceError::Control {
4751                operation: "rewrite",
4752                name: self.identity.name().to_string(),
4753                manager: "launchd",
4754                detail: format!(
4755                    "the install record says --start-at {}, but this daemon runs as {}; the plist \
4756                     is left as it is",
4757                    drift.start_mode,
4758                    if root { "root" } else { "the operator" }
4759                ),
4760            });
4761        }
4762        replace_definition(self.identity.name(), drift)?;
4763        // The definition is now this build's, as after the WSL unit rewrite.
4764        // Best effort: the record says who wrote the plist and nothing reads it
4765        // to decide whether to repair.
4766        if let Ok(Some(mut record)) = InstallRecord::read(&self.paths) {
4767            record.installed_by_version = env!("CARGO_PKG_VERSION").to_string();
4768            let _ = record.write(&self.paths);
4769        }
4770        sys::reload_detached(&self.identity, drift.start_mode, &drift.path)
4771    }
4772
4773    /// Creates or reconciles the runner root this start mode's account needs.
4774    ///
4775    /// The account is not an argument: it is [`ServiceAccount::for_start_mode`],
4776    /// the same function the registration's own principal comes from, so the
4777    /// directory admits exactly the identity the definition registers and a mode
4778    /// change reconciles both together or neither.
4779    ///
4780    /// On macOS and Linux this is a no-op that returns
4781    /// [`RootAccessSummary::NotApplicable`]; see [`crate::runner_root_access`].
4782    fn prepare_runner_root(&self, mode: StartMode) -> Result<RootAccessChange, ServiceError> {
4783        #[cfg(not(windows))]
4784        {
4785            // `runner_root` is consumed here as well as in the Windows arm: a
4786            // field only one platform reads is a dead field on the other.
4787            let _ = (mode, &self.runner_root);
4788            Ok(RootAccessChange::not_applicable())
4789        }
4790        #[cfg(windows)]
4791        {
4792            let wrap = |source| ServiceError::RunnerRoot {
4793                source: Box::new(source),
4794            };
4795            // The test seam, and the two things allowed through it. `cfg!(test)`
4796            // is false in every shipped binary, so what a released build honours
4797            // is the fixture name alone — and a fixture name cannot be the
4798            // product's, which is what keeps a released `service install`
4799            // pointed at the platform default whatever a caller passes.
4800            //
4801            // An overridden root is always reconciled **for this account**,
4802            // which is the foreground admission — a real mode of the product
4803            // rather than a concession invented here. It has to be: a boot-mode
4804            // root admits `SY` and `BA` only, and a test process holding an
4805            // ordinary filtered token is neither, so it could not inspect the
4806            // temporary directory it just supplied nor delete it afterwards.
4807            // What that costs is that the *boot* descriptor is not proved
4808            // through this path; it is proved purely, by this module's
4809            // `the_runner_root_a_boot_registration_needs_admits_only_the_service`
4810            // and by `runner_root_access`'s own tests, and for real by the
4811            // privileged installer test, which runs elevated.
4812            if let Some(root) = self
4813                .runner_root
4814                .as_ref()
4815                .filter(|_| self.identity.is_fixture() || cfg!(test))
4816            {
4817                let admission = RootAdmission::of_this_account().map_err(wrap)?;
4818                return crate::runner_root_access::reconcile(&self.paths, root, &admission)
4819                    .map_err(wrap);
4820            }
4821
4822            let admission = match ServiceAccount::for_start_mode(mode) {
4823                // A boot registration runs as LocalSystem, which the constant
4824                // `SY` ace already names.
4825                ServiceAccount::LocalSystem => RootAdmission::LocalSystem,
4826                // A login registration runs as this account, under a filtered
4827                // token in which Administrators is deny-only — so without an
4828                // ace of its own it would be admitted by nothing.
4829                ServiceAccount::InvokingUser | ServiceAccount::Root => {
4830                    RootAdmission::of_this_account().map_err(wrap)?
4831                }
4832            };
4833            crate::runner_root_access::ensure_default_root(&self.paths, &admission).map_err(wrap)
4834        }
4835    }
4836
4837    /// Takes the single-instance lock, or refuses with `d1`'s own message.
4838    fn refuse_while_an_agent_runs(&self) -> Result<HostLock, ServiceError> {
4839        HostLock::try_acquire(&self.paths, LockKind::SingleInstance).map_err(
4840            |source| match source {
4841                held @ LockError::Held { .. } => ServiceError::LockHeld {
4842                    source: Box::new(held),
4843                },
4844                other => ServiceError::LockUnreadable {
4845                    source: Box::new(other),
4846                },
4847            },
4848        )
4849    }
4850
4851    /// The registration, in whichever domain holds it.
4852    fn find_registration(&self) -> Result<Option<(StartMode, Registration)>, ServiceError> {
4853        for mode in [StartMode::Boot, StartMode::Login] {
4854            let control = self.controls.control(mode)?;
4855            if let Some(registration) = control.query(&self.identity)? {
4856                return Ok(Some((registration.start_mode, registration)));
4857            }
4858        }
4859        Ok(None)
4860    }
4861
4862    /// The registration [`Self::status`] describes, with the directories it
4863    /// runs against when they are not this account's: this account's own
4864    /// registration when either domain holds one, else whichever there is.
4865    ///
4866    /// [`Self::find_registration`] stops at the boot domain, so a boot service
4867    /// made from another data root would hide this account's own login
4868    /// registration. The other domain is asked only in that case, and a
4869    /// failure to ask it is not this status's failure.
4870    fn registration_for_status(
4871        &self,
4872    ) -> Result<Option<(Registration, Option<ServiceDirectories>)>, ServiceError> {
4873        let Some((mode, found)) = self.find_registration()? else {
4874            return Ok(None);
4875        };
4876        let Some(directories) = self.foreign_directories(&found) else {
4877            return Ok(Some((found, None)));
4878        };
4879        let other = match mode {
4880            StartMode::Boot => StartMode::Login,
4881            StartMode::Login => StartMode::Boot,
4882        };
4883        let own = self
4884            .controls
4885            .control(other)
4886            .ok()
4887            .and_then(|control| control.query(&self.identity).ok().flatten())
4888            .filter(|registration| self.foreign_directories(registration).is_none());
4889        Ok(Some(match own {
4890            Some(own) => (own, None),
4891            None => (found, Some(directories)),
4892        }))
4893    }
4894
4895    /// Refuses `operation` when the machine's registration is another
4896    /// installation's. See [`ServiceError::ForeignInstallation`].
4897    fn refuse_foreign(&self, operation: &'static str) -> Result<(), ServiceError> {
4898        match self.registration_for_status()? {
4899            Some((_, Some(directories))) => Err(self.foreign_refusal(operation, directories)),
4900            _ => Ok(()),
4901        }
4902    }
4903
4904    /// [`ServiceError::ForeignInstallation`] for the installation at
4905    /// `directories`.
4906    fn foreign_refusal(
4907        &self,
4908        operation: &'static str,
4909        directories: ServiceDirectories,
4910    ) -> ServiceError {
4911        let foreign = self.foreign_service_at(directories);
4912        ServiceError::ForeignInstallation {
4913            name: self.identity.name().to_string(),
4914            operation,
4915            installed_by: foreign.installed_by(),
4916            act_as: foreign.act_as(),
4917        }
4918    }
4919
4920    /// The directories `registration` runs against, when they are not this
4921    /// account's. See [`ForeignService`].
4922    fn foreign_directories(&self, registration: &Registration) -> Option<ServiceDirectories> {
4923        ServiceDirectories::from_command_line(&registration.command_line)
4924            .filter(|directories| !same_directory(&directories.config, self.paths.config_dir()))
4925    }
4926
4927    /// The machine's service and the installation it belongs to, when that is
4928    /// not this account's, without reading anything of that installation's
4929    /// beyond its path. `None` when the service is this account's, when there
4930    /// is none, or when the service manager cannot be asked.
4931    #[must_use]
4932    pub fn foreign_service(&self) -> Option<ForeignService> {
4933        let (_, directories) = self.registration_for_status().ok()??;
4934        Some(self.foreign_service_at(directories?))
4935    }
4936
4937    /// The [`ForeignService`] for an installation at `directories`, before
4938    /// anything of its own is read.
4939    fn foreign_service_at(&self, directories: ServiceDirectories) -> ForeignService {
4940        let root = directories.root();
4941        let home = host_home();
4942        // This account's own installation under other directories, when the
4943        // four are exactly a layout a command of this account resolves.
4944        let this_account = home
4945            .as_deref()
4946            .filter(|home| contains_directory(home, &root))
4947            .and_then(|_| {
4948                if directories == ServiceDirectories::of(&AppPaths::rooted_at(&root)) {
4949                    Some(format!(
4950                        "with `--data-dir {}`",
4951                        quote_argument(&root.to_string_lossy())
4952                    ))
4953                } else {
4954                    AppPaths::discover()
4955                        .ok()
4956                        .filter(|standard| directories == ServiceDirectories::of(standard))
4957                        .map(|_| "without `--data-dir`".to_owned())
4958                }
4959            });
4960        ForeignService {
4961            owner: data_root_owner(&root, home.as_deref()),
4962            this_account,
4963            root,
4964            directories,
4965            local_root: ServiceDirectories::of(&self.paths).root(),
4966            local_record: InstallRecord::path(&self.paths),
4967            unused_local_record: InstallRecord::read(&self.paths).ok().flatten(),
4968            unreadable: None,
4969        }
4970    }
4971
4972    /// [`Self::status`] for a registration made from another data root.
4973    ///
4974    /// What that installation recorded is read when this account may read it,
4975    /// and the status is then exactly the one its owner would see. When it may
4976    /// not, the status holds what the service manager says and nothing it
4977    /// would have to guess.
4978    fn foreign_status(
4979        &self,
4980        registration: Registration,
4981        directories: ServiceDirectories,
4982    ) -> ServiceStatus {
4983        let service_paths = directories.paths();
4984        let mut foreign = self.foreign_service_at(directories);
4985        let unreadable = match InstallRecord::read(&service_paths) {
4986            Ok(Some(record)) => {
4987                match self.status_at(
4988                    &service_paths,
4989                    Some(record),
4990                    None,
4991                    Some(registration.clone()),
4992                ) {
4993                    Ok(status) => return status.with_foreign(foreign),
4994                    Err(error) => error.to_string(),
4995                }
4996            }
4997            Ok(None) => format!(
4998                "there is no install record at {}",
4999                InstallRecord::path(&service_paths).display()
5000            ),
5001            // Not the error's own words: `RecordNotPermitted` and
5002            // `RecordUnreadable` both end in a `service install` remedy, which
5003            // from this account would re-register the machine's service
5004            // against the wrong directories.
5005            Err(
5006                ServiceError::RecordNotPermitted { path, detail }
5007                | ServiceError::RecordUnreadable { path, detail }
5008                | ServiceError::Record { path, detail, .. },
5009            ) => format!("this account cannot read {} ({detail})", path.display()),
5010            Err(error) => error.to_string(),
5011        };
5012        foreign.unreadable = Some(unreadable);
5013        ServiceStatus::foreign_unread(self.identity.clone(), registration, foreign)
5014    }
5015
5016    /// Moves this account's own configuration aside, when no registration on
5017    /// this machine runs against it: `host forget-local`.
5018    ///
5019    /// Everything in the configuration directory -- the install record, the
5020    /// database with its host row and policies, `runner.env` and the rest -- is
5021    /// **renamed** into a `forgotten-<timestamp>` directory beside it, never
5022    /// deleted, so moving it back restores it exactly. The state, runtime and
5023    /// log directories are left alone even where the platform puts them inside
5024    /// the configuration directory (macOS does), and so is every secret store
5025    /// and every directory a registration names.
5026    ///
5027    /// Both domains are asked, strictly: a domain that cannot be asked might
5028    /// hold this account's own registration, so it refuses rather than reads
5029    /// as empty.
5030    ///
5031    /// # Errors
5032    ///
5033    /// [`ServiceError::LocalConfigurationInUse`] when a registration on this
5034    /// machine runs against these directories, when a registration from
5035    /// another data root names a directory overlapping them, or when an agent
5036    /// holds this account's single-instance lock. [`ServiceError::Record`] when
5037    /// the move fails; whatever had already moved is put back first, and
5038    /// anything that could not be put back is named.
5039    pub fn forget_local(&self, at: DateTime<Utc>) -> Result<ForgottenLocal, ServiceError> {
5040        let config = self.paths.config_dir();
5041        let in_use = |detail: String| ServiceError::LocalConfigurationInUse { detail };
5042        let mut service_root = None;
5043        for mode in [StartMode::Boot, StartMode::Login] {
5044            let registration = self
5045                .controls
5046                .control(mode)
5047                .and_then(|control| control.query(&self.identity))
5048                .map_err(|error| {
5049                    in_use(format!(
5050                        "whether a {mode} registration runs against it could not be read ({error})"
5051                    ))
5052                })?;
5053            let Some(registration) = registration else {
5054                continue;
5055            };
5056            let Some(directories) = self.foreign_directories(&registration) else {
5057                return Err(in_use(format!(
5058                    "{} holds a registration that runs against {}. `service status` describes it",
5059                    registration.manager,
5060                    config.display()
5061                )));
5062            };
5063            if let Some(overlap) = directories.all().into_iter().find(|directory| {
5064                contains_directory(directory, config) || contains_directory(config, directory)
5065            }) {
5066                return Err(in_use(format!(
5067                    "the machine's service runs against {}, which overlaps {}",
5068                    overlap.display(),
5069                    config.display()
5070                )));
5071            }
5072            service_root.get_or_insert_with(|| directories.root());
5073        }
5074        let _lock =
5075            HostLock::try_acquire(&self.paths, LockKind::SingleInstance).map_err(|source| {
5076                in_use(format!(
5077                    "an agent may be running against these directories: {source}"
5078                ))
5079            })?;
5080        let failed = |operation: &'static str, path: &Path| {
5081            let path = path.to_path_buf();
5082            move |error: std::io::Error| ServiceError::Record {
5083                operation,
5084                path,
5085                detail: error.to_string(),
5086            }
5087        };
5088        // Never the agent's own directories, wherever the platform nests them.
5089        let kept = [
5090            self.paths.state_dir(),
5091            self.paths.runtime_dir(),
5092            self.paths.logs_dir(),
5093        ];
5094        let mut entries = Vec::new();
5095        for entry in std::fs::read_dir(config).map_err(failed("list", config))? {
5096            let entry = entry.map_err(failed("list", config))?.path();
5097            let archived = entry
5098                .file_name()
5099                .is_some_and(|name| name.to_string_lossy().starts_with(FORGOTTEN_PREFIX));
5100            if !archived && !kept.iter().any(|kept| contains_directory(&entry, kept)) {
5101                entries.push(entry);
5102            }
5103        }
5104        if entries.is_empty() {
5105            return Ok(ForgottenLocal {
5106                archive: None,
5107                moved: entries,
5108                service_root,
5109            });
5110        }
5111        entries.sort();
5112        let archive = config.join(format!("{FORGOTTEN_PREFIX}{}", at.format("%Y%m%dT%H%M%SZ")));
5113        std::fs::create_dir(&archive).map_err(failed("create", &archive))?;
5114        let inside = |entry: &Path| archive.join(entry.file_name().unwrap_or_default());
5115        for (index, entry) in entries.iter().enumerate() {
5116            if let Err(error) = std::fs::rename(entry, inside(entry)) {
5117                let stranded: Vec<String> = entries[..index]
5118                    .iter()
5119                    .rev()
5120                    .filter(|done| std::fs::rename(inside(done), done).is_err())
5121                    .map(|done| inside(done).display().to_string())
5122                    .collect();
5123                let _ = std::fs::remove_dir(&archive);
5124                let mut detail = error.to_string();
5125                if !stranded.is_empty() {
5126                    detail.push_str(&format!(
5127                        "; these could not be put back and are still in the archive: {}",
5128                        stranded.join(", ")
5129                    ));
5130                }
5131                return Err(ServiceError::Record {
5132                    operation: "archive",
5133                    path: entry.clone(),
5134                    detail,
5135                });
5136            }
5137        }
5138        Ok(ForgottenLocal {
5139            archive: Some(archive),
5140            moved: entries,
5141            service_root,
5142        })
5143    }
5144}
5145
5146/// The name every `host forget-local` archive starts with. A later run leaves
5147/// earlier archives where they are.
5148pub const FORGOTTEN_PREFIX: &str = "forgotten-";
5149
5150/// What [`ServiceOperations::forget_local`] did.
5151#[derive(Debug, Clone, PartialEq, Eq)]
5152pub struct ForgottenLocal {
5153    /// Where the configuration went, or `None` when there was nothing to move.
5154    pub archive: Option<PathBuf>,
5155    /// What was moved, by its old path.
5156    pub moved: Vec<PathBuf>,
5157    /// The data root of the machine's service, when one runs from another
5158    /// installation. Named so the operator can see it was not touched.
5159    pub service_root: Option<PathBuf>,
5160}
5161
5162impl fmt::Display for ForgottenLocal {
5163    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
5164        let Some(archive) = &self.archive else {
5165            return write!(
5166                f,
5167                "Nothing to archive: this account's configuration directory is empty."
5168            );
5169        };
5170        writeln!(f, "Archived this account's unused local configuration")?;
5171        writeln!(f, "  archive                   {}", archive.display())?;
5172        for path in &self.moved {
5173            writeln!(f, "  moved                     {}", path.display())?;
5174        }
5175        if let Some(root) = &self.service_root {
5176            writeln!(
5177                f,
5178                "  service data root         {} (untouched)",
5179                root.display()
5180            )?;
5181        }
5182        write!(
5183            f,
5184            "  restore                   move everything in the archive back into its parent \
5185             directory"
5186        )
5187    }
5188}
5189
5190/// Whether two directory spellings name the same place: the same text on
5191/// this platform's terms, or the same canonical directory.
5192fn same_directory(left: &Path, right: &Path) -> bool {
5193    let text = |path: &Path| {
5194        path.to_string_lossy()
5195            .trim_end_matches(['/', '\\'])
5196            .to_owned()
5197    };
5198    same_path_text(&text(left), &text(right))
5199        || matches!(
5200            (std::fs::canonicalize(left), std::fs::canonicalize(right)),
5201            (Ok(left), Ok(right)) if left == right
5202        )
5203}
5204
5205/// Where the remedy starts in a [`StatusProblem`]'s detail: the sentence
5206/// openings every builder above uses for it.
5207const REMEDY_OPENINGS: [&str; 8] = [
5208    ". Run ",
5209    ". If nothing is waiting",
5210    ". Re-register",
5211    ". Switch the start mode",
5212    " Sign in again",
5213    ". Fix: ",
5214    "; to do it now",
5215    "; if the service manager denies",
5216];
5217
5218/// A problem's diagnosis without its remedy. See
5219/// [`ServiceStatus::with_foreign`].
5220///
5221/// A detail that still names a command that changes the registration or the
5222/// credential after the cut -- a builder this list does not know yet -- is
5223/// reduced to the first clause rather than passed on, because the cost of
5224/// showing too little is a shorter sentence and the cost of showing too much
5225/// is a broken service.
5226fn diagnosis_only(detail: &str) -> String {
5227    let cut = REMEDY_OPENINGS
5228        .iter()
5229        .filter_map(|opening| detail.find(opening))
5230        .min()
5231        .unwrap_or(detail.len());
5232    let diagnosis = detail[..cut].trim_end_matches(['.', ' ']);
5233    let names_a_change = |text: &str| {
5234        [
5235            "service install",
5236            "service uninstall",
5237            "auth login",
5238            "launchctl",
5239            "systemctl",
5240        ]
5241        .iter()
5242        .any(|command| text.contains(command))
5243    };
5244    if !names_a_change(diagnosis) {
5245        return diagnosis.to_owned();
5246    }
5247    // Clause ends followed by a space, so a `.` inside a path does not cut.
5248    let end = [", ", "; ", ". "]
5249        .iter()
5250        .filter_map(|separator| diagnosis.find(separator))
5251        .min()
5252        .unwrap_or(diagnosis.len());
5253    let first = &diagnosis[..end];
5254    if names_a_change(first) {
5255        "the installation's own records report this".to_owned()
5256    } else {
5257        first.to_owned()
5258    }
5259}
5260
5261/// A path's components as this platform compares them: case-insensitively
5262/// on Windows.
5263fn path_key(path: &Path) -> Vec<String> {
5264    path.components()
5265        .map(|component| {
5266            let text = component.as_os_str().to_string_lossy();
5267            if cfg!(windows) {
5268                text.to_lowercase()
5269            } else {
5270                text.into_owned()
5271            }
5272        })
5273        .collect()
5274}
5275
5276/// Whether `inner` is `outer` or a directory under it, by the rule
5277/// [`same_directory`] uses: on this platform's terms, or canonically.
5278fn contains_directory(outer: &Path, inner: &Path) -> bool {
5279    path_key(inner).starts_with(&path_key(outer))
5280        || matches!(
5281            (std::fs::canonicalize(outer), std::fs::canonicalize(inner)),
5282            (Ok(outer), Ok(inner)) if inner.starts_with(&outer)
5283        )
5284}
5285
5286/// The account whose data root `root` is, when the path says: the folder
5287/// under the directory that holds this account's `home`, `root` for root's
5288/// home on Unix, and `LocalSystem` for its profile on Windows. `None` when
5289/// the path names nobody, which is reported as the path alone.
5290#[must_use]
5291pub fn data_root_owner(root: &Path, home: Option<&Path>) -> Option<String> {
5292    let target = path_key(root);
5293    if cfg!(windows) {
5294        if target
5295            .windows(3)
5296            .any(|run| run == ["system32", "config", "systemprofile"])
5297        {
5298            return Some("LocalSystem".to_owned());
5299        }
5300    } else if ["/root", "/var/root", "/private/var/root"]
5301        .iter()
5302        .any(|home| root.starts_with(home))
5303    {
5304        return Some("root".to_owned());
5305    }
5306    let users = path_key(home?.parent()?);
5307    // A home directly under the root (`/root` under `sudo`) names no directory
5308    // of homes: every path would then "belong" to its first component.
5309    if users.len() < 2 {
5310        return None;
5311    }
5312    if target.len() > users.len() && target[..users.len()] == users[..] {
5313        root.components()
5314            .nth(users.len())
5315            .map(|name| name.as_os_str().to_string_lossy().into_owned())
5316    } else {
5317        None
5318    }
5319}
5320
5321/// A registration whose command line names application-data directories that
5322/// are not this account's: **another installation's service**.
5323///
5324/// Every account on a machine sees the one registration its service manager
5325/// holds under the product's name. A second account -- one that ran a
5326/// login-mode install once, say -- has an install record and a database of its
5327/// own, and comparing them against the other installation's registration used
5328/// to report a stale binary, a mismatched start mode and a secret store in the
5329/// wrong scope, with `service uninstall`, `service install` and `auth login`
5330/// as the remedies. Followed from that account, each of them breaks or
5331/// replaces a service that is healthy. So the registration is reported as the
5332/// other installation's, with what can be read of it, and this account's own
5333/// record as the unused configuration it is.
5334#[derive(Debug, Clone, PartialEq, Eq)]
5335pub struct ForeignService {
5336    /// The directories its command line names.
5337    pub directories: ServiceDirectories,
5338    /// The data root they sit under. See [`ServiceDirectories::root`].
5339    pub root: PathBuf,
5340    /// The account that data root belongs to, when its path says.
5341    pub owner: Option<String>,
5342    /// How a command run by this account reaches the installation, when it is
5343    /// this account's own under other directories -- an install made with
5344    /// `--data-dir`, seen from a command run without it, or the other way
5345    /// round. `None` for another account's.
5346    pub this_account: Option<String>,
5347    /// This account's own data root, which no registration runs against.
5348    pub local_root: PathBuf,
5349    /// Where this account's own install record is, or would be.
5350    pub local_record: PathBuf,
5351    /// This account's own install record, from an earlier install.
5352    pub unused_local_record: Option<InstallRecord>,
5353    /// Why the service's own records could not be read, or `None` when they
5354    /// were and the status describes them.
5355    pub unreadable: Option<String>,
5356}
5357
5358impl ForeignService {
5359    /// How to see the service's own details: as its owner, or elevated.
5360    #[must_use]
5361    pub fn where_to_look(&self) -> String {
5362        if let Some(reach) = &self.this_account {
5363            return reach.clone();
5364        }
5365        let elevated = if cfg!(windows) {
5366            "from an elevated terminal (Run as administrator)"
5367        } else {
5368            "with sudo"
5369        };
5370        match &self.owner {
5371            Some(owner) if owner != "LocalSystem" && owner != "root" => {
5372                format!("{elevated}, or signed in as {owner}")
5373            }
5374            _ => elevated.to_owned(),
5375        }
5376    }
5377
5378    /// Where a change to the service has to be made from: the installation's
5379    /// own account, because every remedy a command prints acts on the
5380    /// directories of whoever runs it. Elevation alone reads the service's
5381    /// records but is still this account.
5382    #[must_use]
5383    pub fn act_as(&self) -> String {
5384        if let Some(reach) = &self.this_account {
5385            return reach.clone();
5386        }
5387        match &self.owner {
5388            Some(owner) if owner != "LocalSystem" && owner != "root" => {
5389                format!("signed in as {owner}")
5390            }
5391            _ => format!(
5392                "from the account whose data root {} is",
5393                self.root.display()
5394            ),
5395        }
5396    }
5397
5398    /// Whether the service's own records could not be read, so only what the
5399    /// service manager says is known.
5400    #[must_use]
5401    pub const fn is_unread(&self) -> bool {
5402        self.unreadable.is_some()
5403    }
5404
5405    /// Who installed it, for a sentence.
5406    #[must_use]
5407    pub fn installed_by(&self) -> String {
5408        match &self.owner {
5409            Some(owner) => format!("{owner}'s data root {}", self.root.display()),
5410            None => format!("the data root {}", self.root.display()),
5411        }
5412    }
5413
5414    /// The data root and its owner, for a field: `<root> (<owner>)`.
5415    #[must_use]
5416    pub fn label(&self) -> String {
5417        match &self.owner {
5418            Some(owner) => format!("{} ({owner})", self.root.display()),
5419            None => self.root.display().to_string(),
5420        }
5421    }
5422
5423    /// The note describing this account's own record, when there is one.
5424    #[must_use]
5425    pub fn unused_local_note(&self) -> Option<String> {
5426        let record = self.unused_local_record.as_ref()?;
5427        Some(format!(
5428            "unused local configuration from an earlier install: {} records a {} registration \
5429             made by runner-manager {} on {}, and no service on this machine runs against it. \
5430             `runner-manager host forget-local` moves it aside; it never touches the service's \
5431             own directories or any secret.",
5432            self.local_record.display(),
5433            record.start_mode,
5434            record.installed_by_version,
5435            record.installed_at.format("%Y-%m-%d"),
5436        ))
5437    }
5438}
5439
5440// ---------------------------------------------------------------------------
5441// Status
5442// ---------------------------------------------------------------------------
5443
5444/// One thing `service status` has to report as wrong.
5445#[derive(Debug, Clone, PartialEq, Eq)]
5446pub struct StatusProblem {
5447    /// What it is about.
5448    pub subject: &'static str,
5449    /// What is wrong, and what to do.
5450    pub detail: String,
5451}
5452
5453impl fmt::Display for StatusProblem {
5454    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
5455        write!(f, "{}: {}", self.subject, self.detail)
5456    }
5457}
5458
5459/// Everything `service status` reports.
5460///
5461/// Journey 5 step 4 names four of these — the start mode, the resolved binary
5462/// path, the diagnostic log path, and the last successful GitHub contact — and
5463/// the release gate adds the fifth: *"`service status` reports a stale binary
5464/// path as an error rather than appearing healthy"*. [`Self::is_healthy`] is
5465/// what an exit code should be derived from, and it is false whenever
5466/// [`Self::problems`] is non-empty.
5467#[derive(Debug, Clone)]
5468pub struct ServiceStatus {
5469    identity: ServiceIdentity,
5470    record: Option<InstallRecord>,
5471    registration: Option<Registration>,
5472    binary: Option<BinaryPath>,
5473    log_file: PathBuf,
5474    store: Option<crate::secrets::ActiveStore>,
5475    last_github_contact: Option<DateTime<Utc>>,
5476    runner_root: Option<(PathBuf, RootAccessReport)>,
5477    credential_rejected_since: Option<DateTime<Utc>>,
5478    launches_blocked: Option<crate::launch_health::LaunchesBlocked>,
5479    unattended_gap: Option<UnattendedGap>,
5480    foreign: Option<ForeignService>,
5481    problems: Vec<StatusProblem>,
5482    notes: Vec<String>,
5483}
5484
5485/// Why a registration does not come back by itself after an unattended
5486/// restart, and what an operator can do about it.
5487#[derive(Debug, Clone, PartialEq, Eq)]
5488pub struct UnattendedGap {
5489    /// One or more sentences, the way `service status` prints them.
5490    pub reason: String,
5491    /// What changes it. Never something this product does by itself.
5492    pub remedy: String,
5493}
5494
5495/// The name of the account a registration runs as: the invoking user's own
5496/// for a login-mode one.
5497#[must_use]
5498pub fn service_account_name(account: &ServiceAccount) -> String {
5499    match account {
5500        // Under `sudo` the invoking user is the one sudo names, not root.
5501        ServiceAccount::InvokingUser => std::env::var("SUDO_USER")
5502            .ok()
5503            .filter(|user| !user.is_empty() && crate::host_fitness::is_elevated())
5504            .or_else(crate::unattended_login::current_account)
5505            .unwrap_or_else(|| "the operator".to_owned()),
5506        other => other.as_str().to_owned(),
5507    }
5508}
5509
5510/// Works out the gap for a registration in `mode` running as `account`.
5511///
5512/// `found` is `None` where nothing could be probed (every platform but macOS):
5513/// a login registration then keeps the general statement that it waits for a
5514/// sign-in. On a Mac the statement depends on automatic login and FileVault,
5515/// and a login registration that automatic login brings back has no gap. The
5516/// earlier statement was printed for every login registration, so a Mac set to
5517/// sign its runner account in automatically was told it would not resume.
5518#[must_use]
5519pub fn unattended_gap(
5520    mode: StartMode,
5521    account: &str,
5522    found: Option<&crate::unattended_login::UnattendedLogin>,
5523) -> Option<UnattendedGap> {
5524    let Some(found) = found else {
5525        return (mode == StartMode::Login).then(|| UnattendedGap {
5526            reason: "This registration starts at login, so the agent does not run until the \
5527                     operator signs in; this host does not resume work after an unattended \
5528                     reboot."
5529                .to_owned(),
5530            remedy: "For machine-on operation, move the service and credential to `--start-at \
5531                     boot` from an elevated terminal."
5532                .to_owned(),
5533        });
5534    };
5535    let resume = crate::unattended_login::resume(found, mode, account);
5536    if matches!(resume, crate::unattended_login::Resume::Unknown(_)) {
5537        // `host doctor` says what could not be read; a note here would claim
5538        // a gap nobody has seen.
5539        return None;
5540    }
5541    Some(UnattendedGap {
5542        reason: resume.explain(account)?,
5543        remedy: resume.remedy().unwrap_or_else(|| {
5544            "Run `runner-manager host doctor` on the Mac's desktop, where automatic login and \
5545             FileVault can be read."
5546                .to_owned()
5547        }),
5548    })
5549}
5550
5551impl ServiceStatus {
5552    /// `record_refused` carries a [`ServiceError::RecordNotPermitted`] when
5553    /// there **is** a record and this account may not read it, which is not the
5554    /// same state as having none and must not be reported as one. See
5555    /// [`Operations::status`].
5556    fn compose(
5557        identity: ServiceIdentity,
5558        record: Option<InstallRecord>,
5559        record_refused: Option<&ServiceError>,
5560        registration: Option<Registration>,
5561        last_github_contact: Option<DateTime<Utc>>,
5562        paths: &AppPaths,
5563    ) -> Self {
5564        let mut problems = Vec::new();
5565        let mut notes = Vec::new();
5566
5567        if let Some(refusal) = record_refused {
5568            // A problem rather than a note: `is_healthy` stays false, because
5569            // an operator who cannot read their own install record does have
5570            // something to fix. It is just not a reason to withhold everything
5571            // the service manager knows.
5572            problems.push(StatusProblem {
5573                subject: "install record",
5574                detail: refusal.to_string(),
5575            });
5576        }
5577
5578        let log_file = record.as_ref().map_or_else(
5579            || ServiceDirectories::of(paths).log_file(),
5580            |record| record.log_file.clone(),
5581        );
5582
5583        let binary = record.as_ref().map(|record| {
5584            inspect_binary(
5585                &record.binary,
5586                registration
5587                    .as_ref()
5588                    .and_then(Registration::binary)
5589                    .as_deref(),
5590            )
5591        });
5592        if let Some(state) = &binary
5593            && state.is_error()
5594        {
5595            problems.push(StatusProblem {
5596                subject: "binary",
5597                detail: state.to_string(),
5598            });
5599        }
5600
5601        match (&record, &registration) {
5602            (Some(_), None) => problems.push(StatusProblem {
5603                subject: "registration",
5604                detail: "this host has a service record but no service manager knows the \
5605                         registration. Run `service install` again; it deletes nothing."
5606                    .to_string(),
5607            }),
5608            // Not reported when the record was refused rather than absent: the
5609            // problem above already names the real state, and "there is no
5610            // install record" would be a second, wrong diagnosis of the same
5611            // file, with a `service uninstall` in its remedy.
5612            (None, Some(found)) if record_refused.is_none() => problems.push(StatusProblem {
5613                subject: "record",
5614                detail: format!(
5615                    "{} holds a registration for this service but there is no install record, so \
5616                     the path it was installed from and the directories it was installed against \
5617                     are unknown. Run `service uninstall` and `service install`.",
5618                    found.manager
5619                ),
5620            }),
5621            (None, Some(_)) => {}
5622            (Some(record), Some(found)) => {
5623                if record.start_mode != found.start_mode {
5624                    problems.push(StatusProblem {
5625                        subject: "start mode",
5626                        detail: format!(
5627                            "the record says {} and {} holds a {} registration. Switch the start \
5628                             mode again to make them agree.",
5629                            record.start_mode, found.manager, found.start_mode
5630                        ),
5631                    });
5632                }
5633                if record.start_mode == StartMode::Boot && !found.starts_automatically {
5634                    problems.push(StatusProblem {
5635                        subject: "start mode",
5636                        detail: format!(
5637                            "{} holds the registration but will not start it by itself, so this \
5638                             host does not resume work after a reboot.",
5639                            found.manager
5640                        ),
5641                    });
5642                }
5643                if let Some(actual) = found.restart_delay {
5644                    let expected = record.restart().effective_delay(found.manager);
5645                    if actual != expected {
5646                        problems.push(StatusProblem {
5647                            subject: "restart policy",
5648                            detail: format!(
5649                                "the record says the service restarts after {}s and {} reports \
5650                                 {}s. Something has edited the registration since it was \
5651                                 installed.",
5652                                expected.as_secs(),
5653                                found.manager,
5654                                actual.as_secs()
5655                            ),
5656                        });
5657                    } else if expected != record.restart().delay() {
5658                        // Not a fault. Task Scheduler expresses this in whole
5659                        // minutes, so the delay in force is longer than the one
5660                        // asked for -- never shorter, which is the direction
5661                        // the requirement cares about.
5662                        notes.push(format!(
5663                            "{} expresses the restart delay in whole minutes, so the {}s asked \
5664                             for is enforced as {}s. The service therefore never restarts faster \
5665                             than the configured bound.",
5666                            found.manager,
5667                            record.restart().delay().as_secs(),
5668                            expected.as_secs()
5669                        ));
5670                    }
5671                }
5672                if found.starts_automatically && !found.running {
5673                    problems.push(StatusProblem {
5674                        subject: "runtime",
5675                        detail: format!(
5676                            "{} holds an automatic registration, but the daemon is stopped. Run \
5677                             `runner-manager service install --start-at {}`; if the service \
5678                             manager denies replacement, retry from an elevated terminal. If it \
5679                             stops again, inspect {}.",
5680                            found.manager,
5681                            found.start_mode,
5682                            log_file.display()
5683                        ),
5684                    });
5685                }
5686            }
5687            (None, None) => {}
5688        }
5689
5690        // The two facts here really are independent, which is what `f1`'s
5691        // `status.rs` says this check needs and could not have there: the scope
5692        // is derived from the mode the **record** carries, and it is compared
5693        // against the mode the **service manager** actually holds.
5694        let store = record.as_ref().and_then(|record| {
5695            let registered_mode = registration
5696                .as_ref()
5697                .map_or(record.start_mode, |found| found.start_mode);
5698            crate::secrets::PlatformSecretStore::for_start_mode(record.start_mode)
5699                .ok()
5700                .map(|store| crate::secrets::ActiveStore::of(&store, registered_mode))
5701        });
5702        if let Some(store) = &store
5703            && !store.agrees_with_start_mode()
5704        {
5705            problems.push(StatusProblem {
5706                subject: "secret store",
5707                detail: format!(
5708                    "{store}. Run `auth login` again so the token is stored where the registered \
5709                     start mode can read it."
5710                ),
5711            });
5712        }
5713
5714        if last_github_contact.is_none() {
5715            notes.push(
5716                "GitHub has not been reached successfully since this host's state directory was \
5717                 created."
5718                    .to_string(),
5719            );
5720        }
5721
5722        // The privileged inspection output. Read-only, tolerant of an account
5723        // that may not read the descriptor at all, and never a `problem`: a
5724        // root this account cannot inspect is a true statement about this
5725        // account's rights, not a fault in the registration. What refuses an
5726        // install is the preflight in `runner_root_access`, which runs as the
5727        // installing account and has the authority to know.
5728        let runner_root = crate::runner_root::default_runner_root(paths)
5729            .ok()
5730            .map(|root| {
5731                let path = root.as_path().to_path_buf();
5732                let report = crate::runner_root_access::report(&path);
5733                (path, report)
5734            });
5735        // Reported, and deliberately not a `problem`. The Definition of Done
5736        // asks for a broad root to be "reported and fail the security
5737        // preflight", and the security preflight is `install`'s -- which runs
5738        // as the installing account and refuses outright. `service status` runs
5739        // as whoever typed it, is expected to be readable on a host with
5740        // nothing installed at all, and drives an exit code; turning a
5741        // directory that predates this feature into a non-zero exit for a
5742        // machine that has never installed the service would report a fault
5743        // that is not this registration's.
5744        if let Some((
5745            path,
5746            RootAccessReport::Present {
5747                broad_write: true, ..
5748            },
5749        )) = &runner_root
5750        {
5751            notes.push(format!(
5752                "the platform default runner root {} can be written by ordinary local users, so \
5753                 it is not a safe place to run jobs. `service install` refuses it rather than \
5754                 tightening it, because the contents of a directory anybody could write cannot \
5755                 be trusted: remove or empty it, or choose another root with `runner-manager \
5756                 host set-runtime-root --path <PATH>`.",
5757                path.display()
5758            ));
5759        }
5760
5761        // The surface the agent has nowhere else to reach. A root that refuses
5762        // a launch does so before any attempt row exists, and the daemon's log
5763        // redacts the paths out of the sentence, so without this the operator
5764        // sees `runner_start_failed reason=other` once per poll and nothing
5765        // that names the directory or the fix.
5766        //
5767        // A **note** and not a problem, by the same rule the broad-write root
5768        // above obeys: `service status` runs as whoever typed it, is expected
5769        // to be readable on a host with nothing installed at all, and drives an
5770        // exit code. A problem here would fail that exit code on three hosts
5771        // that are not broken -- one whose root was fixed but which has had no
5772        // queued job since, one whose service was uninstalled without clearing
5773        // `state/`, and one that only ever ran the agent in the foreground --
5774        // and the printed remedy (`service uninstall && service install`) does
5775        // not touch `state/`, so following it would not clear the error either.
5776        // A note is read by the same operator at the same moment and traps
5777        // nobody in a loop.
5778        //
5779        // A read failure is reported and not swallowed, for the reason
5780        // `runner_root_refusals` documents: absence means "every policy is
5781        // placing runners", and answering a parse failure with that would be
5782        // the wrong answer to the question this row exists for.
5783        match runner_root_refusals(paths) {
5784            Ok(refusals) => {
5785                for refusal in refusals {
5786                    notes.push(format!(
5787                        "policy {} started no runner: its runner root {} refused the launch \
5788                         ({}), last at {}. {} This clears when that policy next places a \
5789                         runner.",
5790                        refusal.policy,
5791                        refusal.root,
5792                        refusal.kind,
5793                        refusal.at.to_rfc3339(),
5794                        refusal.detail,
5795                    ));
5796                }
5797            }
5798            Err(error) => notes.push(format!(
5799                "whether the agent could use its runner roots could not be read: {error}"
5800            )),
5801        }
5802
5803        Self {
5804            identity,
5805            record,
5806            registration,
5807            binary,
5808            log_file,
5809            store,
5810            last_github_contact,
5811            runner_root,
5812            credential_rejected_since: None,
5813            launches_blocked: None,
5814            unattended_gap: None,
5815            foreign: None,
5816            problems,
5817            notes,
5818        }
5819    }
5820
5821    /// Marks a status read from another installation's directories as that
5822    /// installation's. See [`ForeignService`].
5823    ///
5824    /// Its problems were written for its owner, and their remedies -- `service
5825    /// install`, `auth login`, re-registering a drifted definition -- act on
5826    /// the directories of whoever runs them. Printed to this account, which
5827    /// reaches here precisely by following "run it elevated", they would
5828    /// re-register the machine's service against this account's data root.
5829    /// So each keeps its diagnosis and loses its remedy, and says where the
5830    /// remedy has to be run instead.
5831    fn with_foreign(mut self, foreign: ForeignService) -> Self {
5832        let act_as = foreign.act_as();
5833        for problem in &mut self.problems {
5834            problem.detail = format!(
5835                "{}. Act on it {act_as}: a remedy run from this account acts on this account's \
5836                 own directories, not the service's.",
5837                diagnosis_only(&problem.detail)
5838            );
5839        }
5840        if let Some(note) = foreign.unused_local_note() {
5841            self.notes.push(note);
5842        }
5843        self.foreign = Some(foreign);
5844        self
5845    }
5846
5847    /// A registration made from another data root whose records this account
5848    /// cannot read: what the service manager reports, and nothing else.
5849    ///
5850    /// The only problems are ones the service manager alone settles -- an
5851    /// automatic registration that is stopped, a boot registration that will
5852    /// not start by itself -- and their remedy is to look as the owner, never
5853    /// to reinstall from here.
5854    fn foreign_unread(
5855        identity: ServiceIdentity,
5856        registration: Registration,
5857        foreign: ForeignService,
5858    ) -> Self {
5859        let mut problems = Vec::new();
5860        let look = foreign.where_to_look();
5861        if registration.starts_automatically && !registration.running {
5862            problems.push(StatusProblem {
5863                subject: "runtime",
5864                detail: format!(
5865                    "{} holds an automatic registration for the service installed from {}, but \
5866                     it is stopped. Run `runner-manager service status` {look} to see why; do not \
5867                     reinstall it from this account.",
5868                    registration.manager,
5869                    foreign.installed_by(),
5870                ),
5871            });
5872        }
5873        if registration.start_mode == StartMode::Boot && !registration.starts_automatically {
5874            problems.push(StatusProblem {
5875                subject: "start mode",
5876                detail: format!(
5877                    "{} holds the boot registration installed from {} but will not start it by \
5878                     itself. Run `runner-manager service status` {look}.",
5879                    registration.manager,
5880                    foreign.installed_by(),
5881                ),
5882            });
5883        }
5884        let mut notes = Vec::new();
5885        if let Some(reason) = &foreign.unreadable {
5886            notes.push(format!(
5887                "its health is not known here: {reason}. Run `runner-manager service status` \
5888                 {look} to see it."
5889            ));
5890        }
5891        notes.extend(foreign.unused_local_note());
5892        Self {
5893            identity,
5894            record: None,
5895            log_file: foreign.directories.log_file(),
5896            registration: Some(registration),
5897            binary: None,
5898            store: None,
5899            last_github_contact: None,
5900            runner_root: None,
5901            credential_rejected_since: None,
5902            launches_blocked: None,
5903            unattended_gap: None,
5904            foreign: Some(foreign),
5905            problems,
5906            notes,
5907        }
5908    }
5909
5910    /// The other installation this machine's service belongs to, when it is
5911    /// not this account's. See [`ForeignService`].
5912    #[must_use]
5913    pub const fn foreign(&self) -> Option<&ForeignService> {
5914        self.foreign.as_ref()
5915    }
5916
5917    /// Whether this is another installation's service whose own records this
5918    /// account could not read, so only the service manager's facts are here.
5919    #[must_use]
5920    pub fn is_unread(&self) -> bool {
5921        self.foreign.as_ref().is_some_and(ForeignService::is_unread)
5922    }
5923
5924    /// The one-word-or-so verdict `service status` ends with.
5925    #[must_use]
5926    pub fn verdict(&self) -> String {
5927        if !self.is_healthy() {
5928            return "NOT healthy".to_owned();
5929        }
5930        match &self.foreign {
5931            Some(foreign) if foreign.is_unread() => format!(
5932                "{}; installed from {}, so its health is read {}",
5933                if self.is_running() {
5934                    "running"
5935                } else {
5936                    "not running"
5937                },
5938                foreign.installed_by(),
5939                foreign.where_to_look()
5940            ),
5941            _ => "healthy".to_owned(),
5942        }
5943    }
5944
5945    /// Reports a definition this build would render differently.
5946    #[must_use]
5947    pub fn with_definition_drift(mut self, drift: Option<DefinitionDrift>) -> Self {
5948        if let Some(drift) = drift {
5949            if drift.manager == DefinitionKind::LaunchdPlist {
5950                // A restart, not `service install`: the running daemon holds the
5951                // single-instance lock, which refuses an install, and a daemon
5952                // that starts with no runner repairs the plist itself.
5953                let restart = match drift.start_mode {
5954                    StartMode::Boot => format!(
5955                        "sudo launchctl kickstart -k system/{}",
5956                        self.identity.launchd_label()
5957                    ),
5958                    StartMode::Login => format!(
5959                        "launchctl kickstart -k gui/$(id -u)/{}",
5960                        self.identity.launchd_label()
5961                    ),
5962                };
5963                self.problems.push(StatusProblem {
5964                    subject: "definition",
5965                    detail: format!(
5966                        "{} was written by runner-manager {} with a `ProcessType` other than \
5967                         `{LAUNCHD_PROCESS_TYPE}`, which launchd applies to every runner the \
5968                         daemon starts, so their jobs run below normal priority. The daemon \
5969                         rewrites it and reloads itself when it starts with no runner, which it \
5970                         does after every upgrade; to do it now, restart the service while no \
5971                         job is running: `{restart}`. Configuration and credentials are kept.",
5972                        drift.path.display(),
5973                        drift.installed_by_version,
5974                    ),
5975                });
5976                return self;
5977            }
5978            let repair = match drift.start_mode {
5979                StartMode::Boot => {
5980                    "`sudo systemctl stop runner-manager && sudo runner-manager service install \
5981                     --start-at boot` (for a managed WSL host, run `runner-manager wsl install \
5982                     --distribution <name>` from Windows)"
5983                }
5984                StartMode::Login => {
5985                    "`systemctl --user stop runner-manager && runner-manager service install \
5986                     --start-at login`"
5987                }
5988            };
5989            self.problems.push(StatusProblem {
5990                subject: "definition",
5991                detail: format!(
5992                    "{} was written by runner-manager {} and differs from what {} renders, so \
5993                     the daemon runs under outdated settings (for example a credential frozen \
5994                     at service start). Re-register it with {repair}; configuration and \
5995                     credentials are kept.",
5996                    drift.path.display(),
5997                    drift.installed_by_version,
5998                    env!("CARGO_PKG_VERSION"),
5999                ),
6000            });
6001        }
6002        self
6003    }
6004
6005    /// Reports that GitHub is rejecting the daemon's credential.
6006    #[must_use]
6007    pub fn with_credential_rejection(mut self, since: Option<DateTime<Utc>>) -> Self {
6008        if let Some(since) = since {
6009            self.credential_rejected_since = Some(since);
6010            self.problems.push(StatusProblem {
6011                subject: "github credential",
6012                detail: format!(
6013                    "GitHub has rejected the service's credential since {}, so the service \
6014                     cannot see queued jobs or start runners. Run `runner-manager auth login` \
6015                     (for a managed WSL host, `runner-manager --host wsl:<name> auth login` \
6016                     from Windows).",
6017                    since.to_rfc3339()
6018                ),
6019            });
6020        }
6021        self
6022    }
6023
6024    /// Since when GitHub has rejected the service's credential, if it has.
6025    #[must_use]
6026    pub const fn credential_rejected_since(&self) -> Option<DateTime<Utc>> {
6027        self.credential_rejected_since
6028    }
6029
6030    /// Reports that the daemon starts no runner because its launches are
6031    /// blocked. See [`crate::launch_health`].
6032    #[must_use]
6033    pub fn with_launches_blocked(
6034        mut self,
6035        blocked: Option<crate::launch_health::LaunchesBlocked>,
6036    ) -> Self {
6037        if let Some(blocked) = &blocked {
6038            self.problems.push(StatusProblem {
6039                subject: LAUNCHES_BLOCKED_SUBJECT,
6040                detail: format!(
6041                    "the service has started no runner {blocked}. Fix: {}.",
6042                    blocked.remedy
6043                ),
6044            });
6045        }
6046        self.launches_blocked = blocked;
6047        self
6048    }
6049
6050    /// Why the service starts no runner, when its launches are blocked.
6051    #[must_use]
6052    pub const fn launches_blocked(&self) -> Option<&crate::launch_health::LaunchesBlocked> {
6053        self.launches_blocked.as_ref()
6054    }
6055
6056    /// Reports that the daemon cannot read its own stored credential, and the
6057    /// exact command that ends it. See [`record_credential_unreadable`].
6058    #[must_use]
6059    pub fn with_credential_unreadable(mut self, since: Option<DateTime<Utc>>) -> Self {
6060        if let Some(since) = since {
6061            let mode = self.start_mode().unwrap_or_default();
6062            self.problems.push(StatusProblem {
6063                subject: CREDENTIAL_UNREADABLE_SUBJECT,
6064                detail: format!(
6065                    "the service has been unable to read its stored GitHub credential since {}, \
6066                     so it starts no runner; the diagnostic log has the store's own answer.{} \
6067                     Sign in again: {}.",
6068                    since.to_rfc3339(),
6069                    if cfg!(target_os = "macos") {
6070                        " That is what a keychain item written by a different build looks like."
6071                    } else {
6072                        ""
6073                    },
6074                    sign_in_instruction(mode)
6075                ),
6076            });
6077        }
6078        self
6079    }
6080
6081    /// What this host's default runner root grants, and to whom.
6082    ///
6083    /// `None` when the platform default could not even be resolved. The
6084    /// descriptor inside has been through
6085    /// [`crate::runner_root_access::redact`], so it names the well-known
6086    /// trustees and says "an account" for everything else — no more identity
6087    /// than the `account` line above it already prints.
6088    #[must_use]
6089    pub fn runner_root(&self) -> Option<(&Path, &RootAccessReport)> {
6090        self.runner_root
6091            .as_ref()
6092            .map(|(path, report)| (path.as_path(), report))
6093    }
6094
6095    /// Whether a registration exists at all.
6096    #[must_use]
6097    pub const fn is_installed(&self) -> bool {
6098        self.registration.is_some() || self.record.is_some()
6099    }
6100
6101    /// Whether the daemon is running now.
6102    #[must_use]
6103    pub fn is_running(&self) -> bool {
6104        self.registration
6105            .as_ref()
6106            .is_some_and(|registration| registration.running)
6107    }
6108
6109    /// **False whenever anything is wrong**, including a stale binary path.
6110    #[must_use]
6111    pub fn is_healthy(&self) -> bool {
6112        self.problems.is_empty()
6113    }
6114
6115    /// Everything that is wrong.
6116    #[must_use]
6117    pub fn problems(&self) -> &[StatusProblem] {
6118        &self.problems
6119    }
6120
6121    /// True statements that are not faults — a login-mode registration not
6122    /// running unattended, or an agent that has not yet reached GitHub.
6123    #[must_use]
6124    pub fn notes(&self) -> &[String] {
6125        &self.notes
6126    }
6127
6128    /// Reports that the daemon starts no runner because a required host check
6129    /// fails from its own session. That is the service not doing its job, so
6130    /// it is a problem: a host whose daemon refused every runner used to read
6131    /// `verdict healthy`.
6132    #[must_use]
6133    pub fn with_host_unfit(mut self, unfit: Option<crate::host_fitness::HostUnfitRecord>) -> Self {
6134        if let Some(unfit) = unfit {
6135            self.problems.push(StatusProblem {
6136                subject: HOST_UNFIT_SUBJECT,
6137                detail: format!(
6138                    "the service has started no runner since {}, because a required host check \
6139                     fails from its own session: {}",
6140                    unfit.since.to_rfc3339(),
6141                    unfit.describe()
6142                ),
6143            });
6144        }
6145        self
6146    }
6147
6148    /// Reports a daemon that is running and has stopped making progress. See
6149    /// [`crate::daemon_heartbeat`]: launchd, systemd and the SCM all report
6150    /// such a daemon running, so this used to read `verdict healthy`.
6151    #[must_use]
6152    pub fn with_stall(mut self, stall: Option<String>) -> Self {
6153        if let Some(detail) = stall {
6154            self.problems.push(StatusProblem {
6155                subject: DAEMON_STALLED_SUBJECT,
6156                detail,
6157            });
6158        }
6159        self
6160    }
6161
6162    /// Records whether this registration comes back by itself after an
6163    /// unattended restart, as a note. See [`unattended_gap`].
6164    #[must_use]
6165    pub fn with_unattended_gap(mut self, gap: Option<UnattendedGap>) -> Self {
6166        if let Some(gap) = &gap {
6167            self.notes.push(gap.reason.clone());
6168        }
6169        self.unattended_gap = gap;
6170        self
6171    }
6172
6173    /// Why this registration does not come back by itself after an
6174    /// unattended restart, when it does not.
6175    #[must_use]
6176    pub const fn unattended_gap(&self) -> Option<&UnattendedGap> {
6177        self.unattended_gap.as_ref()
6178    }
6179
6180    /// The recorded start mode. Journey 5 step 4. For another installation's
6181    /// service whose record this account cannot read, the mode the service
6182    /// manager's domain implies.
6183    #[must_use]
6184    pub fn start_mode(&self) -> Option<StartMode> {
6185        self.record
6186            .as_ref()
6187            .map(|record| record.start_mode)
6188            .or_else(|| {
6189                self.registration
6190                    .as_ref()
6191                    .filter(|_| self.is_unread())
6192                    .map(|registration| registration.start_mode)
6193            })
6194    }
6195
6196    /// The binary the service runs: the recorded one, or for another
6197    /// installation's service whose record this account cannot read, the one
6198    /// the service manager names.
6199    #[must_use]
6200    pub fn service_binary(&self) -> Option<PathBuf> {
6201        self.record
6202            .as_ref()
6203            .map(|record| record.binary.clone())
6204            .or_else(|| {
6205                self.registration
6206                    .as_ref()
6207                    .filter(|_| self.is_unread())
6208                    .and_then(Registration::binary)
6209            })
6210    }
6211
6212    /// The resolved absolute binary path and what became of it. Journey 5
6213    /// step 4 and `05-infrastructure.md` item 6.
6214    #[must_use]
6215    pub const fn binary(&self) -> Option<&BinaryPath> {
6216        self.binary.as_ref()
6217    }
6218
6219    /// The diagnostic log path. `05-infrastructure.md` item 4.
6220    #[must_use]
6221    pub fn log_file(&self) -> &Path {
6222        &self.log_file
6223    }
6224
6225    /// The last successful GitHub contact. Journey 5 step 4.
6226    #[must_use]
6227    pub const fn last_github_contact(&self) -> Option<DateTime<Utc>> {
6228        self.last_github_contact
6229    }
6230
6231    /// Which store the daemon reads, and whether that agrees with the
6232    /// registration.
6233    #[must_use]
6234    pub const fn secret_store(&self) -> Option<&crate::secrets::ActiveStore> {
6235        self.store.as_ref()
6236    }
6237
6238    /// The record `install` wrote.
6239    #[must_use]
6240    pub const fn record(&self) -> Option<&InstallRecord> {
6241        self.record.as_ref()
6242    }
6243
6244    /// What the service manager says.
6245    #[must_use]
6246    pub const fn registration(&self) -> Option<&Registration> {
6247        self.registration.as_ref()
6248    }
6249}
6250
6251impl fmt::Display for ServiceStatus {
6252    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
6253        writeln!(f, "Service: {}", self.identity)?;
6254        match (&self.record, &self.registration) {
6255            (None, None) => {
6256                writeln!(
6257                    f,
6258                    "  installed                 no. `service install` registers `{} {}`.",
6259                    SERVICE_NAME,
6260                    DAEMON_ARGUMENTS.join(" ")
6261                )?;
6262            }
6263            _ => {
6264                let manager = self
6265                    .registration
6266                    .as_ref()
6267                    .map(|registration| registration.manager.manager());
6268                writeln!(
6269                    f,
6270                    "  installed                 {}",
6271                    manager.unwrap_or("yes, but no service manager knows it")
6272                )?;
6273                writeln!(
6274                    f,
6275                    "  state                     {}",
6276                    if self.is_running() {
6277                        "running"
6278                    } else {
6279                        "not running"
6280                    }
6281                )?;
6282            }
6283        }
6284        if let Some(foreign) = &self.foreign {
6285            writeln!(
6286                f,
6287                "  installed from            {} -- another installation's, not this account's ({})",
6288                foreign.installed_by(),
6289                foreign.local_root.display()
6290            )?;
6291            // Without the record, the service manager's own facts.
6292            if foreign.is_unread()
6293                && let Some(registration) = &self.registration
6294            {
6295                writeln!(f, "  start mode                {}", registration.start_mode)?;
6296                if let Some(account) = &registration.account {
6297                    writeln!(f, "  account                   {account}")?;
6298                }
6299                if let Some(binary) = self.service_binary() {
6300                    writeln!(f, "  binary                    {}", binary.display())?;
6301                }
6302            }
6303        }
6304        if let Some(record) = &self.record {
6305            writeln!(f, "  start mode                {}", record.start_mode)?;
6306            writeln!(f, "  account                   {}", record.account)?;
6307            writeln!(f, "  restart on failure        {}", record.restart())?;
6308            writeln!(
6309                f,
6310                "  arguments                 {}",
6311                record.arguments.join(" ")
6312            )?;
6313        }
6314        if let Some(binary) = &self.binary {
6315            writeln!(f, "  binary                    {binary}")?;
6316        }
6317        writeln!(f, "  diagnostic log            {}", self.log_file.display())?;
6318        if let Some((path, report)) = &self.runner_root {
6319            // Named as the *default* rather than as "the runner root", because
6320            // it is only the effective one until an operator runs
6321            // `host set-runtime-root`. This crate cannot see that setting — it
6322            // lives in the application's store — so an unqualified label here
6323            // would contradict the `runner root … (host-configured)` row that
6324            // `status` and `host show` print from the value that is in force.
6325            writeln!(f, "  default runner root       {}", path.display())?;
6326            if *report != RootAccessReport::NotApplicable {
6327                writeln!(f, "  default root access       {report}")?;
6328            }
6329        }
6330        if let Some(store) = &self.store {
6331            writeln!(f, "  secret store              {store}")?;
6332        }
6333        // Not known, rather than "never", for a service whose records were not
6334        // read: its contact file is in a directory this account cannot open.
6335        if !self.is_unread() {
6336            writeln!(
6337                f,
6338                "  last GitHub contact       {}",
6339                match self.last_github_contact {
6340                    Some(at) => at.to_rfc3339(),
6341                    None => "never".to_string(),
6342                }
6343            )?;
6344        }
6345        for note in &self.notes {
6346            writeln!(f, "  note                      {note}")?;
6347        }
6348        for problem in &self.problems {
6349            writeln!(f, "  ERROR                     {problem}")?;
6350        }
6351        write!(f, "  verdict                   {}", self.verdict())
6352    }
6353}
6354
6355// ---------------------------------------------------------------------------
6356// The in-memory double
6357// ---------------------------------------------------------------------------
6358
6359/// A [`ControlFactory`] that registers nothing and remembers everything.
6360///
6361/// Public rather than `#[cfg(test)]` on purpose. `f3` builds three commands on
6362/// [`ServiceOperations`] and has to be able to test them without a service
6363/// manager, and a double that lived behind `#[cfg(test)]` here would be
6364/// invisible from another crate — which would leave `f3` either untested or
6365/// writing a second double that drifts from this one.
6366///
6367/// It is not a simulator. It stores what it was asked to register and reports it
6368/// back, which is exactly enough to exercise the logic that sits *above* a
6369/// service manager: the lock refusal, the record, the stale-path detection, the
6370/// start-mode switch, and the promise that uninstall deletes nothing else.
6371#[derive(Debug, Clone, Default)]
6372pub struct RecordingControls {
6373    state: std::sync::Arc<std::sync::Mutex<RecordingState>>,
6374}
6375
6376#[derive(Debug, Default)]
6377struct RecordingState {
6378    registrations: BTreeMap<(StartMode, String), Registration>,
6379    definitions: BTreeMap<String, ServiceDefinition>,
6380    calls: Vec<String>,
6381    #[cfg(test)]
6382    install_failures: BTreeMap<StartMode, String>,
6383    #[cfg(test)]
6384    after_install: BTreeMap<StartMode, TestInstallSideEffect>,
6385}
6386
6387#[cfg(test)]
6388#[derive(Debug, Clone)]
6389enum TestInstallSideEffect {
6390    HideDirectory { directory: PathBuf, hidden: PathBuf },
6391}
6392
6393impl RecordingControls {
6394    /// A factory holding no registrations.
6395    #[must_use]
6396    pub fn new() -> Self {
6397        Self::default()
6398    }
6399
6400    /// Every call made through this factory, in order, as
6401    /// `"<operation> <name> (<mode>)"`.
6402    #[must_use]
6403    pub fn calls(&self) -> Vec<String> {
6404        self.state.lock().expect("not poisoned").calls.clone()
6405    }
6406
6407    /// Every registration currently held, with the domain it is in.
6408    #[must_use]
6409    pub fn registrations(&self) -> Vec<(StartMode, String, Registration)> {
6410        self.state
6411            .lock()
6412            .expect("not poisoned")
6413            .registrations
6414            .iter()
6415            .map(|((mode, name), registration)| (*mode, name.clone(), registration.clone()))
6416            .collect()
6417    }
6418
6419    /// The definition applied for a registration, if it is held.
6420    #[must_use]
6421    pub fn definition(&self, name: &str) -> Option<ServiceDefinition> {
6422        self.state
6423            .lock()
6424            .expect("not poisoned")
6425            .definitions
6426            .get(name)
6427            .cloned()
6428    }
6429
6430    /// Edits a held registration, as something outside this product would.
6431    ///
6432    /// This is how the divergence and start-type problems are made reachable
6433    /// from a test: `sc config`, `launchctl` and `systemctl` can all change a
6434    /// registration after installation, and a status command that could not be
6435    /// shown detecting that would be a status command nobody had tested against
6436    /// the case it exists for.
6437    ///
6438    /// Does nothing when no registration of that name is held.
6439    pub fn edit(&self, name: &str, edit: impl FnOnce(&mut Registration)) {
6440        let mut state = self.state.lock().expect("not poisoned");
6441        if let Some((_, registration)) = state
6442            .registrations
6443            .iter_mut()
6444            .find(|((_, held), _)| held == name)
6445        {
6446            edit(registration);
6447        }
6448    }
6449
6450    #[cfg(test)]
6451    fn fail_next_install(&self, mode: StartMode, detail: &str) {
6452        self.state
6453            .lock()
6454            .expect("not poisoned")
6455            .install_failures
6456            .insert(mode, detail.to_string());
6457    }
6458
6459    #[cfg(test)]
6460    fn hide_directory_after_install(&self, mode: StartMode, directory: PathBuf, hidden: PathBuf) {
6461        self.state
6462            .lock()
6463            .expect("not poisoned")
6464            .after_install
6465            .insert(
6466                mode,
6467                TestInstallSideEffect::HideDirectory { directory, hidden },
6468            );
6469    }
6470}
6471
6472impl ControlFactory for RecordingControls {
6473    fn control(&self, mode: StartMode) -> Result<Box<dyn ServiceControl>, ServiceError> {
6474        Ok(Box::new(RecordingControl {
6475            mode,
6476            state: std::sync::Arc::clone(&self.state),
6477        }))
6478    }
6479}
6480
6481#[derive(Debug)]
6482struct RecordingControl {
6483    mode: StartMode,
6484    state: std::sync::Arc<std::sync::Mutex<RecordingState>>,
6485}
6486
6487impl RecordingControl {
6488    fn note(&self, operation: &str, name: &str) {
6489        self.state
6490            .lock()
6491            .expect("not poisoned")
6492            .calls
6493            .push(format!("{operation} {name} ({})", self.mode));
6494    }
6495}
6496
6497impl ServiceControl for RecordingControl {
6498    fn manager(&self) -> DefinitionKind {
6499        // The double reports the kind this host's real backend would, so a test
6500        // that asserts on the manager name asserts something true of the
6501        // platform it is running on.
6502        host_definition_kind(self.mode)
6503    }
6504
6505    fn install(&self, plan: &InstallPlan) -> Result<ServiceDefinition, ServiceError> {
6506        self.note("install", plan.identity().name());
6507        #[cfg(test)]
6508        if let Some(detail) = self
6509            .state
6510            .lock()
6511            .expect("not poisoned")
6512            .install_failures
6513            .remove(&self.mode)
6514        {
6515            return Err(ServiceError::Control {
6516                operation: "install",
6517                name: plan.identity().name().to_string(),
6518                manager: "recording control",
6519                detail,
6520            });
6521        }
6522        // The real definition, not a stub. Rendering is pure, so the double can
6523        // afford it -- and a stub would make `Installed::review` report that the
6524        // definition confirms nothing, which is exactly the shape of false
6525        // assurance a test double must never hand back.
6526        let definition = ServiceDefinition::for_host(plan)?;
6527        let mut state = self.state.lock().expect("not poisoned");
6528        state.registrations.insert(
6529            (self.mode, plan.identity().name().to_string()),
6530            Registration {
6531                manager: host_definition_kind(self.mode),
6532                start_mode: self.mode,
6533                command_line: plan.command_line(),
6534                account: Some(plan.account().as_str().to_string()),
6535                running: false,
6536                starts_automatically: true,
6537                restart_delay: Some(plan.restart().delay()),
6538            },
6539        );
6540        state
6541            .definitions
6542            .insert(plan.identity().name().to_string(), definition.clone());
6543        #[cfg(test)]
6544        let side_effect = state.after_install.remove(&self.mode);
6545        drop(state);
6546        #[cfg(test)]
6547        if let Some(TestInstallSideEffect::HideDirectory { directory, hidden }) = side_effect {
6548            std::fs::rename(&directory, &hidden).expect("test fault can hide the record directory");
6549            std::fs::write(&directory, b"blocks recreation")
6550                .expect("test fault can block record directory recreation");
6551        }
6552        Ok(definition)
6553    }
6554
6555    fn uninstall(&self, identity: &ServiceIdentity) -> Result<bool, ServiceError> {
6556        self.note("uninstall", identity.name());
6557        let mut state = self.state.lock().expect("not poisoned");
6558        state.definitions.remove(identity.name());
6559        Ok(state
6560            .registrations
6561            .remove(&(self.mode, identity.name().to_string()))
6562            .is_some())
6563    }
6564
6565    fn query(&self, identity: &ServiceIdentity) -> Result<Option<Registration>, ServiceError> {
6566        self.note("query", identity.name());
6567        Ok(self
6568            .state
6569            .lock()
6570            .expect("not poisoned")
6571            .registrations
6572            .get(&(self.mode, identity.name().to_string()))
6573            .cloned())
6574    }
6575
6576    fn start(&self, identity: &ServiceIdentity) -> Result<(), ServiceError> {
6577        self.note("start", identity.name());
6578        let mut state = self.state.lock().expect("not poisoned");
6579        match state
6580            .registrations
6581            .get_mut(&(self.mode, identity.name().to_string()))
6582        {
6583            Some(registration) => {
6584                registration.running = true;
6585                Ok(())
6586            }
6587            None => Err(ServiceError::NotInstalled {
6588                name: identity.name().to_string(),
6589                operation: "start",
6590            }),
6591        }
6592    }
6593
6594    fn stop(&self, identity: &ServiceIdentity) -> Result<bool, ServiceError> {
6595        self.note("stop", identity.name());
6596        let mut state = self.state.lock().expect("not poisoned");
6597        match state
6598            .registrations
6599            .get_mut(&(self.mode, identity.name().to_string()))
6600        {
6601            Some(registration) => Ok(std::mem::replace(&mut registration.running, false)),
6602            None => Err(ServiceError::NotInstalled {
6603                name: identity.name().to_string(),
6604                operation: "stop",
6605            }),
6606        }
6607    }
6608}
6609
6610impl ControlFactory for HostControls {
6611    fn control(&self, mode: StartMode) -> Result<Box<dyn ServiceControl>, ServiceError> {
6612        sys::control(mode)
6613    }
6614
6615    fn installed_definition(&self, path: &Path) -> Option<String> {
6616        std::fs::read_to_string(path).ok()
6617    }
6618
6619    fn unattended_login(&self) -> Option<crate::unattended_login::UnattendedLogin> {
6620        crate::unattended_login::probe()
6621    }
6622}
6623
6624/// The operator's home directory, when the platform reports one.
6625///
6626/// Only login-mode registrations need it — a LaunchAgent and a systemd user
6627/// unit both live under it — and it is resolved lazily so that a boot-mode
6628/// operation on an account with no profile is not refused for wanting something
6629/// it never uses.
6630#[must_use]
6631pub fn host_home() -> Option<PathBuf> {
6632    directories::BaseDirs::new().map(|dirs| dirs.home_dir().to_path_buf())
6633}
6634
6635/// The Unix backends' name for the same thing.
6636#[cfg(unix)]
6637fn home_directory() -> Option<PathBuf> {
6638    host_home()
6639}
6640
6641/// Runs a command and returns its exit status with both streams as text.
6642///
6643/// The three Unix backends and the Windows Task Scheduler backend all drive a
6644/// stock command-line tool, and all four want the same three things back. The
6645/// tools are chosen for machine-readable, locale-independent output wherever one
6646/// exists — `systemctl is-active`, `launchctl print`, `schtasks /XML` — because
6647/// parsing a localised human-facing table is how a status command starts lying
6648/// on somebody else's machine.
6649fn run(program: &str, arguments: &[&std::ffi::OsStr]) -> std::io::Result<(bool, String, String)> {
6650    let output = std::process::Command::new(program)
6651        .args(arguments)
6652        .output()?;
6653    Ok((
6654        output.status.success(),
6655        String::from_utf8_lossy(&output.stdout).into_owned(),
6656        String::from_utf8_lossy(&output.stderr).into_owned(),
6657    ))
6658}
6659
6660/// Which manager holds a given start-mode domain on the platform this binary
6661/// was built for.
6662#[must_use]
6663pub const fn host_definition_kind(mode: StartMode) -> DefinitionKind {
6664    if cfg!(windows) {
6665        match mode {
6666            StartMode::Boot => DefinitionKind::WindowsService,
6667            StartMode::Login => DefinitionKind::WindowsScheduledTask,
6668        }
6669    } else if cfg!(target_os = "macos") {
6670        DefinitionKind::LaunchdPlist
6671    } else {
6672        DefinitionKind::SystemdUnit
6673    }
6674}
6675
6676/// Enables a launchd label after it has been bootstrapped.
6677///
6678/// `launchctl bootstrap` deliberately preserves a label's disabled bit. An
6679/// `enable` failure therefore means the new registration cannot satisfy its
6680/// start policy. Compensate by booting it out and removing the plist so a
6681/// failed install cannot look installed to `service status`.
6682#[cfg(any(target_os = "macos", test))]
6683fn enable_launchd_registration(
6684    mut launchctl: impl FnMut(&[&std::ffi::OsStr]) -> (bool, String),
6685    domain: &str,
6686    service_target: &str,
6687    plist: &Path,
6688    name: &str,
6689    elevation_remedy: &'static str,
6690) -> Result<(), ServiceError> {
6691    let (enabled, cause) = launchctl(&[
6692        std::ffi::OsStr::new("enable"),
6693        std::ffi::OsStr::new(service_target),
6694    ]);
6695    if enabled {
6696        return Ok(());
6697    }
6698
6699    let (booted_out, bootout_detail) = launchctl(&[
6700        std::ffi::OsStr::new("bootout"),
6701        std::ffi::OsStr::new(service_target),
6702    ]);
6703    let removed = std::fs::remove_file(plist);
6704    if !booted_out || removed.is_err() {
6705        return Err(ServiceError::Rollback {
6706            operation: "enable launchd registration",
6707            name: name.to_string(),
6708            cause,
6709            rollback: format!(
6710                "launchctl bootout {domain}: {}; remove {}: {}",
6711                if booted_out {
6712                    "succeeded".to_string()
6713                } else {
6714                    bootout_detail
6715                },
6716                plist.display(),
6717                removed
6718                    .err()
6719                    .map_or_else(|| "succeeded".to_string(), |error| error.to_string())
6720            ),
6721        });
6722    }
6723
6724    if cause.to_ascii_lowercase().contains("permission denied") {
6725        Err(ServiceError::NeedsElevation {
6726            operation: "enable",
6727            name: name.to_string(),
6728            detail: cause,
6729            remedy: elevation_remedy,
6730        })
6731    } else {
6732        Err(ServiceError::Control {
6733            operation: "enable",
6734            name: name.to_string(),
6735            manager: "launchd",
6736            detail: cause,
6737        })
6738    }
6739}
6740
6741// ---------------------------------------------------------------------------
6742// Windows
6743// ---------------------------------------------------------------------------
6744
6745/// A stop request delivered by the Windows Service Control Manager.
6746///
6747/// The application owns the drain policy; this platform boundary only turns
6748/// `SERVICE_CONTROL_STOP`/`SHUTDOWN` into an awaitable notification.
6749#[derive(Debug, Clone)]
6750pub struct ServiceShutdown(tokio::sync::watch::Receiver<bool>);
6751
6752impl ServiceShutdown {
6753    /// Waits until SCM asks the service to stop or shut down.
6754    pub async fn wait(mut self) {
6755        if !*self.0.borrow() {
6756            let _ = self.0.changed().await;
6757        }
6758    }
6759}
6760
6761/// Runs the production process as a real Windows service host.
6762///
6763/// `StartServiceCtrlDispatcher` must run in the service process's main thread,
6764/// so the callback is handed through a process-global slot to the entrypoint
6765/// invoked by SCM. The slot is single-use by design: one process hosts exactly
6766/// one own-process service.
6767#[cfg(windows)]
6768pub fn run_windows_service_host<F>(run: F) -> Result<u8, ServiceError>
6769where
6770    F: FnOnce(ServiceShutdown) -> u8 + Send + 'static,
6771{
6772    windows_host::run(Box::new(run))
6773}
6774
6775#[cfg(windows)]
6776mod windows_host {
6777    use std::ffi::OsString;
6778    use std::sync::{Arc, Mutex, OnceLock, mpsc};
6779    use std::time::Duration;
6780
6781    use windows_service::service::{
6782        ServiceControl, ServiceControlAccept, ServiceExitCode, ServiceState, ServiceStatus,
6783        ServiceType,
6784    };
6785    use windows_service::service_control_handler::{
6786        self, ServiceControlHandlerResult, ServiceStatusHandle,
6787    };
6788
6789    use super::{SERVICE_NAME, ServiceError, ServiceShutdown};
6790
6791    type Runner = Box<dyn FnOnce(ServiceShutdown) -> u8 + Send>;
6792
6793    struct Invocation {
6794        run: Runner,
6795        result: mpsc::SyncSender<Result<u8, String>>,
6796    }
6797
6798    static INVOCATION: OnceLock<Mutex<Option<Invocation>>> = OnceLock::new();
6799
6800    windows_service::define_windows_service!(ffi_service_main, service_main);
6801
6802    pub(super) fn run(run: Runner) -> Result<u8, ServiceError> {
6803        let (result_tx, result_rx) = mpsc::sync_channel(1);
6804        let slot = INVOCATION.get_or_init(|| Mutex::new(None));
6805        let mut invocation = slot
6806            .lock()
6807            .map_err(|_| host_error("prepare", "the service-host slot is poisoned"))?;
6808        if invocation.is_some() {
6809            return Err(host_error(
6810                "prepare",
6811                "the service-host slot was already used",
6812            ));
6813        }
6814        *invocation = Some(Invocation {
6815            run,
6816            result: result_tx,
6817        });
6818        drop(invocation);
6819
6820        windows_service::service_dispatcher::start("", ffi_service_main)
6821            .map_err(|error| host_error("connect", &error.to_string()))?;
6822        result_rx
6823            .recv()
6824            .map_err(|error| host_error("finish", &error.to_string()))?
6825            .map_err(|detail| host_error("run", &detail))
6826    }
6827
6828    fn service_main(_arguments: Vec<OsString>) {
6829        let Some(invocation) = INVOCATION.get().and_then(|slot| slot.lock().ok()?.take()) else {
6830            return;
6831        };
6832        let result = run_service(invocation.run);
6833        let _ = invocation.result.send(result);
6834    }
6835
6836    fn run_service(run: Runner) -> Result<u8, String> {
6837        let (stop_tx, stop_rx) = tokio::sync::watch::channel(false);
6838        let status: Arc<Mutex<Option<ServiceStatusHandle>>> = Arc::new(Mutex::new(None));
6839        let handler_status = Arc::clone(&status);
6840        let handler = move |control| match control {
6841            ServiceControl::Interrogate => ServiceControlHandlerResult::NoError,
6842            ServiceControl::Stop | ServiceControl::Shutdown => {
6843                if let Some(handle) = handler_status.lock().ok().and_then(|guard| *guard) {
6844                    let _ = handle.set_service_status(service_status(
6845                        ServiceState::StopPending,
6846                        ServiceControlAccept::empty(),
6847                        1,
6848                        Duration::from_secs(300),
6849                        0,
6850                    ));
6851                }
6852                let _ = stop_tx.send(true);
6853                ServiceControlHandlerResult::NoError
6854            }
6855            _ => ServiceControlHandlerResult::NotImplemented,
6856        };
6857        let handle = service_control_handler::register("", handler)
6858            .map_err(|error| format!("cannot register the service control handler: {error}"))?;
6859        *status
6860            .lock()
6861            .map_err(|_| "the service status handle is poisoned".to_string())? = Some(handle);
6862        handle
6863            .set_service_status(service_status(
6864                ServiceState::Running,
6865                ServiceControlAccept::STOP | ServiceControlAccept::SHUTDOWN,
6866                0,
6867                Duration::default(),
6868                0,
6869            ))
6870            .map_err(|error| format!("cannot report SERVICE_RUNNING: {error}"))?;
6871
6872        let exit = run(ServiceShutdown(stop_rx));
6873        handle
6874            .set_service_status(service_status(
6875                ServiceState::Stopped,
6876                ServiceControlAccept::empty(),
6877                0,
6878                Duration::default(),
6879                u32::from(exit),
6880            ))
6881            .map_err(|error| format!("cannot report SERVICE_STOPPED: {error}"))?;
6882        Ok(exit)
6883    }
6884
6885    fn service_status(
6886        state: ServiceState,
6887        accepted: ServiceControlAccept,
6888        checkpoint: u32,
6889        wait_hint: Duration,
6890        exit: u32,
6891    ) -> ServiceStatus {
6892        ServiceStatus {
6893            service_type: ServiceType::OWN_PROCESS,
6894            current_state: state,
6895            controls_accepted: accepted,
6896            exit_code: if exit == 0 {
6897                ServiceExitCode::Win32(0)
6898            } else {
6899                ServiceExitCode::ServiceSpecific(exit)
6900            },
6901            checkpoint,
6902            wait_hint,
6903            process_id: None,
6904        }
6905    }
6906
6907    fn host_error(operation: &'static str, detail: &str) -> ServiceError {
6908        ServiceError::Control {
6909            operation,
6910            name: SERVICE_NAME.to_string(),
6911            manager: "the Windows Service Control Manager",
6912            detail: detail.to_string(),
6913        }
6914    }
6915}
6916
6917#[cfg(windows)]
6918mod sys {
6919    //! Two managers, because Windows has two answers.
6920    //!
6921    //! `--start-at boot` is a service in the Service Control Manager, which is
6922    //! the only Windows facility that starts something before anybody logs in.
6923    //! `--start-at login` is a Task Scheduler task with a logon trigger, which
6924    //! is the only Windows facility that starts something when somebody does.
6925    //! There is no single mechanism that does both: service trigger-start has no
6926    //! logon trigger, and a scheduled task cannot run before a session exists.
6927
6928    use std::ffi::{OsStr, OsString};
6929    use std::time::{Duration, Instant};
6930
6931    use runner_manager_domain::model::StartMode;
6932    use windows_service::service::{
6933        ServiceAccess, ServiceAction, ServiceActionType, ServiceErrorControl,
6934        ServiceFailureActions, ServiceFailureResetPeriod, ServiceInfo, ServiceStartType,
6935        ServiceState, ServiceType,
6936    };
6937    use windows_service::service_manager::{ServiceManager, ServiceManagerAccess};
6938
6939    use super::{
6940        DefinitionKind, InstallPlan, Registration, ServiceControl, ServiceDefinition, ServiceError,
6941        ServiceIdentity, TaskPrincipal, run, windows_service_spec, xml_value,
6942    };
6943
6944    /// `ERROR_SERVICE_DOES_NOT_EXIST`. Not an error here: it is the answer to
6945    /// "is this registered".
6946    const SERVICE_DOES_NOT_EXIST: i32 = 1060;
6947    /// `ERROR_SERVICE_MARKED_FOR_DELETE`. `DeleteService` is asynchronous: the
6948    /// registration remains in this state until every open service handle is
6949    /// closed, and callers must not mistake that short window for a leak.
6950    const SERVICE_MARKED_FOR_DELETE: i32 = 1072;
6951    /// `ERROR_ACCESS_DENIED`.
6952    const ACCESS_DENIED: i32 = 5;
6953    const DELETE_TIMEOUT: Duration = Duration::from_secs(30);
6954    const DELETE_POLL_INTERVAL: Duration = Duration::from_millis(200);
6955
6956    const ELEVATION_REMEDY: &str = "Run the command from an elevated prompt: right-click Windows Terminal or PowerShell and \
6957         choose \"Run as administrator\".";
6958
6959    pub(super) fn control(mode: StartMode) -> Result<Box<dyn ServiceControl>, ServiceError> {
6960        Ok(match mode {
6961            StartMode::Boot => Box::new(ScmControl),
6962            StartMode::Login => Box::new(TaskControl),
6963        })
6964    }
6965
6966    // -- the Service Control Manager -----------------------------------------
6967
6968    #[derive(Debug)]
6969    struct ScmControl;
6970
6971    /// Turns a `windows-service` failure into this module's error, keeping the
6972    /// "needs elevation" case separate: an operator told "access is denied" and
6973    /// an operator told "run this elevated" are not equally well served.
6974    fn scm_error(
6975        operation: &'static str,
6976        name: &str,
6977        error: &windows_service::Error,
6978    ) -> ServiceError {
6979        let raw = match error {
6980            windows_service::Error::Winapi(io) => io.raw_os_error(),
6981            _ => None,
6982        };
6983        let detail = match error {
6984            windows_service::Error::Winapi(io) => io.to_string(),
6985            other => other.to_string(),
6986        };
6987        if raw == Some(ACCESS_DENIED) {
6988            return ServiceError::NeedsElevation {
6989                operation,
6990                name: name.to_string(),
6991                detail,
6992                remedy: ELEVATION_REMEDY,
6993            };
6994        }
6995        ServiceError::Control {
6996            operation,
6997            name: name.to_string(),
6998            manager: "the Windows Service Control Manager",
6999            detail,
7000        }
7001    }
7002
7003    fn open_manager(
7004        access: ServiceManagerAccess,
7005        operation: &'static str,
7006        name: &str,
7007    ) -> Result<ServiceManager, ServiceError> {
7008        ServiceManager::local_computer(None::<&OsStr>, access)
7009            .map_err(|error| scm_error(operation, name, &error))
7010    }
7011
7012    impl ServiceControl for ScmControl {
7013        fn manager(&self) -> DefinitionKind {
7014            DefinitionKind::WindowsService
7015        }
7016
7017        fn install(&self, plan: &InstallPlan) -> Result<ServiceDefinition, ServiceError> {
7018            let spec = windows_service_spec(plan);
7019            let manager = open_manager(
7020                ServiceManagerAccess::CONNECT | ServiceManagerAccess::CREATE_SERVICE,
7021                "install",
7022                &spec.name,
7023            )?;
7024            let info = ServiceInfo {
7025                name: OsString::from(&spec.name),
7026                display_name: OsString::from(&spec.display_name),
7027                // Never INTERACTIVE_PROCESS: see `windows_service_descriptor`.
7028                service_type: ServiceType::OWN_PROCESS,
7029                start_type: if spec.automatic_start {
7030                    ServiceStartType::AutoStart
7031                } else {
7032                    ServiceStartType::OnDemand
7033                },
7034                error_control: ServiceErrorControl::Normal,
7035                executable_path: plan.binary().to_path_buf(),
7036                launch_arguments: plan.arguments().to_vec(),
7037                dependencies: Vec::new(),
7038                // `None` is LocalSystem, which is the account `d2`'s DACL admits.
7039                account_name: spec.account.as_ref().map(OsString::from),
7040                // No password is ever taken, asked for, or stored. An account
7041                // that needed one would be an account this installer refuses.
7042                account_password: None,
7043            };
7044            let service = manager
7045                .create_service(
7046                    &info,
7047                    ServiceAccess::CHANGE_CONFIG
7048                        | ServiceAccess::QUERY_CONFIG
7049                        | ServiceAccess::QUERY_STATUS
7050                        | ServiceAccess::START
7051                        | ServiceAccess::STOP
7052                        | ServiceAccess::DELETE,
7053                )
7054                .map_err(|error| scm_error("install", &spec.name, &error))?;
7055            service
7056                .set_description(&spec.description)
7057                .map_err(|error| scm_error("describe", &spec.name, &error))?;
7058            service
7059                .update_failure_actions(ServiceFailureActions {
7060                    reset_period: ServiceFailureResetPeriod::After(spec.restart.reset_after()),
7061                    reboot_msg: None,
7062                    command: None,
7063                    // Three identical restart actions rather than one: the
7064                    // Service Control Manager applies the first action to the
7065                    // first failure, the second to the second, and the last to
7066                    // every failure after that. One action would leave the
7067                    // second and later failures unhandled, which is a service
7068                    // that comes back once and then stays down.
7069                    actions: Some(vec![
7070                        ServiceAction {
7071                            action_type: ServiceActionType::Restart,
7072                            delay: spec.restart.delay(),
7073                        };
7074                        3
7075                    ]),
7076                })
7077                .map_err(|error| scm_error("set the restart policy of", &spec.name, &error))?;
7078            // Without this the failure actions apply only to a crash, and a
7079            // daemon that exits non-zero is not a crash.
7080            service
7081                .set_failure_actions_on_non_crash_failures(true)
7082                .map_err(|error| scm_error("set the restart policy of", &spec.name, &error))?;
7083            Ok(ServiceDefinition::windows_service(plan))
7084        }
7085
7086        fn uninstall(&self, identity: &ServiceIdentity) -> Result<bool, ServiceError> {
7087            let manager =
7088                open_manager(ServiceManagerAccess::CONNECT, "uninstall", identity.name())?;
7089            let service = match manager.open_service(
7090                identity.name(),
7091                ServiceAccess::QUERY_STATUS | ServiceAccess::STOP | ServiceAccess::DELETE,
7092            ) {
7093                Ok(service) => Some(service),
7094                Err(error) if is_missing(&error) => return Ok(false),
7095                // A previous caller has already asked SCM to delete this
7096                // registration. That is the desired state, not an uninstall
7097                // failure. Wait for SCM to release the name so a following
7098                // `service install` can use it immediately.
7099                Err(error) if is_marked_for_delete(&error) => None,
7100                Err(error) => return Err(scm_error("uninstall", identity.name(), &error)),
7101            };
7102            if let Some(service) = service {
7103                // A running service can be deleted, but it lingers until it stops.
7104                // Stopping first is what makes `uninstall` followed by `install`
7105                // work in one sitting, which is what the start-mode switch needs.
7106                if let Ok(status) = service.query_status()
7107                    && status.current_state != ServiceState::Stopped
7108                {
7109                    let _ = service.stop();
7110                }
7111                match service.delete() {
7112                    Ok(()) => {}
7113                    // DeleteService is idempotent from the operator's point of
7114                    // view. Another process can mark the service between our
7115                    // OpenService and DeleteService calls.
7116                    Err(error) if is_marked_for_delete(&error) => {}
7117                    Err(error) => {
7118                        return Err(scm_error("uninstall", identity.name(), &error));
7119                    }
7120                }
7121                drop(service);
7122            }
7123
7124            // `DeleteService` marks a registration for deletion and returns;
7125            // SCM removes it only after the last service handle closes. Drop
7126            // ours before polling, then wait for the observable postcondition
7127            // promised by `uninstall`: an immediate status check must not see
7128            // a registration that is merely on its way out. This also keeps a
7129            // stop/uninstall/install sequence deterministic on busy hosts.
7130            let absent = wait_until_scm_absent(DELETE_TIMEOUT, DELETE_POLL_INTERVAL, || {
7131                match manager.open_service(identity.name(), ServiceAccess::QUERY_STATUS) {
7132                    Ok(service) => {
7133                        drop(service);
7134                        Ok(false)
7135                    }
7136                    Err(error) if is_missing(&error) => Ok(true),
7137                    Err(error) if is_marked_for_delete(&error) => Ok(false),
7138                    Err(error) => Err(scm_error("verify uninstall of", identity.name(), &error)),
7139                }
7140            })?;
7141            if !absent {
7142                return Err(ServiceError::Control {
7143                    operation: "verify uninstall of",
7144                    name: identity.name().to_string(),
7145                    manager: "the Windows Service Control Manager",
7146                    detail: format!(
7147                        "the registration was still visible {} seconds after DeleteService; \
7148                         retry `service uninstall` from an elevated prompt",
7149                        DELETE_TIMEOUT.as_secs()
7150                    ),
7151                });
7152            }
7153            Ok(true)
7154        }
7155
7156        fn query(&self, identity: &ServiceIdentity) -> Result<Option<Registration>, ServiceError> {
7157            let manager = open_manager(ServiceManagerAccess::CONNECT, "inspect", identity.name())?;
7158            let service = match manager.open_service(
7159                identity.name(),
7160                ServiceAccess::QUERY_CONFIG | ServiceAccess::QUERY_STATUS,
7161            ) {
7162                Ok(service) => service,
7163                Err(error) if is_missing(&error) => return Ok(None),
7164                Err(error) => return Err(scm_error("inspect", identity.name(), &error)),
7165            };
7166            let config = service
7167                .query_config()
7168                .map_err(|error| scm_error("inspect", identity.name(), &error))?;
7169            let status = service
7170                .query_status()
7171                .map_err(|error| scm_error("inspect", identity.name(), &error))?;
7172            let restart_delay = service.get_failure_actions().ok().and_then(|actions| {
7173                actions
7174                    .actions
7175                    .and_then(|actions| actions.into_iter().next())
7176                    .filter(|action| action.action_type == ServiceActionType::Restart)
7177                    .map(|action| action.delay)
7178            });
7179            Ok(Some(Registration {
7180                manager: DefinitionKind::WindowsService,
7181                start_mode: StartMode::Boot,
7182                // `lpBinaryPathName` holds the executable *and* its arguments
7183                // as one string, which is why this module carries its own
7184                // parser rather than trusting `executable_path`'s name.
7185                command_line: config.executable_path.to_string_lossy().into_owned(),
7186                account: config
7187                    .account_name
7188                    .map(|account| account.to_string_lossy().into_owned()),
7189                running: status.current_state == ServiceState::Running,
7190                starts_automatically: config.start_type == ServiceStartType::AutoStart,
7191                restart_delay,
7192            }))
7193        }
7194
7195        fn start(&self, identity: &ServiceIdentity) -> Result<(), ServiceError> {
7196            let manager = open_manager(ServiceManagerAccess::CONNECT, "start", identity.name())?;
7197            let service = manager
7198                .open_service(identity.name(), ServiceAccess::START)
7199                .map_err(|error| scm_error("start", identity.name(), &error))?;
7200            service
7201                .start::<&OsStr>(&[])
7202                .map_err(|error| scm_error("start", identity.name(), &error))
7203        }
7204
7205        fn stop(&self, identity: &ServiceIdentity) -> Result<bool, ServiceError> {
7206            let manager = open_manager(ServiceManagerAccess::CONNECT, "stop", identity.name())?;
7207            let service = manager
7208                .open_service(
7209                    identity.name(),
7210                    ServiceAccess::STOP | ServiceAccess::QUERY_STATUS,
7211                )
7212                .map_err(|error| scm_error("stop", identity.name(), &error))?;
7213            let status = service
7214                .query_status()
7215                .map_err(|error| scm_error("stop", identity.name(), &error))?;
7216            if status.current_state == ServiceState::Stopped {
7217                return Ok(false);
7218            }
7219            service
7220                .stop()
7221                .map_err(|error| scm_error("stop", identity.name(), &error))?;
7222            Ok(true)
7223        }
7224    }
7225
7226    fn is_missing(error: &windows_service::Error) -> bool {
7227        matches!(error, windows_service::Error::Winapi(io)
7228            if io.raw_os_error() == Some(SERVICE_DOES_NOT_EXIST))
7229    }
7230
7231    pub(super) fn is_marked_for_delete(error: &windows_service::Error) -> bool {
7232        matches!(error, windows_service::Error::Winapi(io)
7233            if io.raw_os_error() == Some(SERVICE_MARKED_FOR_DELETE))
7234    }
7235
7236    pub(super) fn wait_until_scm_absent(
7237        timeout: Duration,
7238        poll_interval: Duration,
7239        mut probe_absent: impl FnMut() -> Result<bool, ServiceError>,
7240    ) -> Result<bool, ServiceError> {
7241        let deadline = Instant::now() + timeout;
7242        loop {
7243            if probe_absent()? {
7244                return Ok(true);
7245            }
7246            if Instant::now() >= deadline {
7247                return Ok(false);
7248            }
7249            std::thread::sleep(poll_interval);
7250        }
7251    }
7252
7253    // -- Task Scheduler ------------------------------------------------------
7254
7255    #[derive(Debug)]
7256    struct TaskControl;
7257
7258    fn task_error(operation: &'static str, name: &str, detail: String) -> ServiceError {
7259        if detail.to_ascii_lowercase().contains("access is denied") {
7260            return ServiceError::NeedsElevation {
7261                operation,
7262                name: name.to_string(),
7263                detail,
7264                remedy: ELEVATION_REMEDY,
7265            };
7266        }
7267        ServiceError::Control {
7268            operation,
7269            name: name.to_string(),
7270            manager: "Windows Task Scheduler",
7271            detail,
7272        }
7273    }
7274
7275    fn schtasks(
7276        operation: &'static str,
7277        name: &str,
7278        arguments: &[&OsStr],
7279    ) -> Result<(bool, String), ServiceError> {
7280        match run("schtasks.exe", arguments) {
7281            Ok((ok, stdout, stderr)) => Ok((ok, if ok { stdout } else { stderr })),
7282            Err(error) => Err(task_error(operation, name, error.to_string())),
7283        }
7284    }
7285
7286    /// Task Scheduler reads `/XML` files as UTF-16, so the document is written
7287    /// as UTF-16 little-endian with a byte-order mark rather than as UTF-8.
7288    fn write_utf16(path: &std::path::Path, text: &str) -> std::io::Result<()> {
7289        let mut bytes = vec![0xFF, 0xFE];
7290        for unit in text.encode_utf16() {
7291            bytes.extend_from_slice(&unit.to_le_bytes());
7292        }
7293        std::fs::write(path, bytes)
7294    }
7295
7296    impl ServiceControl for TaskControl {
7297        fn manager(&self) -> DefinitionKind {
7298            DefinitionKind::WindowsScheduledTask
7299        }
7300
7301        fn install(&self, plan: &InstallPlan) -> Result<ServiceDefinition, ServiceError> {
7302            let name = plan.identity().name().to_string();
7303            let principal = TaskPrincipal::current()?;
7304            let definition = ServiceDefinition::windows_scheduled_task(plan, &principal);
7305            let directory = tempfile::tempdir()
7306                .map_err(|error| task_error("install", &name, error.to_string()))?;
7307            let document = directory.path().join("task.xml");
7308            write_utf16(&document, definition.text())
7309                .map_err(|error| task_error("install", &name, error.to_string()))?;
7310            let (ok, message) = schtasks(
7311                "install",
7312                &name,
7313                &[
7314                    OsStr::new("/Create"),
7315                    OsStr::new("/TN"),
7316                    OsStr::new(&name),
7317                    OsStr::new("/XML"),
7318                    document.as_os_str(),
7319                ],
7320            )?;
7321            if !ok {
7322                return Err(task_error("install", &name, message));
7323            }
7324            Ok(definition)
7325        }
7326
7327        fn uninstall(&self, identity: &ServiceIdentity) -> Result<bool, ServiceError> {
7328            if self.query(identity)?.is_none() {
7329                return Ok(false);
7330            }
7331            let name = identity.name().to_string();
7332            let (ok, message) = schtasks(
7333                "uninstall",
7334                &name,
7335                &[
7336                    OsStr::new("/Delete"),
7337                    OsStr::new("/TN"),
7338                    OsStr::new(&name),
7339                    OsStr::new("/F"),
7340                ],
7341            )?;
7342            if !ok {
7343                return Err(task_error("uninstall", &name, message));
7344            }
7345            Ok(true)
7346        }
7347
7348        fn query(&self, identity: &ServiceIdentity) -> Result<Option<Registration>, ServiceError> {
7349            let name = identity.name().to_string();
7350            let (ok, document) = schtasks(
7351                "inspect",
7352                &name,
7353                &[
7354                    OsStr::new("/Query"),
7355                    OsStr::new("/TN"),
7356                    OsStr::new(&name),
7357                    OsStr::new("/XML"),
7358                    OsStr::new("ONE"),
7359                ],
7360            )?;
7361            if !ok {
7362                // `schtasks` reports a missing task and a broken Task Scheduler
7363                // the same way, with a non-zero exit; there is no distinct code.
7364                // Treating it as absence is the safe reading: the caller either
7365                // installs, which will then fail loudly, or reports "not
7366                // installed", which is what an operator with no task sees.
7367                return Ok(None);
7368            }
7369            let command = xml_value(&document, "Command").unwrap_or_default();
7370            let arguments = xml_value(&document, "Arguments").unwrap_or_default();
7371            let command_line = if arguments.is_empty() {
7372                super::quote_argument(&command)
7373            } else {
7374                format!("{} {arguments}", super::quote_argument(&command))
7375            };
7376            Ok(Some(Registration {
7377                manager: DefinitionKind::WindowsScheduledTask,
7378                start_mode: StartMode::Login,
7379                command_line,
7380                account: xml_value(&document, "UserId"),
7381                running: task_is_running(&name),
7382                starts_automatically: super::windows_login_task_starts_automatically(&document),
7383                restart_delay: xml_value(&document, "Interval")
7384                    .as_deref()
7385                    .and_then(parse_iso8601),
7386            }))
7387        }
7388
7389        fn start(&self, identity: &ServiceIdentity) -> Result<(), ServiceError> {
7390            let name = identity.name().to_string();
7391            let (ok, message) = schtasks(
7392                "start",
7393                &name,
7394                &[OsStr::new("/Run"), OsStr::new("/TN"), OsStr::new(&name)],
7395            )?;
7396            if ok {
7397                Ok(())
7398            } else {
7399                Err(task_error("start", &name, message))
7400            }
7401        }
7402
7403        fn stop(&self, identity: &ServiceIdentity) -> Result<bool, ServiceError> {
7404            let running = self
7405                .query(identity)?
7406                .is_some_and(|registration| registration.running);
7407            if !running {
7408                return Ok(false);
7409            }
7410            let name = identity.name().to_string();
7411            let (ok, message) = schtasks(
7412                "stop",
7413                &name,
7414                &[OsStr::new("/End"), OsStr::new("/TN"), OsStr::new(&name)],
7415            )?;
7416            if ok {
7417                Ok(true)
7418            } else {
7419                Err(task_error("stop", &name, message))
7420            }
7421        }
7422    }
7423
7424    /// Whether Task Scheduler reports the task as running.
7425    ///
7426    /// **This is the one value in this module read from localised output.**
7427    /// `schtasks /Query /FO CSV` prints a `Status` column in the machine's
7428    /// display language, and Task Scheduler exposes no locale-independent
7429    /// equivalent short of COM. On a non-English Windows this therefore reports
7430    /// `false` for a task that is in fact running.
7431    ///
7432    /// That is stated rather than hidden because of what does *not* depend on
7433    /// it: Journey 5's liveness question is answered by the last successful
7434    /// GitHub contact, which is a timestamp this product writes itself and no
7435    /// locale can move. Nothing in the Definition of Done rests on this
7436    /// boolean.
7437    fn task_is_running(name: &str) -> bool {
7438        let Ok((true, stdout, _)) = run(
7439            "schtasks.exe",
7440            &[
7441                OsStr::new("/Query"),
7442                OsStr::new("/TN"),
7443                OsStr::new(name),
7444                OsStr::new("/FO"),
7445                OsStr::new("CSV"),
7446                OsStr::new("/NH"),
7447            ],
7448        ) else {
7449            return false;
7450        };
7451        stdout
7452            .lines()
7453            .filter_map(|line| line.rsplit(',').next())
7454            .any(|status| {
7455                status
7456                    .trim()
7457                    .trim_matches('"')
7458                    .eq_ignore_ascii_case("running")
7459            })
7460    }
7461
7462    /// Parses the `PT<n>M` shape this module writes, and the `PT<n>S` shape
7463    /// Task Scheduler would accept if it took seconds. Anything richer -- days,
7464    /// hours, a combination -- is `None` rather than a guess.
7465    fn parse_iso8601(value: &str) -> Option<Duration> {
7466        let rest = value.strip_prefix("PT")?;
7467        if let Some(minutes) = rest.strip_suffix('M') {
7468            return minutes
7469                .parse::<u64>()
7470                .ok()
7471                .map(|minutes| Duration::from_secs(minutes * 60));
7472        }
7473        rest.strip_suffix('S')?
7474            .parse::<u64>()
7475            .ok()
7476            .map(Duration::from_secs)
7477    }
7478}
7479
7480// ---------------------------------------------------------------------------
7481// Unix: shared plumbing
7482// ---------------------------------------------------------------------------
7483
7484/// Writes a definition file, creating its directory, and says when the refusal
7485/// was a permissions one.
7486///
7487/// Both Unix backends write a file into a directory only `root` can write —
7488/// `/Library/LaunchDaemons` and `/etc/systemd/system` — and an operator who ran
7489/// `service install` without `sudo` deserves to be told that rather than handed
7490/// an `EACCES`.
7491#[cfg(unix)]
7492fn write_definition(
7493    operation: &'static str,
7494    name: &str,
7495    path: &Path,
7496    text: &str,
7497    remedy: &'static str,
7498) -> Result<(), ServiceError> {
7499    if let Some(parent) = path.parent()
7500        && let Err(error) = std::fs::create_dir_all(parent)
7501        && error.kind() != std::io::ErrorKind::AlreadyExists
7502    {
7503        return Err(definition_error(operation, name, error, remedy, parent));
7504    }
7505    std::fs::write(path, text)
7506        .map_err(|error| definition_error(operation, name, error, remedy, path))
7507}
7508
7509/// Replaces an installed definition with [`DefinitionDrift::rendered`].
7510///
7511/// Through a temporary file in the same directory and a rename, so a failure
7512/// at any point leaves the old definition in place rather than a truncated
7513/// one. The mode is set to `0644` before the rename, as `service install`
7514/// leaves it: launchd refuses a plist that others can write.
7515#[cfg(all(unix, any(target_os = "macos", test)))]
7516fn replace_definition(name: &str, drift: &DefinitionDrift) -> Result<(), ServiceError> {
7517    use std::io::Write as _;
7518
7519    let path = drift.path.as_path();
7520    let directory = path.parent().unwrap_or_else(|| Path::new("/"));
7521    let failed = |error: std::io::Error, at: &Path| {
7522        definition_error("rewrite", name, error, SUDO_REMEDY, at)
7523    };
7524    let mut staged =
7525        tempfile::NamedTempFile::new_in(directory).map_err(|error| failed(error, directory))?;
7526    staged
7527        .write_all(drift.rendered.as_bytes())
7528        .and_then(|()| staged.as_file().sync_all())
7529        .map_err(|error| failed(error, staged.path()))?;
7530    std::fs::set_permissions(
7531        staged.path(),
7532        <std::fs::Permissions as std::os::unix::fs::PermissionsExt>::from_mode(0o644),
7533    )
7534    .map_err(|error| failed(error, path))?;
7535    staged
7536        .persist(path)
7537        .map(drop)
7538        .map_err(|error| failed(error.error, path))
7539}
7540
7541#[cfg(unix)]
7542fn definition_error(
7543    operation: &'static str,
7544    name: &str,
7545    error: std::io::Error,
7546    remedy: &'static str,
7547    path: &Path,
7548) -> ServiceError {
7549    let detail = format!("{}: {error}", path.display());
7550    if error.kind() == std::io::ErrorKind::PermissionDenied {
7551        ServiceError::NeedsElevation {
7552            operation,
7553            name: name.to_string(),
7554            detail,
7555            remedy,
7556        }
7557    } else {
7558        ServiceError::Control {
7559            operation,
7560            name: name.to_string(),
7561            manager: "the local service manager",
7562            detail,
7563        }
7564    }
7565}
7566
7567/// `sudo` is the remedy on both Unixes, and the message says which command.
7568#[cfg(unix)]
7569const SUDO_REMEDY: &str = "A boot-start registration is machine-wide, so it needs root: run the same command with \
7570     `sudo`. `service install --start-at login` needs no elevation at all, at the cost of the \
7571     agent not running until you sign in.";
7572
7573// ---------------------------------------------------------------------------
7574// macOS
7575// ---------------------------------------------------------------------------
7576
7577#[cfg(target_os = "macos")]
7578mod sys {
7579    //! One manager, two domains. `--start-at boot` is a LaunchDaemon in
7580    //! `/Library/LaunchDaemons`, loaded into launchd's `system` domain;
7581    //! `--start-at login` is a LaunchAgent in the operator's own
7582    //! `~/Library/LaunchAgents`, loaded into `gui/<uid>`.
7583    //!
7584    //! `launchctl print` is the only inspection command here whose output is
7585    //! parsed, and it is English-only on every macOS release — launchd has no
7586    //! localisation for it. The registration's *existence* is decided by the
7587    //! plist file rather than by that output, so a launchd that changed its
7588    //! wording would cost this module the running/not-running line and nothing
7589    //! else.
7590
7591    use std::ffi::OsStr;
7592    use std::path::PathBuf;
7593    use std::time::Duration;
7594
7595    use runner_manager_domain::model::StartMode;
7596
7597    use super::{
7598        DefinitionKind, InstallPlan, LAUNCH_AGENTS_SUBDIR, LAUNCH_DAEMONS_DIR, Registration,
7599        SUDO_REMEDY, ServiceControl, ServiceDefinition, ServiceError, ServiceIdentity,
7600        enable_launchd_registration, home_directory, plist_string_value, quote_argument, run,
7601        write_definition, xml_value,
7602    };
7603
7604    pub(super) fn control(mode: StartMode) -> Result<Box<dyn ServiceControl>, ServiceError> {
7605        Ok(Box::new(LaunchdControl { mode }))
7606    }
7607
7608    #[derive(Debug)]
7609    struct LaunchdControl {
7610        mode: StartMode,
7611    }
7612
7613    impl LaunchdControl {
7614        /// launchd's domain target for this start mode.
7615        fn domain(&self) -> String {
7616            match self.mode {
7617                StartMode::Boot => "system".to_string(),
7618                // SAFETY: `getuid` reads the calling process's real user id and
7619                // cannot fail; it takes no arguments and touches no memory.
7620                StartMode::Login => format!("gui/{}", unsafe { libc::getuid() }),
7621            }
7622        }
7623
7624        fn service_target(&self, identity: &ServiceIdentity) -> String {
7625            format!("{}/{}", self.domain(), identity.launchd_label())
7626        }
7627
7628        /// Where this domain's plist lives, when the domain has a home to put
7629        /// it in.
7630        fn plist_path(&self, identity: &ServiceIdentity) -> Option<PathBuf> {
7631            let file = format!("{}.plist", identity.launchd_label());
7632            match self.mode {
7633                StartMode::Boot => Some(PathBuf::from(LAUNCH_DAEMONS_DIR).join(file)),
7634                StartMode::Login => {
7635                    home_directory().map(|home| home.join(LAUNCH_AGENTS_SUBDIR).join(file))
7636                }
7637            }
7638        }
7639
7640        fn failed(&self, operation: &'static str, name: &str, detail: String) -> ServiceError {
7641            ServiceError::Control {
7642                operation,
7643                name: name.to_string(),
7644                manager: "launchd",
7645                detail,
7646            }
7647        }
7648
7649        fn launchctl(&self, arguments: &[&OsStr]) -> (bool, String) {
7650            match run("launchctl", arguments) {
7651                Ok((ok, stdout, stderr)) => (ok, if ok { stdout } else { stderr }),
7652                Err(error) => (false, error.to_string()),
7653            }
7654        }
7655    }
7656
7657    impl ServiceControl for LaunchdControl {
7658        fn manager(&self) -> DefinitionKind {
7659            DefinitionKind::LaunchdPlist
7660        }
7661
7662        fn install(&self, plan: &InstallPlan) -> Result<ServiceDefinition, ServiceError> {
7663            let name = plan.identity().name().to_string();
7664            let definition = ServiceDefinition::launchd(plan, home_directory().as_deref());
7665            let Some(path) = definition.install_path().map(std::path::Path::to_path_buf) else {
7666                return Err(self.failed(
7667                    "install",
7668                    &name,
7669                    "this account has no home directory, so there is nowhere to put a \
7670                     LaunchAgent. Use --start-at boot, which installs a LaunchDaemon under \
7671                     /Library/LaunchDaemons."
7672                        .to_string(),
7673                ));
7674            };
7675            write_definition("install", &name, &path, definition.text(), SUDO_REMEDY)?;
7676            let target = self.domain();
7677            let (ok, message) = self.launchctl(&[
7678                OsStr::new("bootstrap"),
7679                OsStr::new(&target),
7680                path.as_os_str(),
7681            ]);
7682            if !ok {
7683                // Leave nothing behind: a plist launchd refused is a file that
7684                // would make the next `service status` claim a registration
7685                // that does not exist.
7686                let _ = std::fs::remove_file(&path);
7687                if message.to_ascii_lowercase().contains("permission denied") {
7688                    return Err(ServiceError::NeedsElevation {
7689                        operation: "install",
7690                        name,
7691                        detail: message,
7692                        remedy: SUDO_REMEDY,
7693                    });
7694                }
7695                return Err(self.failed("install", &name, message));
7696            }
7697            // A previously disabled label stays disabled through a bootstrap,
7698            // which is a service that is installed and will never start.
7699            let service_target = self.service_target(plan.identity());
7700            enable_launchd_registration(
7701                |arguments| self.launchctl(arguments),
7702                &target,
7703                &service_target,
7704                &path,
7705                &name,
7706                SUDO_REMEDY,
7707            )?;
7708            Ok(definition)
7709        }
7710
7711        fn uninstall(&self, identity: &ServiceIdentity) -> Result<bool, ServiceError> {
7712            let Some(path) = self.plist_path(identity) else {
7713                return Ok(false);
7714            };
7715            if !path.exists() {
7716                return Ok(false);
7717            }
7718            let target = self.service_target(identity);
7719            let (ok, message) = self.launchctl(&[OsStr::new("bootout"), OsStr::new(&target)]);
7720            if !ok
7721                && !message.to_ascii_lowercase().contains("no such process")
7722                && !message.contains("113")
7723            {
7724                if message.to_ascii_lowercase().contains("permission denied") {
7725                    return Err(ServiceError::NeedsElevation {
7726                        operation: "uninstall",
7727                        name: identity.name().to_string(),
7728                        detail: message,
7729                        remedy: SUDO_REMEDY,
7730                    });
7731                }
7732                return Err(self.failed("uninstall", identity.name(), message));
7733            }
7734            // Only the definition. `05-infrastructure.md` item 5: nothing else
7735            // on this host is touched here or anywhere below it.
7736            std::fs::remove_file(&path).map_err(|error| {
7737                super::definition_error(
7738                    "uninstall",
7739                    identity.name(),
7740                    error,
7741                    SUDO_REMEDY,
7742                    path.as_path(),
7743                )
7744            })?;
7745            Ok(true)
7746        }
7747
7748        fn query(&self, identity: &ServiceIdentity) -> Result<Option<Registration>, ServiceError> {
7749            let Some(path) = self.plist_path(identity) else {
7750                return Ok(None);
7751            };
7752            let Ok(document) = std::fs::read_to_string(&path) else {
7753                return Ok(None);
7754            };
7755            let target = self.service_target(identity);
7756            let (loaded, printed) = self.launchctl(&[OsStr::new("print"), OsStr::new(&target)]);
7757            Ok(Some(Registration {
7758                manager: DefinitionKind::LaunchdPlist,
7759                start_mode: self.mode,
7760                command_line: program_arguments(&document),
7761                account: plist_string_value(&document, "UserName")
7762                    .or_else(|| Some("the invoking user".to_string())),
7763                running: loaded && printed.contains("state = running"),
7764                starts_automatically: document.contains("<key>RunAtLoad</key>")
7765                    && super::plist_bool_value(&document, "RunAtLoad") == Some(true),
7766                restart_delay: xml_value(
7767                    super::plist_value_after_key(&document, "ThrottleInterval").unwrap_or(""),
7768                    "integer",
7769                )
7770                .and_then(|value| value.parse::<u64>().ok())
7771                .map(Duration::from_secs),
7772            }))
7773        }
7774
7775        fn start(&self, identity: &ServiceIdentity) -> Result<(), ServiceError> {
7776            let target = self.service_target(identity);
7777            let (ok, message) = self.launchctl(&[
7778                OsStr::new("kickstart"),
7779                OsStr::new("-k"),
7780                OsStr::new(&target),
7781            ]);
7782            if ok {
7783                Ok(())
7784            } else {
7785                Err(self.failed("start", identity.name(), message))
7786            }
7787        }
7788
7789        fn stop(&self, identity: &ServiceIdentity) -> Result<bool, ServiceError> {
7790            let running = self
7791                .query(identity)?
7792                .is_some_and(|registration| registration.running);
7793            if !running {
7794                return Ok(false);
7795            }
7796            let target = self.service_target(identity);
7797            let (ok, message) = self.launchctl(&[
7798                OsStr::new("kill"),
7799                OsStr::new("SIGTERM"),
7800                OsStr::new(&target),
7801            ]);
7802            if ok {
7803                Ok(true)
7804            } else {
7805                Err(self.failed("stop", identity.name(), message))
7806            }
7807        }
7808    }
7809
7810    /// Boots the job out and bootstraps it again from `plist`, from a shell
7811    /// that outlives this process.
7812    ///
7813    /// `setsid` puts the shell in a session and process group of its own. On a
7814    /// bootout launchd sends SIGTERM to the job's whole process group, the
7815    /// process that asked for the bootout included, and spares a child in
7816    /// another session (measured on macOS 27). Without `setsid` the helper
7817    /// survived that only by luck: one run in four left the job unloaded. The
7818    /// work runs in a background subshell, so the outer shell exits at once and
7819    /// is reaped here rather than left behind. `bootout` blocks until the daemon
7820    /// has exited; the bootstrap is retried for a minute because launchd
7821    /// refuses it while the old job is still being torn down. When the job is
7822    /// still loaded after a failed bootout, every bootstrap fails and the
7823    /// running daemon carries on as it was.
7824    pub(super) fn reload_detached(
7825        identity: &ServiceIdentity,
7826        mode: StartMode,
7827        plist: &std::path::Path,
7828    ) -> Result<(), ServiceError> {
7829        use std::os::unix::process::CommandExt;
7830
7831        const SCRIPT: &str = r#"(
7832  sleep 1
7833  /bin/launchctl bootout "$1"
7834  tries=0
7835  until /bin/launchctl bootstrap "$2" "$3"; do
7836    tries=$((tries + 1))
7837    if [ "$tries" -ge 60 ]; then
7838      echo "runner-manager: launchd did not load $3 again; run: $4" >&2
7839      exit 1
7840    fi
7841    sleep 1
7842  done
7843) </dev/null &"#;
7844
7845        let control = LaunchdControl { mode };
7846        // Once the job is booted out nothing holds the single-instance lock,
7847        // so a reinstall in the same start mode is the remedy.
7848        let reinstall = match mode {
7849            StartMode::Boot => "sudo runner-manager service install --start-at boot",
7850            StartMode::Login => "runner-manager service install --start-at login",
7851        };
7852        let mut command = std::process::Command::new("/bin/sh");
7853        command
7854            .args(["-c", SCRIPT, "runner-manager-reload"])
7855            .arg(control.service_target(identity))
7856            .arg(control.domain())
7857            .arg(plist)
7858            .arg(reinstall)
7859            .stdin(std::process::Stdio::null());
7860        // SAFETY: one async-signal-safe system call between `fork` and `exec`,
7861        // touching no memory, allocator or lock.
7862        unsafe {
7863            command.pre_exec(|| {
7864                if libc::setsid() == -1 {
7865                    return Err(std::io::Error::last_os_error());
7866                }
7867                Ok(())
7868            });
7869        }
7870        let status = command.status().map_err(|error| {
7871            control.failed("reload", identity.name(), format!("/bin/sh: {error}"))
7872        })?;
7873        if status.success() {
7874            Ok(())
7875        } else {
7876            Err(control.failed(
7877                "reload",
7878                identity.name(),
7879                format!("the reload helper exited with {status}"),
7880            ))
7881        }
7882    }
7883
7884    /// Rebuilds the command line from a plist's `ProgramArguments` array.
7885    fn program_arguments(document: &str) -> String {
7886        let Some(rest) = super::plist_value_after_key(document, "ProgramArguments") else {
7887            return String::new();
7888        };
7889        let Some(end) = rest.find("</array>") else {
7890            return String::new();
7891        };
7892        let mut out = Vec::new();
7893        let mut cursor = &rest[..end];
7894        while let Some(open) = cursor.find("<string>") {
7895            let after = &cursor[open + "<string>".len()..];
7896            let Some(close) = after.find("</string>") else {
7897                break;
7898            };
7899            out.push(quote_argument(&super::xml_unescape(&after[..close])));
7900            cursor = &after[close..];
7901        }
7902        out.join(" ")
7903    }
7904}
7905
7906// ---------------------------------------------------------------------------
7907// Linux and the other Unixes
7908// ---------------------------------------------------------------------------
7909
7910#[cfg(all(unix, not(target_os = "macos")))]
7911mod sys {
7912    //! One manager, two domains. `--start-at boot` is a system unit in
7913    //! `/etc/systemd/system`, wanted by `multi-user.target`; `--start-at login`
7914    //! is a user unit in `~/.config/systemd/user`, wanted by `default.target`.
7915    //!
7916    //! Every inspection here goes through `systemctl is-active` and
7917    //! `systemctl is-enabled`, whose output is a fixed machine-readable word
7918    //! rather than a sentence, so nothing in this module depends on the
7919    //! machine's display language.
7920
7921    use std::ffi::OsStr;
7922    use std::path::PathBuf;
7923    use std::time::Duration;
7924
7925    use runner_manager_domain::model::StartMode;
7926
7927    use super::{
7928        DefinitionKind, InstallPlan, Registration, SUDO_REMEDY, SYSTEMD_SYSTEM_DIR,
7929        SYSTEMD_USER_SUBDIR, ServiceControl, ServiceDefinition, ServiceError, ServiceIdentity,
7930        home_directory, ini_directives, run, write_definition,
7931    };
7932
7933    pub(super) fn control(mode: StartMode) -> Result<Box<dyn ServiceControl>, ServiceError> {
7934        Ok(Box::new(SystemdControl { mode }))
7935    }
7936
7937    #[derive(Debug)]
7938    struct SystemdControl {
7939        mode: StartMode,
7940    }
7941
7942    impl SystemdControl {
7943        fn unit_path(&self, identity: &ServiceIdentity) -> Option<PathBuf> {
7944            let file = identity.systemd_unit();
7945            match self.mode {
7946                StartMode::Boot => Some(PathBuf::from(SYSTEMD_SYSTEM_DIR).join(file)),
7947                StartMode::Login => {
7948                    home_directory().map(|home| home.join(SYSTEMD_USER_SUBDIR).join(file))
7949                }
7950            }
7951        }
7952
7953        /// `systemctl`, with `--user` for the login domain.
7954        fn systemctl(&self, arguments: &[&str]) -> (bool, String) {
7955            let mut all: Vec<&OsStr> = Vec::with_capacity(arguments.len() + 1);
7956            if self.mode == StartMode::Login {
7957                all.push(OsStr::new("--user"));
7958            }
7959            all.extend(arguments.iter().map(OsStr::new));
7960            match run("systemctl", &all) {
7961                Ok((ok, stdout, stderr)) => (
7962                    ok,
7963                    if stdout.trim().is_empty() {
7964                        stderr
7965                    } else {
7966                        stdout
7967                    },
7968                ),
7969                Err(error) => (false, error.to_string()),
7970            }
7971        }
7972
7973        fn failed(&self, operation: &'static str, name: &str, detail: String) -> ServiceError {
7974            ServiceError::Control {
7975                operation,
7976                name: name.to_string(),
7977                manager: "systemd",
7978                detail,
7979            }
7980        }
7981    }
7982
7983    impl ServiceControl for SystemdControl {
7984        fn manager(&self) -> DefinitionKind {
7985            DefinitionKind::SystemdUnit
7986        }
7987
7988        fn install(&self, plan: &InstallPlan) -> Result<ServiceDefinition, ServiceError> {
7989            let name = plan.identity().name().to_string();
7990            let definition = ServiceDefinition::systemd(plan, home_directory().as_deref());
7991            let Some(path) = definition.install_path().map(std::path::Path::to_path_buf) else {
7992                return Err(self.failed(
7993                    "install",
7994                    &name,
7995                    "this account has no home directory, so there is nowhere to put a systemd \
7996                     user unit. Use --start-at boot, which installs a system unit under \
7997                     /etc/systemd/system."
7998                        .to_string(),
7999                ));
8000            };
8001            write_definition("install", &name, &path, definition.text(), SUDO_REMEDY)?;
8002            let unit = plan.identity().systemd_unit();
8003            let (reloaded, message) = self.systemctl(&["daemon-reload"]);
8004            if !reloaded {
8005                let _ = std::fs::remove_file(&path);
8006                return Err(self.failed("install", &name, message));
8007            }
8008            let (enabled, message) = self.systemctl(&["enable", &unit]);
8009            if !enabled {
8010                let _ = std::fs::remove_file(&path);
8011                let _ = self.systemctl(&["daemon-reload"]);
8012                return Err(self.failed("install", &name, message));
8013            }
8014            Ok(definition)
8015        }
8016
8017        fn uninstall(&self, identity: &ServiceIdentity) -> Result<bool, ServiceError> {
8018            let Some(path) = self.unit_path(identity) else {
8019                return Ok(false);
8020            };
8021            if !path.exists() {
8022                return Ok(false);
8023            }
8024            let unit = identity.systemd_unit();
8025            // `--now` stops it as well as disabling it. A failure here is not
8026            // fatal: the unit file is about to go, and refusing to remove it
8027            // because `systemctl` disliked something would leave the host with
8028            // a unit nothing manages.
8029            let _ = self.systemctl(&["disable", "--now", &unit]);
8030            std::fs::remove_file(&path).map_err(|error| {
8031                super::definition_error(
8032                    "uninstall",
8033                    identity.name(),
8034                    error,
8035                    SUDO_REMEDY,
8036                    path.as_path(),
8037                )
8038            })?;
8039            let _ = self.systemctl(&["daemon-reload"]);
8040            Ok(true)
8041        }
8042
8043        fn query(&self, identity: &ServiceIdentity) -> Result<Option<Registration>, ServiceError> {
8044            let Some(path) = self.unit_path(identity) else {
8045                return Ok(None);
8046            };
8047            let Ok(document) = std::fs::read_to_string(&path) else {
8048                return Ok(None);
8049            };
8050            let unit = identity.systemd_unit();
8051            let directives = ini_directives(&document, "Service");
8052            let (_, active) = self.systemctl(&["is-active", &unit]);
8053            let (_, enabled) = self.systemctl(&["is-enabled", &unit]);
8054            Ok(Some(Registration {
8055                manager: DefinitionKind::SystemdUnit,
8056                start_mode: self.mode,
8057                command_line: directives.get("ExecStart").cloned().unwrap_or_default(),
8058                account: directives.get("User").cloned().or_else(|| {
8059                    Some(match self.mode {
8060                        StartMode::Boot => "root".to_string(),
8061                        StartMode::Login => "the invoking user".to_string(),
8062                    })
8063                }),
8064                running: active.trim() == "active",
8065                starts_automatically: enabled.trim() == "enabled",
8066                restart_delay: directives
8067                    .get("RestartSec")
8068                    .and_then(|value| value.trim().trim_end_matches('s').parse::<u64>().ok())
8069                    .map(Duration::from_secs),
8070            }))
8071        }
8072
8073        fn start(&self, identity: &ServiceIdentity) -> Result<(), ServiceError> {
8074            let unit = identity.systemd_unit();
8075            let (ok, message) = self.systemctl(&["start", &unit]);
8076            if ok {
8077                Ok(())
8078            } else {
8079                Err(self.failed("start", identity.name(), message))
8080            }
8081        }
8082
8083        fn stop(&self, identity: &ServiceIdentity) -> Result<bool, ServiceError> {
8084            let running = self
8085                .query(identity)?
8086                .is_some_and(|registration| registration.running);
8087            if !running {
8088                return Ok(false);
8089            }
8090            let unit = identity.systemd_unit();
8091            let (ok, message) = self.systemctl(&["stop", &unit]);
8092            if ok {
8093                Ok(true)
8094            } else {
8095                Err(self.failed("stop", identity.name(), message))
8096            }
8097        }
8098    }
8099}
8100
8101/// What the macOS tests that load real launch agents share.
8102#[cfg(all(test, target_os = "macos"))]
8103pub(crate) mod launchd_fixture {
8104    use std::path::{Path, PathBuf};
8105
8106    /// Opts a macOS host into the tests that load launch agents into the
8107    /// caller's GUI session. CI's macOS leg sets it, and
8108    /// `privileged_tests_are_wired_into_ci.rs` asserts that it still does.
8109    const ACCEPTANCE: &str = "RUNNER_MANAGER_LAUNCHD_ACCEPTANCE";
8110
8111    /// Whether this run may load launch agents. Says so when it may not, since
8112    /// the test then passes having measured nothing.
8113    pub(crate) fn enabled() -> bool {
8114        let enabled = std::env::var_os(ACCEPTANCE).is_some_and(|value| value == "1");
8115        if !enabled {
8116            eprintln!("skipped: set {ACCEPTANCE}=1 to load test launch agents");
8117        }
8118        enabled
8119    }
8120
8121    /// A copy of the running test binary in `directory`, for a launchd job to
8122    /// run as its program.
8123    ///
8124    /// A copy, because a launchd job whose program lives on an external volume
8125    /// waits on a privacy approval nobody gives before `main` runs (measured
8126    /// with a target directory under `/Volumes`), and the temporary directory
8127    /// is on the boot volume.
8128    pub(crate) fn job_binary(directory: &Path) -> PathBuf {
8129        let binary = directory.join("launch-agent-job");
8130        std::fs::copy(std::env::current_exe().expect("test binary"), &binary)
8131            .expect("test binary copied");
8132        binary
8133    }
8134
8135    /// A launch agent loaded into the caller's GUI domain, booted out on drop.
8136    pub(crate) struct LoadedAgent(String);
8137
8138    impl LoadedAgent {
8139        /// Bootstraps `plist`, whose `Label` is `label`, and fails the test when
8140        /// launchd refuses it.
8141        pub(crate) fn bootstrap(label: &str, plist: &Path) -> Self {
8142            // SAFETY: `getuid` reads the calling process's real user id and
8143            // cannot fail.
8144            let domain = format!("gui/{}", unsafe { libc::getuid() });
8145            // Armed before the bootstrap, so a half-loaded job is removed too.
8146            let loaded = Self(format!("{domain}/{label}"));
8147            let bootstrap = std::process::Command::new("launchctl")
8148                .arg("bootstrap")
8149                .arg(&domain)
8150                .arg(plist)
8151                .output()
8152                .expect("launchctl runs");
8153            assert!(
8154                bootstrap.status.success(),
8155                "launchctl bootstrap {domain} refused {label}, so nothing was measured: {}",
8156                String::from_utf8_lossy(&bootstrap.stderr)
8157            );
8158            loaded
8159        }
8160
8161        /// `gui/<uid>/<label>`.
8162        pub(crate) fn target(&self) -> &str {
8163            &self.0
8164        }
8165    }
8166
8167    impl Drop for LoadedAgent {
8168        fn drop(&mut self) {
8169            let _ = std::process::Command::new("launchctl")
8170                .args(["bootout", &self.0])
8171                .output();
8172        }
8173    }
8174}
8175
8176#[cfg(test)]
8177mod tests {
8178    use super::*;
8179
8180    use std::collections::BTreeMap;
8181
8182    #[test]
8183    fn task_scheduler_omitting_default_enabled_still_means_automatic() {
8184        assert!(windows_login_task_starts_automatically(
8185            "<Task><Triggers><LogonTrigger></LogonTrigger></Triggers></Task>"
8186        ));
8187        assert!(windows_login_task_starts_automatically(
8188            "<Task><LogonTrigger><Enabled>true</Enabled></LogonTrigger></Task>"
8189        ));
8190        assert!(!windows_login_task_starts_automatically(
8191            "<Task><LogonTrigger><Enabled>false</Enabled></LogonTrigger></Task>"
8192        ));
8193        assert!(!windows_login_task_starts_automatically(
8194            "<Task><BootTrigger></BootTrigger></Task>"
8195        ));
8196    }
8197
8198    // -----------------------------------------------------------------------
8199    // Fixtures
8200    // -----------------------------------------------------------------------
8201
8202    /// A plan against paths that exist on no platform, so that every renderer
8203    /// can be asserted on every leg of the CI matrix.
8204    fn linux_plan(mode: StartMode) -> InstallPlan {
8205        InstallPlan::unchecked(
8206            ServiceIdentity::product(),
8207            mode,
8208            "/opt/runner-manager/bin/runner-manager",
8209            ServiceDirectories {
8210                config: PathBuf::from("/var/lib/runner-manager/config"),
8211                state: PathBuf::from("/var/lib/runner-manager/state"),
8212                runtime: PathBuf::from("/var/lib/runner-manager/runtime"),
8213                logs: PathBuf::from("/var/log/runner-manager"),
8214            },
8215        )
8216        .with_secret_guard("/var/lib/runner-manager/secrets/user-access-token")
8217    }
8218
8219    fn windows_plan(mode: StartMode) -> InstallPlan {
8220        InstallPlan::unchecked(
8221            ServiceIdentity::product(),
8222            mode,
8223            "C:\\Program Files\\runner-manager\\runner-manager.exe",
8224            ServiceDirectories {
8225                config: PathBuf::from("C:\\Users\\op\\AppData\\Local\\rm\\config"),
8226                state: PathBuf::from("C:\\Users\\op\\AppData\\Local\\rm\\state"),
8227                runtime: PathBuf::from("C:\\Users\\op\\AppData\\Local\\rm\\runtime"),
8228                logs: PathBuf::from("C:\\Users\\op\\AppData\\Local\\rm\\logs"),
8229            },
8230        )
8231    }
8232
8233    /// Replaces one fragment of a rendered definition, and **fails the test if
8234    /// the fragment was not there**.
8235    ///
8236    /// Without the assertion a renderer change would quietly turn every
8237    /// "widened definition is rejected" test below into a test that reviews the
8238    /// unmodified definition and passes for the wrong reason. This is the
8239    /// single guard that keeps that class of vacuity out of this file.
8240    fn edited(text: &str, from: &str, to: &str) -> String {
8241        assert!(
8242            text.contains(from),
8243            "the rendered definition does not contain `{from}`, so this test would assert \
8244             nothing about a widened one"
8245        );
8246        text.replace(from, to)
8247    }
8248
8249    /// Every file under `root`, keyed by its path relative to `root`.
8250    fn snapshot(roots: &[&Path]) -> BTreeMap<PathBuf, Vec<u8>> {
8251        fn walk(directory: &Path, out: &mut BTreeMap<PathBuf, Vec<u8>>) {
8252            let Ok(entries) = std::fs::read_dir(directory) else {
8253                return;
8254            };
8255            for entry in entries.flatten() {
8256                let path = entry.path();
8257                if path.is_dir() {
8258                    walk(&path, out);
8259                } else if let Ok(bytes) = std::fs::read(&path) {
8260                    out.insert(path, bytes);
8261                }
8262            }
8263        }
8264        let mut out = BTreeMap::new();
8265        for root in roots {
8266            walk(root, &mut out);
8267        }
8268        out
8269    }
8270
8271    struct Host {
8272        _root: tempfile::TempDir,
8273        paths: AppPaths,
8274        binary: PathBuf,
8275        /// A runner root inside this host's own temporary tree.
8276        ///
8277        /// Without it every `install` in this module would create and
8278        /// re-permission the **real** `%SystemDrive%\rman` on the machine
8279        /// running the tests, and would fail outright on any machine where that
8280        /// directory already exists with the access control `C:\` gives it.
8281        /// A sibling of the four application-data directories rather than a
8282        /// child of one, so `b1`'s overlap check has nothing to object to.
8283        runner_root: LocalAbsolutePath,
8284        controls: RecordingControls,
8285    }
8286
8287    impl Host {
8288        fn new() -> Self {
8289            let root = tempfile::tempdir().expect("a temporary directory");
8290            let paths = AppPaths::rooted_at(root.path());
8291            paths.create_all().expect("the four directories");
8292            let binary = root.path().join(if cfg!(windows) {
8293                "runner-manager.exe"
8294            } else {
8295                "runner-manager"
8296            });
8297            std::fs::write(&binary, b"not a real binary").expect("a stand-in binary");
8298            let runner_root = LocalAbsolutePath::new(
8299                root.path()
8300                    .join("runner-root")
8301                    .to_str()
8302                    .expect("a unicode temporary path"),
8303            )
8304            .expect("a local absolute path");
8305            Self {
8306                _root: root,
8307                paths,
8308                binary,
8309                runner_root,
8310                controls: RecordingControls::new(),
8311            }
8312        }
8313
8314        fn operations(&self) -> ServiceOperations {
8315            ServiceOperations::with_controls(
8316                self.paths.clone(),
8317                ServiceIdentity::product(),
8318                std::sync::Arc::new(self.controls.clone()),
8319            )
8320            .with_runner_root(self.runner_root.clone())
8321        }
8322
8323        fn request(&self, mode: StartMode) -> InstallRequest {
8324            InstallRequest::new(mode).for_binary(&self.binary)
8325        }
8326    }
8327
8328    #[cfg(windows)]
8329    #[test]
8330    fn windows_uninstall_waits_through_the_marked_for_deletion_window() {
8331        let probes = std::cell::Cell::new(0);
8332        let absent =
8333            super::sys::wait_until_scm_absent(Duration::from_secs(1), Duration::ZERO, || {
8334                let next = probes.get() + 1;
8335                probes.set(next);
8336                Ok(next == 3)
8337            })
8338            .expect("the simulated SCM probe succeeds");
8339
8340        assert!(absent);
8341        assert_eq!(
8342            probes.get(),
8343            3,
8344            "uninstall must recheck after transient presence instead of treating it as a leak"
8345        );
8346    }
8347
8348    #[cfg(windows)]
8349    #[test]
8350    fn windows_recognises_an_already_pending_service_deletion() {
8351        let error = windows_service::Error::Winapi(std::io::Error::from_raw_os_error(1072));
8352
8353        assert!(super::sys::is_marked_for_delete(&error));
8354    }
8355
8356    #[test]
8357    fn launchd_enable_failure_is_returned_and_removes_the_bootstrapped_registration() {
8358        let root = tempfile::tempdir().expect("a temporary directory");
8359        let plist = root.path().join("fixture.plist");
8360        std::fs::write(&plist, b"fixture").expect("a plist fixture");
8361        let calls = std::cell::RefCell::new(Vec::new());
8362
8363        let error = enable_launchd_registration(
8364            |arguments| {
8365                let call = arguments
8366                    .iter()
8367                    .map(|argument| argument.to_string_lossy().into_owned())
8368                    .collect::<Vec<_>>();
8369                let operation = call[0].clone();
8370                calls.borrow_mut().push(call);
8371                if operation == "enable" {
8372                    (false, "label remains disabled".to_string())
8373                } else {
8374                    (true, String::new())
8375                }
8376            },
8377            "system",
8378            "system/com.openai.runner-manager-selftest",
8379            &plist,
8380            "runner-manager-selftest",
8381            "rerun with administrative rights",
8382        )
8383        .expect_err("enable failure must fail the install");
8384
8385        assert!(
8386            matches!(
8387                error,
8388                ServiceError::Control {
8389                    operation: "enable",
8390                    ..
8391                }
8392            ),
8393            "{error}"
8394        );
8395        assert_eq!(calls.borrow().len(), 2);
8396        assert_eq!(calls.borrow()[0][0], "enable");
8397        assert_eq!(calls.borrow()[1][0], "bootout");
8398        assert!(!plist.exists(), "rollback must remove the plist");
8399    }
8400
8401    // -----------------------------------------------------------------------
8402    // Quoting, and reading a command line back
8403    // -----------------------------------------------------------------------
8404
8405    #[test]
8406    fn a_path_with_spaces_survives_a_round_trip_through_a_command_line() {
8407        let plan = windows_plan(StartMode::Boot);
8408        let command_line = plan.command_line();
8409        assert!(
8410            command_line.starts_with('"'),
8411            "a path with a space must be quoted, got {command_line}"
8412        );
8413        assert_eq!(
8414            executable_from_command_line(&command_line).as_deref(),
8415            Some(plan.binary())
8416        );
8417    }
8418
8419    #[test]
8420    fn a_path_without_spaces_is_not_quoted_and_still_reads_back() {
8421        let plan = linux_plan(StartMode::Boot);
8422        let command_line = plan.command_line();
8423        assert!(!command_line.starts_with('"'), "got {command_line}");
8424        assert_eq!(
8425            executable_from_command_line(&command_line).as_deref(),
8426            Some(plan.binary())
8427        );
8428    }
8429
8430    #[test]
8431    fn a_quoted_path_containing_a_quote_reads_back_verbatim() {
8432        // Not a path anybody has, but it is the input that separates a real
8433        // implementation of Windows' quoting rules from a `split_whitespace`.
8434        let awkward = r#"C:\odd "name"\rm.exe"#;
8435        let quoted = quote_argument(awkward);
8436        assert_eq!(
8437            executable_from_command_line(&format!("{quoted} daemon run"))
8438                .as_deref()
8439                .map(Path::to_string_lossy)
8440                .as_deref(),
8441            Some(awkward)
8442        );
8443    }
8444
8445    #[test]
8446    fn an_empty_command_line_has_no_executable() {
8447        assert_eq!(executable_from_command_line("   "), None);
8448        assert_eq!(executable_from_command_line(""), None);
8449    }
8450
8451    #[test]
8452    fn xml_escaping_round_trips_the_characters_a_path_or_an_account_may_hold() {
8453        let awkward = r#"DOMAIN\R&D <team> "ops""#;
8454        assert_eq!(
8455            xml_escape(awkward),
8456            "DOMAIN\\R&amp;D &lt;team&gt; &quot;ops&quot;"
8457        );
8458        assert_eq!(xml_unescape(&xml_escape(awkward)), awkward);
8459    }
8460
8461    // -----------------------------------------------------------------------
8462    // The restart policy is bounded at both ends
8463    // -----------------------------------------------------------------------
8464
8465    #[test]
8466    fn a_restart_delay_under_the_floor_is_refused() {
8467        let error = RestartPolicy::new(Duration::from_millis(500), Duration::from_secs(60))
8468            .expect_err("half a second is under the one-second floor");
8469        assert!(
8470            matches!(error, ServiceError::RestartDelay { .. }),
8471            "{error}"
8472        );
8473    }
8474
8475    #[test]
8476    fn a_restart_delay_over_the_ceiling_is_refused() {
8477        let error = RestartPolicy::new(Duration::from_secs(3600), Duration::from_secs(7200))
8478            .expect_err("an hour is over the five-minute ceiling");
8479        assert!(
8480            matches!(error, ServiceError::RestartDelay { .. }),
8481            "{error}"
8482        );
8483    }
8484
8485    #[test]
8486    fn a_reset_window_no_longer_than_the_delay_is_refused() {
8487        let error = RestartPolicy::new(Duration::from_secs(15), Duration::from_secs(15))
8488            .expect_err("a window equal to the delay can never elapse between restarts");
8489        assert!(
8490            matches!(error, ServiceError::RestartResetWindow { .. }),
8491            "{error}"
8492        );
8493    }
8494
8495    #[test]
8496    fn a_delay_inside_the_bound_is_accepted() {
8497        let policy = RestartPolicy::new(Duration::from_secs(20), Duration::from_secs(300))
8498            .expect("twenty seconds is inside the bound");
8499        assert_eq!(policy.delay(), Duration::from_secs(20));
8500        assert_eq!(policy.reset_after(), Duration::from_secs(300));
8501    }
8502
8503    // -----------------------------------------------------------------------
8504    // Identity
8505    // -----------------------------------------------------------------------
8506
8507    #[test]
8508    fn a_fixture_identity_can_never_be_the_product_identity() {
8509        let fixture = ServiceIdentity::fixture("abc123");
8510        assert!(fixture.is_fixture());
8511        assert!(!ServiceIdentity::product().is_fixture());
8512        assert_ne!(fixture.name(), ServiceIdentity::product().name());
8513        assert_ne!(
8514            fixture.launchd_label(),
8515            ServiceIdentity::product().launchd_label()
8516        );
8517        assert_ne!(
8518            fixture.systemd_unit(),
8519            ServiceIdentity::product().systemd_unit()
8520        );
8521    }
8522
8523    #[test]
8524    fn a_fixture_tag_is_reduced_to_characters_every_manager_accepts() {
8525        let fixture = ServiceIdentity::fixture("A b/c:\\d");
8526        assert!(
8527            fixture
8528                .name()
8529                .chars()
8530                .all(|c| c.is_ascii_lowercase() || c.is_ascii_digit() || c == '-'),
8531            "got {}",
8532            fixture.name()
8533        );
8534    }
8535
8536    // -----------------------------------------------------------------------
8537    // The systemd unit
8538    // -----------------------------------------------------------------------
8539
8540    #[test]
8541    fn the_boot_unit_restarts_on_failure_after_the_bounded_delay() {
8542        let unit = systemd_unit(&linux_plan(StartMode::Boot));
8543        assert!(unit.contains("KillMode=process\n"), "{unit}");
8544        assert!(unit.contains("Restart=on-failure\n"), "{unit}");
8545        assert!(unit.contains("RestartSec=15\n"), "{unit}");
8546        assert!(unit.contains("StartLimitIntervalSec=600\n"), "{unit}");
8547        assert!(unit.contains("StartLimitBurst=5\n"), "{unit}");
8548        assert!(unit.contains("WantedBy=multi-user.target\n"), "{unit}");
8549    }
8550
8551    #[test]
8552    fn the_boot_unit_reads_the_live_store_instead_of_a_frozen_systemd_copy() {
8553        let unit = systemd_unit(&linux_plan(StartMode::Boot));
8554        assert!(
8555            !unit.contains("LoadCredential="),
8556            "a startup snapshot would shadow every rotated credential until restart:\n{unit}"
8557        );
8558    }
8559
8560    #[test]
8561    fn a_login_unit_carries_no_machine_credential_and_wants_the_session_target() {
8562        let unit = systemd_unit(&linux_plan(StartMode::Login));
8563        assert!(
8564            !unit.contains("LoadCredential="),
8565            "a user unit must not name a root-owned credential file, got:\n{unit}"
8566        );
8567        assert!(unit.contains("WantedBy=default.target\n"), "{unit}");
8568    }
8569
8570    #[test]
8571    fn the_unit_can_atomically_replace_the_credential_and_write_only_required_directories() {
8572        let plan = linux_plan(StartMode::Boot);
8573        let unit = systemd_unit(&plan);
8574        let directives = ini_directives(&unit, "Service");
8575        let listed = split_quoted(
8576            directives
8577                .get("ReadWritePaths")
8578                .expect("the unit names its writable paths"),
8579        );
8580        assert_eq!(listed.len(), 5, "{listed:?}");
8581        for path in plan.directories().all() {
8582            assert!(
8583                listed.iter().any(|entry| entry == &path.to_string_lossy()),
8584                "{} is missing from {listed:?}",
8585                path.display()
8586            );
8587        }
8588        assert!(
8589            listed
8590                .iter()
8591                .any(|entry| entry == "/var/lib/runner-manager/secrets"),
8592            "atomic credential replacement needs its parent directory in {listed:?}"
8593        );
8594    }
8595
8596    #[test]
8597    fn the_unit_records_the_absolute_binary_path() {
8598        let plan = linux_plan(StartMode::Boot);
8599        let unit = systemd_unit(&plan);
8600        assert!(
8601            unit.contains("ExecStart=/opt/runner-manager/bin/runner-manager daemon run\n"),
8602            "{unit}"
8603        );
8604    }
8605
8606    // -----------------------------------------------------------------------
8607    // The launchd property list
8608    // -----------------------------------------------------------------------
8609
8610    #[test]
8611    fn the_daemon_restarts_only_after_an_unsuccessful_exit() {
8612        let plist = launchd_plist(&linux_plan(StartMode::Boot));
8613        // A bare `KeepAlive` would also restart a job the operator stopped
8614        // deliberately, which turns `service stop` into a fight with launchd.
8615        assert!(
8616            plist.contains(
8617                "<key>KeepAlive</key>\n  <dict>\n    <key>SuccessfulExit</key>\n    <false/>\n"
8618            ),
8619            "{plist}"
8620        );
8621        assert!(
8622            plist.contains("<key>ThrottleInterval</key>\n  <integer>15</integer>"),
8623            "{plist}"
8624        );
8625    }
8626
8627    #[test]
8628    fn a_launch_daemon_names_root_and_a_launch_agent_names_nobody() {
8629        let daemon = launchd_plist(&linux_plan(StartMode::Boot));
8630        assert_eq!(
8631            plist_string_value(&daemon, "UserName").as_deref(),
8632            Some("root")
8633        );
8634        assert_eq!(plist_bool_value(&daemon, "SessionCreate"), Some(false));
8635
8636        let agent = launchd_plist(&linux_plan(StartMode::Login));
8637        assert_eq!(
8638            plist_string_value(&agent, "UserName"),
8639            None,
8640            "a LaunchAgent already runs as the operator:\n{agent}"
8641        );
8642    }
8643
8644    #[test]
8645    fn the_plist_runs_the_daemon_and_its_runners_as_an_interactive_job() {
8646        for mode in [StartMode::Boot, StartMode::Login] {
8647            let plan = linux_plan(mode);
8648            let plist = launchd_plist(&plan);
8649            assert_eq!(
8650                plist_string_value(&plist, "ProcessType").as_deref(),
8651                Some("Interactive"),
8652                "launchd applies ProcessType to every runner the daemon starts:\n{plist}"
8653            );
8654            let review = review_least_privilege(
8655                &ServiceDefinition::from_text(DefinitionKind::LaunchdPlist, plist),
8656                &plan,
8657            );
8658            assert!(
8659                review
8660                    .controls()
8661                    .iter()
8662                    .any(|control| control == "ProcessType=Interactive"),
8663                "{review}"
8664            );
8665            assert!(
8666                !review
8667                    .findings()
8668                    .iter()
8669                    .any(|finding| finding.subject == "ProcessType"),
8670                "{review}"
8671            );
8672        }
8673    }
8674
8675    /// The value 0.4.31 and earlier installed, and launchd's default, both
8676    /// throttle every runner. That is a shortfall the update repairs, not an
8677    /// excess: a scheduling class grants no authority.
8678    #[test]
8679    fn a_plist_that_throttles_its_runners_is_a_shortfall() {
8680        let plan = linux_plan(StartMode::Login);
8681        let rendered = launchd_plist(&plan);
8682        let interactive = "<key>ProcessType</key>\n  <string>Interactive</string>\n";
8683        for (label, throttled) in [
8684            (
8685                "Background",
8686                edited(
8687                    &rendered,
8688                    interactive,
8689                    "<key>ProcessType</key>\n  <string>Background</string>\n",
8690                ),
8691            ),
8692            ("absent", edited(&rendered, interactive, "")),
8693        ] {
8694            let review = review_least_privilege(
8695                &ServiceDefinition::from_text(DefinitionKind::LaunchdPlist, throttled),
8696                &plan,
8697            );
8698            assert!(review.is_least_privilege(), "{label}: {review}");
8699            assert!(
8700                review.findings().iter().any(|finding| {
8701                    finding.kind == FindingKind::Shortfall
8702                        && finding.subject == "ProcessType"
8703                        && finding.detail.contains(label)
8704                }),
8705                "{label}: {review}"
8706            );
8707        }
8708    }
8709
8710    #[test]
8711    fn a_plist_with_an_old_process_type_is_drift_and_an_operator_edit_is_not() {
8712        let host = Host::new();
8713        let identity = ServiceIdentity::product();
8714        let plan = InstallPlan::resolve(
8715            identity.clone(),
8716            &host.request(StartMode::Login),
8717            ServiceDirectories::of(&host.paths),
8718        )
8719        .expect("a resolvable plan");
8720        let installed = launchd_plist(&plan);
8721        let definition =
8722            ServiceDefinition::from_text(DefinitionKind::LaunchdPlist, installed.clone());
8723        let mut record = InstallRecord::of(&plan, &definition, Utc::now());
8724        record.installed_by_version = "0.4.31".into();
8725        let path = PathBuf::from("/Users/op/Library/LaunchAgents/runner-manager.plist");
8726
8727        assert_eq!(
8728            launchd_definition_drift(&identity, &record, path.clone(), &installed),
8729            None
8730        );
8731        let edited_by_operator = edited(
8732            &installed,
8733            "</dict>\n</plist>\n",
8734            "  <key>Nice</key>\n  <integer>0</integer>\n</dict>\n</plist>\n",
8735        );
8736        assert_eq!(
8737            launchd_definition_drift(&identity, &record, path.clone(), &edited_by_operator),
8738            None,
8739            "the daemon repairs this drift itself, so it must not key on edits it did not cause"
8740        );
8741
8742        let written_by_0_4_31 = edited(
8743            &installed,
8744            "<string>Interactive</string>",
8745            "<string>Background</string>",
8746        );
8747        let drift = launchd_definition_drift(&identity, &record, path, &written_by_0_4_31)
8748            .expect("a Background plist is drift");
8749        assert_eq!(drift.manager, DefinitionKind::LaunchdPlist);
8750        assert_eq!(drift.start_mode, StartMode::Login);
8751        assert_eq!(
8752            drift.rendered, installed,
8753            "the repair must write exactly what `service install` writes"
8754        );
8755
8756        let status = ServiceStatus::compose(identity, None, None, None, None, &host.paths)
8757            .with_definition_drift(Some(drift));
8758        let problem = status
8759            .problems()
8760            .iter()
8761            .find(|problem| problem.subject == "definition")
8762            .expect("the drift is reported");
8763        // The remedy is a restart: a running daemon holds the lock that refuses
8764        // `service install`.
8765        assert!(
8766            problem.detail.contains("0.4.31")
8767                && problem.detail.contains(
8768                    "launchctl kickstart -k gui/$(id -u)/io.github.IvanMurzak.runner-manager`"
8769                )
8770                && !problem.detail.contains("service install")
8771                && !problem.detail.contains("systemctl")
8772                && !problem.detail.contains("sudo"),
8773            "{}",
8774            problem.detail
8775        );
8776    }
8777
8778    #[cfg(unix)]
8779    #[test]
8780    fn a_replaced_definition_is_the_rendering_and_readable_by_launchd() {
8781        use std::os::unix::fs::PermissionsExt as _;
8782
8783        let root = tempfile::tempdir().expect("a temporary directory");
8784        let path = root
8785            .path()
8786            .join("io.github.IvanMurzak.runner-manager.plist");
8787        std::fs::write(&path, "<string>Background</string>").expect("old plist");
8788        std::fs::set_permissions(&path, std::fs::Permissions::from_mode(0o600)).unwrap();
8789        let drift = DefinitionDrift {
8790            path: path.clone(),
8791            installed_by_version: "0.4.31".into(),
8792            rendered: launchd_plist(&linux_plan(StartMode::Login)),
8793            start_mode: StartMode::Login,
8794            manager: DefinitionKind::LaunchdPlist,
8795        };
8796
8797        replace_definition("runner-manager", &drift).expect("replaced");
8798
8799        assert_eq!(std::fs::read_to_string(&path).unwrap(), drift.rendered);
8800        assert_eq!(
8801            std::fs::metadata(&path).unwrap().permissions().mode() & 0o777,
8802            0o644
8803        );
8804        assert_eq!(
8805            std::fs::read_dir(root.path()).unwrap().count(),
8806            1,
8807            "no staging file is left beside the plist"
8808        );
8809    }
8810
8811    /// The job of [`a_job_reloads_itself_from_its_rewritten_plist`], run by
8812    /// launchd and by nothing else.
8813    ///
8814    /// It does what the daemon does when it starts: when its own plist (kept
8815    /// in its working directory) does not carry [`LAUNCHD_PROCESS_TYPE`], it
8816    /// rewrites the plist and asks for the reload from inside the job that the
8817    /// reload boots out. Either way it then stays up, to be booted out by the
8818    /// helper or, once reloaded, by the test.
8819    #[cfg(target_os = "macos")]
8820    #[test]
8821    #[ignore = "run only by launchd, as the job of the detached-reload test"]
8822    fn detached_reload_job() {
8823        let label = std::env::var("XPC_SERVICE_NAME").unwrap_or_default();
8824        let tag = label
8825            .split_once("-selftest-")
8826            .map(|(_, tag)| tag.to_string())
8827            .unwrap_or_else(|| panic!("run only by launchd for the reload test, not {label:?}"));
8828        let identity = ServiceIdentity::fixture(&tag);
8829        assert_eq!(identity.launchd_label(), label);
8830        let path = std::env::current_dir()
8831            .expect("working directory")
8832            .join(format!("{label}.plist"));
8833        let installed = std::fs::read_to_string(&path).expect("own plist");
8834        if plist_string_value(&installed, "ProcessType").as_deref() != Some(LAUNCHD_PROCESS_TYPE) {
8835            let drift = DefinitionDrift {
8836                path: path.clone(),
8837                installed_by_version: "0.4.31".into(),
8838                rendered: installed.replace(
8839                    "<string>Background</string>",
8840                    &format!("<string>{LAUNCHD_PROCESS_TYPE}</string>"),
8841                ),
8842                start_mode: StartMode::Login,
8843                manager: DefinitionKind::LaunchdPlist,
8844            };
8845            replace_definition(identity.name(), &drift).expect("plist replaced");
8846            super::sys::reload_detached(&identity, StartMode::Login, &path)
8847                .expect("helper started");
8848        }
8849        std::thread::sleep(Duration::from_secs(120));
8850    }
8851
8852    /// A job that rewrites its own plist and asks for the reload comes back
8853    /// under the rewritten plist.
8854    ///
8855    /// The request comes from inside the job, as the daemon's does, so the
8856    /// helper has to outlive the bootout of the very job that started it.
8857    /// launchd sends SIGTERM to the job's whole process group, so a helper that
8858    /// stayed in it would sometimes die before its bootstrap and leave the job
8859    /// unloaded. Measured with `setsid` removed, that failed one run in four
8860    /// here: this test catches it, though not on every run.
8861    #[cfg(target_os = "macos")]
8862    #[test]
8863    fn a_job_reloads_itself_from_its_rewritten_plist() {
8864        if !super::launchd_fixture::enabled() {
8865            return;
8866        }
8867        let root = tempfile::tempdir().expect("a temporary directory");
8868        let identity = ServiceIdentity::fixture(&format!("reload-{}", std::process::id()));
8869        let label = identity.launchd_label();
8870        let path = root.path().join(format!("{label}.plist"));
8871        let binary = super::launchd_fixture::job_binary(root.path());
8872        let mut plist = String::from("<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n");
8873        plist.push_str("<plist version=\"1.0\">\n<dict>\n");
8874        plist.push_str(&plist_string("Label", &label));
8875        plist.push_str("  <key>ProgramArguments</key>\n  <array>\n");
8876        for argument in [
8877            binary.to_string_lossy().as_ref(),
8878            "--ignored",
8879            "--exact",
8880            "service::tests::detached_reload_job",
8881            "--test-threads=1",
8882        ] {
8883            plist.push_str(&format!("    <string>{}</string>\n", xml_escape(argument)));
8884        }
8885        plist.push_str("  </array>\n  <key>RunAtLoad</key>\n  <true/>\n");
8886        plist.push_str(&plist_string(
8887            "WorkingDirectory",
8888            &root.path().to_string_lossy(),
8889        ));
8890        plist.push_str(&plist_string("ProcessType", "Background"));
8891        plist.push_str("</dict>\n</plist>\n");
8892        std::fs::write(&path, plist).expect("plist written");
8893
8894        let loaded = super::launchd_fixture::LoadedAgent::bootstrap(&label, &path);
8895        let spawn_type = || {
8896            let printed = std::process::Command::new("launchctl")
8897                .args(["print", loaded.target()])
8898                .output()
8899                .expect("launchctl runs");
8900            String::from_utf8_lossy(&printed.stdout)
8901                .lines()
8902                .find_map(|line| {
8903                    line.trim()
8904                        .strip_prefix("spawn type = ")
8905                        .map(str::to_string)
8906                })
8907        };
8908        assert_eq!(spawn_type().as_deref(), Some("background (5)"));
8909
8910        let deadline = std::time::Instant::now() + Duration::from_secs(90);
8911        while spawn_type().as_deref() != Some("interactive (4)") {
8912            assert!(
8913                std::time::Instant::now() < deadline,
8914                "the job did not come back under its rewritten plist; launchd reports {:?}",
8915                spawn_type()
8916            );
8917            std::thread::sleep(Duration::from_millis(250));
8918        }
8919        let rewritten = std::fs::read_to_string(&path).expect("plist readable");
8920        assert_eq!(
8921            plist_string_value(&rewritten, "ProcessType").as_deref(),
8922            Some(LAUNCHD_PROCESS_TYPE)
8923        );
8924    }
8925
8926    #[test]
8927    fn the_plist_records_the_absolute_binary_path_and_the_daemon_arguments() {
8928        let plist = launchd_plist(&linux_plan(StartMode::Boot));
8929        assert!(
8930            plist.contains("<string>/opt/runner-manager/bin/runner-manager</string>"),
8931            "{plist}"
8932        );
8933        assert!(plist.contains("<string>daemon</string>"), "{plist}");
8934        assert!(plist.contains("<string>run</string>"), "{plist}");
8935    }
8936
8937    #[test]
8938    fn the_launchd_label_is_the_product_identity_in_reverse_domain_form() {
8939        assert_eq!(
8940            ServiceIdentity::product().launchd_label(),
8941            "io.github.IvanMurzak.runner-manager"
8942        );
8943    }
8944
8945    // -----------------------------------------------------------------------
8946    // The Task Scheduler document
8947    // -----------------------------------------------------------------------
8948
8949    #[test]
8950    fn the_task_runs_at_least_privilege_on_a_logon_trigger() {
8951        let xml = windows_scheduled_task_xml(
8952            &windows_plan(StartMode::Login),
8953            &TaskPrincipal::named("HOST\\operator"),
8954        );
8955        assert!(xml.contains("<LogonTrigger>"), "{xml}");
8956        assert!(xml.contains("<RunLevel>LeastPrivilege</RunLevel>"), "{xml}");
8957        assert!(
8958            xml.contains("<LogonType>InteractiveToken</LogonType>"),
8959            "{xml}"
8960        );
8961        assert!(xml.contains("<UserId>HOST\\operator</UserId>"), "{xml}");
8962        assert!(
8963            xml.contains("<Interval>PT1M</Interval>"),
8964            "Task Scheduler takes whole minutes only, and rejects the registration outright \
8965             for anything finer:\n{xml}"
8966        );
8967    }
8968
8969    #[test]
8970    fn task_schedulers_minute_granularity_only_ever_rounds_the_delay_up() {
8971        // Measured against the real Task Scheduler, which answers `PT15S` with
8972        // "The task XML contains a value which is incorrectly formatted or out
8973        // of range". Rounding *up* is what keeps "does not restart-loop faster
8974        // than that bound" true on this manager.
8975        for (asked, enforced) in [(1u64, 60u64), (15, 60), (60, 60), (61, 120), (300, 300)] {
8976            let policy =
8977                RestartPolicy::new(Duration::from_secs(asked), Duration::from_secs(asked + 600))
8978                    .expect("inside the supported range");
8979            assert_eq!(
8980                policy
8981                    .effective_delay(DefinitionKind::WindowsScheduledTask)
8982                    .as_secs(),
8983                enforced,
8984                "a {asked}s delay must be enforced as {enforced}s"
8985            );
8986            assert!(
8987                policy.effective_delay(DefinitionKind::WindowsScheduledTask) >= policy.delay(),
8988                "rounding must never shorten the bound"
8989            );
8990        }
8991    }
8992
8993    #[test]
8994    fn every_other_manager_enforces_the_delay_exactly_as_configured() {
8995        let policy = RestartPolicy::default();
8996        for kind in [
8997            DefinitionKind::WindowsService,
8998            DefinitionKind::LaunchdPlist,
8999            DefinitionKind::SystemdUnit,
9000        ] {
9001            assert_eq!(
9002                policy.effective_delay(kind),
9003                policy.delay(),
9004                "{kind:?} takes seconds and enforces exactly what it is given"
9005            );
9006        }
9007    }
9008
9009    #[test]
9010    fn a_task_whose_manager_reports_the_rounded_delay_is_not_a_fault() {
9011        let host = Host::new();
9012        let operations = host.operations();
9013        operations
9014            .install(&host.request(StartMode::Login))
9015            .expect("an install at login");
9016        // What a real Task Scheduler reports back for a 15-second policy.
9017        host.controls.edit("runner-manager", |registration| {
9018            registration.manager = DefinitionKind::WindowsScheduledTask;
9019            registration.restart_delay = Some(Duration::from_secs(60));
9020        });
9021
9022        let status = operations.status().expect("a status");
9023        assert!(
9024            status.is_healthy(),
9025            "minute granularity is the manager's, not a mis-registration: {status}"
9026        );
9027        assert!(
9028            status
9029                .notes()
9030                .iter()
9031                .any(|note| note.contains("whole minutes")),
9032            "but the operator must be told why 15 became 60: {status}"
9033        );
9034
9035        // The discriminator: a delay that is neither the configured one nor its
9036        // rounding is still a fault.
9037        host.controls.edit("runner-manager", |registration| {
9038            registration.restart_delay = Some(Duration::from_secs(1));
9039        });
9040        assert!(
9041            !operations.status().expect("a status").is_healthy(),
9042            "a one-second delay is not what any manager was asked for"
9043        );
9044    }
9045
9046    #[test]
9047    fn the_task_records_the_absolute_binary_path_and_the_daemon_arguments() {
9048        let plan = windows_plan(StartMode::Login);
9049        let xml = windows_scheduled_task_xml(&plan, &TaskPrincipal::named("HOST\\operator"));
9050        assert_eq!(
9051            xml_value(&xml, "Command").as_deref(),
9052            Some("C:\\Program Files\\runner-manager\\runner-manager.exe"),
9053            "{xml}"
9054        );
9055        assert_eq!(xml_value(&xml, "Arguments").as_deref(), Some("daemon run"));
9056    }
9057
9058    #[test]
9059    fn an_account_name_holding_xml_punctuation_is_escaped() {
9060        let xml = windows_scheduled_task_xml(
9061            &windows_plan(StartMode::Login),
9062            &TaskPrincipal::named("R&D\\ops"),
9063        );
9064        assert!(xml.contains("<UserId>R&amp;D\\ops</UserId>"), "{xml}");
9065        assert_eq!(xml_value(&xml, "UserId").as_deref(), Some("R&D\\ops"));
9066    }
9067
9068    // -----------------------------------------------------------------------
9069    // The Windows service descriptor
9070    // -----------------------------------------------------------------------
9071
9072    #[test]
9073    fn the_service_starts_automatically_under_the_account_the_store_admits() {
9074        let text = windows_service_descriptor(&windows_plan(StartMode::Boot));
9075        let directives = ini_directives(&text, "windows-service");
9076        assert_eq!(
9077            directives.get("StartType").map(String::as_str),
9078            Some("AutoStart")
9079        );
9080        assert_eq!(
9081            directives.get("Account").map(String::as_str),
9082            Some("NT AUTHORITY\\SYSTEM")
9083        );
9084        assert_eq!(
9085            directives.get("ServiceType").map(String::as_str),
9086            Some("OWN_PROCESS")
9087        );
9088        assert_eq!(
9089            directives
9090                .get("FailureActionRestartDelaySecs")
9091                .map(String::as_str),
9092            Some("15")
9093        );
9094        assert_eq!(
9095            directives
9096                .get("FailureActionsOnNonCrashFailures")
9097                .map(String::as_str),
9098            Some("true"),
9099            "without this flag a non-zero exit is not a failure the manager restarts"
9100        );
9101    }
9102
9103    #[test]
9104    fn the_service_spec_leaves_the_account_unnamed_so_the_api_means_local_system() {
9105        let spec = windows_service_spec(&windows_plan(StartMode::Boot));
9106        assert_eq!(spec.account, None);
9107        assert!(spec.automatic_start);
9108        assert!(
9109            spec.command_line.contains("daemon run"),
9110            "{}",
9111            spec.command_line
9112        );
9113    }
9114
9115    // -----------------------------------------------------------------------
9116    // Where each definition is installed
9117    // -----------------------------------------------------------------------
9118
9119    #[test]
9120    fn each_definition_goes_where_its_platform_expects_it() {
9121        let home = PathBuf::from("/home/op");
9122        assert_eq!(
9123            ServiceDefinition::launchd(&linux_plan(StartMode::Boot), Some(&home)).install_path(),
9124            Some(Path::new(
9125                "/Library/LaunchDaemons/io.github.IvanMurzak.runner-manager.plist"
9126            ))
9127        );
9128        assert_eq!(
9129            ServiceDefinition::launchd(&linux_plan(StartMode::Login), Some(&home)).install_path(),
9130            Some(Path::new(
9131                "/home/op/Library/LaunchAgents/io.github.IvanMurzak.runner-manager.plist"
9132            ))
9133        );
9134        assert_eq!(
9135            ServiceDefinition::systemd(&linux_plan(StartMode::Boot), Some(&home)).install_path(),
9136            Some(Path::new("/etc/systemd/system/runner-manager.service"))
9137        );
9138        assert_eq!(
9139            ServiceDefinition::systemd(&linux_plan(StartMode::Login), Some(&home)).install_path(),
9140            Some(Path::new(
9141                "/home/op/.config/systemd/user/runner-manager.service"
9142            ))
9143        );
9144        assert_eq!(
9145            ServiceDefinition::windows_service(&windows_plan(StartMode::Boot)).install_path(),
9146            None,
9147            "the Service Control Manager has no file"
9148        );
9149    }
9150
9151    #[test]
9152    fn a_login_definition_without_a_home_directory_has_nowhere_to_go() {
9153        assert_eq!(
9154            ServiceDefinition::systemd(&linux_plan(StartMode::Login), None).install_path(),
9155            None
9156        );
9157        assert_eq!(
9158            ServiceDefinition::launchd(&linux_plan(StartMode::Login), None).install_path(),
9159            None
9160        );
9161    }
9162
9163    // -----------------------------------------------------------------------
9164    // The least-privilege review: the passing case, then every failing one
9165    // -----------------------------------------------------------------------
9166
9167    #[test]
9168    fn the_rendered_definitions_are_all_least_privilege() {
9169        let linux = linux_plan(StartMode::Boot);
9170        let windows = windows_plan(StartMode::Boot);
9171        for (definition, plan) in [
9172            (ServiceDefinition::systemd(&linux, None), &linux),
9173            (ServiceDefinition::launchd(&linux, None), &linux),
9174            (ServiceDefinition::windows_service(&windows), &windows),
9175        ] {
9176            let review = review_least_privilege(&definition, plan);
9177            assert!(
9178                review.is_least_privilege(),
9179                "{:?} should be least privilege, got:\n{review}",
9180                definition.kind()
9181            );
9182            assert!(
9183                !review.controls().is_empty(),
9184                "a review that confirms nothing proves nothing: {:?}",
9185                definition.kind()
9186            );
9187        }
9188    }
9189
9190    #[test]
9191    fn the_rendered_task_is_least_privilege_and_says_what_it_checked() {
9192        let plan = windows_plan(StartMode::Login);
9193        let definition =
9194            ServiceDefinition::windows_scheduled_task(&plan, &TaskPrincipal::named("HOST\\op"));
9195        let review = review_least_privilege(&definition, &plan);
9196        assert!(review.is_least_privilege(), "{review}");
9197        assert!(
9198            review
9199                .controls()
9200                .iter()
9201                .any(|control| control.contains("LeastPrivilege")),
9202            "{review}"
9203        );
9204    }
9205
9206    #[test]
9207    fn a_unit_that_makes_one_more_directory_writable_is_not_least_privilege() {
9208        let plan = linux_plan(StartMode::Boot);
9209        let rendered = systemd_unit(&plan);
9210        let widened = edited(
9211            &rendered,
9212            "ReadWritePaths=/var/lib/runner-manager/config",
9213            "ReadWritePaths=/etc /var/lib/runner-manager/config",
9214        );
9215        let review = review_least_privilege(
9216            &ServiceDefinition::from_text(DefinitionKind::SystemdUnit, widened),
9217            &plan,
9218        );
9219        assert!(!review.is_least_privilege(), "{review}");
9220        assert!(
9221            review
9222                .excesses()
9223                .iter()
9224                .any(|finding| finding.detail.contains("/etc")),
9225            "the review must name the directory it objects to: {review}"
9226        );
9227    }
9228
9229    #[test]
9230    fn a_unit_that_drops_a_hardening_directive_is_not_least_privilege() {
9231        let plan = linux_plan(StartMode::Boot);
9232        let rendered = systemd_unit(&plan);
9233        let weakened = edited(&rendered, "NoNewPrivileges=yes\n", "");
9234        let review = review_least_privilege(
9235            &ServiceDefinition::from_text(DefinitionKind::SystemdUnit, weakened),
9236            &plan,
9237        );
9238        assert!(!review.is_least_privilege(), "{review}");
9239        assert!(
9240            review
9241                .excesses()
9242                .iter()
9243                .any(|finding| finding.subject == "NoNewPrivileges"),
9244            "{review}"
9245        );
9246    }
9247
9248    #[test]
9249    fn a_unit_that_keeps_capabilities_is_not_least_privilege() {
9250        let plan = linux_plan(StartMode::Boot);
9251        let rendered = systemd_unit(&plan);
9252        let widened = edited(
9253            &rendered,
9254            "CapabilityBoundingSet=\n",
9255            "CapabilityBoundingSet=CAP_NET_ADMIN CAP_SYS_ADMIN\n",
9256        );
9257        let review = review_least_privilege(
9258            &ServiceDefinition::from_text(DefinitionKind::SystemdUnit, widened),
9259            &plan,
9260        );
9261        assert!(!review.is_least_privilege(), "{review}");
9262        assert!(
9263            review
9264                .excesses()
9265                .iter()
9266                .any(|finding| finding.subject == "CapabilityBoundingSet"),
9267            "{review}"
9268        );
9269    }
9270
9271    #[test]
9272    fn a_unit_that_opens_a_listening_socket_is_not_least_privilege() {
9273        let plan = linux_plan(StartMode::Boot);
9274        let rendered = systemd_unit(&plan);
9275        let widened = edited(
9276            &rendered,
9277            "[Install]",
9278            "ListenStream=127.0.0.1:9000\n\n[Install]",
9279        );
9280        let review = review_least_privilege(
9281            &ServiceDefinition::from_text(DefinitionKind::SystemdUnit, widened),
9282            &plan,
9283        );
9284        assert!(
9285            !review.is_least_privilege(),
9286            "07-security.md rule 2 forbids any inbound surface: {review}"
9287        );
9288    }
9289
9290    #[test]
9291    fn a_unit_that_makes_a_directory_unwritable_is_a_shortfall_not_an_excess() {
9292        let plan = linux_plan(StartMode::Boot);
9293        let rendered = systemd_unit(&plan);
9294        let narrowed = edited(&rendered, " /var/lib/runner-manager/runtime", "");
9295        let review = review_least_privilege(
9296            &ServiceDefinition::from_text(DefinitionKind::SystemdUnit, narrowed),
9297            &plan,
9298        );
9299        assert!(
9300            review.is_least_privilege(),
9301            "too little authority is not an excess: {review}"
9302        );
9303        assert!(
9304            review
9305                .findings()
9306                .iter()
9307                .any(|finding| finding.kind == FindingKind::Shortfall
9308                    && finding.detail.contains("runtime")),
9309            "{review}"
9310        );
9311    }
9312
9313    #[test]
9314    fn a_launch_agent_that_names_an_account_is_not_least_privilege() {
9315        let plan = linux_plan(StartMode::Login);
9316        let rendered = launchd_plist(&plan);
9317        let widened = edited(
9318            &rendered,
9319            "<key>ProcessType</key>",
9320            "<key>UserName</key>\n  <string>root</string>\n  <key>ProcessType</key>",
9321        );
9322        let review = review_least_privilege(
9323            &ServiceDefinition::from_text(DefinitionKind::LaunchdPlist, widened),
9324            &plan,
9325        );
9326        assert!(!review.is_least_privilege(), "{review}");
9327        assert!(
9328            review
9329                .excesses()
9330                .iter()
9331                .any(|finding| finding.subject == "UserName"),
9332            "{review}"
9333        );
9334    }
9335
9336    #[test]
9337    fn a_launch_daemon_that_asks_for_a_session_is_not_least_privilege() {
9338        let plan = linux_plan(StartMode::Boot);
9339        let rendered = launchd_plist(&plan);
9340        let widened = edited(
9341            &rendered,
9342            "<key>SessionCreate</key>\n  <false/>",
9343            "<key>SessionCreate</key>\n  <true/>",
9344        );
9345        let review = review_least_privilege(
9346            &ServiceDefinition::from_text(DefinitionKind::LaunchdPlist, widened),
9347            &plan,
9348        );
9349        assert!(!review.is_least_privilege(), "{review}");
9350        assert!(
9351            review
9352                .excesses()
9353                .iter()
9354                .any(|finding| finding.subject == "SessionCreate"),
9355            "{review}"
9356        );
9357    }
9358
9359    #[test]
9360    fn a_launchd_job_that_publishes_a_mach_service_is_not_least_privilege() {
9361        let plan = linux_plan(StartMode::Boot);
9362        let rendered = launchd_plist(&plan);
9363        let widened = edited(
9364            &rendered,
9365            "<key>ProcessType</key>",
9366            "<key>MachServices</key>\n  <dict/>\n  <key>ProcessType</key>",
9367        );
9368        let review = review_least_privilege(
9369            &ServiceDefinition::from_text(DefinitionKind::LaunchdPlist, widened),
9370            &plan,
9371        );
9372        assert!(!review.is_least_privilege(), "{review}");
9373    }
9374
9375    #[test]
9376    fn a_task_asking_for_the_highest_available_token_is_not_least_privilege() {
9377        let plan = windows_plan(StartMode::Login);
9378        let rendered = windows_scheduled_task_xml(&plan, &TaskPrincipal::named("HOST\\op"));
9379        let widened = edited(
9380            &rendered,
9381            "<RunLevel>LeastPrivilege</RunLevel>",
9382            "<RunLevel>HighestAvailable</RunLevel>",
9383        );
9384        let review = review_least_privilege(
9385            &ServiceDefinition::from_text(DefinitionKind::WindowsScheduledTask, widened),
9386            &plan,
9387        );
9388        assert!(!review.is_least_privilege(), "{review}");
9389        assert!(
9390            review
9391                .excesses()
9392                .iter()
9393                .any(|finding| finding.subject == "RunLevel"),
9394            "{review}"
9395        );
9396    }
9397
9398    #[test]
9399    fn a_task_that_would_store_a_password_is_not_least_privilege() {
9400        let plan = windows_plan(StartMode::Login);
9401        let rendered = windows_scheduled_task_xml(&plan, &TaskPrincipal::named("HOST\\op"));
9402        let widened = edited(
9403            &rendered,
9404            "<LogonType>InteractiveToken</LogonType>",
9405            "<LogonType>Password</LogonType>",
9406        );
9407        let review = review_least_privilege(
9408            &ServiceDefinition::from_text(DefinitionKind::WindowsScheduledTask, widened),
9409            &plan,
9410        );
9411        assert!(!review.is_least_privilege(), "{review}");
9412    }
9413
9414    #[test]
9415    fn an_interactive_windows_service_is_not_least_privilege() {
9416        let plan = windows_plan(StartMode::Boot);
9417        let rendered = windows_service_descriptor(&plan);
9418        let widened = edited(
9419            &rendered,
9420            "ServiceType=OWN_PROCESS",
9421            "ServiceType=OWN_PROCESS|INTERACTIVE_PROCESS",
9422        );
9423        let review = review_least_privilege(
9424            &ServiceDefinition::from_text(DefinitionKind::WindowsService, widened),
9425            &plan,
9426        );
9427        assert!(!review.is_least_privilege(), "{review}");
9428        assert!(
9429            review
9430                .excesses()
9431                .iter()
9432                .any(|finding| finding.subject == "ServiceType"),
9433            "{review}"
9434        );
9435    }
9436
9437    #[test]
9438    fn a_windows_service_under_an_account_the_store_dacl_does_not_name_is_reported() {
9439        let plan = windows_plan(StartMode::Boot);
9440        let rendered = windows_service_descriptor(&plan);
9441        let changed = edited(
9442            &rendered,
9443            "Account=NT AUTHORITY\\SYSTEM",
9444            "Account=NT AUTHORITY\\LocalService",
9445        );
9446        let review = review_least_privilege(
9447            &ServiceDefinition::from_text(DefinitionKind::WindowsService, changed),
9448            &plan,
9449        );
9450        // LocalService is *less* privileged, so this is not an excess. It is
9451        // still wrong, because `d2`'s DACL names SY, BA and OW and nothing else:
9452        // the daemon would start and find no credential. Reporting it as a
9453        // shortfall rather than as an excess is the difference between a check
9454        // that understands the requirement and one that counts adjectives.
9455        assert!(review.is_least_privilege(), "{review}");
9456        assert!(
9457            review.findings().iter().any(|finding| {
9458                finding.kind == FindingKind::Shortfall && finding.subject == "Account"
9459            }),
9460            "{review}"
9461        );
9462    }
9463
9464    // -----------------------------------------------------------------------
9465    // The recorded absolute path, and what became of it
9466    // -----------------------------------------------------------------------
9467
9468    #[test]
9469    fn a_recorded_path_that_is_still_there_is_current() {
9470        let host = Host::new();
9471        let state = inspect_binary(&host.binary, Some(&host.binary));
9472        assert!(!state.is_error(), "{state}");
9473        assert!(matches!(state, BinaryPath::Current { .. }), "{state}");
9474    }
9475
9476    #[test]
9477    fn the_npm_upgrade_case_reports_a_stale_path_as_an_error() {
9478        let host = Host::new();
9479        // An `npm i -g` binary lives under the active Node installation's
9480        // global prefix. Switching Node versions moves the prefix, and the
9481        // recorded path stops existing while the registration survives.
9482        let recorded = host.binary.clone();
9483        let healthy = inspect_binary(&recorded, Some(&recorded));
9484        assert!(
9485            !healthy.is_error(),
9486            "the discriminator: before the binary moves, this must be healthy"
9487        );
9488
9489        std::fs::remove_file(&recorded).expect("the binary moves out from under the record");
9490
9491        let state = inspect_binary(&recorded, Some(&recorded));
9492        assert!(state.is_error(), "{state}");
9493        assert!(matches!(state, BinaryPath::Missing { .. }), "{state}");
9494        assert!(
9495            state.to_string().contains("npm"),
9496            "the message must name the cause an operator will not otherwise connect: {state}"
9497        );
9498    }
9499
9500    #[test]
9501    fn a_directory_at_the_recorded_path_is_not_something_the_manager_can_start() {
9502        let root = tempfile::tempdir().expect("a temporary directory");
9503        let state = inspect_binary(root.path(), None);
9504        assert!(state.is_error(), "{state}");
9505        assert!(matches!(state, BinaryPath::NotExecutable { .. }), "{state}");
9506    }
9507
9508    #[test]
9509    fn a_registration_naming_a_different_binary_is_a_divergence() {
9510        let host = Host::new();
9511        let other = host.binary.with_file_name("something-else");
9512        let state = inspect_binary(&host.binary, Some(&other));
9513        assert!(state.is_error(), "{state}");
9514        assert!(matches!(state, BinaryPath::Diverged { .. }), "{state}");
9515    }
9516
9517    #[test]
9518    fn absence_is_reported_before_divergence() {
9519        let host = Host::new();
9520        let recorded = host.binary.clone();
9521        std::fs::remove_file(&recorded).expect("removable");
9522        let other = recorded.with_file_name("something-else");
9523        // Both faults are true at once. The operator needs to hear about the
9524        // missing file, not about a disagreement between two paths of which one
9525        // does not exist.
9526        assert!(matches!(
9527            inspect_binary(&recorded, Some(&other)),
9528            BinaryPath::Missing { .. }
9529        ));
9530    }
9531
9532    // -----------------------------------------------------------------------
9533    // The record
9534    // -----------------------------------------------------------------------
9535
9536    #[test]
9537    fn the_record_round_trips_through_toml() {
9538        let host = Host::new();
9539        let plan = InstallPlan::resolve(
9540            ServiceIdentity::product(),
9541            &host.request(StartMode::Boot),
9542            ServiceDirectories::of(&host.paths),
9543        )
9544        .expect("a resolvable plan");
9545        let definition = ServiceDefinition::from_text(DefinitionKind::SystemdUnit, "[Service]\n");
9546        let record = InstallRecord::of(&plan, &definition, Utc::now());
9547        record.write(&host.paths).expect("a writable record");
9548        let read = InstallRecord::read(&host.paths)
9549            .expect("a readable record")
9550            .expect("a record is there");
9551        assert_eq!(read, record);
9552        assert_eq!(read.binary, host.binary);
9553        assert!(read.binary.is_absolute());
9554    }
9555
9556    #[test]
9557    fn a_unit_written_by_an_older_build_is_reported_and_a_current_one_is_not() {
9558        let host = Host::new();
9559        let identity = ServiceIdentity::product();
9560        let plan = InstallPlan::resolve(
9561            identity.clone(),
9562            &host.request(StartMode::Boot),
9563            ServiceDirectories::of(&host.paths),
9564        )
9565        .expect("a resolvable plan");
9566        let installed = systemd_unit(&plan);
9567        let definition =
9568            ServiceDefinition::from_text(DefinitionKind::SystemdUnit, installed.clone());
9569        let mut record = InstallRecord::of(&plan, &definition, Utc::now());
9570        let path = PathBuf::from("/etc/systemd/system/runner-manager.service");
9571
9572        assert_eq!(
9573            systemd_definition_drift(&identity, &record, path.clone(), &installed),
9574            None,
9575            "the record must rebuild exactly the unit `service install` wrote"
9576        );
9577
9578        record.installed_by_version = "0.4.7".into();
9579        let frozen = installed.replace(
9580            "[Service]\n",
9581            "[Service]\nLoadCredential=runner-manager.user-access-token:/var/lib/runner-manager/secrets/user-access-token\n",
9582        );
9583        let drift = systemd_definition_drift(&identity, &record, path, &frozen)
9584            .expect("an outdated unit is drift");
9585        assert_eq!(drift.installed_by_version, "0.4.7");
9586        assert_eq!(drift.rendered, installed);
9587
9588        let status = ServiceStatus::compose(identity, None, None, None, None, &host.paths)
9589            .with_definition_drift(Some(drift));
9590        assert!(!status.is_healthy());
9591        assert!(
9592            status
9593                .problems()
9594                .iter()
9595                .any(|problem| problem.subject == "definition" && problem.detail.contains("0.4.7")),
9596            "{status}"
9597        );
9598    }
9599
9600    #[test]
9601    fn a_rejected_credential_keeps_its_first_moment_and_makes_the_service_unhealthy() {
9602        let host = Host::new();
9603        assert_eq!(github_credential_rejected_since(&host.paths).unwrap(), None);
9604        let first = Utc::now() - chrono::Duration::hours(9);
9605        record_github_credential_rejected(&host.paths, first).unwrap();
9606        record_github_credential_rejected(&host.paths, Utc::now()).unwrap();
9607        let since = github_credential_rejected_since(&host.paths).unwrap();
9608        assert_eq!(since, Some(first));
9609
9610        let status = ServiceStatus::compose(
9611            ServiceIdentity::product(),
9612            None,
9613            None,
9614            None,
9615            None,
9616            &host.paths,
9617        )
9618        .with_credential_rejection(since);
9619        assert!(!status.is_healthy());
9620        assert_eq!(status.credential_rejected_since(), Some(first));
9621        assert!(status.to_string().contains("auth login"), "{status}");
9622
9623        clear_github_credential_rejection(&host.paths).unwrap();
9624        clear_github_credential_rejection(&host.paths).unwrap();
9625        assert_eq!(github_credential_rejected_since(&host.paths).unwrap(), None);
9626    }
9627
9628    /// A guest that started nothing for 13 days while this said `healthy`.
9629    #[test]
9630    fn blocked_launches_make_the_service_unhealthy_with_their_remedy() {
9631        use crate::launch_health::{
9632            BlockCause, Viewpoint, clear_launches_blocked, launches_blocked,
9633            record_launches_blocked,
9634        };
9635
9636        let host = Host::new();
9637        let compose = || {
9638            ServiceStatus::compose(
9639                ServiceIdentity::product(),
9640                None,
9641                None,
9642                None,
9643                None,
9644                &host.paths,
9645            )
9646            .with_launches_blocked(launches_blocked(&host.paths, Utc::now()))
9647        };
9648        assert!(
9649            !compose()
9650                .problems()
9651                .iter()
9652                .any(|problem| problem.subject == LAUNCHES_BLOCKED_SUBJECT)
9653        );
9654        let blocked = BlockCause::FenceHeld {
9655            directory: PathBuf::from("launch-fence"),
9656            owner: None,
9657        }
9658        .blocked_since(Utc::now(), Viewpoint::Guest);
9659        record_launches_blocked(&host.paths, &blocked, Utc::now()).unwrap();
9660
9661        let status = compose();
9662        assert!(!status.is_healthy());
9663        assert_eq!(status.launches_blocked(), Some(&blocked));
9664        let problem = status
9665            .problems()
9666            .iter()
9667            .find(|problem| problem.subject == LAUNCHES_BLOCKED_SUBJECT)
9668            .expect("reported under its own subject");
9669        assert!(problem.detail.contains("systemctl restart"), "{status}");
9670
9671        clear_launches_blocked(&host.paths).unwrap();
9672        assert_eq!(compose().launches_blocked(), None);
9673    }
9674
9675    /// The daemon's own "I cannot read my credential", which used to exist
9676    /// only in its log while `service status` said `healthy`.
9677    #[test]
9678    fn an_unreadable_credential_keeps_its_first_moment_and_names_the_exact_command() {
9679        let host = Host::new();
9680        assert_eq!(credential_unreadable_since(&host.paths).unwrap(), None);
9681        let first = Utc::now() - chrono::Duration::minutes(40);
9682        record_credential_unreadable(&host.paths, first).unwrap();
9683        record_credential_unreadable(&host.paths, Utc::now()).unwrap();
9684        let since = credential_unreadable_since(&host.paths).unwrap();
9685        assert_eq!(since, Some(first), "a restart loop must not move the start");
9686
9687        let status = ServiceStatus::compose(
9688            ServiceIdentity::product(),
9689            None,
9690            None,
9691            None,
9692            None,
9693            &host.paths,
9694        )
9695        .with_credential_unreadable(since);
9696        assert!(!status.is_healthy());
9697        let problem = status
9698            .problems()
9699            .iter()
9700            .find(|problem| problem.subject == CREDENTIAL_UNREADABLE_SUBJECT)
9701            .expect("reported under its own subject");
9702        // No record, so the default start mode, whose exact command is named.
9703        assert!(
9704            problem
9705                .detail
9706                .contains(&sign_in_instruction(StartMode::default())),
9707            "{status}"
9708        );
9709
9710        clear_credential_unreadable(&host.paths).unwrap();
9711        clear_credential_unreadable(&host.paths).unwrap();
9712        assert_eq!(credential_unreadable_since(&host.paths).unwrap(), None);
9713        let healthy = ServiceStatus::compose(
9714            ServiceIdentity::product(),
9715            None,
9716            None,
9717            None,
9718            None,
9719            &host.paths,
9720        )
9721        .with_credential_unreadable(None);
9722        assert!(
9723            !healthy
9724                .problems()
9725                .iter()
9726                .any(|problem| problem.subject == CREDENTIAL_UNREADABLE_SUBJECT),
9727            "{healthy}"
9728        );
9729    }
9730
9731    #[test]
9732    fn the_sign_in_instruction_names_the_start_mode_it_writes() {
9733        assert!(sign_in_instruction(StartMode::Login).contains("auth login --start-at login`"));
9734        assert!(sign_in_instruction(StartMode::Boot).contains("auth login --start-at boot`"));
9735        assert_eq!(
9736            sign_in_instruction(StartMode::Boot).starts_with("`sudo "),
9737            cfg!(unix),
9738            "a machine-scoped store needs root on Unix"
9739        );
9740        assert_eq!(
9741            sign_in_instruction(StartMode::Login).contains("SSH"),
9742            cfg!(target_os = "macos"),
9743            "only the macOS login keychain is out of an SSH session's reach"
9744        );
9745    }
9746
9747    /// The record is readable by the account whose directory it is in.
9748    ///
9749    /// A boot-mode `service install` runs under `sudo`, and `NamedTempFile`
9750    /// creates at `0600`, so the record landed `root`-owned and unreadable by
9751    /// the operator — `service status` then failed on their own host with
9752    /// `Permission denied`. Ownership cannot be reproduced in an unprivileged
9753    /// test, but the mode that made ownership fatal can, and it is the half
9754    /// this code controls.
9755    #[cfg(unix)]
9756    #[test]
9757    fn the_record_is_not_written_readable_only_by_whoever_installed_it() {
9758        use std::os::unix::fs::PermissionsExt as _;
9759
9760        let host = Host::new();
9761        let plan = InstallPlan::resolve(
9762            ServiceIdentity::product(),
9763            &host.request(StartMode::Boot),
9764            ServiceDirectories::of(&host.paths),
9765        )
9766        .expect("a resolvable plan");
9767        let definition = ServiceDefinition::from_text(DefinitionKind::SystemdUnit, "[Service]\n");
9768        InstallRecord::of(&plan, &definition, Utc::now())
9769            .write(&host.paths)
9770            .expect("a writable record");
9771
9772        let mode = std::fs::metadata(InstallRecord::path(&host.paths))
9773            .expect("the record is there")
9774            .permissions()
9775            .mode()
9776            & 0o777;
9777        assert_eq!(
9778            mode, 0o644,
9779            "the record is mode {mode:04o}; at 0600 an operator cannot read a record `sudo \
9780             service install` wrote, and `service status` fails on their own host. It holds no \
9781             credential and sits in a 0700 directory, so 0644 discloses nothing"
9782        );
9783    }
9784
9785    /// A record written before the service ran a copy of its own must still
9786    /// load. It reads as `None`, which is the truth about it: that registration
9787    /// names the package manager's file and cannot be upgraded under itself.
9788    #[test]
9789    fn a_record_without_a_source_binary_still_reads_and_says_it_has_none() {
9790        let host = Host::new();
9791        let path = InstallRecord::path(&host.paths);
9792        std::fs::write(
9793            &path,
9794            format!(
9795                "schema_version = {RECORD_SCHEMA_VERSION}
9796service_name = \"runner-manager\"
9797                 manager = \"systemd\"
9798start_mode = \"boot\"
9799account = \"root\"
9800                 binary = \"/x\"
9801arguments = []
9802restart_delay_secs = 15
9803                 restart_reset_secs = 600
9804log_file = \"/x\"
9805                 installed_at = \"2026-01-01T00:00:00Z\"
9806installed_by_version = \"0.1.0\"
9807                 [directories]
9808config = \"/a\"
9809state = \"/b\"
9810runtime = \"/c\"
9811logs = \"/d\"
9812"
9813            ),
9814        )
9815        .expect("a writable record");
9816        let read = InstallRecord::read(&host.paths)
9817            .expect("a record missing an optional field is still readable")
9818            .expect("a record is there");
9819        assert_eq!(
9820            read.source_binary, None,
9821            "the legacy layout has no source, and must not invent one"
9822        );
9823    }
9824
9825    /// The field that makes an upgrade possible survives the write.
9826    #[test]
9827    fn a_registration_remembers_the_file_it_was_copied_from() {
9828        let host = Host::new();
9829        let source = host.binary.with_file_name("npm-installed-runner-manager");
9830        std::fs::copy(&host.binary, &source).expect("a second file to stand in for the package");
9831        let plan = InstallPlan::resolve(
9832            ServiceIdentity::product(),
9833            &host.request(StartMode::Boot).copied_from(&source),
9834            ServiceDirectories::of(&host.paths),
9835        )
9836        .expect("a resolvable plan");
9837        let definition = ServiceDefinition::from_text(
9838            DefinitionKind::SystemdUnit,
9839            "[Service]
9840",
9841        );
9842        let record = InstallRecord::of(&plan, &definition, Utc::now());
9843        record.write(&host.paths).expect("a writable record");
9844
9845        let read = InstallRecord::read(&host.paths)
9846            .expect("a readable record")
9847            .expect("a record is there");
9848        assert_eq!(read.source_binary.as_deref(), Some(source.as_path()));
9849        assert_ne!(
9850            read.source_binary.as_deref(),
9851            Some(read.binary.as_path()),
9852            "the whole point is that the two are different files: one the service holds open,              one the package manager is free to replace"
9853        );
9854    }
9855
9856    #[test]
9857    fn a_record_from_a_schema_this_build_cannot_read_is_refused_with_a_remedy() {
9858        let host = Host::new();
9859        let path = InstallRecord::path(&host.paths);
9860        std::fs::write(
9861            &path,
9862            format!(
9863                "schema_version = {}\nservice_name = \"runner-manager\"\nmanager = \"systemd\"\n\
9864                 start_mode = \"boot\"\naccount = \"root\"\nbinary = \"/x\"\narguments = []\n\
9865                 restart_delay_secs = 15\nrestart_reset_secs = 600\nlog_file = \"/x\"\n\
9866                 installed_at = \"2026-01-01T00:00:00Z\"\ninstalled_by_version = \"0.1.0\"\n\
9867                 [directories]\nconfig = \"/a\"\nstate = \"/b\"\nruntime = \"/c\"\nlogs = \"/d\"\n",
9868                RECORD_SCHEMA_VERSION + 1
9869            ),
9870        )
9871        .expect("a writable record");
9872        let error = InstallRecord::read(&host.paths).expect_err("a future schema is refused");
9873        assert!(
9874            matches!(error, ServiceError::RecordUnreadable { .. }),
9875            "{error}"
9876        );
9877        assert!(
9878            error.to_string().contains("service uninstall"),
9879            "the message must say how to recover: {error}"
9880        );
9881    }
9882
9883    #[test]
9884    fn no_record_is_not_an_error() {
9885        let host = Host::new();
9886        assert_eq!(InstallRecord::read(&host.paths).expect("no record"), None);
9887        assert!(!InstallRecord::remove(&host.paths).expect("nothing to remove"));
9888    }
9889
9890    // -----------------------------------------------------------------------
9891    // The last successful GitHub contact
9892    // -----------------------------------------------------------------------
9893
9894    #[test]
9895    fn no_heartbeat_reads_as_never_rather_than_as_the_epoch() {
9896        let host = Host::new();
9897        assert_eq!(last_github_contact(&host.paths).expect("readable"), None);
9898    }
9899
9900    #[test]
9901    fn the_heartbeat_round_trips_to_the_second() {
9902        let host = Host::new();
9903        let at = DateTime::parse_from_rfc3339("2026-08-22T10:11:12Z")
9904            .expect("a valid timestamp")
9905            .with_timezone(&Utc);
9906        record_github_contact(&host.paths, at).expect("a writable heartbeat");
9907        assert_eq!(
9908            last_github_contact(&host.paths).expect("readable"),
9909            Some(at)
9910        );
9911    }
9912
9913    #[test]
9914    fn a_malformed_heartbeat_is_an_error_and_not_silently_never() {
9915        let host = Host::new();
9916        std::fs::write(contact_path(&host.paths), b"this is not toml \x00").expect("writable");
9917        let error = last_github_contact(&host.paths)
9918            .expect_err("a heartbeat that cannot be parsed is not the same as no heartbeat");
9919        assert!(matches!(error, ServiceError::Record { .. }), "{error}");
9920    }
9921
9922    /// The surface the three-hour outage did not have.
9923    ///
9924    /// A root that refuses a launch does so before any attempt row exists, and
9925    /// the daemon's log scrubs the paths out of the sentence, so this file is
9926    /// the only place the directory and the remediation can reach the operator.
9927    #[test]
9928    fn a_runner_root_refusal_round_trips_for_service_status() {
9929        let host = Host::new();
9930        let at = DateTime::from_timestamp(1_760_000_000, 0).expect("a valid instant");
9931
9932        assert!(
9933            runner_root_refusals(&host.paths)
9934                .expect("readable")
9935                .is_empty(),
9936            "no record means every policy is placing runners"
9937        );
9938
9939        record_runner_root_refusal(
9940            &host.paths,
9941            "policy-a",
9942            at,
9943            "denied_by_privacy_policy",
9944            "/Volumes/NVME/runners",
9945            "the runner root /Volumes/NVME/runners cannot be used: ... Grant Full Disk Access",
9946        )
9947        .expect("a writable record");
9948
9949        let refusals = runner_root_refusals(&host.paths).expect("readable");
9950        assert_eq!(refusals.len(), 1);
9951        assert_eq!(refusals[0].policy, "policy-a");
9952        assert_eq!(refusals[0].at, at);
9953        assert_eq!(refusals[0].kind, "denied_by_privacy_policy");
9954        assert_eq!(refusals[0].root, "/Volumes/NVME/runners");
9955        assert!(
9956            refusals[0].detail.contains("/Volumes/NVME/runners")
9957                && refusals[0].detail.contains("Full Disk Access"),
9958            "the path and the remediation are the whole point of this file: {refusals:?}"
9959        );
9960    }
9961
9962    /// The regression that would put the outage straight back.
9963    ///
9964    /// A host runs several policies and they do not share a fate. When the
9965    /// record was one host-wide slot, the policy that placed a runner deleted
9966    /// the record of the policy that could not — on the same reconcile pass —
9967    /// and `service status` reported a healthy host that had started zero
9968    /// runners for an entire target.
9969    #[test]
9970    fn one_policy_placing_a_runner_does_not_clear_another_policys_refusal() {
9971        let host = Host::new();
9972        let at = DateTime::from_timestamp(1_760_000_000, 0).expect("a valid instant");
9973        record_runner_root_refusal(
9974            &host.paths,
9975            "broken",
9976            at,
9977            "denied_by_privacy_policy",
9978            "/Volumes/NVME/runners",
9979            "detail",
9980        )
9981        .expect("a writable record");
9982        record_runner_root_refusal(
9983            &host.paths,
9984            "also-broken",
9985            at,
9986            "not_writable",
9987            "/srv/other",
9988            "detail",
9989        )
9990        .expect("a writable record");
9991
9992        // The policy with its own working root succeeds and clears only itself.
9993        clear_runner_root_refusal(&host.paths, "healthy")
9994            .expect("clearing an absent policy is fine");
9995        clear_runner_root_refusal(&host.paths, "also-broken").expect("that policy recovered");
9996
9997        let refusals = runner_root_refusals(&host.paths).expect("readable");
9998        assert_eq!(
9999            refusals
10000                .iter()
10001                .map(|r| r.policy.as_str())
10002                .collect::<Vec<_>>(),
10003            vec!["broken"],
10004            "the policy that is still refused must keep its record"
10005        );
10006    }
10007
10008    /// Clearing is what keeps a fixed host from reporting a stale fault, and the
10009    /// file goes when the last entry does.
10010    #[test]
10011    fn clearing_the_last_refusal_removes_the_file_and_is_idempotent() {
10012        let host = Host::new();
10013        record_runner_root_refusal(
10014            &host.paths,
10015            "p",
10016            Utc::now(),
10017            "not_writable",
10018            "/srv/x",
10019            "detail",
10020        )
10021        .expect("a writable record");
10022
10023        clear_runner_root_refusal(&host.paths, "p").expect("the record is removed");
10024        assert!(
10025            !root_refusal_path(&host.paths).exists(),
10026            "a host with nothing refused leaves nothing behind"
10027        );
10028        clear_runner_root_refusal(&host.paths, "p").expect("removing what is gone is not an error");
10029    }
10030
10031    /// Same rule as the heartbeat: absence means "every policy is placing
10032    /// runners", so a record that cannot be parsed must not be reported as one.
10033    #[test]
10034    fn a_malformed_refusal_is_an_error_and_not_silently_none() {
10035        let host = Host::new();
10036        std::fs::write(root_refusal_path(&host.paths), b"not toml \x00").expect("writable");
10037        let error = runner_root_refusals(&host.paths)
10038            .expect_err("an unparseable record is not the same as no record");
10039        assert!(matches!(error, ServiceError::Record { .. }), "{error}");
10040    }
10041
10042    /// The half of this feature that turns the file into a diagnosis, and the
10043    /// half that had no test: `service status` must actually read it.
10044    ///
10045    /// A **note** and not a problem, deliberately. `service status` runs as
10046    /// whoever typed it, must stay readable on a host with nothing installed,
10047    /// and drives an exit code -- and nothing an operator can type clears this
10048    /// record, so a problem would fail that exit code on a host whose root was
10049    /// fixed but which has had no queued job since, with a printed remedy
10050    /// (`service uninstall && service install`) that does not touch `state/`.
10051    #[test]
10052    fn service_status_reports_a_refusal_as_a_note_and_stays_healthy() {
10053        let host = Host::new();
10054        record_runner_root_refusal(
10055            &host.paths,
10056            "policy-a",
10057            DateTime::from_timestamp(1_760_000_000, 0).expect("a valid instant"),
10058            "denied_by_privacy_policy",
10059            "/Volumes/NVME/runners",
10060            "Grant Full Disk Access to the program that runs the service",
10061        )
10062        .expect("a writable record");
10063
10064        let status = host.operations().status().expect("a readable status");
10065        let notes = status.notes().join("\n");
10066
10067        assert!(
10068            notes.contains("/Volumes/NVME/runners") && notes.contains("Full Disk Access"),
10069            "the directory and the remediation the log had to scrub must appear here: {notes}"
10070        );
10071        assert!(
10072            notes.contains("policy-a"),
10073            "the operator has to know which target placed no runner: {notes}"
10074        );
10075        assert!(
10076            !status
10077                .problems()
10078                .iter()
10079                .any(|problem| problem.subject == "runner root"),
10080            "a record nothing an operator types can clear must not drive the exit code"
10081        );
10082    }
10083
10084    // -----------------------------------------------------------------------
10085    // -----------------------------------------------------------------------
10086    // Install
10087    // -----------------------------------------------------------------------
10088
10089    #[test]
10090    fn install_records_the_absolute_binary_path_and_the_four_directories() {
10091        let host = Host::new();
10092        let installed = host
10093            .operations()
10094            .install(&host.request(StartMode::Boot))
10095            .expect("an install against the recording controls");
10096
10097        assert_eq!(installed.record.binary, host.binary);
10098        assert!(installed.record.binary.is_absolute());
10099        assert_eq!(installed.record.start_mode, StartMode::Boot);
10100        assert_eq!(installed.record.arguments, vec!["daemon", "run"]);
10101        assert_eq!(
10102            installed.record.directories,
10103            ServiceDirectories::of(&host.paths)
10104        );
10105        assert_eq!(
10106            installed.record.log_file,
10107            host.paths.logs_dir().join(LOG_FILE_STEM)
10108        );
10109        assert_eq!(installed.record.restart_delay_secs, 15);
10110
10111        let registrations = host.controls.registrations();
10112        assert_eq!(registrations.len(), 1);
10113        assert_eq!(registrations[0].0, StartMode::Boot);
10114        assert_eq!(registrations[0].1, "runner-manager");
10115    }
10116
10117    #[test]
10118    fn install_is_refused_while_the_single_instance_lock_is_held() {
10119        let host = Host::new();
10120        // The discriminator: with the lock free, the same call succeeds.
10121        {
10122            let operations = host.operations();
10123            operations
10124                .install(&host.request(StartMode::Boot))
10125                .expect("an install with the lock free");
10126            operations.uninstall().expect("a clean slate");
10127        }
10128
10129        let _held = HostLock::try_acquire(&host.paths, LockKind::SingleInstance)
10130            .expect("this process takes the lock first");
10131
10132        let error = host
10133            .operations()
10134            .install(&host.request(StartMode::Boot))
10135            .expect_err("a second agent must not be registered while one is running");
10136        assert!(matches!(error, ServiceError::LockHeld { .. }), "{error}");
10137        assert!(
10138            error.to_string().contains("already running"),
10139            "the message must be actionable: {error}"
10140        );
10141        assert!(
10142            host.controls.registrations().is_empty(),
10143            "a refused install must register nothing"
10144        );
10145        assert_eq!(
10146            InstallRecord::read(&host.paths).expect("readable"),
10147            None,
10148            "a refused install must write no record"
10149        );
10150    }
10151
10152    /// Installing over the same start mode replaces the registration.
10153    ///
10154    /// It used to be refused, and the refusal is what broke a real host: the
10155    /// caller has already swapped the binary the service runs by the time this
10156    /// is reached — that is the documented way to upgrade by hand — so refusing
10157    /// left a daemon running an unregistered binary and, on macOS, one the
10158    /// stored credential's keychain grant did not name.
10159    #[test]
10160    fn installing_over_the_same_start_mode_replaces_the_registration() {
10161        let host = Host::new();
10162        let operations = host.operations();
10163        operations
10164            .install(&host.request(StartMode::Boot))
10165            .expect("the first install");
10166
10167        let again = operations
10168            .install(&host.request(StartMode::Boot))
10169            .expect("an install over the same mode replaces rather than refusing");
10170
10171        assert!(
10172            again.replaced_existing,
10173            "the operator is told this replaced something rather than made it"
10174        );
10175        assert_eq!(
10176            host.controls.registrations().len(),
10177            1,
10178            "replacing must not leave two registrations behind"
10179        );
10180        assert_eq!(
10181            InstallRecord::read(&host.paths)
10182                .expect("readable")
10183                .expect("a record")
10184                .start_mode,
10185            StartMode::Boot
10186        );
10187    }
10188
10189    /// The *other* mode is still refused, because changing it moves the
10190    /// registration between two service managers and changes the account and
10191    /// the secret store with it. `set_start_mode` is what does that.
10192    #[test]
10193    fn installing_over_the_other_start_mode_is_refused() {
10194        let host = Host::new();
10195        let operations = host.operations();
10196        operations
10197            .install(&host.request(StartMode::Boot))
10198            .expect("the first install");
10199
10200        let error = operations
10201            .install(&host.request(StartMode::Login))
10202            .expect_err("a mode change is not an install");
10203        assert!(
10204            matches!(
10205                error,
10206                ServiceError::AlreadyInstalled {
10207                    existing: StartMode::Boot,
10208                    requested: StartMode::Login,
10209                    ..
10210                }
10211            ),
10212            "{error}"
10213        );
10214        assert!(
10215            !error
10216                .to_string()
10217                .contains("switch the start mode in place,"),
10218            "the old remedy named a capability no command offers; the terminal UI is where \
10219             the start mode moves: {error}"
10220        );
10221        assert_eq!(host.controls.registrations().len(), 1);
10222    }
10223
10224    #[test]
10225    fn install_rolls_back_the_registration_when_record_persistence_fails() {
10226        let host = Host::new();
10227        let record_path = InstallRecord::path(&host.paths);
10228        std::fs::create_dir(&record_path).expect("a directory blocks the record file");
10229
10230        let error = host
10231            .operations()
10232            .install(&host.request(StartMode::Boot))
10233            .expect_err("record persistence must fail");
10234
10235        assert!(matches!(error, ServiceError::Record { .. }), "{error}");
10236        assert!(
10237            host.controls.registrations().is_empty(),
10238            "a failed install must not leave a live unrecorded registration"
10239        );
10240        assert!(
10241            host.controls
10242                .calls()
10243                .iter()
10244                .any(|call| call == "uninstall runner-manager (boot)"),
10245            "the registration must be explicitly rolled back: {:?}",
10246            host.controls.calls()
10247        );
10248        assert!(
10249            !host.runner_root.as_path().exists(),
10250            "the rollback must take the runner root this install created with it; a directory \
10251             prepared for a registration that does not exist is litter, and on Windows it is \
10252             litter with a security descriptor"
10253        );
10254    }
10255
10256    // -----------------------------------------------------------------------
10257    // The runner root (b2)
10258    // -----------------------------------------------------------------------
10259
10260    #[test]
10261    fn the_runner_root_a_boot_registration_needs_admits_only_the_service() {
10262        use crate::runner_root_access::{RootAdmission, default_root_sddl, grants_broad_write};
10263
10264        // The mapping `prepare_runner_root` reads, asserted against the same
10265        // function the registration's own principal comes from — which is the
10266        // point of deriving it there rather than restating it.
10267        assert_eq!(
10268            ServiceAccount::for_definition(DefinitionKind::WindowsService, StartMode::Boot),
10269            ServiceAccount::LocalSystem
10270        );
10271        assert_eq!(
10272            ServiceAccount::for_definition(DefinitionKind::WindowsScheduledTask, StartMode::Login),
10273            ServiceAccount::InvokingUser
10274        );
10275
10276        let boot = default_root_sddl(&RootAdmission::LocalSystem);
10277        assert!(!grants_broad_write(&boot), "{boot}");
10278        assert!(
10279            !boot.contains("S-1-5-21"),
10280            "a boot registration runs as LocalSystem, so its root names no operator: {boot}"
10281        );
10282
10283        // A login task runs under a *filtered* token — `RunLevel` is
10284        // `LeastPrivilege`, see `windows_scheduled_task_xml` — in which
10285        // Administrators is deny-only. Without an ace of its own the account
10286        // the task runs as would be admitted by nothing at all.
10287        let login = default_root_sddl(&RootAdmission::Account("S-1-5-21-1-2-3-1001".to_owned()));
10288        assert!(login.contains("S-1-5-21-1-2-3-1001"), "{login}");
10289        assert!(!grants_broad_write(&login), "{login}");
10290    }
10291
10292    #[test]
10293    fn an_install_reports_the_runner_root_it_prepared() {
10294        let host = Host::new();
10295        let installed = host
10296            .operations()
10297            .install(&host.request(StartMode::Boot))
10298            .expect("an install");
10299        let rendered = installed.runner_root.to_string();
10300        assert!(
10301            !rendered.contains("S-1-5-21"),
10302            "the report must add no identity to the output: {rendered}"
10303        );
10304        if cfg!(windows) {
10305            assert_eq!(
10306                installed.runner_root.path(),
10307                Some(host.runner_root.as_path())
10308            );
10309            assert!(
10310                host.runner_root.as_path().is_dir(),
10311                "the directory jobs would run in has to exist once the service is registered"
10312            );
10313        } else {
10314            assert_eq!(
10315                installed.runner_root,
10316                crate::runner_root_access::RootAccessSummary::NotApplicable,
10317                "macOS and Linux keep the runtime directory they have always used"
10318            );
10319        }
10320    }
10321
10322    #[test]
10323    fn switching_start_mode_reconciles_the_runner_root_for_the_new_account() {
10324        let host = Host::new();
10325        let operations = host.operations();
10326        operations
10327            .install(&host.request(StartMode::Boot))
10328            .expect("an install at boot");
10329
10330        let change = operations
10331            .set_start_mode(StartMode::Login)
10332            .expect("a switch to login");
10333
10334        assert!(change.changed);
10335        if cfg!(windows) {
10336            assert_eq!(change.runner_root.path(), Some(host.runner_root.as_path()));
10337            assert!(
10338                host.runner_root.as_path().is_dir(),
10339                "the switch must not remove the directory it reconciled"
10340            );
10341        }
10342        // The operator is told, because the account that may write there has
10343        // moved with the mode.
10344        assert!(
10345            change.to_string().contains("runner root"),
10346            "{}",
10347            change.to_string()
10348        );
10349    }
10350
10351    #[test]
10352    fn switching_to_the_mode_already_in_force_touches_no_runner_root() {
10353        let host = Host::new();
10354        let operations = host.operations();
10355        operations
10356            .install(&host.request(StartMode::Boot))
10357            .expect("an install at boot");
10358
10359        let change = operations
10360            .set_start_mode(StartMode::Boot)
10361            .expect("a switch to the mode already in force");
10362
10363        assert!(!change.changed);
10364        assert_eq!(
10365            change.runner_root,
10366            crate::runner_root_access::RootAccessSummary::NotApplicable,
10367            "nothing moves, so nothing about the root's access control has to; reconciling here \
10368             would turn a no-op command into one that can fail on a permission it does not need"
10369        );
10370    }
10371
10372    #[cfg(windows)]
10373    #[test]
10374    fn a_registration_the_manager_refuses_leaves_no_runner_root_behind() {
10375        let host = Host::new();
10376        host.controls
10377            .fail_next_install(StartMode::Boot, "injected registration failure");
10378
10379        let error = host
10380            .operations()
10381            .install(&host.request(StartMode::Boot))
10382            .expect_err("the manager refuses the registration");
10383
10384        assert!(matches!(error, ServiceError::Control { .. }), "{error}");
10385        assert!(
10386            !host.runner_root.as_path().exists(),
10387            "the directory was created for a registration that does not exist"
10388        );
10389    }
10390
10391    #[cfg(windows)]
10392    #[test]
10393    fn an_existing_broad_runner_root_refuses_the_install_before_anything_is_registered() {
10394        let host = Host::new();
10395        // What a directory created below `C:\` with inheritance left on looks
10396        // like, built deliberately because a per-account `%TEMP%` never
10397        // produces one by accident.
10398        crate::runner_root_access::create_with_descriptor_for_tests(
10399            host.runner_root.as_path(),
10400            "D:P(A;OICI;FA;;;SY)(A;OICI;FA;;;WD)",
10401        )
10402        .expect("a deliberately open runner root");
10403        let before = crate::runner_root_access::report(host.runner_root.as_path());
10404
10405        let error = host
10406            .operations()
10407            .install(&host.request(StartMode::Boot))
10408            .expect_err("an open runner root is refused");
10409
10410        assert!(matches!(error, ServiceError::RunnerRoot { .. }), "{error}");
10411        assert!(
10412            error.to_string().contains("nothing was registered"),
10413            "{error}"
10414        );
10415        assert!(
10416            host.controls.registrations().is_empty(),
10417            "the refusal has to come before the platform is asked to register anything: {:?}",
10418            host.controls.calls()
10419        );
10420        assert_eq!(
10421            crate::runner_root_access::report(host.runner_root.as_path()),
10422            before,
10423            "an open directory is refused rather than tightened: its contents cannot be trusted, \
10424             so adopting it would be worse than declining it"
10425        );
10426    }
10427
10428    #[cfg(windows)]
10429    #[test]
10430    fn uninstall_leaves_the_runner_root_exactly_where_it_is() {
10431        let host = Host::new();
10432        let operations = host.operations();
10433        operations
10434            .install(&host.request(StartMode::Boot))
10435            .expect("an install");
10436        assert!(host.runner_root.as_path().is_dir());
10437
10438        operations.uninstall().expect("an uninstall");
10439
10440        assert!(
10441            host.runner_root.as_path().is_dir(),
10442            "`05-infrastructure.md` item 5: uninstall deregisters and deletes nothing else. A \
10443             runner root may hold an operator's retained workspaces."
10444        );
10445    }
10446
10447    #[test]
10448    fn install_reviews_what_it_registered() {
10449        let host = Host::new();
10450        let installed = host
10451            .operations()
10452            .install(&host.request(StartMode::Boot))
10453            .expect("an install");
10454        assert!(
10455            installed.review.is_least_privilege(),
10456            "{}",
10457            installed.review
10458        );
10459        assert!(
10460            !installed.review.controls().is_empty(),
10461            "a review that confirms nothing proves nothing: {}",
10462            installed.review
10463        );
10464        assert_eq!(
10465            installed.review.kind(),
10466            host_definition_kind(StartMode::Boot),
10467            "the review must be of the definition this host's manager was given"
10468        );
10469        assert!(
10470            !installed.review.account().justification().is_empty(),
10471            "a privileged account with no stated reason is an unreviewed one"
10472        );
10473    }
10474
10475    // -----------------------------------------------------------------------
10476    // Uninstall deletes nothing else
10477    // -----------------------------------------------------------------------
10478
10479    #[test]
10480    fn uninstall_leaves_configuration_sqlite_secrets_and_cache_exactly_as_they_were() {
10481        let host = Host::new();
10482        let operations = host.operations();
10483        operations
10484            .install(&host.request(StartMode::Boot))
10485            .expect("an install");
10486
10487        // The files `05-infrastructure.md` says must survive. Written *after*
10488        // the install so that the snapshot below is of a host in the state an
10489        // operator's would be in.
10490        let config = host.paths.config_dir();
10491        std::fs::write(config.join("runner-manager.db"), b"sqlite fixture").expect("writable");
10492        std::fs::write(config.join("config.toml"), b"host_capacity = 2").expect("writable");
10493        std::fs::create_dir_all(host.paths.state_dir().join("packages/2.330.0")).expect("writable");
10494        std::fs::write(
10495            host.paths
10496                .state_dir()
10497                .join("packages/2.330.0/runner.tar.gz"),
10498            b"cached package",
10499        )
10500        .expect("writable");
10501        std::fs::create_dir_all(host.paths.state_dir().join("secrets")).expect("writable");
10502        std::fs::write(
10503            host.paths.state_dir().join("secrets/user-access-token"),
10504            b"a stand-in for the stored credential",
10505        )
10506        .expect("writable");
10507        std::fs::write(
10508            host.paths.logs_dir().join("runner-manager.log.2026-08-22"),
10509            b"diagnostics",
10510        )
10511        .expect("writable");
10512
10513        let roots: Vec<PathBuf> = host
10514            .paths
10515            .all()
10516            .iter()
10517            .map(|(_, path)| (*path).to_path_buf())
10518            .collect();
10519        let roots: Vec<&Path> = roots.iter().map(PathBuf::as_path).collect();
10520        let before = snapshot(&roots);
10521
10522        // Non-vacuity: a comparison of two empty maps would pass whatever
10523        // `uninstall` did.
10524        assert!(
10525            before.len() >= 6,
10526            "the fixture must actually contain the files this test is about, got {before:#?}"
10527        );
10528        let record_path = InstallRecord::path(&host.paths);
10529        assert!(
10530            before.contains_key(&record_path),
10531            "the install record must be present before uninstall"
10532        );
10533
10534        let uninstalled = operations.uninstall().expect("an uninstall");
10535        assert!(uninstalled.removed_registration);
10536        assert!(uninstalled.removed_record);
10537
10538        let after = snapshot(&roots);
10539
10540        // Exactly one thing changed, and it is the registration's own record.
10541        let mut expected = before.clone();
10542        expected.remove(&record_path);
10543        assert_eq!(
10544            after, expected,
10545            "uninstall must remove its own record and nothing else"
10546        );
10547        assert!(
10548            !record_path.exists(),
10549            "the record itself must go, or `uninstall` did nothing at all"
10550        );
10551        assert!(
10552            uninstalled
10553                .preserved
10554                .iter()
10555                .all(|path| roots.contains(&path.as_path())),
10556            "the preserved list must name the four directories: {uninstalled}"
10557        );
10558    }
10559
10560    #[test]
10561    fn uninstall_on_a_host_with_no_registration_is_not_a_failure() {
10562        let host = Host::new();
10563        let uninstalled = host.operations().uninstall().expect("a no-op uninstall");
10564        assert!(!uninstalled.removed_registration);
10565        assert!(!uninstalled.removed_record);
10566    }
10567
10568    #[test]
10569    fn uninstall_removes_a_registration_even_when_the_record_is_gone() {
10570        let host = Host::new();
10571        let operations = host.operations();
10572        operations
10573            .install(&host.request(StartMode::Boot))
10574            .expect("an install");
10575        std::fs::remove_file(InstallRecord::path(&host.paths)).expect("the record is lost");
10576
10577        let uninstalled = operations.uninstall().expect("an uninstall");
10578        assert!(
10579            uninstalled.removed_registration,
10580            "a lost record must not strand a registration"
10581        );
10582        assert!(host.controls.registrations().is_empty());
10583    }
10584
10585    // -----------------------------------------------------------------------
10586    // Switching start mode
10587    // -----------------------------------------------------------------------
10588
10589    #[test]
10590    fn switching_start_mode_reuses_the_recorded_path_and_re_resolves_nothing() {
10591        let host = Host::new();
10592        let operations = host.operations();
10593        operations
10594            .install(&host.request(StartMode::Boot))
10595            .expect("an install at boot");
10596
10597        // The discriminator. If the switch re-resolved the binary the way
10598        // `install` does, it would either fail here or silently record the test
10599        // binary instead. Removing the file makes the difference visible.
10600        std::fs::remove_file(&host.binary).expect("the installed binary goes away");
10601
10602        let change = operations
10603            .set_start_mode(StartMode::Login)
10604            .expect("a switch that does not reinstall the product");
10605        assert!(change.changed);
10606        assert_eq!(change.from, StartMode::Boot);
10607        assert_eq!(change.to, StartMode::Login);
10608        assert_eq!(change.store_scope, crate::secrets::SecretScope::User);
10609
10610        let record = InstallRecord::read(&host.paths)
10611            .expect("readable")
10612            .expect("a record");
10613        assert_eq!(record.start_mode, StartMode::Login);
10614        assert_eq!(
10615            record.binary, host.binary,
10616            "the recorded path must survive the switch untouched"
10617        );
10618
10619        let registrations = host.controls.registrations();
10620        assert_eq!(registrations.len(), 1, "{registrations:?}");
10621        assert_eq!(registrations[0].0, StartMode::Login);
10622        assert!(
10623            registrations[0]
10624                .2
10625                .command_line
10626                .contains(&host.binary.to_string_lossy().into_owned()),
10627            "{:?}",
10628            registrations[0].2
10629        );
10630    }
10631
10632    #[test]
10633    fn switching_start_mode_keeps_the_live_registration_when_target_install_fails() {
10634        let host = Host::new();
10635        let operations = host.operations();
10636        operations
10637            .install(&host.request(StartMode::Boot))
10638            .expect("an install at boot");
10639        let record_before = std::fs::read(InstallRecord::path(&host.paths)).expect("the record");
10640        host.controls
10641            .fail_next_install(StartMode::Login, "injected target failure");
10642
10643        let error = operations
10644            .set_start_mode(StartMode::Login)
10645            .expect_err("the target manager refuses the install");
10646
10647        assert!(matches!(error, ServiceError::Control { .. }), "{error}");
10648        assert_eq!(
10649            std::fs::read(InstallRecord::path(&host.paths)).expect("the old record survives"),
10650            record_before
10651        );
10652        let registrations = host.controls.registrations();
10653        assert_eq!(registrations.len(), 1, "{registrations:?}");
10654        assert_eq!(registrations[0].0, StartMode::Boot);
10655    }
10656
10657    #[test]
10658    fn switching_start_mode_rolls_back_target_when_record_persistence_fails() {
10659        let host = Host::new();
10660        let operations = host.operations();
10661        operations
10662            .install(&host.request(StartMode::Boot))
10663            .expect("an install at boot");
10664        let record_before = std::fs::read(InstallRecord::path(&host.paths)).expect("the record");
10665        let config = host.paths.config_dir().to_path_buf();
10666        let hidden = config.with_file_name("config-hidden-by-fault");
10667        host.controls.hide_directory_after_install(
10668            StartMode::Login,
10669            config.clone(),
10670            hidden.clone(),
10671        );
10672
10673        let error = operations
10674            .set_start_mode(StartMode::Login)
10675            .expect_err("the injected filesystem fault prevents persistence");
10676
10677        std::fs::remove_file(&config).expect("remove the injected blocker");
10678        std::fs::rename(&hidden, &config).expect("restore the record directory");
10679        assert!(matches!(error, ServiceError::Record { .. }), "{error}");
10680        assert_eq!(
10681            std::fs::read(InstallRecord::path(&host.paths)).expect("the old record survives"),
10682            record_before
10683        );
10684        let registrations = host.controls.registrations();
10685        assert_eq!(registrations.len(), 1, "{registrations:?}");
10686        assert_eq!(registrations[0].0, StartMode::Boot);
10687        assert!(
10688            host.controls
10689                .calls()
10690                .iter()
10691                .any(|call| call == "uninstall runner-manager (login)"),
10692            "the target must be rolled back: {:?}",
10693            host.controls.calls()
10694        );
10695    }
10696
10697    #[test]
10698    fn switching_to_the_mode_already_in_force_registers_nothing_again() {
10699        let host = Host::new();
10700        let operations = host.operations();
10701        operations
10702            .install(&host.request(StartMode::Boot))
10703            .expect("an install");
10704        let before = host.controls.calls().len();
10705
10706        let change = operations
10707            .set_start_mode(StartMode::Boot)
10708            .expect("a no-op switch");
10709        assert!(!change.changed);
10710        assert_eq!(
10711            host.controls.calls().len(),
10712            before,
10713            "a no-op switch must not touch the service manager"
10714        );
10715    }
10716
10717    #[test]
10718    fn switching_start_mode_on_a_host_with_no_registration_is_refused() {
10719        let host = Host::new();
10720        let error = host
10721            .operations()
10722            .set_start_mode(StartMode::Login)
10723            .expect_err("there is nothing to switch");
10724        assert!(
10725            matches!(error, ServiceError::NotInstalled { .. }),
10726            "{error}"
10727        );
10728    }
10729
10730    // -----------------------------------------------------------------------
10731    // Status
10732    // -----------------------------------------------------------------------
10733
10734    #[test]
10735    fn status_reports_the_four_facts_journey_five_asks_for() {
10736        let host = Host::new();
10737        let operations = host.operations();
10738        operations
10739            .install(&host.request(StartMode::Boot))
10740            .expect("an install");
10741        let at = DateTime::parse_from_rfc3339("2026-08-22T09:00:00Z")
10742            .expect("a valid timestamp")
10743            .with_timezone(&Utc);
10744        record_github_contact(&host.paths, at).expect("a heartbeat");
10745
10746        let status = operations.status().expect("a status");
10747        assert_eq!(status.start_mode(), Some(StartMode::Boot));
10748        assert_eq!(
10749            status.binary().map(BinaryPath::recorded),
10750            Some(host.binary.as_path())
10751        );
10752        assert_eq!(status.log_file(), host.paths.logs_dir().join(LOG_FILE_STEM));
10753        assert_eq!(status.last_github_contact(), Some(at));
10754        assert!(status.is_installed());
10755        assert!(status.is_healthy(), "{status}");
10756
10757        let printed = status.to_string();
10758        for fragment in [
10759            "start mode",
10760            "diagnostic log",
10761            "last GitHub contact",
10762            "binary",
10763        ] {
10764            assert!(printed.contains(fragment), "{printed}");
10765        }
10766    }
10767
10768    /// A record this account may not read is reported, and `status` keeps
10769    /// going.
10770    ///
10771    /// It used to end the command with `Permission denied`, which is exactly
10772    /// what an operator met on a boot-mode host: `sudo service install` wrote
10773    /// the record `0600` and `root`-owned, and `service status` — a command
10774    /// that asks no privilege of anybody — then printed nothing at all about a
10775    /// registration the service manager could describe perfectly well.
10776    ///
10777    /// Ownership cannot be reproduced without privileges; the mode that made it
10778    /// fatal can.
10779    #[cfg(unix)]
10780    #[test]
10781    fn status_reports_a_record_it_may_not_read_and_still_reports_the_registration() {
10782        use std::os::unix::fs::PermissionsExt as _;
10783
10784        // `root` is not subject to the mode bits and would read the record
10785        // regardless, leaving the assertions below testing nothing.
10786        // SAFETY: `geteuid` takes no argument and cannot fail.
10787        if unsafe { libc::geteuid() } == 0 {
10788            return;
10789        }
10790
10791        let host = Host::new();
10792        let operations = host.operations();
10793        operations
10794            .install(&host.request(StartMode::Boot))
10795            .expect("an install");
10796        assert!(
10797            operations.status().expect("a status").is_healthy(),
10798            "the discriminator: healthy before the record is made unreadable"
10799        );
10800
10801        let record = InstallRecord::path(&host.paths);
10802        std::fs::set_permissions(&record, std::fs::Permissions::from_mode(0o000))
10803            .expect("the mode is applied");
10804
10805        let status = operations
10806            .status()
10807            .expect("a record this account may not read is reported, not thrown");
10808        assert!(status.is_installed(), "{status}");
10809        assert!(!status.is_healthy(), "{status}");
10810
10811        let printed = status.to_string();
10812        assert!(
10813            printed.contains("this account may not read it"),
10814            "the operator is told which of the two states this is: {printed}"
10815        );
10816        assert!(
10817            !printed.contains("there is no install record"),
10818            "a record that is there and unreadable is not a record that is missing, and the \
10819             missing one's remedy starts with `service uninstall`: {printed}"
10820        );
10821
10822        // Left readable so the fixture's own cleanup is not the thing that
10823        // fails on a host where TempDir removal walks the tree.
10824        std::fs::set_permissions(&record, std::fs::Permissions::from_mode(0o644))
10825            .expect("the mode is restored");
10826    }
10827
10828    #[test]
10829    fn status_reports_a_stale_binary_as_an_error_rather_than_appearing_healthy() {
10830        let host = Host::new();
10831        let operations = host.operations();
10832        operations
10833            .install(&host.request(StartMode::Boot))
10834            .expect("an install");
10835
10836        // The discriminator: healthy first, so a `is_healthy` that always
10837        // returned false could not pass this test.
10838        assert!(
10839            operations.status().expect("a status").is_healthy(),
10840            "the freshly installed host must be healthy"
10841        );
10842
10843        std::fs::remove_file(&host.binary).expect("the binary moves out from under the record");
10844
10845        let status = operations.status().expect("a status");
10846        assert!(!status.is_healthy(), "{status}");
10847        assert!(
10848            status
10849                .problems()
10850                .iter()
10851                .any(|problem| problem.subject == "binary"),
10852            "{status}"
10853        );
10854        assert!(status.to_string().contains("STALE"), "{status}");
10855    }
10856
10857    #[test]
10858    fn status_reports_a_registration_that_would_not_start_at_boot() {
10859        let host = Host::new();
10860        let operations = host.operations();
10861        operations
10862            .install(&host.request(StartMode::Boot))
10863            .expect("an install");
10864        assert!(operations.status().expect("a status").is_healthy());
10865
10866        host.controls.edit("runner-manager", |registration| {
10867            registration.starts_automatically = false;
10868        });
10869
10870        let status = operations.status().expect("a status");
10871        assert!(!status.is_healthy(), "{status}");
10872        assert!(
10873            status
10874                .problems()
10875                .iter()
10876                .any(|problem| problem.detail.contains("after a reboot")),
10877            "{status}"
10878        );
10879    }
10880
10881    #[test]
10882    fn install_starts_the_registration_and_a_later_stop_is_unhealthy() {
10883        let host = Host::new();
10884        let operations = host.operations();
10885        operations
10886            .install(&host.request(StartMode::Login))
10887            .expect("install and immediate start");
10888
10889        assert!(
10890            operations.status().expect("running status").is_running(),
10891            "install must not wait for the next login trigger"
10892        );
10893        operations.stop().expect("stop the registration");
10894        let status = operations.status().expect("stopped status");
10895        assert!(!status.is_healthy(), "{status}");
10896        assert!(
10897            status
10898                .problems()
10899                .iter()
10900                .any(|problem| problem.subject == "runtime"),
10901            "{status}"
10902        );
10903        let rendered = status.to_string();
10904        assert!(
10905            rendered.contains("runner-manager service install --start-at login"),
10906            "{rendered}"
10907        );
10908        assert!(
10909            !rendered.contains("runner-manager service start"),
10910            "{rendered}"
10911        );
10912    }
10913
10914    #[test]
10915    fn status_reports_a_restart_policy_something_else_edited() {
10916        let host = Host::new();
10917        let operations = host.operations();
10918        operations
10919            .install(&host.request(StartMode::Boot))
10920            .expect("an install");
10921        assert!(operations.status().expect("a status").is_healthy());
10922
10923        host.controls.edit("runner-manager", |registration| {
10924            registration.restart_delay = Some(Duration::from_secs(1));
10925        });
10926
10927        let status = operations.status().expect("a status");
10928        assert!(!status.is_healthy(), "{status}");
10929        assert!(
10930            status
10931                .problems()
10932                .iter()
10933                .any(|problem| problem.subject == "restart policy"),
10934            "{status}"
10935        );
10936    }
10937
10938    #[test]
10939    fn status_reports_a_registration_naming_a_binary_the_record_does_not() {
10940        let host = Host::new();
10941        let operations = host.operations();
10942        operations
10943            .install(&host.request(StartMode::Boot))
10944            .expect("an install");
10945        let other = host.binary.with_file_name("someone-elses.exe");
10946        std::fs::write(&other, b"x").expect("writable");
10947
10948        host.controls.edit("runner-manager", |registration| {
10949            registration.command_line = quote_argument(&other.to_string_lossy());
10950        });
10951
10952        let status = operations.status().expect("a status");
10953        assert!(!status.is_healthy(), "{status}");
10954        assert!(
10955            matches!(status.binary(), Some(BinaryPath::Diverged { .. })),
10956            "{status}"
10957        );
10958    }
10959
10960    #[test]
10961    fn status_reports_a_record_no_service_manager_knows_about() {
10962        let host = Host::new();
10963        let operations = host.operations();
10964        operations
10965            .install(&host.request(StartMode::Boot))
10966            .expect("an install");
10967        // Something removed the registration behind this product's back.
10968        for mode in [StartMode::Boot, StartMode::Login] {
10969            host.controls
10970                .control(mode)
10971                .expect("a control")
10972                .uninstall(&ServiceIdentity::product())
10973                .expect("removed");
10974        }
10975
10976        let status = operations.status().expect("a status");
10977        assert!(!status.is_healthy(), "{status}");
10978        assert!(
10979            status
10980                .problems()
10981                .iter()
10982                .any(|problem| problem.subject == "registration"),
10983            "{status}"
10984        );
10985    }
10986
10987    #[test]
10988    fn status_on_a_host_with_nothing_installed_is_neither_healthy_nor_broken() {
10989        let host = Host::new();
10990        let status = host.operations().status().expect("a status");
10991        assert!(!status.is_installed());
10992        assert!(
10993            status.is_healthy(),
10994            "a host that never installed the service has no fault to report: {status}"
10995        );
10996        assert!(status.to_string().contains("installed"), "{status}");
10997    }
10998
10999    #[test]
11000    fn status_says_a_login_registration_does_not_resume_after_an_unattended_reboot() {
11001        let host = Host::new();
11002        let operations = host.operations();
11003        operations
11004            .install(&host.request(StartMode::Login))
11005            .expect("an install at login");
11006        let status = operations.status().expect("a status");
11007        assert!(
11008            status
11009                .notes()
11010                .iter()
11011                .any(|note| note.contains("does not run until the operator signs in")),
11012            "05-infrastructure.md requires `service status` to say so: {status}"
11013        );
11014    }
11015
11016    /// A Mac that signs its runner account in automatically does resume, and
11017    /// `service status` used to tell it otherwise.
11018    #[test]
11019    fn a_mac_with_automatic_login_as_the_service_account_resumes_after_a_reboot() {
11020        use crate::unattended_login::{AutoLogin, UnattendedLogin};
11021        let automatic = UnattendedLogin {
11022            auto_login: AutoLogin::As("ivan".into()),
11023            filevault_on: Some(false),
11024        };
11025        assert_eq!(
11026            unattended_gap(StartMode::Login, "ivan", Some(&automatic)),
11027            None
11028        );
11029
11030        let off = UnattendedLogin {
11031            auto_login: AutoLogin::Off,
11032            filevault_on: Some(false),
11033        };
11034        let gap = unattended_gap(StartMode::Login, "ivan", Some(&off)).expect("a gap");
11035        assert!(gap.reason.contains("automatic login is off"), "{gap:?}");
11036        assert!(gap.reason.contains("Automatically log in as"), "{gap:?}");
11037
11038        let locked = UnattendedLogin {
11039            auto_login: AutoLogin::As("ivan".into()),
11040            filevault_on: Some(true),
11041        };
11042        let gap = unattended_gap(StartMode::Boot, "root", Some(&locked)).expect("a gap");
11043        assert!(gap.reason.contains("FileVault is on"), "{gap:?}");
11044
11045        // An unanswered probe is no gap: `host doctor` says what it could not
11046        // read.
11047        let unread = UnattendedLogin {
11048            auto_login: AutoLogin::As("ivan".into()),
11049            filevault_on: None,
11050        };
11051        assert_eq!(
11052            unattended_gap(StartMode::Login, "ivan", Some(&unread)),
11053            None
11054        );
11055
11056        // Where nothing can be probed, a login registration still says so.
11057        let gap = unattended_gap(StartMode::Login, "ivan", None).expect("a gap");
11058        assert!(
11059            gap.reason
11060                .contains("does not run until the operator signs in")
11061        );
11062        assert_eq!(unattended_gap(StartMode::Boot, "root", None), None);
11063    }
11064
11065    /// A daemon blocked in a file system call is still "running" to the
11066    /// service manager; its heartbeat is what says it is stuck.
11067    #[test]
11068    fn a_daemon_that_stopped_beating_makes_the_service_unhealthy() {
11069        let host = Host::new();
11070        let operations = host.operations();
11071        operations
11072            .install(&host.request(StartMode::Login))
11073            .expect("an install at login");
11074        let stalled_problem = |operations: &ServiceOperations| {
11075            operations
11076                .status()
11077                .expect("a status")
11078                .problems()
11079                .iter()
11080                .find(|problem| problem.subject == DAEMON_STALLED_SUBJECT)
11081                .map(|problem| problem.detail.clone())
11082        };
11083        crate::daemon_heartbeat::beat(&operations.paths, Utc::now()).unwrap();
11084        assert_eq!(stalled_problem(&operations), None, "beating");
11085
11086        crate::daemon_heartbeat::beat(
11087            &operations.paths,
11088            Utc::now() - chrono::Duration::minutes(20),
11089        )
11090        .unwrap();
11091        let detail = stalled_problem(&operations).expect("a stall is a problem");
11092        assert!(detail.contains("has made no progress since"), "{detail}");
11093        assert!(detail.contains("runner-manager service stop"), "{detail}");
11094    }
11095
11096    /// A daemon refusing every runner is not a healthy service.
11097    #[test]
11098    fn a_daemon_that_finds_the_host_unfit_makes_the_service_unhealthy() {
11099        let host = Host::new();
11100        let operations = host.operations();
11101        operations
11102            .install(&host.request(StartMode::Login))
11103            .expect("an install at login");
11104        let paths = operations.paths.clone();
11105        crate::host_fitness::record_host_unfit(
11106            &paths,
11107            &[crate::host_fitness::UnfitFinding {
11108                id: "host.runner_root_responsive".into(),
11109                detail: "macOS has not let the service list /Volumes/NVME/runners".into(),
11110                remedy: Some("click Allow".into()),
11111            }],
11112            Utc::now(),
11113        )
11114        .unwrap();
11115        let status = operations.status().expect("a status");
11116        assert!(!status.is_healthy(), "{status}");
11117        let problem = status
11118            .problems()
11119            .iter()
11120            .find(|problem| problem.subject == HOST_UNFIT_SUBJECT)
11121            .expect("the refusal is a problem");
11122        assert!(
11123            problem.detail.contains("/Volumes/NVME/runners"),
11124            "{problem:?}"
11125        );
11126        assert!(problem.detail.contains("Fix: click Allow"), "{problem:?}");
11127
11128        // A record nobody has re-stamped for a while is a stopped daemon's,
11129        // not a refusal in force.
11130        crate::host_fitness::clear_host_unfit(&paths).unwrap();
11131        crate::host_fitness::record_host_unfit(
11132            &paths,
11133            &[],
11134            Utc::now() - chrono::Duration::hours(2),
11135        )
11136        .unwrap();
11137        let status = operations.status().expect("a status");
11138        assert!(
11139            !status
11140                .problems()
11141                .iter()
11142                .any(|problem| problem.subject == HOST_UNFIT_SUBJECT),
11143            "{status}"
11144        );
11145    }
11146
11147    // -----------------------------------------------------------------------
11148    // Start and stop
11149    // -----------------------------------------------------------------------
11150
11151    #[test]
11152    fn start_and_stop_reach_the_domain_that_holds_the_registration() {
11153        let host = Host::new();
11154        let operations = host.operations();
11155        operations
11156            .install(&host.request(StartMode::Login))
11157            .expect("an install at login");
11158        operations.start().expect("a start");
11159        assert!(operations.status().expect("a status").is_running());
11160        assert!(operations.stop().expect("a stop"));
11161        assert!(!operations.status().expect("a status").is_running());
11162    }
11163
11164    #[test]
11165    fn starting_a_host_with_no_registration_is_refused() {
11166        let host = Host::new();
11167        let error = host.operations().start().expect_err("nothing to start");
11168        assert!(
11169            matches!(error, ServiceError::NotInstalled { .. }),
11170            "{error}"
11171        );
11172    }
11173
11174    // -----------------------------------------------------------------------
11175    // The seam with `d2`
11176    // -----------------------------------------------------------------------
11177
11178    /// Requirement 2 is *"an account that can **read the secret store**"*, and
11179    /// on Windows what an account may read is decided by a DACL in a file this
11180    /// task does not own. Asserting the account name alone would be asserting
11181    /// this module against itself; this reads `d2`'s protection back off a real
11182    /// store and checks that it admits the account this installer registers.
11183    ///
11184    /// No privileges are needed. `d2` documents that a rooted machine-scoped
11185    /// store *"is protected and encrypted exactly as the standard one is"* —
11186    /// the Windows backend picks its DACL from the scope, not from the site —
11187    /// so a store in a temporary directory carries the access control the real
11188    /// one does.
11189    #[cfg(windows)]
11190    #[test]
11191    fn the_account_this_installer_registers_is_one_the_stores_own_dacl_admits() {
11192        use crate::secrets::{PlatformSecretStore, SecretScope, SecretStore as _};
11193
11194        let root = tempfile::tempdir().expect("a temporary directory");
11195        let store = PlatformSecretStore::rooted_at(SecretScope::Machine, root.path())
11196            .expect("a rooted machine-scoped store");
11197        store
11198            .store(&secrecy::SecretString::from("a stand-in for the token"))
11199            .expect("the store accepts a value");
11200        let protection = store.protection().expect("the DACL can be read back");
11201
11202        assert!(
11203            protection.description().contains(";;;SY)"),
11204            "the machine-scoped store must admit LocalSystem, or a boot-start service cannot \
11205             read the token. `d2` writes this DACL and it is not this task's to widen. Got: {}",
11206            protection.description()
11207        );
11208        assert_eq!(
11209            ServiceAccount::for_definition(DefinitionKind::WindowsService, StartMode::Boot),
11210            ServiceAccount::LocalSystem,
11211            "and that is the account this installer registers, which is why SY is what matters"
11212        );
11213        assert!(
11214            !protection.readable_by_other_local_users(),
11215            "the same DACL must still exclude ordinary local users: {}",
11216            protection.description()
11217        );
11218
11219        // The other half of the analysis in `docs/service-account.md`, and the
11220        // reason LocalService and NetworkService are not used: the DACL names
11221        // three trustees and neither of them is among them.
11222        for rejected in [";;;LS)", ";;;NS)"] {
11223            assert!(
11224                !protection.description().contains(rejected),
11225                "if the store ever admitted {rejected}, the least-privilege analysis in \
11226                 docs/service-account.md would need redoing: {}",
11227                protection.description()
11228            );
11229        }
11230    }
11231
11232    // -- another installation's service ---------------------------------------
11233
11234    /// A request that records `host`'s own four directories on the command
11235    /// line, the way `service install` registers one.
11236    fn request_with_directories(host: &Host, mode: StartMode) -> InstallRequest {
11237        let mut arguments: Vec<OsString> = DAEMON_ARGUMENTS.iter().map(OsString::from).collect();
11238        arguments.extend(ServiceDirectories::of(&host.paths).arguments());
11239        host.request(mode).with_arguments(arguments)
11240    }
11241
11242    /// The operations a second account on the same machine gets: its own
11243    /// directories, and the machine's one service manager.
11244    fn second_account(account: &Host, machine: &Host) -> ServiceOperations {
11245        ServiceOperations::with_controls(
11246            account.paths.clone(),
11247            ServiceIdentity::product(),
11248            std::sync::Arc::new(machine.controls.clone()),
11249        )
11250        .with_runner_root(account.runner_root.clone())
11251    }
11252
11253    /// The owner's boot service, and a second account holding the record of
11254    /// an earlier login install of its own -- the layout an owner reported on
11255    /// a Windows host with two local accounts.
11256    fn a_machine_with_a_stale_second_account() -> (Host, Host) {
11257        let owner = Host::new();
11258        owner
11259            .operations()
11260            .install(&request_with_directories(&owner, StartMode::Boot))
11261            .expect("the owner's boot install");
11262        let earlier = Host::new();
11263        earlier
11264            .operations()
11265            .install(&request_with_directories(&earlier, StartMode::Login))
11266            .expect("the second account's earlier login install");
11267        assert!(
11268            InstallRecord::read(&earlier.paths).unwrap().is_some(),
11269            "the earlier install left its record behind"
11270        );
11271        (owner, earlier)
11272    }
11273
11274    /// The advice that, followed from the second account, breaks or replaces
11275    /// the owner's healthy service.
11276    const DESTRUCTIVE_ADVICE: [&str; 4] = [
11277        "service uninstall",
11278        "service install`",
11279        "service install --",
11280        "auth login",
11281    ];
11282
11283    #[test]
11284    fn a_service_installed_from_another_data_root_is_reported_as_that_installations() {
11285        let (owner, earlier) = a_machine_with_a_stale_second_account();
11286
11287        let status = second_account(&earlier, &owner).status().expect("a status");
11288
11289        let foreign = status.foreign().expect("the service is the owner's");
11290        assert_eq!(foreign.directories, ServiceDirectories::of(&owner.paths));
11291        assert_eq!(foreign.root, ServiceDirectories::of(&owner.paths).root());
11292        assert!(foreign.unreadable.is_none(), "{status}");
11293        assert!(
11294            status.is_healthy(),
11295            "the owner's service is healthy, and a stale record elsewhere is not its fault: \
11296             {status}"
11297        );
11298        for subject in [
11299            "binary",
11300            "start mode",
11301            "secret store",
11302            "record",
11303            "registration",
11304        ] {
11305            assert!(
11306                !status
11307                    .problems()
11308                    .iter()
11309                    .any(|problem| problem.subject == subject),
11310                "{subject} compared the second account's record against the owner's \
11311                 registration: {status}"
11312            );
11313        }
11314        let rendered = status.to_string();
11315        for advice in DESTRUCTIVE_ADVICE {
11316            assert!(!rendered.contains(advice), "{advice}: {rendered}");
11317        }
11318        assert!(rendered.contains("another installation's"), "{rendered}");
11319        assert!(
11320            status
11321                .notes()
11322                .iter()
11323                .any(|note| note.contains("unused local configuration")
11324                    && note.contains("host forget-local")),
11325            "{status}"
11326        );
11327        // What the owner recorded, not what the second account did.
11328        assert_eq!(status.start_mode(), Some(StartMode::Boot));
11329    }
11330
11331    #[test]
11332    fn a_foreign_service_whose_records_cannot_be_read_reports_only_what_the_manager_says() {
11333        let (owner, earlier) = a_machine_with_a_stale_second_account();
11334        // Stands in for a profile this account may not open: there is nothing
11335        // to read where the registration points.
11336        InstallRecord::remove(&owner.paths).expect("the owner's record removed");
11337
11338        let operations = second_account(&earlier, &owner);
11339        let status = operations.status().expect("a status");
11340        let foreign = status.foreign().expect("the service is the owner's");
11341        assert!(foreign.unreadable.is_some(), "{status}");
11342        assert!(
11343            status.is_healthy(),
11344            "running, and nothing it could judge: {status}"
11345        );
11346        let rendered = status.to_string();
11347        for advice in DESTRUCTIVE_ADVICE {
11348            assert!(!rendered.contains(advice), "{advice}: {rendered}");
11349        }
11350        assert!(rendered.contains("running; installed from"), "{rendered}");
11351        assert!(!rendered.contains("last GitHub contact"), "{rendered}");
11352
11353        // Stopped, it is a problem -- with a place to look, not a reinstall.
11354        owner.operations().stop().expect("stopped");
11355        let stopped = operations.status().expect("a status");
11356        assert!(!stopped.is_healthy(), "{stopped}");
11357        let rendered = stopped.to_string();
11358        for advice in DESTRUCTIVE_ADVICE {
11359            assert!(!rendered.contains(advice), "{advice}: {rendered}");
11360        }
11361    }
11362
11363    #[test]
11364    fn this_accounts_own_registration_still_reports_every_mismatch() {
11365        let host = Host::new();
11366        let operations = host.operations();
11367        operations
11368            .install(&request_with_directories(&host, StartMode::Boot))
11369            .expect("an install");
11370        let other = host.binary.with_file_name("someone-elses");
11371        std::fs::write(&other, b"x").expect("writable");
11372        // Something edited the registration, keeping its directories.
11373        host.controls.edit("runner-manager", |registration| {
11374            let mut arguments = split_command_line(&registration.command_line);
11375            arguments[0] = other.to_string_lossy().into_owned();
11376            registration.command_line = arguments
11377                .iter()
11378                .map(|argument| quote_argument(argument))
11379                .collect::<Vec<_>>()
11380                .join(" ");
11381            registration.start_mode = StartMode::Login;
11382        });
11383
11384        let status = operations.status().expect("a status");
11385        assert!(status.foreign().is_none(), "{status}");
11386        assert!(
11387            matches!(status.binary(), Some(BinaryPath::Diverged { .. })),
11388            "{status}"
11389        );
11390        assert!(
11391            status
11392                .problems()
11393                .iter()
11394                .any(|problem| problem.subject == "start mode"),
11395            "{status}"
11396        );
11397        assert!(status.to_string().contains("service uninstall"), "{status}");
11398    }
11399
11400    #[test]
11401    fn directories_read_back_from_a_command_line_with_spaces() {
11402        let root = Path::new("/Users/someone/Library/Application Support/runner manager");
11403        let directories = ServiceDirectories {
11404            config: root.join("config"),
11405            state: root.join("data").join("state"),
11406            runtime: root.join("data").join("runtime"),
11407            logs: root.join("data").join("logs"),
11408        };
11409        let mut command_line = quote_argument("/usr/local/bin/runner-manager");
11410        for argument in ["daemon", "run"]
11411            .map(OsString::from)
11412            .into_iter()
11413            .chain(directories.arguments())
11414        {
11415            command_line.push(' ');
11416            command_line.push_str(&quote_argument(&argument.to_string_lossy()));
11417        }
11418        assert_eq!(
11419            ServiceDirectories::from_command_line(&command_line),
11420            Some(directories.clone())
11421        );
11422        assert_eq!(directories.root(), root);
11423        assert_eq!(
11424            ServiceDirectories::from_command_line("runner-manager daemon run"),
11425            None,
11426            "a registration that names no directories cannot be judged foreign"
11427        );
11428    }
11429
11430    #[test]
11431    fn the_owner_of_a_data_root_is_read_from_its_path() {
11432        let users = Path::new(if cfg!(windows) { r"C:\Users" } else { "/home" });
11433        let home = users.join("ivand");
11434        assert_eq!(
11435            data_root_owner(&users.join("yuriv").join("data"), Some(&home)).as_deref(),
11436            Some("yuriv")
11437        );
11438        assert_eq!(
11439            data_root_owner(
11440                Path::new(if cfg!(windows) { r"D:\rm" } else { "/opt/rm" }),
11441                Some(&home)
11442            ),
11443            None
11444        );
11445        if cfg!(windows) {
11446            assert_eq!(
11447                data_root_owner(
11448                    Path::new(r"C:\Windows\System32\config\systemprofile\AppData"),
11449                    Some(&home)
11450                )
11451                .as_deref(),
11452                Some("LocalSystem")
11453            );
11454        } else {
11455            assert_eq!(
11456                data_root_owner(Path::new("/root/.config/runner-manager"), Some(&home)).as_deref(),
11457                Some("root")
11458            );
11459        }
11460    }
11461
11462    #[test]
11463    fn forget_local_archives_the_unused_record_and_never_touches_the_service_root() {
11464        let (owner, earlier) = a_machine_with_a_stale_second_account();
11465        let owner_before = snapshot(&[owner._root.path()]);
11466        let config = earlier.paths.config_dir().to_path_buf();
11467        let record = std::fs::read(InstallRecord::path(&earlier.paths)).unwrap();
11468
11469        let forgotten = second_account(&earlier, &owner)
11470            .forget_local(Utc::now())
11471            .expect("archived");
11472
11473        assert_eq!(
11474            snapshot(&[owner._root.path()]),
11475            owner_before,
11476            "the owner's directories must not change by one byte"
11477        );
11478        let archive = forgotten.archive.expect("something was archived");
11479        assert!(archive.starts_with(&config), "{}", archive.display());
11480        assert_eq!(
11481            std::fs::read(archive.join(RECORD_FILE)).unwrap(),
11482            record,
11483            "moved, not rewritten or deleted"
11484        );
11485        assert!(InstallRecord::read(&earlier.paths).unwrap().is_none());
11486        assert_eq!(
11487            forgotten.service_root,
11488            Some(ServiceDirectories::of(&owner.paths).root())
11489        );
11490        // Afterwards the second account's status has no unused record to note.
11491        let status = second_account(&earlier, &owner).status().expect("status");
11492        assert!(
11493            !status
11494                .notes()
11495                .iter()
11496                .any(|note| note.contains("unused local configuration")),
11497            "{status}"
11498        );
11499    }
11500
11501    #[test]
11502    fn forget_local_refuses_a_configuration_a_service_runs_against() {
11503        let (owner, earlier) = a_machine_with_a_stale_second_account();
11504        let before = snapshot(&[owner._root.path()]);
11505        let refused = owner.operations().forget_local(Utc::now());
11506        assert!(
11507            matches!(refused, Err(ServiceError::LocalConfigurationInUse { .. })),
11508            "{refused:?}"
11509        );
11510        assert_eq!(snapshot(&[owner._root.path()]), before);
11511
11512        // Nor while an agent holds the second account's own lock.
11513        let _agent = HostLock::try_acquire(&earlier.paths, LockKind::SingleInstance)
11514            .expect("the agent's lock");
11515        let refused = second_account(&earlier, &owner).forget_local(Utc::now());
11516        assert!(
11517            matches!(refused, Err(ServiceError::LocalConfigurationInUse { .. })),
11518            "{refused:?}"
11519        );
11520        assert!(InstallRecord::read(&earlier.paths).unwrap().is_some());
11521    }
11522
11523    #[test]
11524    fn a_readable_foreign_service_keeps_its_diagnoses_and_loses_its_remedies() {
11525        let (owner, earlier) = a_machine_with_a_stale_second_account();
11526        // The owner's service has every problem whose remedy acts on the
11527        // directories of whoever runs it.
11528        owner.operations().stop().expect("stopped");
11529        record_github_credential_rejected(&owner.paths, Utc::now()).expect("rejected");
11530        record_credential_unreadable(&owner.paths, Utc::now()).expect("unreadable");
11531
11532        let status = second_account(&earlier, &owner).status().expect("a status");
11533        assert!(status.foreign().is_some_and(|foreign| !foreign.is_unread()));
11534        let subjects: Vec<_> = status.problems().iter().map(|p| p.subject).collect();
11535        for subject in [
11536            "runtime",
11537            "github credential",
11538            CREDENTIAL_UNREADABLE_SUBJECT,
11539        ] {
11540            assert!(subjects.contains(&subject), "{subject}: {status}");
11541        }
11542        let rendered = status.to_string();
11543        for advice in DESTRUCTIVE_ADVICE {
11544            assert!(!rendered.contains(advice), "{advice}: {rendered}");
11545        }
11546        for problem in status.problems() {
11547            assert!(problem.detail.contains("Act on it"), "{problem}");
11548        }
11549    }
11550
11551    #[test]
11552    fn no_command_of_this_account_changes_another_installations_service() {
11553        let (owner, earlier) = a_machine_with_a_stale_second_account();
11554        let operations = second_account(&earlier, &owner);
11555        let foreign = |result: Result<(), ServiceError>| {
11556            assert!(
11557                matches!(result, Err(ServiceError::ForeignInstallation { .. })),
11558                "{result:?}"
11559            );
11560        };
11561        foreign(
11562            operations
11563                .install(&request_with_directories(&earlier, StartMode::Boot))
11564                .map(drop),
11565        );
11566        foreign(operations.uninstall().map(drop));
11567        foreign(operations.set_start_mode(StartMode::Boot).map(drop));
11568        foreign(operations.start());
11569        foreign(operations.stop().map(drop));
11570
11571        let registration = owner
11572            .controls
11573            .control(StartMode::Boot)
11574            .unwrap()
11575            .query(&ServiceIdentity::product())
11576            .unwrap()
11577            .expect("the owner's registration is still there");
11578        assert_eq!(
11579            ServiceDirectories::from_command_line(&registration.command_line),
11580            Some(ServiceDirectories::of(&owner.paths)),
11581            "and still runs against the owner's directories"
11582        );
11583        assert!(registration.running, "and was never stopped");
11584        assert!(
11585            InstallRecord::read(&earlier.paths).unwrap().is_some(),
11586            "the refused uninstall kept this account's record"
11587        );
11588    }
11589
11590    #[test]
11591    fn forget_local_leaves_agent_directories_nested_in_the_configuration_alone() {
11592        // macOS puts state, runtime and logs inside the configuration
11593        // directory: `config_local_dir` and `data_local_dir` are one folder.
11594        let root = tempfile::tempdir().unwrap();
11595        let config = root.path().join("Application Support");
11596        let paths = AppPaths::from_directories(
11597            &config,
11598            config.join("state"),
11599            config.join("runtime"),
11600            config.join("logs"),
11601        );
11602        paths.create_all().unwrap();
11603        std::fs::write(config.join(RECORD_FILE), b"an earlier install").unwrap();
11604        std::fs::write(paths.logs_dir().join("runner-manager.log"), b"log").unwrap();
11605        let operations = ServiceOperations::with_controls(
11606            paths.clone(),
11607            ServiceIdentity::product(),
11608            std::sync::Arc::new(RecordingControls::new()),
11609        );
11610
11611        let forgotten = operations.forget_local(Utc::now()).expect("archived");
11612
11613        assert_eq!(forgotten.moved, vec![config.join(RECORD_FILE)]);
11614        for kept in [paths.state_dir(), paths.runtime_dir(), paths.logs_dir()] {
11615            assert!(kept.is_dir(), "{} was moved", kept.display());
11616        }
11617        assert!(paths.logs_dir().join("runner-manager.log").is_file());
11618    }
11619
11620    #[test]
11621    fn a_remedy_is_cut_from_every_detail_a_builder_writes() {
11622        for (detail, diagnosis) in [
11623            (
11624                "x holds an automatic registration, but the daemon is stopped. Run `runner-manager \
11625                 service install --start-at boot`; if the service manager denies replacement",
11626                "x holds an automatic registration, but the daemon is stopped",
11627            ),
11628            (
11629                "store -- MISMATCH. Run `auth login` again so the token is stored",
11630                "store -- MISMATCH",
11631            ),
11632            (
11633                "unit differs, so the daemon runs under outdated settings. Re-register it with \
11634                 `sudo runner-manager service install --start-at boot`",
11635                "unit differs, so the daemon runs under outdated settings",
11636            ),
11637            (
11638                "it starts no runner; the log has it. Sign in again: `runner-manager auth login`",
11639                "it starts no runner; the log has it",
11640            ),
11641            (
11642                "C:\\a\\runner-manager.exe -- STALE: moved. Run `service uninstall` and `service \
11643                 install`",
11644                "C:\\a\\runner-manager.exe -- STALE: moved",
11645            ),
11646            (
11647                "a new builder: run `runner-manager service install` here, or not",
11648                "the installation's own records report this",
11649            ),
11650        ] {
11651            assert_eq!(diagnosis_only(detail), diagnosis, "{detail}");
11652        }
11653    }
11654}