Skip to main content

runner_manager_github/
lib.rs

1// owner: c2-device-flow-auth
2//
3// c2 also owns the shared authenticated HTTP client that lives at this crate
4// root; c3 owns `rest`, and c4 owns `demand` and `jit`.
5
6//! The GitHub gateway.
7//!
8//! This crate holds every line of code in the product that talks to GitHub, and
9//! the crate root holds the one client all of it goes through.
10//!
11//! * [`device_flow`] — the OAuth 2.0 Device Authorization Grant, which is the
12//!   *only* way this product ever obtains a credential (D3, D16).
13//! * [`AuthenticatedClient`] — the shared `api.github.com` client. Every request
14//!   in this crate is built by it, which is what makes "sets
15//!   `X-GitHub-Api-Version` and an explicit `Accept`" a property of the design
16//!   rather than of each call site, and what lets the authentication-failure
17//!   taxonomy be implemented exactly once.
18//! * [`rest`], [`demand`], [`jit`] — typed adapters owned by `c3` and `c4`,
19//!   built on [`AuthenticatedClient`].
20//!
21//! # Three properties this crate is required to keep
22//!
23//! **It holds no client secret, and it renews without one.** This paragraph
24//! used to say the opposite — that the App opts out of user-token expiration,
25//! that no renewal token is ever issued, and that renewing would require a
26//! client secret a public client cannot hold. All three were wrong, and a test
27//! in this crate enforced the error by forbidding the word "refresh token".
28//!
29//! The published App has user-token expiration **on**: a device-flow exchange
30//! returns `expires_in: 28800` and a `refresh_token`. GitHub requires the
31//! client secret to refresh *"unless the user access token was generated using
32//! the device flow"*, and every one of this product's is. So the credential
33//! renews itself, no server appears in the design, and the eight-hour life of
34//! an access token is invisible to a daemon that runs for months. See
35//! [`AuthenticatedClient::renew_once`], and
36//! `docs/spikes/token-expiry-and-renewal.md` for the two renewals that
37//! confirmed it on real hosts.
38//!
39//! [`AuthenticatedClient::revalidate`] is what still happens on a `401` for a
40//! credential with no refresh half — every one issued before 0.1.11.
41//!
42//! **It persists nothing.** [`device_flow::DeviceFlow::complete`] *returns* the
43//! token; it never writes it anywhere. The machine-scoped secret store is `d2`
44//! and the wiring is `f1`. That boundary is why this crate has no dependency on
45//! `runner-manager-platform` and performs no filesystem write outside its own
46//! tests — and it is what lets the whole gateway be tested with no platform
47//! dependency at all.
48//!
49//! **It never renders a secret.** The device code, the user access token, and
50//! every header carrying either are absent from `Debug`, from `Display`, from
51//! errors, and from tracing output. Every type here that holds one wraps it in
52//! [`secrecy::SecretString`] *and* implements [`fmt::Debug`] by hand, because a
53//! `#[derive(Debug)]` added later to a struct with a plain `String` field is
54//! precisely how this control is lost. `tests/no_secret_reaches_the_logs.rs`
55//! drives a whole login and an authenticated round trip through a capturing
56//! `tracing` subscriber and fails if any of the three appears.
57//!
58//! That scan is a **separate test binary**, and deliberately so. As a unit test
59//! it silently stopped working: `tracing` caches each callsite's `Interest`
60//! process-wide while `with_default` installs a subscriber on one *thread*, and
61//! run **concurrently** with the crate's other unit tests the scan captured
62//! only its own handful of events — passing with a real device-code leak on the
63//! live path. A binary holding one test has no concurrency to be poisoned by.
64//! The word "concurrently" is load-bearing and was measured;
65//! `tests/no_secret_reaches_the_logs.rs` records the numbers and what they rule
66//! out.
67
68pub mod conditional;
69pub mod demand;
70pub mod device_flow;
71pub mod jit;
72pub mod rest;
73pub mod traffic;
74
75use std::{
76    fmt,
77    sync::{
78        Arc,
79        atomic::{AtomicU64, Ordering},
80    },
81    time::Duration,
82};
83
84use chrono::{DateTime, Utc};
85use reqwest::{Method, StatusCode};
86use runner_manager_domain::model::{Clock, Org, OwnerRepo, Timestamp};
87
88/// Re-exported because [`GithubError::headers`] and [`ApiResponse::headers`]
89/// return one, and a consumer cannot *name* a type it has no path to.
90///
91/// `a1` owns every manifest in this workspace, so a crate outside this one that
92/// wanted to hold a `HeaderMap` from this seam would otherwise need `reqwest`
93/// added to its own dependencies — turning a `c2` seam into an `a1` change, and
94/// putting `reqwest`'s version in two places at once. [`GithubError::retry_after`]
95/// and [`GithubError::rate_limit`] exist precisely so that the common cases need
96/// no path at all; this is for the ones that do.
97///
98/// It is re-exported under its own name rather than an alias so that the type a
99/// consumer imports is the type the signatures already show.
100pub use reqwest::header::HeaderMap;
101use secrecy::{ExposeSecret, SecretString};
102use serde::{Deserialize, Serialize, de::DeserializeOwned};
103use url::Url;
104
105// ---------------------------------------------------------------------------
106// Constants
107// ---------------------------------------------------------------------------
108
109/// The REST API version every request pins.
110///
111/// `04-subsystem-contracts.md`: "All requests set `X-GitHub-Api-Version` and an
112/// explicit `Accept` header." Both spikes ran against this version.
113pub const GITHUB_API_VERSION: &str = "2022-11-28";
114
115/// The media type every request asks for, stated rather than defaulted.
116pub const GITHUB_ACCEPT: &str = "application/vnd.github+json";
117
118/// Production `api.github.com`.
119pub const GITHUB_API_BASE: &str = "https://api.github.com/";
120
121/// Production `github.com`, which hosts the device-flow endpoints. They are on
122/// the web host, not the API host.
123pub const GITHUB_WEB_BASE: &str = "https://github.com/";
124
125/// The canonical page a user types their code into.
126///
127/// This constant *is* the phishing control (`07-security.md`, threat table): the
128/// tool prints this URL and never proxies, embeds, or imitates the approval
129/// page. [`Endpoints::verification_url`] derives from the configured web base so
130/// a test server can be pointed at, and
131/// [`device_flow::DeviceAuthorization::verification_uri`] is checked against it
132/// so a response that tries to send the user somewhere else is rejected rather
133/// than displayed.
134pub const DEVICE_VERIFICATION_PATH: &str = "login/device";
135
136/// What a `401` re-validates the held credential against.
137///
138/// `GET /user/installations` rather than `GET /user`, because it is the call the
139/// D18 spike actually made with a user-to-server token and observed `200` from
140/// (`docs/spikes/d18-org-jit-verification.md`, "The permission that authorized
141/// it"), and because a successful re-validation then carries the same answer
142/// [`AuthenticatedClient::discover_installations`] needs.
143pub const REVALIDATION_PATH: &str = "/user/installations";
144
145/// How long a lockout backs off for when GitHub sends no `retry-after`.
146///
147/// `03-control-flows.md` flow 4.3 requires a back-off but names no duration.
148/// Sixty seconds is GitHub's own documented floor for its secondary rate limits.
149pub const DEFAULT_LOCKOUT_BACKOFF: Duration = Duration::from_secs(60);
150
151/// The longest a lockout may silence this client, whatever `Retry-After` said.
152///
153/// A back-off is a *safety* mechanism, and an unclamped one is a denial of
154/// service with extra steps: `Retry-After: 86400` would latch a silent
155/// twenty-four-hour outage of the agent's reconciliation loop, clearable only by
156/// [`AuthenticatedClient::clear_lockout`]. Fifteen minutes is far longer than any
157/// back-off GitHub documents for the authentication lockout this latches on, and
158/// short enough that a hostile or simply wrong header cannot take the product
159/// down for a shift. Honouring a header without a ceiling is trusting a remote
160/// party with the product's availability.
161pub const MAX_LOCKOUT_BACKOFF: Duration = Duration::from_secs(15 * 60);
162
163/// The most pages either pagination loop follows before giving up.
164///
165/// A `Link: rel="next"` that points back at the page it arrived on — a proxy
166/// rewriting the header, or a bug at the other end — is an infinite loop inside
167/// the agent's reconciliation loop, which is the one place in this product that
168/// must not be able to wedge. At `per_page=100` this ceiling is ten thousand
169/// installations or repositories, past any real account by orders of magnitude,
170/// so it bounds the pathological case without truncating a legitimate one.
171pub const MAX_PAGES: usize = 100;
172
173/// Per-request ceiling, so one wedged connection cannot stall the agent's
174/// reconciliation loop forever.
175pub const DEFAULT_REQUEST_TIMEOUT: Duration = Duration::from_secs(30);
176
177/// The `User-Agent` GitHub requires on every API request.
178pub const USER_AGENT: &str = concat!("runner-manager/", env!("CARGO_PKG_VERSION"));
179
180// ---------------------------------------------------------------------------
181// The published App
182// ---------------------------------------------------------------------------
183
184/// The published GitHub App this product authenticates as (D3, D16).
185///
186/// Both fields are **public by design**. `07-security.md`'s credential inventory
187/// lists the `client_id` as "Not secret … may appear in logs and documentation",
188/// which is exactly what makes the device flow serverless: a public client
189/// cannot secure a client secret, and this design never tries to.
190///
191/// The concrete values are *not* compiled in here. Registering and publishing
192/// the App is Phase 0 of `06-migration-rollout.md` and has not happened, so
193/// there is no honest value to write; `f1` supplies both when it wires the CLI.
194/// Committing a placeholder that looked real would be worse than requiring the
195/// caller to pass one.
196#[derive(Debug, Clone, PartialEq, Eq)]
197pub struct AppRegistration {
198    client_id: String,
199    slug: String,
200}
201
202impl AppRegistration {
203    /// # Errors
204    /// An empty `client_id` or an empty `slug`.
205    pub fn new(client_id: impl Into<String>, slug: impl Into<String>) -> Result<Self, ConfigError> {
206        let client_id = client_id.into();
207        let slug = slug.into();
208        if client_id.trim().is_empty() {
209            return Err(ConfigError::Empty { what: "client_id" });
210        }
211        if slug.trim().is_empty() {
212            return Err(ConfigError::Empty { what: "app slug" });
213        }
214        Ok(Self { client_id, slug })
215    }
216
217    #[must_use]
218    pub fn client_id(&self) -> &str {
219        &self.client_id
220    }
221
222    #[must_use]
223    pub fn slug(&self) -> &str {
224        &self.slug
225    }
226
227    /// The canonical URL a user with no installation must visit.
228    ///
229    /// `03-control-flows.md` flow 1.1: "If the published App is not yet installed
230    /// on any repository, it prints the installation URL."
231    ///
232    /// # Panics
233    /// Never, for a registration built through [`AppRegistration::new`]: the slug
234    /// is non-empty and is percent-encoded into the path.
235    #[must_use]
236    pub fn install_url(&self, endpoints: &Endpoints) -> Url {
237        endpoints
238            .web_base
239            .join("apps/")
240            .and_then(|u| u.join(&format!("{}/", encode_path_segment(&self.slug))))
241            .and_then(|u| u.join("installations/new"))
242            .expect("a non-empty encoded slug always joins onto the web base")
243    }
244}
245
246fn encode_path_segment(raw: &str) -> String {
247    raw.chars()
248        .map(|c| {
249            if c.is_ascii_alphanumeric() || matches!(c, '-' | '_' | '.' | '~') {
250                c.to_string()
251            } else {
252                let mut buf = [0_u8; 4];
253                c.encode_utf8(&mut buf)
254                    .as_bytes()
255                    .iter()
256                    .map(|b| format!("%{b:02X}"))
257                    .collect()
258            }
259        })
260        .collect()
261}
262
263/// Where GitHub is.
264///
265/// Two bases rather than one, because the device grant lives on `github.com`
266/// while every API call lives on `api.github.com`. Tests point both at one
267/// `wiremock` server; the paths do not collide.
268#[derive(Debug, Clone, PartialEq, Eq)]
269pub struct Endpoints {
270    api_base: Url,
271    web_base: Url,
272}
273
274impl Endpoints {
275    /// Production GitHub.
276    ///
277    /// # Panics
278    /// Never: both constants are parsed at every call and are valid URLs.
279    #[must_use]
280    pub fn production() -> Self {
281        Self {
282            api_base: Url::parse(GITHUB_API_BASE).expect("GITHUB_API_BASE is a valid URL"),
283            web_base: Url::parse(GITHUB_WEB_BASE).expect("GITHUB_WEB_BASE is a valid URL"),
284        }
285    }
286
287    /// Both bases are normalised to end in `/` so that relative joins keep the
288    /// whole base path instead of replacing its last segment.
289    #[must_use]
290    pub fn new(api_base: Url, web_base: Url) -> Self {
291        Self {
292            api_base: with_trailing_slash(api_base),
293            web_base: with_trailing_slash(web_base),
294        }
295    }
296
297    /// Point every endpoint at one test server.
298    ///
299    /// # Errors
300    /// `root` not being a parseable absolute URL.
301    pub fn for_test_server(root: &str) -> Result<Self, ConfigError> {
302        let root = Url::parse(root).map_err(|_| ConfigError::Empty {
303            what: "test server URL",
304        })?;
305        Ok(Self::new(root.clone(), root))
306    }
307
308    #[must_use]
309    pub fn api_base(&self) -> &Url {
310        &self.api_base
311    }
312
313    #[must_use]
314    pub fn web_base(&self) -> &Url {
315        &self.web_base
316    }
317
318    /// # Panics
319    /// Never: the path is a constant and the base ends in `/`.
320    #[must_use]
321    pub fn device_code_url(&self) -> Url {
322        self.web_base
323            .join("login/device/code")
324            .expect("a constant path joins onto a normalised base")
325    }
326
327    /// # Panics
328    /// Never: the path is a constant and the base ends in `/`.
329    #[must_use]
330    pub fn access_token_url(&self) -> Url {
331        self.web_base
332            .join("login/oauth/access_token")
333            .expect("a constant path joins onto a normalised base")
334    }
335
336    /// The canonical page the user code is typed into, and the only device-flow
337    /// URL this product ever prints.
338    ///
339    /// # Panics
340    /// Never: the path is a constant and the base ends in `/`.
341    #[must_use]
342    pub fn verification_url(&self) -> Url {
343        self.web_base
344            .join(DEVICE_VERIFICATION_PATH)
345            .expect("a constant path joins onto a normalised base")
346    }
347}
348
349impl Default for Endpoints {
350    fn default() -> Self {
351        Self::production()
352    }
353}
354
355fn with_trailing_slash(mut url: Url) -> Url {
356    if !url.path().ends_with('/') {
357        let path = format!("{}/", url.path());
358        url.set_path(&path);
359    }
360    url
361}
362
363/// A configuration value this crate refuses to start with.
364#[derive(Debug, Clone, PartialEq, Eq, thiserror::Error)]
365pub enum ConfigError {
366    #[error("{what} must not be empty")]
367    Empty { what: &'static str },
368}
369
370// ---------------------------------------------------------------------------
371// The credential
372// ---------------------------------------------------------------------------
373
374/// A user access token obtained from the device flow.
375///
376/// **Non-expiring, and non-renewable.** The published App opts out of
377/// user-token expiration, so GitHub issues no renewal token alongside this one
378/// and there is nothing to renew — see the crate documentation. The token is
379/// invalidated only by the user uninstalling the App or revoking the
380/// authorization at GitHub.
381///
382/// `Debug` is written by hand. Deriving it here would put the token into every
383/// `tracing` field, every `unwrap()` panic message, and every `anyhow` chain
384/// that ever carries one, which is the exact leak `07-security.md` gates on.
385#[derive(Clone)]
386pub struct UserAccessToken {
387    token: SecretString,
388    token_type: String,
389    scope: Option<String>,
390    /// The renewal half, when the App issues one.
391    ///
392    /// # Why this is an `Option` rather than a second type
393    ///
394    /// Whether a credential can renew itself is a setting on the *App*, not a
395    /// property of this build: an App with user-token expiration off returns an
396    /// access token and nothing else, and one with it on returns a pair. Both
397    /// shapes reach this type, and a host holding either must keep working --
398    /// otherwise flipping that setting would strand every installation that had
399    /// not upgraded, which for a published product is an outage nobody asked
400    /// for.
401    ///
402    /// `None` is therefore not a defect. It is the credential this product held
403    /// for its whole life until now: non-expiring, unrenewable, replaced only
404    /// by an interactive `auth login`.
405    renewal: Option<Renewal>,
406}
407
408/// What a token needs in order to replace itself without a person.
409#[derive(Clone)]
410pub struct Renewal {
411    refresh_token: SecretString,
412    /// When the access token stops being accepted, if it was stated.
413    pub access_expires_at: Option<DateTime<Utc>>,
414    /// When the *refresh* token stops working. Past this, only an interactive
415    /// sign-in helps -- and it is six months from the last renewal, not from
416    /// the first sign-in, so a host that runs at all never reaches it.
417    pub refresh_expires_at: Option<DateTime<Utc>>,
418}
419
420impl Renewal {
421    /// The refresh token itself. Kept behind a method for the same reason the
422    /// access token is: it is the more dangerous half of the pair, because it
423    /// mints access tokens indefinitely.
424    #[must_use]
425    pub fn refresh_token(&self) -> &SecretString {
426        &self.refresh_token
427    }
428}
429
430impl fmt::Debug for Renewal {
431    /// Written by hand, like [`UserAccessToken`]'s, and for a stronger reason:
432    /// a leaked refresh token does not expire in eight hours.
433    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
434        f.debug_struct("Renewal")
435            .field("refresh_token", &"[redacted]")
436            .field("access_expires_at", &self.access_expires_at)
437            .field("refresh_expires_at", &self.refresh_expires_at)
438            .finish()
439    }
440}
441
442impl UserAccessToken {
443    #[must_use]
444    pub fn new(token: SecretString) -> Self {
445        Self {
446            token,
447            token_type: "bearer".to_string(),
448            scope: None,
449            renewal: None,
450        }
451    }
452
453    /// The whole credential as the token endpoint returned it. `pub(crate)`
454    /// because [`device_flow`] is the only thing in the product entitled to mint
455    /// one — every other path receives a token rather than constructing it.
456    pub(crate) fn from_parts(
457        token: SecretString,
458        token_type: String,
459        scope: Option<String>,
460    ) -> Self {
461        Self {
462            token,
463            token_type,
464            scope,
465            renewal: None,
466        }
467    }
468
469    /// Attach the renewal half, if the App issued one.
470    ///
471    /// The two durations are seconds-from-now as GitHub states them, turned
472    /// into instants here so that nothing downstream has to remember when
473    /// "now" was.
474    #[must_use]
475    pub(crate) fn with_renewal(
476        mut self,
477        refresh_token: Option<SecretString>,
478        access_expires_in: Option<u64>,
479        refresh_expires_in: Option<u64>,
480    ) -> Self {
481        self.renewal = refresh_token.map(|refresh_token| {
482            let at = |secs: Option<u64>| {
483                secs.and_then(|s| i64::try_from(s).ok())
484                    .and_then(|s| Utc::now().checked_add_signed(chrono::TimeDelta::seconds(s)))
485            };
486            Renewal {
487                refresh_token,
488                access_expires_at: at(access_expires_in),
489                refresh_expires_at: at(refresh_expires_in),
490            }
491        });
492        self
493    }
494
495    /// The renewal half, when there is one.
496    #[must_use]
497    pub fn renewal(&self) -> Option<&Renewal> {
498        self.renewal.as_ref()
499    }
500
501    /// Rebuild the credential `d2` handed back, for `f1`.
502    #[must_use]
503    pub fn from_stored(token: SecretString) -> Self {
504        Self::from_stored_document(&token)
505    }
506
507    /// Reads whichever of the two stored shapes is there.
508    ///
509    /// # Why the store holds a document now, and why the old shape still loads
510    ///
511    /// A renewable credential is three values -- access token, refresh token,
512    /// and when each stops working -- where there used to be one string. The
513    /// secret store takes one opaque value per host, so the document goes
514    /// inside it rather than the store growing a schema: no platform change, no
515    /// migration step, and the same DPAPI blob or keychain item as before.
516    ///
517    /// **A value that is not this document is a bare access token**, which is
518    /// what every host stored until now. That is not a fallback for tidiness:
519    /// upgrading must not log anybody out, and the App's expiration setting can
520    /// be turned on -- or back off -- without stranding hosts that are mid-way
521    /// through either. A token has no internal structure to confuse with JSON,
522    /// so the discrimination is unambiguous.
523    #[must_use]
524    pub fn from_stored_document(stored: &SecretString) -> Self {
525        #[derive(Deserialize)]
526        struct Document {
527            access_token: String,
528            #[serde(default)]
529            refresh_token: Option<String>,
530            #[serde(default)]
531            access_expires_at: Option<DateTime<Utc>>,
532            #[serde(default)]
533            refresh_expires_at: Option<DateTime<Utc>>,
534        }
535
536        match serde_json::from_str::<Document>(stored.expose_secret()) {
537            Ok(document) => Self {
538                token: SecretString::from(document.access_token),
539                token_type: "bearer".to_string(),
540                scope: None,
541                renewal: document.refresh_token.map(|refresh_token| Renewal {
542                    refresh_token: SecretString::from(refresh_token),
543                    access_expires_at: document.access_expires_at,
544                    refresh_expires_at: document.refresh_expires_at,
545                }),
546            },
547            Err(_) => Self::new(stored.clone()),
548        }
549    }
550
551    /// The value to hand the secret store.
552    ///
553    /// Always the document, even for a credential with no renewal half: one
554    /// shape written means one shape to reason about, and reading still accepts
555    /// the bare token that older versions wrote.
556    #[must_use]
557    pub fn to_stored_document(&self) -> SecretString {
558        #[derive(Serialize)]
559        struct Document<'a> {
560            access_token: &'a str,
561            #[serde(skip_serializing_if = "Option::is_none")]
562            refresh_token: Option<&'a str>,
563            #[serde(skip_serializing_if = "Option::is_none")]
564            access_expires_at: Option<DateTime<Utc>>,
565            #[serde(skip_serializing_if = "Option::is_none")]
566            refresh_expires_at: Option<DateTime<Utc>>,
567        }
568
569        let document = Document {
570            access_token: self.token.expose_secret(),
571            refresh_token: self
572                .renewal
573                .as_ref()
574                .map(|r| r.refresh_token.expose_secret()),
575            access_expires_at: self.renewal.as_ref().and_then(|r| r.access_expires_at),
576            refresh_expires_at: self.renewal.as_ref().and_then(|r| r.refresh_expires_at),
577        };
578        // Serialising a struct of `&str` cannot fail; the fallback keeps the
579        // access token usable rather than inventing an error path nobody can
580        // act on.
581        SecretString::from(
582            serde_json::to_string(&document)
583                .unwrap_or_else(|_| self.token.expose_secret().to_string()),
584        )
585    }
586
587    /// The token itself. Every call site of this is a place a secret can escape,
588    /// so there are deliberately few: the `Authorization` header, and `d2`'s
589    /// store call.
590    #[must_use]
591    pub fn secret(&self) -> &SecretString {
592        &self.token
593    }
594
595    #[must_use]
596    pub fn token_type(&self) -> &str {
597        &self.token_type
598    }
599
600    #[must_use]
601    pub fn scope(&self) -> Option<&str> {
602        self.scope.as_deref()
603    }
604
605    /// The token's four-character family prefix — `ghu_` for an App
606    /// user-to-server token — and nothing else.
607    ///
608    /// This exists so diagnostics can answer "did the device flow return the
609    /// kind of token we expected?" without exposing the token. The D17 spike
610    /// asserted exactly this and no more.
611    #[must_use]
612    pub fn family(&self) -> &str {
613        let raw = self.token.expose_secret();
614        match raw.find('_') {
615            Some(idx) if idx < 8 => &raw[..=idx],
616            _ => "",
617        }
618    }
619
620    /// `true` for the `ghu_` family the published App issues.
621    #[must_use]
622    pub fn is_user_to_server(&self) -> bool {
623        self.family() == "ghu_"
624    }
625}
626
627impl fmt::Debug for UserAccessToken {
628    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
629        f.debug_struct("UserAccessToken")
630            .field("token", &"[REDACTED]")
631            .field("family", &self.family())
632            .finish_non_exhaustive()
633    }
634}
635
636impl PartialEq for UserAccessToken {
637    /// Equality exists so [`device_flow::PollOutcome`] can carry a token and
638    /// still be compared in a test. Production code never compares two
639    /// credentials, and this is not a constant-time comparison.
640    fn eq(&self, other: &Self) -> bool {
641        self.token.expose_secret() == other.token.expose_secret()
642            && self.token_type == other.token_type
643            && self.scope == other.scope
644    }
645}
646
647impl Eq for UserAccessToken {}
648
649// ---------------------------------------------------------------------------
650// Errors
651// ---------------------------------------------------------------------------
652
653/// Everything [`AuthenticatedClient`] can fail with.
654///
655/// The first three variants are the taxonomy `03-control-flows.md` flow 4.3
656/// requires, and they are separate variants because `c3` and `f1` both act on
657/// the distinction: [`GithubError::AuthenticationFailed`] moves a policy to
658/// `authentication_failed` and tells the operator to run `auth login`,
659/// [`GithubError::AuthenticationLockout`] must *wait* and tell the operator
660/// nothing is wrong with their credential, and [`GithubError::Forbidden`] is a
661/// permissions answer that re-authenticating will not change.
662///
663/// # Why the failing variants carry response headers
664///
665/// Rate-limit *policy* is `c3`'s and is deliberately not implemented here. But a
666/// policy needs evidence, and the evidence — `retry-after`,
667/// `x-ratelimit-remaining`, `x-ratelimit-reset` — only exists on the response
668/// that failed. An error taxonomy that dropped those headers would leave `c3`
669/// with no way to honour a `429` except by editing this file, which is exactly
670/// the conflict the `c2`/`c3` ownership split exists to prevent. So
671/// [`GithubError::Status`] and [`GithubError::Forbidden`] carry the headers
672/// verbatim and interpret none of them; see [`GithubError::headers`].
673#[derive(Debug, thiserror::Error)]
674pub enum GithubError {
675    /// GitHub rejected the credential and a single re-validation confirmed it.
676    /// Terminal: only an interactive `auth login` clears this.
677    #[error(
678        "GitHub rejected the stored credential; run `runner-manager auth login` to sign in again"
679    )]
680    AuthenticationFailed,
681
682    /// GitHub answered `403` after `401`s — its temporary authentication
683    /// lockout, not a permissions change. Back off; do not re-authenticate and
684    /// do not retry.
685    #[error(
686        "GitHub has temporarily locked out authentication for this credential; \
687         back off for {}s and do not retry — the credential itself is not the problem",
688        retry_after.as_secs()
689    )]
690    AuthenticationLockout { retry_after: Duration },
691
692    /// A `403` that is not the lockout: a permissions answer, or GitHub's own
693    /// rate limit. `c3` tells the two apart from `headers`; this crate does not,
694    /// because which of them is worth retrying is rate-limit policy.
695    #[error(
696        "GitHub denied {method} {path}: the App installation does not grant it{}",
697        message.as_deref().map(|m| format!(" ({m})")).unwrap_or_default()
698    )]
699    Forbidden {
700        method: String,
701        path: String,
702        message: Option<String>,
703        /// The response headers, verbatim and uninterpreted.
704        headers: Box<HeaderMap>,
705    },
706
707    #[error(
708        "GitHub returned {status} for {method} {path}{}",
709        message.as_deref().map(|m| format!(": {m}")).unwrap_or_default()
710    )]
711    Status {
712        status: u16,
713        method: String,
714        path: String,
715        message: Option<String>,
716        /// The response headers, verbatim and uninterpreted. A `429` reaches
717        /// `c3` through this variant, and its `retry-after` survives with it.
718        headers: Box<HeaderMap>,
719    },
720
721    /// The request never got an answer. The URL is stripped from the source
722    /// error before it is stored: a device-flow URL never carries a secret, but
723    /// stripping it costs nothing and removes a whole class of future leak.
724    #[error("GitHub was unreachable")]
725    Transport(#[source] reqwest::Error),
726
727    #[error("a {what} response from GitHub could not be decoded as {expected}")]
728    Decode {
729        what: &'static str,
730        expected: &'static str,
731        #[source]
732        source: serde_json::Error,
733    },
734
735    #[error("GitHub returned {value:?} for {what}, which this client cannot use")]
736    Malformed { what: &'static str, value: String },
737
738    #[error(transparent)]
739    Config(#[from] ConfigError),
740}
741
742impl GithubError {
743    /// `true` for the two authentication outcomes, which callers handle
744    /// differently from every other failure.
745    #[must_use]
746    pub fn is_authentication(&self) -> bool {
747        matches!(
748            self,
749            Self::AuthenticationFailed | Self::AuthenticationLockout { .. }
750        )
751    }
752
753    /// `true` only for the lockout, which is the one authentication outcome that
754    /// resolves by waiting rather than by signing in again.
755    #[must_use]
756    pub fn is_lockout(&self) -> bool {
757        matches!(self, Self::AuthenticationLockout { .. })
758    }
759
760    /// The failing response's headers, for the variants that have them.
761    ///
762    /// This is the whole of `c2`'s contribution to rate limiting: it hands `c3`
763    /// the evidence and stops there. Nothing in this crate reads
764    /// `x-ratelimit-remaining` to decide anything.
765    #[must_use]
766    pub fn headers(&self) -> Option<&HeaderMap> {
767        match self {
768            Self::Status { headers, .. } | Self::Forbidden { headers, .. } => Some(headers),
769            _ => None,
770        }
771    }
772
773    /// The failing response's `retry-after`, in seconds, if it sent one.
774    ///
775    /// Reading a documented header is evidence, not policy: what to *do* with a
776    /// `retry-after` — wait, shed load, surface it to an operator — is `c3`'s.
777    #[must_use]
778    pub fn retry_after(&self) -> Option<Duration> {
779        self.headers().and_then(retry_after)
780    }
781
782    /// The `x-ratelimit-remaining` / `x-ratelimit-reset` pair, when present.
783    ///
784    /// Returned as the raw numbers GitHub sent. `reset` is a Unix timestamp in
785    /// seconds, which is what the header carries; it is deliberately not turned
786    /// into a [`Timestamp`] here, because comparing it against a clock is the
787    /// first step of a policy decision and that decision is `c3`'s.
788    #[must_use]
789    pub fn rate_limit(&self) -> Option<RateLimitEvidence> {
790        let headers = self.headers()?;
791        let read = |name: &str| {
792            headers
793                .get(name)
794                .and_then(|v| v.to_str().ok())
795                .and_then(|v| v.trim().parse::<u64>().ok())
796        };
797        let remaining = read("x-ratelimit-remaining");
798        let reset = read("x-ratelimit-reset");
799        if remaining.is_none() && reset.is_none() {
800            return None;
801        }
802        Some(RateLimitEvidence {
803            remaining,
804            reset_unix_secs: reset,
805            retry_after: self.retry_after(),
806        })
807    }
808}
809
810/// What GitHub said about its own rate limit on a response that failed.
811///
812/// Evidence, carried across the `c2`/`c3` seam. Every field is what the wire
813/// said, and none of them has been interpreted.
814#[derive(Debug, Clone, Copy, PartialEq, Eq)]
815pub struct RateLimitEvidence {
816    /// `x-ratelimit-remaining`. Zero is GitHub's primary rate limit.
817    pub remaining: Option<u64>,
818    /// `x-ratelimit-reset`, a Unix timestamp in seconds.
819    pub reset_unix_secs: Option<u64>,
820    /// `retry-after`, which secondary rate limits send instead.
821    pub retry_after: Option<Duration>,
822}
823
824fn transport(err: reqwest::Error) -> GithubError {
825    GithubError::Transport(err.without_url())
826}
827
828// ---------------------------------------------------------------------------
829// Requests and responses
830// ---------------------------------------------------------------------------
831
832/// One `api.github.com` request, before authentication headers are applied.
833///
834/// `Debug` is written by hand and never renders the body: `c4` posts
835/// `generate-jitconfig` requests through this type, and a JIT configuration is
836/// a sensitive short-lived value (`07-security.md`, credential inventory).
837#[derive(Clone)]
838pub struct ApiRequest {
839    method: Method,
840    /// Either a path relative to the API base, or an absolute URL — which is
841    /// what a `Link: rel="next"` page is.
842    path: String,
843    query: Vec<(String, String)>,
844    body: Option<serde_json::Value>,
845    /// What to send as `If-None-Match` / `If-Modified-Since`, when this is a
846    /// conditional `GET`. See [`crate::conditional`].
847    validators: Option<Validators>,
848}
849
850impl ApiRequest {
851    #[must_use]
852    pub fn get(path: impl Into<String>) -> Self {
853        Self::new(Method::GET, path)
854    }
855
856    #[must_use]
857    pub fn delete(path: impl Into<String>) -> Self {
858        Self::new(Method::DELETE, path)
859    }
860
861    #[must_use]
862    pub fn new(method: Method, path: impl Into<String>) -> Self {
863        Self {
864            method,
865            path: path.into(),
866            query: Vec::new(),
867            body: None,
868            validators: None,
869        }
870    }
871
872    /// # Errors
873    /// `body` failing to serialize.
874    pub fn post_json<T: Serialize>(path: impl Into<String>, body: &T) -> Result<Self, GithubError> {
875        let value = serde_json::to_value(body).map_err(|source| GithubError::Decode {
876            what: "request",
877            expected: "JSON",
878            source,
879        })?;
880        Ok(Self {
881            method: Method::POST,
882            path: path.into(),
883            query: Vec::new(),
884            body: Some(value),
885            validators: None,
886        })
887    }
888
889    #[must_use]
890    pub fn query(mut self, key: impl Into<String>, value: impl fmt::Display) -> Self {
891        self.query.push((key.into(), value.to_string()));
892        self
893    }
894
895    /// Make this a conditional request: GitHub answers `304 Not Modified`,
896    /// with no body, while the resource still matches `validators`.
897    ///
898    /// Only meaningful on a `GET`. GitHub: "Conditional requests for unsafe
899    /// methods, such as `POST`, `PUT`, `PATCH`, and `DELETE` are not supported
900    /// unless otherwise noted in the documentation."
901    #[must_use]
902    pub fn conditional(mut self, validators: Validators) -> Self {
903        self.validators = (!validators.is_empty()).then_some(validators);
904        self
905    }
906
907    /// Whether this request carries validators, and so may be answered `304`.
908    #[must_use]
909    pub const fn is_conditional(&self) -> bool {
910        self.validators.is_some()
911    }
912
913    /// The resource this request reads: method, path, and query in the order
914    /// it was added. An `ETag` is scoped to exactly this — GitHub: "Use the
915    /// same parameters every time you poll the same data. A different page
916    /// size, page number, or filter produces a different response with a
917    /// different `etag`."
918    #[must_use]
919    pub fn cache_key(&self) -> String {
920        let mut key = format!("{} {}", self.method.as_str(), self.path);
921        for (index, (name, value)) in self.query.iter().enumerate() {
922            key.push(if index == 0 { '?' } else { '&' });
923            key.push_str(name);
924            key.push('=');
925            key.push_str(value);
926        }
927        key
928    }
929
930    #[must_use]
931    pub fn method(&self) -> &Method {
932        &self.method
933    }
934
935    #[must_use]
936    pub fn path(&self) -> &str {
937        &self.path
938    }
939}
940
941impl fmt::Debug for ApiRequest {
942    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
943        f.debug_struct("ApiRequest")
944            .field("method", &self.method.as_str())
945            .field("path", &self.path)
946            .field(
947                "query_keys",
948                &self.query.iter().map(|(k, _)| k).collect::<Vec<_>>(),
949            )
950            .field(
951                "body",
952                &self.body.as_ref().map_or("none", |_| "[REDACTED JSON]"),
953            )
954            .field("conditional", &self.validators.is_some())
955            .finish()
956    }
957}
958
959/// A response's cache validators: its `ETag` and `Last-Modified`.
960///
961/// Sent back verbatim, a weak `W/"…"` tag included, because that is what
962/// GitHub compares against. Measured on `api.github.com` before this was
963/// written: the workflow-run, job and runner listings each return a weak
964/// `ETag`, and an `If-None-Match` carrying it is answered `304` for as long as
965/// the listing is unchanged.
966#[derive(Debug, Clone, Default, PartialEq, Eq)]
967pub struct Validators {
968    pub etag: Option<String>,
969    pub last_modified: Option<String>,
970}
971
972impl Validators {
973    /// The validators a response carries, if any.
974    #[must_use]
975    pub fn of(response: &ApiResponse) -> Self {
976        let read = |name: &str| {
977            response
978                .header(name)
979                .map(str::trim)
980                .filter(|value| !value.is_empty())
981                .map(str::to_string)
982        };
983        Self {
984            etag: read("etag"),
985            last_modified: read("last-modified"),
986        }
987    }
988
989    #[must_use]
990    pub const fn is_empty(&self) -> bool {
991        self.etag.is_none() && self.last_modified.is_none()
992    }
993}
994
995/// One buffered `api.github.com` response.
996///
997/// Buffered rather than streamed because every API response this crate reads is
998/// small JSON. The one large download in the product — the runner package — is
999/// `e2`'s and uses its own streaming client.
1000///
1001/// `Debug` renders the status and the body's *length*, never the body: a
1002/// `generate-jitconfig` response body is an encoded JIT configuration.
1003#[derive(Clone)]
1004pub struct ApiResponse {
1005    status: StatusCode,
1006    headers: HeaderMap,
1007    body: Vec<u8>,
1008}
1009
1010impl ApiResponse {
1011    #[must_use]
1012    pub fn status(&self) -> StatusCode {
1013        self.status
1014    }
1015
1016    /// `304 Not Modified`: the answer to a conditional request whose resource
1017    /// has not changed. It has no body; [`crate::conditional`] substitutes the
1018    /// cached one.
1019    #[must_use]
1020    pub fn is_not_modified(&self) -> bool {
1021        self.status == StatusCode::NOT_MODIFIED
1022    }
1023
1024    #[must_use]
1025    pub fn headers(&self) -> &HeaderMap {
1026        &self.headers
1027    }
1028
1029    #[must_use]
1030    pub fn header(&self, name: &str) -> Option<&str> {
1031        self.headers.get(name).and_then(|v| v.to_str().ok())
1032    }
1033
1034    /// # Errors
1035    /// A body that is not the expected JSON shape.
1036    pub fn json<T: DeserializeOwned>(&self) -> Result<T, GithubError> {
1037        serde_json::from_slice(&self.body).map_err(|source| GithubError::Decode {
1038            what: "response",
1039            expected: std::any::type_name::<T>(),
1040            source,
1041        })
1042    }
1043
1044    /// The next page of a paginated collection, from the `Link` header.
1045    ///
1046    /// `04-subsystem-contracts.md`: "Pagination is mandatory; the dashboard must
1047    /// not treat a first page as a complete inventory." It lives on the shared
1048    /// response type so that `c3`'s inventory and this module's installation
1049    /// discovery cannot disagree about how a `Link` header is read.
1050    #[must_use]
1051    pub fn next_page(&self) -> Option<Url> {
1052        self.header("link").and_then(parse_link_next)
1053    }
1054}
1055
1056impl fmt::Debug for ApiResponse {
1057    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
1058        f.debug_struct("ApiResponse")
1059            .field("status", &self.status.as_u16())
1060            .field("body_bytes", &self.body.len())
1061            .finish_non_exhaustive()
1062    }
1063}
1064
1065/// The `rel="next"` target of an RFC 8288 `Link` header, or `None`.
1066///
1067/// # Why this scans rather than splits
1068///
1069/// A comma separates one link-value from the next, but a comma is also a legal
1070/// character *inside* a URL, and GitHub sends such URLs routinely — a runner
1071/// query carries `labels=self-hosted,windows`. Splitting the whole header on `,`
1072/// first tears that URL in half, neither half parses as `<...>`, and the
1073/// relation is silently lost. The caller then treats page 1 as the whole
1074/// inventory, which is the specific outcome `04-subsystem-contracts.md` forbids:
1075/// "the dashboard must not treat a first page as a complete inventory".
1076///
1077/// So the target is located by its `<`…`>` delimiters, and only a comma that
1078/// actually begins the next link-value — one followed by optional whitespace and
1079/// `<` — ends the parameter section.
1080fn parse_link_next(link: &str) -> Option<Url> {
1081    let mut rest = link;
1082    while let Some(open) = rest.find('<') {
1083        let after_open = &rest[open + 1..];
1084        let Some(close) = after_open.find('>') else {
1085            // An unterminated `<` cannot be a link-value; nothing after it is
1086            // interpretable either.
1087            return None;
1088        };
1089        let target = after_open[..close].trim();
1090        let tail = &after_open[close + 1..];
1091
1092        // The parameters run to the start of the next link-value.
1093        let cut = tail
1094            .match_indices(',')
1095            .find(|(i, _)| tail[i + 1..].trim_start().starts_with('<'))
1096            .map_or(tail.len(), |(i, _)| i);
1097        let (params, next) = tail.split_at(cut);
1098
1099        let is_next = params.split(';').any(|param| {
1100            let param = param.trim().replace(['"', '\''], "");
1101            param.eq_ignore_ascii_case("rel=next")
1102        });
1103        if is_next {
1104            return Url::parse(target).ok();
1105        }
1106        rest = next.strip_prefix(',').unwrap_or(next);
1107    }
1108    None
1109}
1110
1111#[derive(Debug, Deserialize)]
1112struct ErrorEnvelope {
1113    message: Option<String>,
1114}
1115
1116/// GitHub's own message for a failure, and never the raw body.
1117fn error_message(body: &[u8]) -> Option<String> {
1118    serde_json::from_slice::<ErrorEnvelope>(body)
1119        .ok()
1120        .and_then(|e| e.message)
1121        .filter(|m| !m.is_empty())
1122}
1123
1124fn retry_after(headers: &HeaderMap) -> Option<Duration> {
1125    headers
1126        .get("retry-after")
1127        .and_then(|v| v.to_str().ok())
1128        .and_then(|v| v.trim().parse::<u64>().ok())
1129        .map(Duration::from_secs)
1130}
1131
1132// ---------------------------------------------------------------------------
1133// Sleeping
1134// ---------------------------------------------------------------------------
1135
1136/// The one way anything in this crate waits.
1137///
1138/// A port rather than a direct `tokio::time::sleep`, for the same reason the
1139/// domain has a `Clock`: the device flow's `slow_down` handling is a *timing*
1140/// behaviour, and a timing behaviour tested by actually waiting is either
1141/// untested or slow. A test substitutes a sleeper that records the requested
1142/// durations and returns immediately, which turns "`slow_down` increases the
1143/// poll interval" into an equality assertion on a `Vec<Duration>` rather than a
1144/// stopwatch reading.
1145#[async_trait::async_trait]
1146pub trait Sleeper: Send + Sync + fmt::Debug {
1147    async fn sleep(&self, duration: Duration);
1148}
1149
1150/// The production adapter.
1151#[derive(Debug, Clone, Copy, Default)]
1152pub struct TokioSleeper;
1153
1154#[async_trait::async_trait]
1155impl Sleeper for TokioSleeper {
1156    async fn sleep(&self, duration: Duration) {
1157        tokio::time::sleep(duration).await;
1158    }
1159}
1160
1161// ---------------------------------------------------------------------------
1162// The shared authenticated client
1163// ---------------------------------------------------------------------------
1164
1165/// What a single re-validation of the held credential concluded.
1166#[derive(Debug, Clone, Copy, PartialEq, Eq)]
1167pub enum Revalidation {
1168    /// GitHub still accepts the credential, so the `401` was about the request
1169    /// rather than about the token. One retry is warranted.
1170    Valid,
1171    /// GitHub rejects the credential. Terminal; only `auth login` clears it.
1172    Rejected,
1173    /// The probe itself could not be completed — GitHub was unreachable, or
1174    /// answered something neither `2xx` nor `401`. Nothing was learned, so the
1175    /// caller still gets its one retry.
1176    Unavailable,
1177}
1178
1179#[derive(Debug)]
1180struct LockoutState {
1181    until: Option<Timestamp>,
1182    backoff: Duration,
1183}
1184
1185/// The one client every `api.github.com` request in this crate goes through.
1186///
1187/// It exists to make four things structural rather than remembered:
1188///
1189/// 1. `X-GitHub-Api-Version`, `Accept`, `User-Agent`, and `Authorization` are
1190///    set on every request because they are set *here*.
1191/// 2. The `401` / `403` taxonomy of `03-control-flows.md` flow 4.3 is
1192///    implemented once. `c3` and `f1` both branch on the distinction, and two
1193///    implementations of it would eventually disagree.
1194/// 3. A `401` storm produces **one** credential re-validation, not one per
1195///    caller — see [`AuthenticatedClient::revalidate`].
1196/// 4. A lockout stops traffic. Once GitHub answers `403` after `401`s, this
1197///    client issues no further HTTP at all until the back-off elapses.
1198pub struct AuthenticatedClient {
1199    http: reqwest::Client,
1200    endpoints: Endpoints,
1201    /// Swappable, because a renewable credential replaces itself while this
1202    /// client is in use. A `Mutex` rather than a lock-free cell: it is read
1203    /// once per request and written once every eight hours, so contention is
1204    /// not the concern -- being obviously correct is.
1205    credential: std::sync::Mutex<UserAccessToken>,
1206    /// Where to re-read the credential when a `401` outlives renewal.
1207    ///
1208    /// `None` for a short-lived client, which is every one that is not the
1209    /// daemon's: a command that runs for a second cannot be outlived by a
1210    /// sign-in.
1211    source: Option<Arc<dyn CredentialSource>>,
1212    /// How a credential replaces itself, when it can.
1213    ///
1214    /// `None` for a client whose credential has no renewal half, which is every
1215    /// client until an App turns user-token expiration on, and for the paths
1216    /// that hold a token for one call and never outlive its eight hours.
1217    renewal: Option<Arc<dyn CredentialRenewal>>,
1218    clock: Arc<dyn Clock>,
1219
1220    /// Bumped once per completed re-validation. A caller that took a `401`
1221    /// samples this *before* queuing on the gate; if it changed while the caller
1222    /// waited, some other caller already did the work and this one must not
1223    /// repeat it. This is the whole single-flight mechanism.
1224    revalidation_generation: AtomicU64,
1225    revalidation_gate: tokio::sync::Mutex<()>,
1226    last_revalidation: std::sync::Mutex<Revalidation>,
1227    revalidations_performed: AtomicU64,
1228
1229    /// `401`s seen since the last successful caller response.
1230    ///
1231    /// **Nothing in production reads this.** It is incremented on every `401`
1232    /// and cleared by a successful caller response, and that is the whole of
1233    /// what it does. It used to be documented as the lockout's test — "a `403`
1234    /// while this is non-zero is a lockout" — and
1235    /// [`AuthenticatedClient::is_lockout_403`] stopped consulting it when that
1236    /// rule was replaced by position plus GitHub's own evidence:
1237    /// [`Attempt::Retry`] already implies this request's own `401` incremented
1238    /// it moments ago, so reading it added no signal, and it did add a race that
1239    /// failed open — a concurrent success clearing the count downgraded a real
1240    /// lockout to a permissions answer.
1241    ///
1242    /// The field is kept on purpose, and only the tests read it: they drive it
1243    /// to values that *would* change the answer if it were still consulted, and
1244    /// assert that the answer does not change. Deleting it would delete the
1245    /// ability to make that assertion, which is the only thing standing between
1246    /// the conjunct and its reintroduction. See
1247    /// `a_concurrent_success_cannot_downgrade_a_lockout_to_a_permissions_answer`.
1248    consecutive_unauthorized: AtomicU64,
1249    lockout: std::sync::Mutex<LockoutState>,
1250    /// Every response this client received, counted. Observation only:
1251    /// nothing in this client decides anything from it. See
1252    /// [`crate::traffic`].
1253    traffic: traffic::ApiTraffic,
1254}
1255
1256impl fmt::Debug for AuthenticatedClient {
1257    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
1258        // `try_lock`, not `is_locked_out()`. `std::sync::Mutex` is not
1259        // reentrant, and `latch_lockout` holds this one; the moment anybody adds
1260        // a `tracing` call inside that function — which is a natural thing to
1261        // want there — rendering the client would deadlock the whole agent. A
1262        // `Debug` impl must never be able to block, so it reports what it can
1263        // see and says so when it cannot.
1264        let locked_out = match self.lockout.try_lock() {
1265            Ok(state) => {
1266                if state.until.is_some_and(|until| self.clock.now() < until) {
1267                    "yes"
1268                } else {
1269                    "no"
1270                }
1271            }
1272            Err(_) => "unknown (the lockout state is being updated)",
1273        };
1274        f.debug_struct("AuthenticatedClient")
1275            .field("api_base", &self.endpoints.api_base.as_str())
1276            .field(
1277                "credential",
1278                &self
1279                    .credential
1280                    .try_lock()
1281                    .map_or("[in use]", |_| "[redacted]"),
1282            )
1283            .field(
1284                "revalidations_performed",
1285                &self.revalidations_performed.load(Ordering::Relaxed),
1286            )
1287            .field("locked_out", &locked_out)
1288            .finish_non_exhaustive()
1289    }
1290}
1291
1292/// How a credential replaces itself.
1293///
1294/// # Why this is a port rather than a method
1295///
1296/// Renewal is two acts that must happen in one order: exchange the refresh
1297/// token with GitHub, then **persist the new pair before anything uses it**.
1298/// GitHub rotates on use -- the old pair dies the instant the new one is
1299/// issued -- so a response that is used but not stored leaves the host holding
1300/// a credential it will forget, and the one it forgot is already dead. There
1301/// is no retry: a spent refresh token answers `incorrect_client_credentials`,
1302/// a message about the client id and secret that is about neither.
1303///
1304/// The exchange belongs to `c2` and the store belongs to `d2`, and this crate
1305/// owns neither. So the ordering lives with whoever implements this, in one
1306/// place, rather than being a rule each caller has to remember.
1307#[async_trait::async_trait]
1308pub trait CredentialRenewal: fmt::Debug + Send + Sync {
1309    /// Exchange `refresh_token` for a fresh pair and persist it.
1310    ///
1311    /// # Errors
1312    /// Any failure; the caller treats every one the same way, by keeping the
1313    /// credential it has and letting the next `401` try again. The variant
1314    /// decides only how loudly that is reported; see [`RenewalError`].
1315    async fn renew(&self, refresh_token: &SecretString) -> Result<UserAccessToken, RenewalError>;
1316}
1317
1318/// Why [`CredentialRenewal::renew`] handed back no fresh pair.
1319#[derive(Debug, thiserror::Error)]
1320pub enum RenewalError {
1321    /// This process chose not to spend the refresh token, and GitHub was not
1322    /// asked.
1323    ///
1324    /// The pair is intact, so this is not a credential problem: a process that
1325    /// cannot store what the exchange returns leaves renewal to one that can --
1326    /// on a boot-mode host, the service. Reported quietly, because it recurs
1327    /// on every attempt for as long as the renewal window is open and nothing
1328    /// about it needs an operator.
1329    #[error("{0}")]
1330    Declined(String),
1331    /// The renewal was attempted and did not produce a stored pair.
1332    #[error("{0}")]
1333    Failed(String),
1334}
1335
1336impl From<String> for RenewalError {
1337    fn from(detail: String) -> Self {
1338        Self::Failed(detail)
1339    }
1340}
1341
1342/// Where a client can go to find out that the stored credential changed under
1343/// it.
1344///
1345/// # Why a long-running client needs this
1346///
1347/// A daemon reads the store once, at startup, and holds the result for as long
1348/// as it runs. That was invisible while the only way to change the store was to
1349/// stop the daemon — but `auth login` does not stop anything, so a host whose
1350/// credential died before its daemon started stays dead through every sign-in
1351/// meant to fix it. The operator does the right thing, watches it not work, and
1352/// has nothing to tell them why.
1353///
1354/// Watched on 2026-08-29: a Windows daemon started at `04:08Z` holding an
1355/// already-expired token, a sign-in at `10:43Z` that wrote a good pair, and 180
1356/// `unauthorized` events an hour for 28 hours without a single minute's pause
1357/// across the sign-in. See `docs/spikes/token-expiry-and-renewal.md`.
1358///
1359/// # Why it is not a file watch
1360///
1361/// This is consulted on `401` and nowhere else, so a store that never changes
1362/// costs nothing and a daemon that is working never reads the disk. It also
1363/// covers the case a watch would miss on macOS, where the credential lives in a
1364/// keychain rather than at a path.
1365pub trait CredentialSource: fmt::Debug + Send + Sync {
1366    /// The credential the store holds *now*, or `None` if it cannot be read.
1367    ///
1368    /// Infallible by design: every failure — missing, unreadable, corrupt —
1369    /// means the same thing to the caller, which is that there is nothing new
1370    /// to try and the `401` stands.
1371    fn reload(&self) -> Option<UserAccessToken>;
1372}
1373
1374impl AuthenticatedClient {
1375    /// # Errors
1376    /// The HTTP client failing to build — a TLS backend that will not
1377    /// initialise, in practice.
1378    pub fn new(
1379        endpoints: Endpoints,
1380        credential: UserAccessToken,
1381        clock: Arc<dyn Clock>,
1382    ) -> Result<Self, GithubError> {
1383        let http = reqwest::Client::builder()
1384            .timeout(DEFAULT_REQUEST_TIMEOUT)
1385            .build()
1386            .map_err(transport)?;
1387        Ok(Self::with_http_client(http, endpoints, credential, clock))
1388    }
1389
1390    #[must_use]
1391    pub fn with_http_client(
1392        http: reqwest::Client,
1393        endpoints: Endpoints,
1394        credential: UserAccessToken,
1395        clock: Arc<dyn Clock>,
1396    ) -> Self {
1397        Self {
1398            http,
1399            endpoints,
1400            credential: std::sync::Mutex::new(credential),
1401            source: None,
1402            renewal: None,
1403            clock,
1404            revalidation_generation: AtomicU64::new(0),
1405            revalidation_gate: tokio::sync::Mutex::new(()),
1406            last_revalidation: std::sync::Mutex::new(Revalidation::Valid),
1407            revalidations_performed: AtomicU64::new(0),
1408            consecutive_unauthorized: AtomicU64::new(0),
1409            lockout: std::sync::Mutex::new(LockoutState {
1410                until: None,
1411                backoff: DEFAULT_LOCKOUT_BACKOFF,
1412            }),
1413            traffic: traffic::ApiTraffic::new(),
1414        }
1415    }
1416
1417    #[must_use]
1418    pub fn endpoints(&self) -> &Endpoints {
1419        &self.endpoints
1420    }
1421
1422    /// What this client has received from GitHub, by class, and the newest
1423    /// reading of the account's hourly quota.
1424    #[must_use]
1425    pub const fn traffic(&self) -> &traffic::ApiTraffic {
1426        &self.traffic
1427    }
1428
1429    /// How many credential re-validations this client has performed.
1430    ///
1431    /// Public because it is the observable the single-flight requirement is
1432    /// stated in terms of: "concurrent callers hitting `401` together produce
1433    /// **one** attempt, not N".
1434    #[must_use]
1435    pub fn revalidations_performed(&self) -> u64 {
1436        self.revalidations_performed.load(Ordering::SeqCst)
1437    }
1438
1439    /// `true` while a lockout back-off is still running, during which this
1440    /// client issues no HTTP at all.
1441    ///
1442    /// # Panics
1443    /// If a previous holder panicked while the lockout lock was held.
1444    #[must_use]
1445    pub fn is_locked_out(&self) -> bool {
1446        self.lockout_remaining().is_some()
1447    }
1448
1449    /// How much of the lockout back-off is left, or `None` when not locked out.
1450    ///
1451    /// # Panics
1452    /// If a previous holder panicked while the lockout lock was held.
1453    #[must_use]
1454    pub fn lockout_remaining(&self) -> Option<Duration> {
1455        let state = self.lockout.lock().expect("lockout lock poisoned");
1456        let until = state.until?;
1457        let now = self.clock.now();
1458        if now >= until {
1459            return None;
1460        }
1461        (until - now).to_std().ok()
1462    }
1463
1464    /// Clear a lockout early. `f1` does not need this — the back-off expires on
1465    /// its own against the clock — but a successful interactive `auth login`
1466    /// legitimately invalidates the whole lockout premise.
1467    ///
1468    /// # Panics
1469    /// If a previous holder panicked while the lockout lock was held.
1470    pub fn clear_lockout(&self) {
1471        self.lockout.lock().expect("lockout lock poisoned").until = None;
1472        self.consecutive_unauthorized.store(0, Ordering::SeqCst);
1473    }
1474
1475    /// Send one request, applying the authentication taxonomy.
1476    ///
1477    /// On `401` this performs a single-flight credential re-validation and then
1478    /// **one** retry — never more, and never a token renewal, because there is
1479    /// nothing to renew (see [`AuthenticatedClient::revalidate`]).
1480    ///
1481    /// # Errors
1482    /// Every variant of [`GithubError`].
1483    pub async fn send(&self, request: &ApiRequest) -> Result<ApiResponse, GithubError> {
1484        if let Some(remaining) = self.lockout_remaining() {
1485            // "backs off without further attempts": no socket is opened at all.
1486            tracing::debug!(
1487                method = request.method.as_str(),
1488                path = %request.path,
1489                remaining_secs = remaining.as_secs(),
1490                "suppressed a request: GitHub authentication lockout is still backing off"
1491            );
1492            return Err(GithubError::AuthenticationLockout {
1493                retry_after: remaining,
1494            });
1495        }
1496
1497        let first = self.send_raw(request).await?;
1498        match self.classify(request, &first, Attempt::First) {
1499            Classified::Ok => Ok(first),
1500            Classified::Unauthorized => self.revalidate_and_retry_once(request).await,
1501            Classified::Error(err) => Err(err),
1502        }
1503    }
1504
1505    /// Deserialize a `GET` in one step.
1506    ///
1507    /// # Errors
1508    /// Every variant of [`GithubError`].
1509    pub async fn get_json<T: DeserializeOwned>(&self, path: &str) -> Result<T, GithubError> {
1510        self.send(&ApiRequest::get(path)).await?.json()
1511    }
1512
1513    /// The access token to send, as a string, for exactly one request.
1514    ///
1515    /// Cloned out of the lock rather than borrowed through it: the value is
1516    /// about to go into a header, and holding the lock across the request would
1517    /// serialise every call in the process behind one mutex.
1518    fn bearer(&self) -> String {
1519        self.credential
1520            .lock()
1521            .unwrap_or_else(std::sync::PoisonError::into_inner)
1522            .secret()
1523            .expose_secret()
1524            .to_string()
1525    }
1526
1527    /// Attach a way for this client's credential to replace itself.
1528    #[must_use]
1529    pub fn with_renewal(mut self, renewal: Arc<dyn CredentialRenewal>) -> Self {
1530        self.renewal = Some(renewal);
1531        self
1532    }
1533
1534    /// Attach the store this client's credential came from, so a `401` it
1535    /// cannot renew its way out of can still notice a sign-in that already
1536    /// happened.
1537    #[must_use]
1538    pub fn with_credential_source(mut self, source: Arc<dyn CredentialSource>) -> Self {
1539        self.source = Some(source);
1540        self
1541    }
1542
1543    /// Hand this client a different credential, for the next request on.
1544    ///
1545    /// For a holder that mints its own short-lived credentials and replaces
1546    /// them before they expire -- the coordinator's GitHub App installation
1547    /// tokens, which last an hour. Keeping the client (rather than building a
1548    /// new one per token) keeps its conditional cache and rate-limit state.
1549    /// Returns whether the credential changed.
1550    ///
1551    /// Taken under the same gate as a `401`'s renewal, so the two never race.
1552    pub async fn replace_credential(&self, fresh: UserAccessToken) -> bool {
1553        let _gate = self.revalidation_gate.lock().await;
1554        self.swap_in(fresh)
1555    }
1556
1557    /// Installs `fresh` unless it is the credential already held, and bumps
1558    /// the generation a queued `401` retry checks. The caller holds
1559    /// `revalidation_gate`.
1560    fn swap_in(&self, fresh: UserAccessToken) -> bool {
1561        {
1562            let mut guard = self
1563                .credential
1564                .lock()
1565                .unwrap_or_else(std::sync::PoisonError::into_inner);
1566            if guard.secret().expose_secret() == fresh.secret().expose_secret() {
1567                return false;
1568            }
1569            *guard = fresh;
1570        }
1571        self.revalidation_generation.fetch_add(1, Ordering::SeqCst);
1572        true
1573    }
1574
1575    /// Replace the credential with whatever `produce` finds, once, however many
1576    /// callers asked at the same moment.
1577    ///
1578    /// Answers whether the credential in hand is now a *different* one, which
1579    /// is the caller's cue to retry. `false` means there was nothing new, and
1580    /// the `401` stands.
1581    ///
1582    /// # Why both ways in share this
1583    ///
1584    /// [`Self::renew_once`] and [`Self::reload_once`] ask one question —
1585    /// *can this `401` be retried with something else* — and differ only in
1586    /// where the something else comes from. Written separately they were two
1587    /// copies of the same `SeqCst` generation protocol, and the copies had
1588    /// already drifted: the equality check below existed in one of them and not
1589    /// the other, so a renewal that handed back an identical token reported a
1590    /// change that had not happened.
1591    ///
1592    /// # Why the comparison, and not just a swap
1593    ///
1594    /// A swap on every `401` would answer `true` forever and turn the one retry
1595    /// into an endless pair of requests against a credential that is genuinely
1596    /// dead. Only a different token is evidence that retrying might go
1597    /// differently.
1598    async fn swap_credential_once<F>(&self, produce: F, note: &'static str) -> bool
1599    where
1600        F: AsyncFnOnce(&Self) -> Option<UserAccessToken>,
1601    {
1602        let before = self.revalidation_generation.load(Ordering::SeqCst);
1603        let _gate = self.revalidation_gate.lock().await;
1604        if self.revalidation_generation.load(Ordering::SeqCst) != before {
1605            // Somebody swapped while this caller queued, so there is already
1606            // something new to retry with. Producing again would be wasted at
1607            // best and, for a renewal, would burn the pair they just minted.
1608            return true;
1609        }
1610
1611        let Some(fresh) = produce(self).await else {
1612            return false;
1613        };
1614        if !self.swap_in(fresh) {
1615            return false;
1616        }
1617        tracing::info!("{note}");
1618        true
1619    }
1620
1621    /// Pick up a credential somebody else stored.
1622    ///
1623    /// `false` when there is no source, the store cannot be read, or what it
1624    /// holds is the token that just failed.
1625    async fn reload_once(&self) -> bool {
1626        let Some(source) = self.source.clone() else {
1627            return false;
1628        };
1629        self.swap_credential_once(
1630            async |_| source.reload(),
1631            "the stored credential changed and was picked up without a restart",
1632        )
1633        .await
1634    }
1635
1636    /// Spend the refresh half for a fresh pair.
1637    ///
1638    /// # Ordering
1639    ///
1640    /// The implementation of [`CredentialRenewal::renew`] persists before
1641    /// returning, so by the time the swap happens the new pair is already
1642    /// durable. A crash between the two loses nothing: the store holds the pair
1643    /// that works, and the next start reads it.
1644    async fn renew_once(&self) -> bool {
1645        let Some(renewal) = self.renewal.clone() else {
1646            return false;
1647        };
1648        self.swap_credential_once(
1649            async |client: &Self| {
1650                let refresh = {
1651                    let guard = client
1652                        .credential
1653                        .lock()
1654                        .unwrap_or_else(std::sync::PoisonError::into_inner);
1655                    guard.renewal().map(|r| r.refresh_token().clone())
1656                }?;
1657                match renewal.renew(&refresh).await {
1658                    Ok(fresh) => Some(fresh),
1659                    Err(RenewalError::Declined(reason)) => {
1660                        tracing::debug!(
1661                            %reason,
1662                            "left the user access token for a process that can store its renewal"
1663                        );
1664                        None
1665                    }
1666                    Err(RenewalError::Failed(error)) => {
1667                        tracing::warn!(
1668                            %error,
1669                            "the user access token could not be renewed; an interactive \
1670                             sign-in may be required"
1671                        );
1672                        None
1673                    }
1674                }
1675            },
1676            "the user access token was renewed",
1677        )
1678        .await
1679    }
1680
1681    /// Proactively replace a renewable credential when either half is close
1682    /// to expiry.
1683    ///
1684    /// Ordinary request handling also renews after a `401`, but a daemon with
1685    /// no GitHub work can make no request for months. In that state the refresh
1686    /// token itself eventually expires and an interactive sign-in is the only
1687    /// recovery. A long-running owner calls this periodically so every
1688    /// successful rotation extends the credential even while it is otherwise
1689    /// idle.
1690    ///
1691    /// The stored credential is checked first for the same reason as the `401`
1692    /// path: another process may already have rotated the pair, and spending
1693    /// its predecessor after that would be a refresh-token replay.
1694    ///
1695    /// Returns `true` when a different credential was loaded or minted. A
1696    /// credential without a refresh half is left untouched. Missing expiry
1697    /// metadata is treated as due once so that the exchange can repair an old
1698    /// stored document.
1699    pub async fn renew_if_expiring_within(&self, window: Duration) -> bool {
1700        let due = {
1701            let credential = self
1702                .credential
1703                .lock()
1704                .unwrap_or_else(std::sync::PoisonError::into_inner);
1705            let Some(renewal) = credential.renewal() else {
1706                return false;
1707            };
1708            let Some(deadline) = self.clock.now().checked_add_signed(
1709                chrono::TimeDelta::from_std(window).unwrap_or(chrono::TimeDelta::MAX),
1710            ) else {
1711                return true;
1712            };
1713            renewal
1714                .access_expires_at
1715                .is_none_or(|expires| expires <= deadline)
1716                || renewal
1717                    .refresh_expires_at
1718                    .is_some_and(|expires| expires <= deadline)
1719        };
1720
1721        due && (self.reload_once().await || self.renew_once().await)
1722    }
1723
1724    /// Serialize, `POST`, and deserialize in one step.
1725    ///
1726    /// # Errors
1727    /// Every variant of [`GithubError`].
1728    pub async fn post_json<B: Serialize, T: DeserializeOwned>(
1729        &self,
1730        path: &str,
1731        body: &B,
1732    ) -> Result<T, GithubError> {
1733        self.send(&ApiRequest::post_json(path, body)?).await?.json()
1734    }
1735
1736    /// Re-validate the credential once, no matter how many callers ask at once.
1737    ///
1738    /// # This is not a token renewal — renewal is [`Self::renew_once`]
1739    ///
1740    /// `03-control-flows.md` flow 4.3 says a `401` "triggers one refresh under
1741    /// a single-flight mutex, then one retry", and that is implemented
1742    /// literally: one attempt shared by every concurrent caller, then one retry
1743    /// each.
1744    ///
1745    /// This method used to claim the word "refresh" could not be meant, because
1746    /// renewing needs a client secret and the App issues no renewal token. Both
1747    /// halves were false; see this module's header. Renewal exists, it runs
1748    /// first in [`Self::revalidate_and_retry_once`], and what is left here is
1749    /// the path for a credential that has no refresh half to spend — one stored
1750    /// before 0.1.11, or issued while the App had expiration switched off.
1751    ///
1752    /// So the single thing that happens under the mutex is a re-validation of
1753    /// the credential already held: one `GET /user/installations` with the same
1754    /// token, asking GitHub whether it still accepts it. A
1755    /// [`Revalidation::Rejected`] answer is terminal
1756    /// [`GithubError::AuthenticationFailed`] requiring an interactive
1757    /// `auth login`; [`Revalidation::Valid`] and [`Revalidation::Unavailable`]
1758    /// both spend the one retry.
1759    ///
1760    /// # Position, and what it no longer decides on its own
1761    ///
1762    /// This method is the caller's own probe: it is a **first** attempt by
1763    /// construction, whatever happened on some other request minutes ago, so it
1764    /// passes [`Attempt::First`] down.
1765    ///
1766    /// That used to settle the matter. This heading read "why this entry point
1767    /// may not latch a lockout", and the text said the probe it drives *cannot*
1768    /// latch — an accurate description of the position rule as it then stood,
1769    /// and a false statement about the product. A lockout that outlives one
1770    /// back-off continues on a **first** attempt by construction, because this
1771    /// client's retry never happened: the request never reached the wire.
1772    /// Refusing to latch there stopped the back-off entirely and hammered a
1773    /// credential GitHub had asked to be left alone.
1774    ///
1775    /// So position alone no longer decides.
1776    /// [`AuthenticatedClient::is_lockout_403`] reads GitHub's own evidence in
1777    /// the first position instead: a first attempt latches when, and only when,
1778    /// the response carries `retry-after` and no parseable GitHub message body.
1779    /// A permissions refusal names what is not accessible, so it still does not
1780    /// latch, which is what keeps [`GithubError::Forbidden`] reachable from
1781    /// here.
1782    ///
1783    /// `revalidate_and_retry_once` uses the private
1784    /// [`AuthenticatedClient::revalidate_after_unauthorized`] instead, which is
1785    /// in the retry position, where any `403` that is not a rate limit is the
1786    /// lockout regardless of what the body says.
1787    ///
1788    /// # This call can latch a lockout, and then it says so
1789    ///
1790    /// Because a first attempt can latch, this call can leave the whole client
1791    /// backed off for up to [`MAX_LOCKOUT_BACKOFF`]. It reports that as
1792    /// [`GithubError::AuthenticationLockout`] rather than answering
1793    /// `Ok(`[`Revalidation::Unavailable`]`)` and leaving the caller to discover
1794    /// it through a separate [`AuthenticatedClient::is_locked_out`] call. An
1795    /// `auth status` that printed "could not determine" while the client it had
1796    /// just silenced sat mute for fifteen minutes would be reporting the wrong
1797    /// event, and reporting it as the milder one.
1798    ///
1799    /// # Errors
1800    /// [`GithubError::AuthenticationLockout`] if this client is already backing
1801    /// off when the call arrives, **or** if this call's own probe latches one.
1802    ///
1803    /// A credential GitHub has rejected outright is *not* an error here: that
1804    /// comes back as `Ok(`[`Revalidation::Rejected`]`)`, and what to do about it
1805    /// — prompt for `auth login` — is the caller's decision, not this method's.
1806    ///
1807    /// # Panics
1808    /// If a previous holder panicked while the re-validation result lock was
1809    /// held.
1810    pub async fn revalidate(&self) -> Result<Revalidation, GithubError> {
1811        self.revalidate_from(Attempt::First).await
1812    }
1813
1814    /// The re-validation that `send` runs between a `401` and its one retry.
1815    ///
1816    /// Identical to [`AuthenticatedClient::revalidate`] except for position:
1817    /// this one *is* the retry path, so a `403` on its probe is the lockout and
1818    /// is latched.
1819    async fn revalidate_after_unauthorized(&self) -> Result<Revalidation, GithubError> {
1820        self.revalidate_from(Attempt::Retry).await
1821    }
1822
1823    async fn revalidate_from(&self, attempt: Attempt) -> Result<Revalidation, GithubError> {
1824        // "A lockout stops traffic. This client issues no further HTTP at all
1825        // until the back-off elapses" is a property of the client, not of
1826        // `send`, and the probe is HTTP like any other. `send` has already
1827        // returned by the time it calls in here, so this guard only bites a
1828        // caller that probes on its own — and one that did would otherwise be
1829        // the single exception to the rule, which is how a rule stops holding.
1830        if let Some(retry_after) = self.lockout_remaining() {
1831            return Err(GithubError::AuthenticationLockout { retry_after });
1832        }
1833
1834        // Sample before queuing. If this changes while we wait for the gate,
1835        // someone else's re-validation covers us and we must not repeat it.
1836        let sampled = self.revalidation_generation.load(Ordering::SeqCst);
1837        let _guard = self.revalidation_gate.lock().await;
1838        let outcome = if self.revalidation_generation.load(Ordering::SeqCst) != sampled {
1839            let shared = *self
1840                .last_revalidation
1841                .lock()
1842                .expect("re-validation lock poisoned");
1843            tracing::debug!(
1844                outcome = ?shared,
1845                "reused an in-flight credential re-validation instead of starting another"
1846            );
1847            shared
1848        } else {
1849            self.revalidations_performed.fetch_add(1, Ordering::SeqCst);
1850            let fresh = self.probe_credential(attempt).await;
1851            *self
1852                .last_revalidation
1853                .lock()
1854                .expect("re-validation lock poisoned") = fresh;
1855            self.revalidation_generation.fetch_add(1, Ordering::SeqCst);
1856            tracing::info!(
1857                outcome = ?fresh,
1858                "re-validated the stored credential; it carries no refresh half to renew"
1859            );
1860            fresh
1861        };
1862
1863        // The probe is HTTP, and since the continuation rule re-widened
1864        // `Attempt::First`, HTTP from *this* entry point can latch a lockout.
1865        // `probe_credential` reports a `403` as `Unavailable` whether or not it
1866        // latched one, and `Unavailable` on its own reads as "this taught us
1867        // nothing about the credential" — so without this check `revalidate`
1868        // answers `Ok(Unavailable)` having just silenced the entire client for
1869        // up to `MAX_LOCKOUT_BACKOFF`, and the only way for `f1` to find that
1870        // out is a separate `is_locked_out()` call it has no reason to make.
1871        //
1872        // `revalidate_and_retry_once` has always re-checked after its own probe.
1873        // Doing it here instead makes the two entry points agree: before, being
1874        // told about the lockout depended on *who latched it* — the guard above
1875        // reports one latched by someone else's traffic, this reports one
1876        // latched by the caller's own probe — and that distinction is invisible
1877        // from outside and actionable by nobody.
1878        //
1879        // This changes what is reported, not when a lockout latches:
1880        // `latch_lockout` is reached on exactly the paths it was before.
1881        //
1882        // It also covers the shared branch above, which the retry path's own
1883        // check never could. A second caller arriving while the first one's
1884        // probe is in flight takes the cached `Unavailable` without probing at
1885        // all, and is just as silenced by the lockout that probe latched.
1886        if let Some(retry_after) = self.lockout_remaining() {
1887            return Err(GithubError::AuthenticationLockout { retry_after });
1888        }
1889        Ok(outcome)
1890    }
1891
1892    /// One `GET /user/installations` with the credential already held, asking
1893    /// GitHub whether it still accepts it.
1894    ///
1895    /// `attempt` is the *caller's* position, not the probe's own. A probe driven
1896    /// by `send`'s `401` handling is part of that request's retry; a probe a
1897    /// caller asked for through [`AuthenticatedClient::revalidate`] is a first
1898    /// attempt.
1899    ///
1900    /// Either may latch a lockout, on different evidence. This used to say "only
1901    /// the former may", which was the position rule before the continuation rule
1902    /// re-widened `Attempt::First`: a retry `403` that is not a rate limit is
1903    /// the lockout outright, and a first-attempt `403` is the lockout when
1904    /// GitHub's own evidence says so — `retry-after` present, no parseable
1905    /// message. See [`AuthenticatedClient::is_lockout_403`], which both this and
1906    /// `classify` go through, so the rule is stated once instead of twice.
1907    async fn probe_credential(&self, attempt: Attempt) -> Revalidation {
1908        let probe = ApiRequest::get(REVALIDATION_PATH).query("per_page", 1);
1909        match self.send_raw(&probe).await {
1910            // Deliberately does **not** reset `consecutive_unauthorized`. The
1911            // probe is this client's own diagnostic, not the caller's traffic,
1912            // and a successful probe is exactly the state a lockout arrives in:
1913            // GitHub still accepts the credential, and answers the *next* real
1914            // request with `403`. Only a successful caller request clears the
1915            // count, in `classify`.
1916            //
1917            // The reason used to be given as "resetting here would erase the
1918            // evidence the `403` is classified against", and that stopped being
1919            // true when `is_lockout_403` stopped consulting the count. There is
1920            // no such evidence to erase now — nothing in production reads the
1921            // field. The line stays because the field's one remaining job is to
1922            // let the tests prove it is *not* consulted, and a probe that
1923            // quietly rewrote it would make those tests assert against a counter
1924            // value no caller path actually produces.
1925            Ok(response) if response.status.is_success() => Revalidation::Valid,
1926            Ok(response) if response.status == StatusCode::UNAUTHORIZED => {
1927                self.consecutive_unauthorized.fetch_add(1, Ordering::SeqCst);
1928                Revalidation::Rejected
1929            }
1930            Ok(response) if response.status == StatusCode::FORBIDDEN => {
1931                // A `403` on a probe that *is* this request's retry is the
1932                // lockout outright. A `403` on a probe a caller asked for
1933                // directly is the lockout only when the response itself says so
1934                // — `retry-after` present, no parseable message — because that
1935                // is what a lockout continuing past one back-off looks like, and
1936                // it necessarily arrives in the first position.
1937                //
1938                // The comment that used to sit here claimed "the probe only ever
1939                // runs after a `401`, so it is always in the retry position",
1940                // and publishing `revalidate` is what made that untrue. Its
1941                // replacement then said a direct probe "is not" the lockout,
1942                // which the continuation rule in turn made untrue. Both were
1943                // position asserted as a conclusion; the position now arrives as
1944                // an argument and the conclusion is drawn in one place, by
1945                // `is_lockout_403`.
1946                if self.is_lockout_403(&response, attempt) {
1947                    self.latch_lockout(&response.headers);
1948                }
1949                Revalidation::Unavailable
1950            }
1951            Ok(_) | Err(_) => Revalidation::Unavailable,
1952        }
1953    }
1954
1955    async fn revalidate_and_retry_once(
1956        &self,
1957        request: &ApiRequest,
1958    ) -> Result<ApiResponse, GithubError> {
1959        // ------------------------------------------------------------------
1960        // RENEWAL COMES FIRST, AND IT DID NOT USED TO EXIST.
1961        // ------------------------------------------------------------------
1962        // This method's documentation once said a `401` could only ever be
1963        // re-validated, never renewed, because renewing needs a confidential
1964        // client credential and a published binary cannot carry one. The first
1965        // half of that is wrong: GitHub requires one *"unless the user access
1966        // token was generated using the device flow"*, and this product's
1967        // always are. Verified against live GitHub before this was written.
1968        //
1969        // So a credential that carries a refresh token replaces itself here,
1970        // silently, and the caller's request is retried with the new one. That
1971        // is what lets an eight-hour token serve a daemon that runs for months
1972        // -- and what lets two machines hold their own credentials at once,
1973        // because each renews its own pair instead of re-authorising and
1974        // revoking the other's.
1975        //
1976        // A credential with no refresh token falls through to exactly the
1977        // behaviour that was here before.
1978        //
1979        // ------------------------------------------------------------------
1980        // THEN THE STORE, FOR THE 401 RENEWAL CANNOT ANSWER.
1981        // ------------------------------------------------------------------
1982        // Renewal covers a token that expired under a daemon holding a
1983        // *renewable* pair. It cannot cover a daemon that started holding a
1984        // dead bare token, because there is no refresh half to spend -- and
1985        // that daemon will not recover on its own no matter how many times an
1986        // operator runs `auth login`, because it never looks at the store
1987        // again. Consulting it here is what makes the obvious remedy work.
1988        //
1989        // First, not second: if multiple processes (e.g., daemon and TUI) share the
1990        // same token, they will both get 401 at the same time. The first one to renew
1991        // will write the new token to disk. If the second process tries to renew with
1992        // its in-memory refresh token *before* reloading, GitHub will detect a refresh
1993        // token replay and instantly revoke the entire token chain. So we must always
1994        // check disk for a newer token first.
1995        //
1996        // To further prevent a race condition if two processes hit 401 at the exact
1997        // same millisecond, we introduce a pseudo-random jitter based on the OS
1998        // process ID. This ensures one process wakes up first, finishes the renewal,
1999        // and writes to disk, so the second process sees the new file during its
2000        // `reload_once()`. All threads in the *same* process compute the exact same
2001        // jitter, preserving their ability to coalesce behind `revalidation_gate`.
2002        let jitter = (std::process::id() % 1500) as u64 + 50;
2003        tokio::time::sleep(std::time::Duration::from_millis(jitter)).await;
2004
2005        if self.reload_once().await || self.renew_once().await {
2006            let second = self.send_raw(request).await?;
2007            return match self.classify(request, &second, Attempt::Retry) {
2008                Classified::Ok => Ok(second),
2009                // The renewed credential was rejected too. Nothing here can
2010                // help: this is a sign-in, not a token, that has gone.
2011                Classified::Unauthorized => Err(GithubError::AuthenticationFailed),
2012                Classified::Error(err) => Err(err),
2013            };
2014        }
2015        match self.revalidate_after_unauthorized().await? {
2016            Revalidation::Rejected => {
2017                tracing::warn!(
2018                    method = request.method.as_str(),
2019                    path = %request.path,
2020                    "GitHub rejected the stored credential; re-authentication is required"
2021                );
2022                Err(GithubError::AuthenticationFailed)
2023            }
2024            Revalidation::Valid | Revalidation::Unavailable => {
2025                // `revalidate_from` now converts a lockout that its own probe
2026                // latched, so this no longer catches that case. It stays for the
2027                // one it still catches: a *concurrent* request latching between
2028                // that check and this one. Sending the retry into a live lockout
2029                // is the thing the back-off exists to prevent, and this is the
2030                // last point at which it can be declined.
2031                if let Some(remaining) = self.lockout_remaining() {
2032                    return Err(GithubError::AuthenticationLockout {
2033                        retry_after: remaining,
2034                    });
2035                }
2036                let second = self.send_raw(request).await?;
2037                match self.classify(request, &second, Attempt::Retry) {
2038                    Classified::Ok => Ok(second),
2039                    // The one retry is spent. A second `401` is terminal.
2040                    Classified::Unauthorized => Err(GithubError::AuthenticationFailed),
2041                    Classified::Error(err) => Err(err),
2042                }
2043            }
2044        }
2045    }
2046
2047    fn classify(
2048        &self,
2049        request: &ApiRequest,
2050        response: &ApiResponse,
2051        attempt: Attempt,
2052    ) -> Classified {
2053        let status = response.status;
2054        // A `304` is a success only as the answer to a conditional request:
2055        // it means "what you hold is still current". Unasked for, it is not an
2056        // answer this client can hand on, and it stays an error.
2057        if status.is_success() || (status == StatusCode::NOT_MODIFIED && request.is_conditional()) {
2058            self.consecutive_unauthorized.store(0, Ordering::SeqCst);
2059            return Classified::Ok;
2060        }
2061        if status == StatusCode::UNAUTHORIZED {
2062            self.consecutive_unauthorized.fetch_add(1, Ordering::SeqCst);
2063            return Classified::Unauthorized;
2064        }
2065        if status == StatusCode::FORBIDDEN && self.is_lockout_403(response, attempt) {
2066            let backoff = self.latch_lockout(&response.headers);
2067            tracing::warn!(
2068                method = request.method.as_str(),
2069                path = %request.path,
2070                backoff_secs = backoff.as_secs(),
2071                "GitHub answered 403 after 401s: temporary authentication lockout, backing off"
2072            );
2073            return Classified::Error(GithubError::AuthenticationLockout {
2074                retry_after: backoff,
2075            });
2076        }
2077        let headers = Box::new(response.headers.clone());
2078        let message = error_message(&response.body);
2079        if status == StatusCode::FORBIDDEN {
2080            return Classified::Error(GithubError::Forbidden {
2081                method: request.method.as_str().to_string(),
2082                path: request.path.clone(),
2083                message,
2084                headers,
2085            });
2086        }
2087        Classified::Error(GithubError::Status {
2088            status: status.as_u16(),
2089            method: request.method.as_str().to_string(),
2090            path: request.path.clone(),
2091            message,
2092            headers,
2093        })
2094    }
2095
2096    /// Whether a `403` is GitHub's temporary *authentication* lockout, as
2097    /// opposed to a permissions answer or a rate limit.
2098    ///
2099    /// # It must not be a rate limit
2100    ///
2101    /// `classify` used to reach the `403` branch before anything looked at the
2102    /// rate-limit headers, so a primary rate limit arriving during a `401` storm
2103    /// was reported as `AuthenticationLockout` — telling the operator "the
2104    /// credential itself is not the problem" about a response that never
2105    /// mentioned the credential. Recognising GitHub's own rate-limit evidence is
2106    /// not rate-limit *policy*; it is declining to make an assertion the
2107    /// evidence contradicts. What to do about the rate limit stays `c3`'s, which
2108    /// is why this only changes which variant carries the headers onward.
2109    ///
2110    /// # Then one of two positions, and the second one is a fix for the first
2111    ///
2112    /// **The retry.** `consecutive_unauthorized` counts `401`s since the last
2113    /// successful caller response and — correctly — does not decay: a request
2114    /// that ends in `404`, `422` or `500` leaves it set. In the agent's
2115    /// long-lived reconciliation loop that meant a single `401` from minutes ago
2116    /// converted the *next* genuine permissions `403` into a fake lockout: sixty
2117    /// seconds of silence plus an operator message insisting the credential is
2118    /// fine, when in truth `generate-jitconfig` was missing
2119    /// `Administration: write`. The lockout's signature is narrower than "a
2120    /// `403` while the count is set" — it is a `403` on the one retry this
2121    /// client itself issued after this request's own `401`.
2122    ///
2123    /// The count is deliberately *not* consulted. [`Attempt::Retry`] already
2124    /// means this request's own `401` incremented it moments ago, so reading it
2125    /// adds no signal — and does add a race that fails open: any concurrent
2126    /// request succeeding between the `401` and the retry `store(0)`s the
2127    /// counter, and a real lockout is then reported as a plain permissions
2128    /// refusal. A conjunct that can only ever weaken a safety check is worse
2129    /// than no conjunct.
2130    ///
2131    /// **The continuation.** Narrowing to the retry position opened a hole at
2132    /// the far end of the same back-off. When the back-off elapses and GitHub is
2133    /// still locking the credential out, the next request is a *first* attempt
2134    /// by construction — this client's retry never happened, because the request
2135    /// never reached the wire. The position rule then declined to call it a
2136    /// lockout, `classify` fell through to [`GithubError::Forbidden`] — whose
2137    /// documented reading is "the App installation does not grant it" — and the
2138    /// client **stopped backing off entirely**, hammering a credential GitHub
2139    /// had asked it to leave alone. That is the exact inverse of the
2140    /// Definition of Done's "backs off without retrying", and it failed for
2141    /// every lockout outliving one back-off.
2142    ///
2143    /// No counter is needed for that case either, because the response says so
2144    /// itself. GitHub's lockout carries `retry-after` and no parseable message;
2145    /// a permissions refusal carries a message naming what is not accessible and
2146    /// no `retry-after`. Requiring **both** halves of that signature is what
2147    /// keeps this from degenerating into "every `403` is a lockout": a
2148    /// permissions answer has a message, so it never matches, and a secondary
2149    /// rate limit has both a message and `retry-after`, so `is_rate_limited`
2150    /// takes it first.
2151    ///
2152    /// "No parseable message" is deliberately wider than "an empty body", which
2153    /// is how this used to be stated. See [`is_lockout_continuation`] for what
2154    /// else falls into it — a proxy's HTML error page most notably — and for why
2155    /// the resulting false positives are accepted rather than tightened away.
2156    ///
2157    /// This also settles a standing worry about [`MAX_LOCKOUT_BACKOFF`]. With
2158    /// the continuation recognised, the ceiling no longer decides whether the
2159    /// product ever gives up — it only decides how often it re-asks. A lockout
2160    /// longer than the ceiling now re-latches instead of being reported as a
2161    /// permissions failure, so the value is a polling interval rather than a
2162    /// deadline.
2163    fn is_lockout_403(&self, response: &ApiResponse, attempt: Attempt) -> bool {
2164        if is_rate_limited(response) {
2165            return false;
2166        }
2167        match attempt {
2168            Attempt::Retry => true,
2169            Attempt::First => is_lockout_continuation(response),
2170        }
2171    }
2172
2173    fn latch_lockout(&self, headers: &HeaderMap) -> Duration {
2174        // Clamp before latching. An unclamped `Retry-After` is a remote party
2175        // deciding how long this product stays down.
2176        let requested = retry_after(headers).unwrap_or(DEFAULT_LOCKOUT_BACKOFF);
2177        let clamped = requested.min(MAX_LOCKOUT_BACKOFF);
2178
2179        // A span too large for `chrono` must fall back to the default, never to
2180        // `None`: the old code's `.ok()` turned an absurd `Retry-After` into "no
2181        // lockout at all", which fails *open* — the exact inverse of what a
2182        // back-off is for, and reachable by a header alone. The clamp above
2183        // already makes this branch unreachable; it stays because the invariant
2184        // it protects ("latching always latches") is worth more than the line.
2185        let delta = chrono::TimeDelta::from_std(clamped).unwrap_or_else(|_| {
2186            chrono::TimeDelta::from_std(DEFAULT_LOCKOUT_BACKOFF)
2187                .expect("sixty seconds is a representable span")
2188        });
2189        // No third clamp. `clamped` is already `<= MAX_LOCKOUT_BACKOFF`, and
2190        // `TimeDelta` round-trips it exactly, so re-clamping here was dead twice
2191        // over — it could only ever re-apply a bound already applied, and the
2192        // fallback it guarded is `DEFAULT_LOCKOUT_BACKOFF`, which is smaller
2193        // than the ceiling by construction.
2194        let backoff = delta.to_std().unwrap_or(DEFAULT_LOCKOUT_BACKOFF);
2195
2196        let mut state = self.lockout.lock().expect("lockout lock poisoned");
2197        state.backoff = backoff;
2198        state.until = Some(self.clock.now() + delta);
2199        backoff
2200    }
2201
2202    /// One HTTP round trip with the standard headers applied and no
2203    /// interpretation of the result.
2204    async fn send_raw(&self, request: &ApiRequest) -> Result<ApiResponse, GithubError> {
2205        let url = self.resolve(&request.path)?;
2206        let mut builder = self
2207            .http
2208            .request(request.method.clone(), url)
2209            .header(reqwest::header::ACCEPT, GITHUB_ACCEPT)
2210            .header(reqwest::header::USER_AGENT, USER_AGENT)
2211            .header("X-GitHub-Api-Version", GITHUB_API_VERSION)
2212            // The only place the token is ever written onto the wire. It is
2213            // never logged, and `reqwest` does not render headers in its errors.
2214            .header(
2215                reqwest::header::AUTHORIZATION,
2216                format!("Bearer {}", self.bearer()),
2217            );
2218        if !request.query.is_empty() {
2219            builder = builder.query(&request.query);
2220        }
2221        if let Some(body) = &request.body {
2222            builder = builder.json(body);
2223        }
2224        if let Some(validators) = &request.validators {
2225            if let Some(etag) = &validators.etag {
2226                builder = builder.header(reqwest::header::IF_NONE_MATCH, etag);
2227            }
2228            if let Some(modified) = &validators.last_modified {
2229                builder = builder.header(reqwest::header::IF_MODIFIED_SINCE, modified);
2230            }
2231        }
2232
2233        let response = builder.send().await.map_err(transport)?;
2234        let status = response.status();
2235        let headers = response.headers().clone();
2236        // Counted on the headers, before the body: GitHub charged the request
2237        // whether or not the body then arrives whole.
2238        self.traffic.record(status, &headers, self.clock.now());
2239        let body = response.bytes().await.map_err(transport)?.to_vec();
2240
2241        tracing::debug!(
2242            method = request.method.as_str(),
2243            path = %request.path,
2244            status = status.as_u16(),
2245            body_bytes = body.len(),
2246            "github api request"
2247        );
2248
2249        Ok(ApiResponse {
2250            status,
2251            headers,
2252            body,
2253        })
2254    }
2255
2256    fn resolve(&self, path: &str) -> Result<Url, GithubError> {
2257        if path.starts_with("http://") || path.starts_with("https://") {
2258            return Url::parse(path).map_err(|_| GithubError::Malformed {
2259                what: "an absolute request URL",
2260                value: path.to_string(),
2261            });
2262        }
2263        self.endpoints
2264            .api_base
2265            .join(path.trim_start_matches('/'))
2266            .map_err(|_| GithubError::Malformed {
2267                what: "a request path",
2268                value: path.to_string(),
2269            })
2270    }
2271}
2272
2273enum Classified {
2274    Ok,
2275    Unauthorized,
2276    Error(GithubError),
2277}
2278
2279/// Which of a request's at-most-two attempts produced a response.
2280///
2281/// The authentication lockout is defined by *position*, not just by status: it
2282/// is what GitHub answers the retry that follows a `401`. Passing this in makes
2283/// that explicit at both call sites instead of inferring it from a counter that
2284/// outlives the request.
2285#[derive(Debug, Clone, Copy, PartialEq, Eq)]
2286enum Attempt {
2287    First,
2288    Retry,
2289}
2290
2291/// Whether GitHub attributed a failing response to its own rate limit.
2292///
2293/// Reading the evidence, and nothing else — see [`GithubError`]'s note on why
2294/// the headers travel with the error. `c3` decides what to do about it.
2295fn is_rate_limited(response: &ApiResponse) -> bool {
2296    if response.status == StatusCode::TOO_MANY_REQUESTS {
2297        return true;
2298    }
2299    // The primary rate limit's documented signature.
2300    if response
2301        .header("x-ratelimit-remaining")
2302        .is_some_and(|v| v.trim() == "0")
2303    {
2304        return true;
2305    }
2306    // A secondary rate limit sends `retry-after` — but so does the
2307    // authentication lockout, so that header alone cannot tell them apart.
2308    // GitHub's own message ("You have exceeded a secondary rate limit") can.
2309    error_message(&response.body).is_some_and(|m| m.to_ascii_lowercase().contains("rate limit"))
2310}
2311
2312/// Whether a `403` on a *first* attempt is GitHub continuing an authentication
2313/// lockout that outlived this client's back-off.
2314///
2315/// The two halves are both required, and both are GitHub's own evidence rather
2316/// than this client's memory:
2317///
2318/// * **`retry-after` is present.** GitHub sends it when it wants to be left
2319///   alone. A permissions refusal never does — there is nothing to wait for.
2320///
2321///   The header's *presence* is what is tested, not whether it parses, and that
2322///   is a fix rather than laziness. [`retry_after`] reads **integer seconds
2323///   only**, while RFC 9110 §10.2.3 also permits an HTTP-date. Gating detection
2324///   on `retry_after(..).is_some()` meant a date-form header was not recognised
2325///   as a continuation at all, and the bug this function exists to fix came
2326///   straight back for that shape — silently, since the response still looks
2327///   like an ordinary [`GithubError::Forbidden`] on the way out.
2328///
2329///   How long to wait stays a separate question, still answered by the integer
2330///   parse: [`AuthenticatedClient::latch_lockout`] already falls back to
2331///   [`DEFAULT_LOCKOUT_BACKOFF`] for a header it cannot read, so a date-form
2332///   header now latches sixty seconds instead of latching nothing. GitHub sends
2333///   integer seconds in practice; the point is not to depend on that.
2334/// * **The body carries no parseable GitHub message.** A permissions refusal
2335///   always names what is not accessible ("Resource not accessible by
2336///   integration"); the lockout's does not. This is the half that stops the rule
2337///   from swallowing [`GithubError::Forbidden`] entirely.
2338///
2339///   "No parseable GitHub message" is wider than "the body is empty", which is
2340///   how this used to be written, and the difference is worth stating because it
2341///   is what the code actually tests. [`error_message`] returns `None` for *any*
2342///   body that is not JSON carrying a non-empty `message`: an HTML error page
2343///   from a proxy or a CDN, a JSON body carrying only `documentation_url`, plain
2344///   text, a truncated response. Proxies routinely send `Retry-After` too, so a
2345///   `403` that never came from GitHub at all can read as an authentication
2346///   lockout here.
2347///
2348///   That is accepted rather than missed. The cost is bounded — the client
2349///   waits, clamped by [`MAX_LOCKOUT_BACKOFF`], then re-asks — and the direction
2350///   is the safe one: treating a strange `403` as "wait" costs latency, while
2351///   treating a real lockout as a permissions answer costs the back-off
2352///   entirely and tells the operator to fix a grant that is not missing.
2353///   Tightening it would mean asserting the body *is* GitHub's, which is
2354///   precisely the assertion an intercepting proxy makes false.
2355///
2356/// Callers reach this through [`AuthenticatedClient::is_lockout_403`], which
2357/// rules out a rate limit first — a secondary rate limit carries `retry-after`
2358/// *and* a message, so it fails this test on the second half anyway, but the
2359/// ordering makes the precedence explicit rather than incidental.
2360fn is_lockout_continuation(response: &ApiResponse) -> bool {
2361    response.headers.contains_key("retry-after") && error_message(&response.body).is_none()
2362}
2363
2364// ---------------------------------------------------------------------------
2365// Installation discovery
2366// ---------------------------------------------------------------------------
2367
2368/// Whether an installation can reach every repository on its account, or only
2369/// the ones the user picked.
2370#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
2371#[serde(rename_all = "snake_case")]
2372pub enum RepositorySelection {
2373    /// Every repository on the account, including ones created later.
2374    All,
2375    /// Only the repositories the user chose at install time.
2376    Selected,
2377}
2378
2379impl RepositorySelection {
2380    /// `07-security.md`: "`auth status` shows which repositories the token can
2381    /// reach, so an over-broad installation is visible rather than assumed."
2382    /// This is the flag that makes it visible.
2383    #[must_use]
2384    pub fn is_over_broad(self) -> bool {
2385        matches!(self, Self::All)
2386    }
2387}
2388
2389/// Whose account an installation sits on.
2390#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
2391#[serde(rename_all = "snake_case")]
2392pub enum InstallationAccount {
2393    User(String),
2394    Organization(Org),
2395    /// An enterprise account.
2396    ///
2397    /// It is its own variant rather than a [`InstallationAccount::User`]
2398    /// because it is not one, and `auth status` says out loud whose account
2399    /// each installation sits on. Everything GitHub reports without
2400    /// `type: "Organization"` used to fall into `User`, so an enterprise was
2401    /// labelled a user — a wrong statement about the operator's own account, on
2402    /// the one screen that exists to tell them what their credential reaches.
2403    ///
2404    /// It contributes nothing to [`ReachableTargets::organizations`], and that
2405    /// is correct rather than a second bug: an enterprise is not an
2406    /// organization, and `GET /orgs/{org}/actions/runners` does not accept one.
2407    /// The distinction is only visible now because the label is.
2408    Enterprise(String),
2409}
2410
2411impl InstallationAccount {
2412    #[must_use]
2413    pub fn login(&self) -> &str {
2414        match self {
2415            Self::User(login) | Self::Enterprise(login) => login,
2416            Self::Organization(org) => org.as_str(),
2417        }
2418    }
2419
2420    /// The organization, when the account is one. An organization account is a
2421    /// reachable *target* in its own right (D18): a policy may scale for the
2422    /// whole organization.
2423    #[must_use]
2424    pub fn organization(&self) -> Option<&Org> {
2425        match self {
2426            Self::Organization(org) => Some(org),
2427            Self::User(_) | Self::Enterprise(_) => None,
2428        }
2429    }
2430
2431    /// What to call this account in `auth status`. `f1` renders it; nothing in
2432    /// this crate branches on it.
2433    #[must_use]
2434    pub fn kind(&self) -> &'static str {
2435        match self {
2436            Self::User(_) => "user",
2437            Self::Organization(_) => "organization",
2438            Self::Enterprise(_) => "enterprise",
2439        }
2440    }
2441}
2442
2443impl fmt::Display for InstallationAccount {
2444    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
2445        f.write_str(self.login())
2446    }
2447}
2448
2449/// One installation of the published App, and what it can actually reach.
2450///
2451/// Serializable, as is everything [`InstallationDiscovery`] holds, so that a
2452/// service can answer a discovery for a command that cannot read the
2453/// credential itself (an SSH session on a Mac, whose login keychain it cannot
2454/// unlock). Nothing in it is a secret.
2455#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
2456pub struct Installation {
2457    pub id: u64,
2458    pub account: InstallationAccount,
2459    pub repository_selection: RepositorySelection,
2460    pub repositories: Vec<OwnerRepo>,
2461    /// The permissions GitHub reports for this installation, as
2462    /// `name -> level`. Surfaced verbatim so `auth status` can show a grant the
2463    /// user did not expect rather than assert the published set was applied.
2464    pub permissions: Vec<(String, String)>,
2465}
2466
2467impl Installation {
2468    #[must_use]
2469    pub fn is_over_broad(&self) -> bool {
2470        self.repository_selection.is_over_broad()
2471    }
2472}
2473
2474/// Everything the stored credential can reach.
2475#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
2476pub struct ReachableTargets {
2477    installations: Vec<Installation>,
2478    skipped: usize,
2479}
2480
2481impl ReachableTargets {
2482    #[must_use]
2483    pub fn installations(&self) -> &[Installation] {
2484        &self.installations
2485    }
2486
2487    /// How many installations GitHub reported that this client could not
2488    /// describe, and therefore left out of everything above.
2489    ///
2490    /// Non-zero means this report is **incomplete**, not merely small: whatever
2491    /// those installations reach is absent from
2492    /// [`ReachableTargets::repositories`] and
2493    /// [`ReachableTargets::organizations`]. `auth status` should say so, because
2494    /// the alternative is an operator reading a short list as a complete one.
2495    #[must_use]
2496    pub fn skipped(&self) -> usize {
2497        self.skipped
2498    }
2499
2500    /// Every repository the credential can reach, sorted and de-duplicated.
2501    #[must_use]
2502    pub fn repositories(&self) -> Vec<OwnerRepo> {
2503        let mut all: Vec<OwnerRepo> = self
2504            .installations
2505            .iter()
2506            .flat_map(|i| i.repositories.iter().cloned())
2507            .collect();
2508        all.sort();
2509        all.dedup();
2510        all
2511    }
2512
2513    /// Every organization the App is installed on, sorted and de-duplicated.
2514    #[must_use]
2515    pub fn organizations(&self) -> Vec<Org> {
2516        let mut all: Vec<Org> = self
2517            .installations
2518            .iter()
2519            .filter_map(|i| i.account.organization().cloned())
2520            .collect();
2521        all.sort();
2522        all.dedup();
2523        all
2524    }
2525
2526    /// The installations that hold `repository_selection: all`.
2527    #[must_use]
2528    pub fn over_broad(&self) -> Vec<&Installation> {
2529        self.installations
2530            .iter()
2531            .filter(|i| i.is_over_broad())
2532            .collect()
2533    }
2534
2535    #[must_use]
2536    pub fn is_empty(&self) -> bool {
2537        self.repositories().is_empty() && self.organizations().is_empty()
2538    }
2539}
2540
2541/// What `auth status` and `auth login` show after a successful sign-in.
2542#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
2543#[serde(rename_all = "snake_case")]
2544pub enum InstallationDiscovery {
2545    /// The credential is valid, GitHub reported nothing this client could not
2546    /// describe, and still nothing is reachable: the App is installed nowhere,
2547    /// or on nothing. `03-control-flows.md` flow 1.1 requires the installation
2548    /// URL here, and the URL is the remediation.
2549    NotInstalled {
2550        #[serde(with = "url_text")]
2551        install_url: Url,
2552    },
2553    /// Nothing is reachable, but at least one installation was **skipped**, so
2554    /// this client cannot tell "not installed" from "installed on something it
2555    /// could not describe".
2556    ///
2557    /// # Why this variant exists at all
2558    ///
2559    /// Skipping an unnameable installation is the right trade — one odd
2560    /// installation must not take down `auth status` for every other one — but
2561    /// it was made silently, and the silence flipped a verdict. An account this
2562    /// client cannot name, on the *only* installation the credential has, used
2563    /// to collapse to [`InstallationDiscovery::NotInstalled`], and `auth status`
2564    /// then handed an already-installed operator the "install the App" URL. That
2565    /// is a wrong remediation on the only authentication path there is,
2566    /// contradicted by nothing louder than a `warn!` in a log the operator is
2567    /// not reading.
2568    ///
2569    /// So the skip stays and the verdict does not flip. There is deliberately no
2570    /// `install_url` here: the whole point is that this client does not know
2571    /// whether installing is the remedy, and offering the URL anyway would put
2572    /// the wrong answer back one field over. `f1` says "1 installation could not
2573    /// be described" and stops there, which is true.
2574    Indeterminate { skipped: usize },
2575    /// The credential reaches at least one repository or organization. It may
2576    /// still be an incomplete picture — see [`ReachableTargets::skipped`].
2577    Installed(ReachableTargets),
2578}
2579
2580impl InstallationDiscovery {
2581    #[must_use]
2582    pub fn targets(&self) -> Option<&ReachableTargets> {
2583        match self {
2584            Self::Installed(t) => Some(t),
2585            Self::NotInstalled { .. } | Self::Indeterminate { .. } => None,
2586        }
2587    }
2588
2589    /// The installation URL, and *only* when installing is actually the
2590    /// remediation. See [`InstallationDiscovery::Indeterminate`].
2591    #[must_use]
2592    pub fn install_url(&self) -> Option<&Url> {
2593        match self {
2594            Self::NotInstalled { install_url } => Some(install_url),
2595            Self::Installed(_) | Self::Indeterminate { .. } => None,
2596        }
2597    }
2598
2599    /// How many installations GitHub reported that this client could not
2600    /// describe, whichever verdict was reached. One call for `f1`, so that
2601    /// "this report is incomplete" does not depend on which variant it landed
2602    /// in.
2603    #[must_use]
2604    pub fn skipped(&self) -> usize {
2605        match self {
2606            Self::NotInstalled { .. } => 0,
2607            Self::Indeterminate { skipped } => *skipped,
2608            Self::Installed(targets) => targets.skipped(),
2609        }
2610    }
2611}
2612
2613/// A [`Url`] as its text: this workspace builds `url` without its `serde`
2614/// feature.
2615mod url_text {
2616    use serde::{Deserialize as _, Deserializer, Serializer};
2617    use url::Url;
2618
2619    pub(super) fn serialize<S: Serializer>(url: &Url, serializer: S) -> Result<S::Ok, S::Error> {
2620        serializer.serialize_str(url.as_str())
2621    }
2622
2623    pub(super) fn deserialize<'de, D: Deserializer<'de>>(deserializer: D) -> Result<Url, D::Error> {
2624        let text = String::deserialize(deserializer)?;
2625        Url::parse(&text).map_err(serde::de::Error::custom)
2626    }
2627}
2628
2629#[derive(Debug, Deserialize)]
2630struct InstallationsPage {
2631    /// GitHub reports the size of the whole collection on every page. Decoding
2632    /// it costs nothing and turns silent under-collection into a visible
2633    /// warning — see [`under_collected`].
2634    #[serde(default)]
2635    total_count: Option<u64>,
2636    #[serde(default)]
2637    installations: Vec<RawInstallation>,
2638}
2639
2640#[derive(Debug, Deserialize)]
2641struct RawInstallation {
2642    id: u64,
2643    /// **Nullable.** GitHub's published `installation` schema types `account` as
2644    /// nullable, so a required field here would fail the *whole* decode — and
2645    /// with it all of `discover_installations`, which is all of `auth status` —
2646    /// over one installation whose account this client did not need to name.
2647    #[serde(default)]
2648    account: Option<RawAccount>,
2649    #[serde(default)]
2650    repository_selection: Option<String>,
2651    #[serde(default)]
2652    permissions: std::collections::BTreeMap<String, String>,
2653}
2654
2655/// An installation's account, which is *not* always a simple user.
2656///
2657/// GitHub's schema makes `account` either a simple-user or an enterprise, and an
2658/// enterprise carries `slug` and `name` where a user carries `login`. Requiring
2659/// `login` therefore made an enterprise installation a hard decode failure of
2660/// the entire response. All three are optional here and
2661/// [`RawAccount::display_login`] takes the first usable one.
2662#[derive(Debug, Deserialize)]
2663struct RawAccount {
2664    #[serde(default)]
2665    login: Option<String>,
2666    /// An enterprise account's stable identifier.
2667    #[serde(default)]
2668    slug: Option<String>,
2669    /// An enterprise account's display name, the last resort.
2670    #[serde(default)]
2671    name: Option<String>,
2672    #[serde(rename = "type", default)]
2673    account_type: Option<String>,
2674}
2675
2676impl RawAccount {
2677    fn display_login(&self) -> Option<&str> {
2678        [
2679            self.login.as_deref(),
2680            self.slug.as_deref(),
2681            self.name.as_deref(),
2682        ]
2683        .into_iter()
2684        .flatten()
2685        .find(|value| !value.is_empty())
2686    }
2687
2688    /// An account with no `login` that still names itself is an enterprise:
2689    /// `slug`/`name` is the enterprise shape, and every simple-user and
2690    /// organization account carries `login`.
2691    fn is_enterprise_shaped(&self) -> bool {
2692        self.login.as_deref().is_none_or(str::is_empty)
2693            && (self.slug.as_deref().is_some_and(|s| !s.is_empty())
2694                || self.name.as_deref().is_some_and(|s| !s.is_empty()))
2695    }
2696}
2697
2698#[derive(Debug, Deserialize)]
2699struct RepositoriesPage {
2700    #[serde(default)]
2701    total_count: Option<u64>,
2702    #[serde(default)]
2703    repositories: Vec<RawRepository>,
2704}
2705
2706#[derive(Debug, Deserialize)]
2707struct RawRepository {
2708    full_name: String,
2709}
2710
2711/// How many items a paginated collection said it had, when that is more than
2712/// arrived.
2713///
2714/// This is the cheapest possible check and it is worth more than it looks. The
2715/// `Link`-header parser used to lose the relation whenever a page URL contained
2716/// a comma, which stopped pagination at page 1 — and *nothing* noticed, because
2717/// a short answer and a complete answer are the same shape. Cross-checking the
2718/// count GitHub itself reported turns that class of bug from a wrong answer into
2719/// a logged warning. A collection larger than `total_count` is not reported:
2720/// GitHub can legitimately grow a collection between pages.
2721fn under_collected(collected: usize, total_count: Option<u64>) -> Option<u64> {
2722    let total = total_count?;
2723    (total > collected as u64).then_some(total)
2724}
2725
2726impl AuthenticatedClient {
2727    /// Which repositories and organizations the stored credential can actually
2728    /// reach.
2729    ///
2730    /// Two calls, both paginated: `GET /user/installations`, then
2731    /// `GET /user/installations/{id}/repositories` per installation. The shapes
2732    /// are the ones the D18 spike observed live
2733    /// (`docs/spikes/d18-org-jit-verification.md`, "The permission that
2734    /// authorized it").
2735    ///
2736    /// An installation is reported even when it is broader than the user
2737    /// expected — [`Installation::is_over_broad`] — because `07-security.md`
2738    /// requires that an over-broad installation be *visible* rather than
2739    /// assumed. Nothing here narrows or hides one.
2740    ///
2741    /// # Errors
2742    /// Every variant of [`GithubError`]. A `401` here goes through the same
2743    /// single-flight re-validation as any other request.
2744    pub async fn discover_installations(
2745        &self,
2746        app: &AppRegistration,
2747    ) -> Result<InstallationDiscovery, GithubError> {
2748        let mut installations = Vec::new();
2749        let mut skipped = 0_usize;
2750        for raw in self.all_installations().await? {
2751            // A null or nameless account is skipped rather than fatal. GitHub
2752            // types this field as nullable, and one unnameable installation must
2753            // not take down `auth status` for every other one — but it is also
2754            // not something to swallow quietly, because the repositories behind
2755            // it are then absent from the reported reach. The count is what
2756            // carries that out of here; a `warn!` alone let the skip change the
2757            // verdict with nothing to say so.
2758            let Some(login) = raw.account.as_ref().and_then(RawAccount::display_login) else {
2759                skipped += 1;
2760                tracing::warn!(
2761                    installation_id = raw.id,
2762                    "skipping an installation GitHub reported with no nameable account; \
2763                     anything it reaches is missing from this report"
2764                );
2765                continue;
2766            };
2767            let account_type = raw.account.as_ref().and_then(|a| a.account_type.as_deref());
2768            let account = match account_type {
2769                Some("Organization") => {
2770                    InstallationAccount::Organization(Org::new(login).map_err(|_| {
2771                        GithubError::Malformed {
2772                            what: "an installation account login",
2773                            value: login.to_string(),
2774                        }
2775                    })?)
2776                }
2777                Some("Enterprise") => InstallationAccount::Enterprise(login.to_string()),
2778                // An enterprise is also reported with no `type` at all, carrying
2779                // `slug`/`name` where a user carries `login` — which is the
2780                // shape D18 observed and the shape `display_login` exists for.
2781                // Recognising it by that shape is what stops it being labelled a
2782                // user by default.
2783                _ if raw
2784                    .account
2785                    .as_ref()
2786                    .is_some_and(RawAccount::is_enterprise_shaped) =>
2787                {
2788                    InstallationAccount::Enterprise(login.to_string())
2789                }
2790                _ => InstallationAccount::User(login.to_string()),
2791            };
2792            let repository_selection = match raw.repository_selection.as_deref() {
2793                Some("all") => RepositorySelection::All,
2794                _ => RepositorySelection::Selected,
2795            };
2796            installations.push(Installation {
2797                id: raw.id,
2798                account,
2799                repository_selection,
2800                repositories: self.installation_repositories(raw.id).await?,
2801                permissions: raw.permissions.into_iter().collect(),
2802            });
2803        }
2804
2805        let targets = ReachableTargets {
2806            installations,
2807            skipped,
2808        };
2809        if targets.is_empty() {
2810            // "Nothing reachable" and "nothing this client could describe" are
2811            // different answers, and only the first one is fixed by installing
2812            // the App. Reporting them as the same answer is how an
2813            // already-installed operator was handed an install URL.
2814            if skipped > 0 {
2815                tracing::warn!(
2816                    skipped,
2817                    "every installation GitHub reported was skipped; whether the App is \
2818                     installed cannot be determined from this credential"
2819                );
2820                return Ok(InstallationDiscovery::Indeterminate { skipped });
2821            }
2822            let install_url = app.install_url(&self.endpoints);
2823            tracing::info!(
2824                install_url = %install_url,
2825                "the published App is not installed on anything this credential can reach"
2826            );
2827            return Ok(InstallationDiscovery::NotInstalled { install_url });
2828        }
2829        tracing::info!(
2830            repositories = targets.repositories().len(),
2831            organizations = targets.organizations().len(),
2832            over_broad = targets.over_broad().len(),
2833            skipped,
2834            "discovered the targets this credential can reach"
2835        );
2836        Ok(InstallationDiscovery::Installed(targets))
2837    }
2838
2839    async fn all_installations(&self) -> Result<Vec<RawInstallation>, GithubError> {
2840        let mut out = Vec::new();
2841        let mut total_count = None;
2842        let mut next = Some(ApiRequest::get("/user/installations").query("per_page", 100));
2843        let mut pages = 0_usize;
2844        while let Some(request) = next.take() {
2845            let response = self.send(&request).await?;
2846            let page: InstallationsPage = response.json()?;
2847            total_count = page.total_count.or(total_count);
2848            out.extend(page.installations);
2849
2850            pages += 1;
2851            if pages >= MAX_PAGES {
2852                tracing::warn!(
2853                    pages,
2854                    collected = out.len(),
2855                    "stopped following installation pages at the ceiling; a `Link: rel=next` \
2856                     that never ends would otherwise loop forever"
2857                );
2858                break;
2859            }
2860            next = response
2861                .next_page()
2862                .map(|url| ApiRequest::get(url.as_str()));
2863        }
2864        if let Some(expected) = under_collected(out.len(), total_count) {
2865            tracing::warn!(
2866                expected,
2867                collected = out.len(),
2868                "GitHub reported more installations than pagination collected; the reported \
2869                 reach is incomplete"
2870            );
2871        }
2872        Ok(out)
2873    }
2874
2875    async fn installation_repositories(&self, id: u64) -> Result<Vec<OwnerRepo>, GithubError> {
2876        let mut out = Vec::new();
2877        let mut total_count = None;
2878        let mut next = Some(
2879            ApiRequest::get(format!("/user/installations/{id}/repositories"))
2880                .query("per_page", 100),
2881        );
2882        let mut pages = 0_usize;
2883        while let Some(request) = next.take() {
2884            let response = self.send(&request).await?;
2885            let page: RepositoriesPage = response.json()?;
2886            total_count = page.total_count.or(total_count);
2887            for repo in page.repositories {
2888                out.push(OwnerRepo::parse(&repo.full_name).map_err(|_| {
2889                    GithubError::Malformed {
2890                        what: "a repository full_name",
2891                        value: repo.full_name.clone(),
2892                    }
2893                })?);
2894            }
2895
2896            pages += 1;
2897            if pages >= MAX_PAGES {
2898                tracing::warn!(
2899                    installation_id = id,
2900                    pages,
2901                    collected = out.len(),
2902                    "stopped following repository pages at the ceiling; a `Link: rel=next` \
2903                     that never ends would otherwise loop forever"
2904                );
2905                break;
2906            }
2907            next = response
2908                .next_page()
2909                .map(|url| ApiRequest::get(url.as_str()));
2910        }
2911        if let Some(expected) = under_collected(out.len(), total_count) {
2912            tracing::warn!(
2913                installation_id = id,
2914                expected,
2915                collected = out.len(),
2916                "GitHub reported more repositories than pagination collected; this \
2917                 installation's reach is under-reported"
2918            );
2919        }
2920        Ok(out)
2921    }
2922}
2923
2924/// Test support shared by this file and [`device_flow`].
2925///
2926/// It lives inline rather than in `src/testing.rs` on purpose. `a1` laid out
2927/// this crate's five source files — `lib.rs`, `device_flow.rs`, `rest.rs`,
2928/// `demand.rs`, `jit.rs` — and owns every manifest; `c3` and `c4` are working in
2929/// the same directory in parallel, and a new file there is a merge conflict
2930/// waiting to happen for no benefit. An inline `#[cfg(test)]` module is
2931/// reachable as `crate::testing` from every module in the crate and adds nothing
2932/// to a release build.
2933///
2934/// It does not live in `runner-manager-testkit` either, and that one is
2935/// mechanical: `testkit` depends on `runner-manager-github`, so a unit test
2936/// inside this crate that used a `testkit` helper would link a *second* instance
2937/// of this library and the two instances' types would not unify — the same
2938/// hazard `testkit`'s own crate documentation records for `domain`.
2939#[cfg(test)]
2940pub(crate) mod testing {
2941    use super::*;
2942    use serde_json::{Value, json};
2943    use std::sync::{Mutex, atomic::AtomicUsize};
2944    use wiremock::{Request, Respond, ResponseTemplate};
2945
2946    /// Shaped like a real `ghu_` token, and unmistakably not one.
2947    pub const FIXTURE_TOKEN: &str = "ghu_fixtureTOKENnotARealCredential00";
2948    /// Shaped like a real device code, and unmistakably not one.
2949    pub const FIXTURE_DEVICE_CODE: &str = "fixture-device-code-0e37a9c1b4d84f2a";
2950    /// The example user code from RFC 8628.
2951    pub const FIXTURE_USER_CODE: &str = "WDJB-MJHT";
2952
2953    /// A clock the test moves.
2954    ///
2955    /// Deliberately not `runner_manager_testkit::clock::FakeClock`; see this
2956    /// module's documentation for why a `testkit` import is not available here.
2957    #[derive(Debug)]
2958    pub struct TestClock {
2959        now: Mutex<Timestamp>,
2960    }
2961
2962    impl TestClock {
2963        /// # Panics
2964        /// If a previous holder panicked while the lock was held.
2965        pub fn advance_secs(&self, secs: i64) {
2966            let mut now = self.now.lock().expect("TestClock lock poisoned");
2967            *now += chrono::TimeDelta::seconds(secs);
2968        }
2969    }
2970
2971    impl Default for TestClock {
2972        fn default() -> Self {
2973            // 2026-08-21T00:00:00Z, the date this taskflow's decisions were
2974            // locked — the same epoch `testkit`'s clock starts at.
2975            Self {
2976                now: Mutex::new(
2977                    chrono::DateTime::from_timestamp(1_787_270_400, 0).expect("a valid instant"),
2978                ),
2979            }
2980        }
2981    }
2982
2983    impl Clock for TestClock {
2984        fn now(&self) -> Timestamp {
2985            *self.now.lock().expect("TestClock lock poisoned")
2986        }
2987    }
2988
2989    /// A sleeper that records what it was asked to wait and returns at once.
2990    ///
2991    /// This is what turns "`slow_down` demonstrably increases the poll interval"
2992    /// into an equality assertion on a `Vec<Duration>`.
2993    #[derive(Debug, Default)]
2994    pub struct RecordingSleeper {
2995        recorded: Mutex<Vec<Duration>>,
2996    }
2997
2998    impl RecordingSleeper {
2999        /// # Panics
3000        /// If a previous holder panicked while the lock was held.
3001        pub fn recorded(&self) -> Vec<Duration> {
3002            self.recorded.lock().expect("sleeper lock poisoned").clone()
3003        }
3004    }
3005
3006    #[async_trait::async_trait]
3007    impl Sleeper for RecordingSleeper {
3008        async fn sleep(&self, duration: Duration) {
3009            self.recorded
3010                .lock()
3011                .expect("sleeper lock poisoned")
3012                .push(duration);
3013        }
3014    }
3015
3016    /// Answers from a fixed script, one entry per call, repeating the last.
3017    pub struct Script {
3018        responses: Vec<ResponseTemplate>,
3019        calls: AtomicUsize,
3020    }
3021
3022    impl Script {
3023        #[must_use]
3024        pub fn new(responses: Vec<ResponseTemplate>) -> Self {
3025            assert!(
3026                !responses.is_empty(),
3027                "a script needs at least one response"
3028            );
3029            Self {
3030                responses,
3031                calls: AtomicUsize::new(0),
3032            }
3033        }
3034    }
3035
3036    impl Respond for Script {
3037        fn respond(&self, _: &Request) -> ResponseTemplate {
3038            let i = self.calls.fetch_add(1, Ordering::SeqCst);
3039            self.responses[i.min(self.responses.len() - 1)].clone()
3040        }
3041    }
3042
3043    /// `POST https://github.com/login/device/code` → `200`, in the shape both
3044    /// spikes observed (`docs/spikes/d17-spike.ps1`).
3045    #[must_use]
3046    pub fn device_code_body(server_uri: &str, interval: u64, expires_in: u64) -> Value {
3047        json!({
3048            "device_code": FIXTURE_DEVICE_CODE,
3049            "user_code": FIXTURE_USER_CODE,
3050            "verification_uri": format!("{server_uri}/login/device"),
3051            "expires_in": expires_in,
3052            "interval": interval
3053        })
3054    }
3055
3056    /// `POST .../login/oauth/access_token` → `200` with an `error` field, which
3057    /// is how GitHub answers every state in the matrix.
3058    #[must_use]
3059    pub fn error_body(code: &str, interval: Option<u64>) -> Value {
3060        let mut body = json!({
3061            "error": code,
3062            "error_description": "see the OAuth 2.0 Device Authorization Grant",
3063            "error_uri": "https://docs.github.com/developers/apps/authorizing-oauth-apps"
3064        });
3065        if let Some(interval) = interval {
3066            body["interval"] = json!(interval);
3067        }
3068        body
3069    }
3070
3071    /// `POST .../login/oauth/access_token` → `200` with an approved token.
3072    #[must_use]
3073    pub fn token_body() -> Value {
3074        json!({ "access_token": FIXTURE_TOKEN, "token_type": "bearer", "scope": "" })
3075    }
3076
3077    /// `GET /user/installations` → `200`. The permission set is the one D18 read
3078    /// back from the live installation.
3079    #[must_use]
3080    pub fn installations_body(entries: &[(u64, &str, &str, &str)]) -> Value {
3081        let installations: Vec<Value> = entries
3082            .iter()
3083            .map(|(id, login, account_type, selection)| {
3084                json!({
3085                    "id": id,
3086                    "account": { "login": login, "type": account_type },
3087                    "repository_selection": selection,
3088                    "permissions": {
3089                        "actions": "read",
3090                        "administration": "write",
3091                        "metadata": "read",
3092                        "organization_self_hosted_runners": "write"
3093                    }
3094                })
3095            })
3096            .collect();
3097        json!({ "total_count": installations.len(), "installations": installations })
3098    }
3099
3100    /// `GET /user/installations/{id}/repositories` → `200`.
3101    #[must_use]
3102    pub fn repositories_body(full_names: &[&str]) -> Value {
3103        let repositories: Vec<Value> = full_names
3104            .iter()
3105            .map(|full_name| json!({ "full_name": full_name }))
3106            .collect();
3107        json!({ "total_count": repositories.len(), "repositories": repositories })
3108    }
3109
3110    // The `tracing` capture subscriber that used to live here now lives in
3111    // `tests/no_secret_reaches_the_logs.rs`, and the move is the point rather
3112    // than tidying. `tracing` caches a callsite's `Interest` process-wide while
3113    // `with_default` installs a subscriber only on the calling *thread*, so a
3114    // scan running alongside the crate's other unit tests captured nothing but
3115    // its own handful of events and passed with a real device-code leak in the
3116    // flow. A scan that is the only test in its process has no concurrent
3117    // thread to be poisoned by, and no `#[cfg(test)]` module here can offer
3118    // that guarantee.
3119    //
3120    // The blinding is a *concurrency* effect and not a permanent
3121    // first-registration one — see that file's header for the measurement that
3122    // separates the two. The distinction matters here because only the
3123    // concurrency reading implies what this comment concludes: that one test
3124    // per process is the fix.
3125}
3126
3127#[cfg(test)]
3128mod tests {
3129
3130    /// Upgrading must not log anybody out, and the App's expiration setting must
3131    /// be safe to turn on -- or back off -- with hosts mid-way through either.
3132    #[test]
3133    fn both_stored_shapes_load_and_a_pair_survives_a_round_trip() {
3134        // What every host stored before renewal existed: a bare token.
3135        let legacy = UserAccessToken::from_stored_document(&SecretString::from("ghu_legacy123"));
3136        assert_eq!(legacy.secret().expose_secret(), "ghu_legacy123");
3137        assert!(
3138            legacy.renewal().is_none(),
3139            "a bare token has no renewal half, and inventing one would make the client try to              refresh a credential the App never issued a refresh token for"
3140        );
3141
3142        // A pair, written and read back.
3143        let pair = UserAccessToken::new(SecretString::from("ghu_new")).with_renewal(
3144            Some(SecretString::from("ghr_new")),
3145            Some(28_800),
3146            Some(15_897_600),
3147        );
3148        let stored = pair.to_stored_document();
3149        let read = UserAccessToken::from_stored_document(&stored);
3150        assert_eq!(read.secret().expose_secret(), "ghu_new");
3151        let renewal = read.renewal().expect("the pair survives the round trip");
3152        assert_eq!(renewal.refresh_token().expose_secret(), "ghr_new");
3153        assert!(renewal.access_expires_at.is_some());
3154        assert!(renewal.refresh_expires_at.is_some());
3155
3156        // A credential with no renewal still writes the document shape, and
3157        // still reads back as having none.
3158        let bare_round_trip = UserAccessToken::from_stored_document(&legacy.to_stored_document());
3159        assert_eq!(bare_round_trip.secret().expose_secret(), "ghu_legacy123");
3160        assert!(bare_round_trip.renewal().is_none());
3161    }
3162
3163    /// The refresh token is the more dangerous half -- it mints access tokens
3164    /// for six months -- so it must not reach a log through `Debug`.
3165    #[test]
3166    fn a_refresh_token_never_appears_in_debug_output() {
3167        let pair = UserAccessToken::new(SecretString::from("ghu_x")).with_renewal(
3168            Some(SecretString::from("ghr_SUPERSECRET")),
3169            Some(1),
3170            Some(2),
3171        );
3172        let rendered = format!("{:?}", pair.renewal().expect("a renewal"));
3173        assert!(!rendered.contains("ghr_SUPERSECRET"), "{rendered}");
3174        assert!(rendered.contains("redacted"), "{rendered}");
3175    }
3176    use super::*;
3177    use crate::testing::{FIXTURE_TOKEN, Script, TestClock, installations_body, repositories_body};
3178    use serde_json::json;
3179    use wiremock::{
3180        Mock, MockServer, ResponseTemplate,
3181        matchers::{header, method, path},
3182    };
3183
3184    fn client(server: &MockServer, clock: Arc<TestClock>) -> AuthenticatedClient {
3185        AuthenticatedClient::new(
3186            Endpoints::for_test_server(&server.uri()).unwrap(),
3187            UserAccessToken::new(SecretString::from(FIXTURE_TOKEN)),
3188            clock,
3189        )
3190        .unwrap()
3191    }
3192
3193    fn app() -> AppRegistration {
3194        AppRegistration::new("Iv23liTESTCLIENTID", "runner-manager").unwrap()
3195    }
3196
3197    // -- picking up a credential somebody else stored -------------------------
3198
3199    /// A [`CredentialSource`] over a fixed answer, which is what a store looks
3200    /// like from here.
3201    #[derive(Debug)]
3202    struct StoreHolding(Option<&'static str>);
3203
3204    impl CredentialSource for StoreHolding {
3205        fn reload(&self) -> Option<UserAccessToken> {
3206            self.0
3207                .map(|token| UserAccessToken::new(SecretString::from(token)))
3208        }
3209    }
3210
3211    /// The 28-hour failure, as a test: a daemon holding a dead bare token, an
3212    /// operator who signs in, and nothing that tells the daemon.
3213    ///
3214    /// Bare on purpose. A pair would renew and never reach the store at all,
3215    /// which is why renewal alone did not cover this.
3216    #[tokio::test]
3217    async fn a_daemon_picks_up_a_sign_in_that_happened_after_it_started() {
3218        let server = MockServer::start().await;
3219        Mock::given(method("GET"))
3220            .and(path("/repos/acme/app"))
3221            .and(header("authorization", "Bearer ghu_dead"))
3222            .respond_with(ResponseTemplate::new(401))
3223            .expect(1)
3224            .mount(&server)
3225            .await;
3226        Mock::given(method("GET"))
3227            .and(path("/repos/acme/app"))
3228            .and(header("authorization", "Bearer ghu_freshly_signed_in"))
3229            .respond_with(ResponseTemplate::new(200).set_body_json(json!({"id": 1})))
3230            .expect(1)
3231            .mount(&server)
3232            .await;
3233
3234        let client = AuthenticatedClient::new(
3235            Endpoints::for_test_server(&server.uri()).unwrap(),
3236            UserAccessToken::new(SecretString::from("ghu_dead")),
3237            Arc::new(TestClock::default()),
3238        )
3239        .unwrap()
3240        .with_credential_source(Arc::new(StoreHolding(Some("ghu_freshly_signed_in"))));
3241
3242        client
3243            .send(&ApiRequest::get("/repos/acme/app"))
3244            .await
3245            .expect(
3246                "the 401 is retried with what the store holds now, without anybody \n                 restarting the daemon",
3247            );
3248    }
3249
3250    /// The other half, and the reason for the comparison in `reload_once`: a
3251    /// store that still holds the token that just failed is not news.
3252    ///
3253    /// Without the check, every `401` would answer "something changed, retry"
3254    /// and a genuinely revoked credential would spend two requests per poll
3255    /// forever instead of being reported.
3256    #[tokio::test]
3257    async fn a_store_holding_the_same_dead_token_is_not_worth_a_retry() {
3258        let server = MockServer::start().await;
3259        Mock::given(method("GET"))
3260            .and(path("/repos/acme/app"))
3261            .respond_with(ResponseTemplate::new(401))
3262            .expect(1)
3263            .mount(&server)
3264            .await;
3265        // The re-validation probe that runs once reload declines.
3266        Mock::given(method("GET"))
3267            .and(path("/user/installations"))
3268            .respond_with(ResponseTemplate::new(401))
3269            .expect(1)
3270            .mount(&server)
3271            .await;
3272
3273        let client = AuthenticatedClient::new(
3274            Endpoints::for_test_server(&server.uri()).unwrap(),
3275            UserAccessToken::new(SecretString::from("ghu_revoked")),
3276            Arc::new(TestClock::default()),
3277        )
3278        .unwrap()
3279        .with_credential_source(Arc::new(StoreHolding(Some("ghu_revoked"))));
3280
3281        let failure = client
3282            .send(&ApiRequest::get("/repos/acme/app"))
3283            .await
3284            .expect_err("a revoked credential is still revoked when the store agrees");
3285        assert!(
3286            matches!(failure, GithubError::AuthenticationFailed),
3287            "{failure:?}"
3288        );
3289    }
3290
3291    /// An unreadable store leaves the `401` exactly where it was, rather than
3292    /// turning a rejection into a different kind of error.
3293    #[tokio::test]
3294    async fn an_unreadable_store_changes_nothing_about_the_rejection() {
3295        let server = MockServer::start().await;
3296        Mock::given(method("GET"))
3297            .and(path("/repos/acme/app"))
3298            .respond_with(ResponseTemplate::new(401))
3299            .expect(1)
3300            .mount(&server)
3301            .await;
3302        Mock::given(method("GET"))
3303            .and(path("/user/installations"))
3304            .respond_with(ResponseTemplate::new(401))
3305            .expect(1)
3306            .mount(&server)
3307            .await;
3308
3309        let client = AuthenticatedClient::new(
3310            Endpoints::for_test_server(&server.uri()).unwrap(),
3311            UserAccessToken::new(SecretString::from("ghu_revoked")),
3312            Arc::new(TestClock::default()),
3313        )
3314        .unwrap()
3315        .with_credential_source(Arc::new(StoreHolding(None)));
3316
3317        let failure = client
3318            .send(&ApiRequest::get("/repos/acme/app"))
3319            .await
3320            .expect_err("nothing to pick up means the rejection stands");
3321        assert!(
3322            matches!(failure, GithubError::AuthenticationFailed),
3323            "{failure:?}"
3324        );
3325    }
3326
3327    // -- headers ------------------------------------------------------------
3328
3329    #[tokio::test]
3330    async fn every_request_states_its_api_version_and_accept_header() {
3331        let server = MockServer::start().await;
3332        Mock::given(method("GET"))
3333            .and(path("/user/installations"))
3334            .and(header("x-github-api-version", GITHUB_API_VERSION))
3335            .and(header("accept", GITHUB_ACCEPT))
3336            .and(header("authorization", format!("Bearer {FIXTURE_TOKEN}")))
3337            .and(header("user-agent", USER_AGENT))
3338            .respond_with(ResponseTemplate::new(200).set_body_json(installations_body(&[])))
3339            .expect(1)
3340            .mount(&server)
3341            .await;
3342
3343        let client = client(&server, Arc::new(TestClock::default()));
3344        client
3345            .send(&ApiRequest::get("/user/installations"))
3346            .await
3347            .expect("the mock only matches when all four headers are present");
3348    }
3349
3350    // -- the 401 path -------------------------------------------------------
3351
3352    #[tokio::test]
3353    async fn a_401_revalidates_once_and_retries_once_then_succeeds() {
3354        let server = MockServer::start().await;
3355        Mock::given(method("GET"))
3356            .and(path("/orgs/acme/actions/runners"))
3357            .respond_with(Script::new(vec![
3358                ResponseTemplate::new(401).set_body_json(json!({"message": "Bad credentials"})),
3359                ResponseTemplate::new(200).set_body_json(json!({"total_count": 0})),
3360            ]))
3361            .expect(2)
3362            .mount(&server)
3363            .await;
3364        Mock::given(method("GET"))
3365            .and(path("/user/installations"))
3366            .respond_with(ResponseTemplate::new(200).set_body_json(installations_body(&[])))
3367            .expect(1)
3368            .mount(&server)
3369            .await;
3370
3371        let client = client(&server, Arc::new(TestClock::default()));
3372        let response = client
3373            .send(&ApiRequest::get("/orgs/acme/actions/runners"))
3374            .await
3375            .expect("the retry succeeds");
3376
3377        assert_eq!(response.status(), StatusCode::OK);
3378        assert_eq!(
3379            client.revalidations_performed(),
3380            1,
3381            "one 401 must produce exactly one re-validation"
3382        );
3383    }
3384
3385    #[tokio::test]
3386    async fn a_second_401_after_the_retry_is_terminal_authentication_failure() {
3387        let server = MockServer::start().await;
3388        Mock::given(method("GET"))
3389            .and(path("/orgs/acme/actions/runners"))
3390            .respond_with(ResponseTemplate::new(401))
3391            .expect(2)
3392            .mount(&server)
3393            .await;
3394        Mock::given(method("GET"))
3395            .and(path("/user/installations"))
3396            .respond_with(ResponseTemplate::new(200).set_body_json(installations_body(&[])))
3397            .mount(&server)
3398            .await;
3399
3400        let client = client(&server, Arc::new(TestClock::default()));
3401        let err = client
3402            .send(&ApiRequest::get("/orgs/acme/actions/runners"))
3403            .await
3404            .expect_err("two 401s is terminal");
3405
3406        assert!(matches!(err, GithubError::AuthenticationFailed), "{err:?}");
3407        assert!(err.is_authentication());
3408        assert!(!err.is_lockout());
3409    }
3410
3411    #[tokio::test]
3412    async fn a_rejected_revalidation_fails_without_spending_the_retry() {
3413        let server = MockServer::start().await;
3414        Mock::given(method("GET"))
3415            .and(path("/orgs/acme/actions/runners"))
3416            .respond_with(ResponseTemplate::new(401))
3417            // Exactly one: a credential GitHub has confirmed dead must not be
3418            // used for a retry.
3419            .expect(1)
3420            .mount(&server)
3421            .await;
3422        Mock::given(method("GET"))
3423            .and(path("/user/installations"))
3424            .respond_with(ResponseTemplate::new(401))
3425            .expect(1)
3426            .mount(&server)
3427            .await;
3428
3429        let client = client(&server, Arc::new(TestClock::default()));
3430        let err = client
3431            .send(&ApiRequest::get("/orgs/acme/actions/runners"))
3432            .await
3433            .expect_err("the credential is dead");
3434        assert!(matches!(err, GithubError::AuthenticationFailed), "{err:?}");
3435    }
3436
3437    /// The Definition of Done's concurrency claim, tested with real concurrent
3438    /// callers on a multi-threaded runtime rather than by reasoning about the
3439    /// mutex.
3440    ///
3441    /// Two things make the assertion deterministic rather than lucky. A barrier
3442    /// releases all eight callers into `send` together, so all eight take their
3443    /// `401` before any of them reaches the gate; and the re-validation endpoint
3444    /// is delayed, so the first caller still holds the gate while the other
3445    /// seven sample the generation counter. Without the delay a caller could
3446    /// legitimately arrive after the first re-validation completed, which is a
3447    /// *new* `401` storm and correctly earns its own attempt.
3448    #[tokio::test(flavor = "multi_thread", worker_threads = 8)]
3449    async fn eight_concurrent_401s_produce_one_revalidation_not_eight() {
3450        const CALLERS: usize = 8;
3451
3452        let server = MockServer::start().await;
3453        Mock::given(method("GET"))
3454            .and(path("/orgs/acme/actions/runners"))
3455            .respond_with(ResponseTemplate::new(401))
3456            .mount(&server)
3457            .await;
3458        Mock::given(method("GET"))
3459            .and(path("/user/installations"))
3460            .respond_with(
3461                ResponseTemplate::new(200)
3462                    .set_body_json(installations_body(&[]))
3463                    .set_delay(Duration::from_millis(250)),
3464            )
3465            .expect(1)
3466            .mount(&server)
3467            .await;
3468
3469        let client = Arc::new(client(&server, Arc::new(TestClock::default())));
3470        let barrier = Arc::new(tokio::sync::Barrier::new(CALLERS));
3471        let mut tasks = Vec::new();
3472        for _ in 0..CALLERS {
3473            let client = Arc::clone(&client);
3474            let barrier = Arc::clone(&barrier);
3475            tasks.push(tokio::spawn(async move {
3476                barrier.wait().await;
3477                client
3478                    .send(&ApiRequest::get("/orgs/acme/actions/runners"))
3479                    .await
3480                    .expect_err("every caller sees a dead endpoint")
3481            }));
3482        }
3483
3484        let mut outcomes = Vec::new();
3485        for task in tasks {
3486            outcomes.push(task.await.expect("no caller panicked"));
3487        }
3488
3489        assert_eq!(outcomes.len(), CALLERS);
3490        for err in &outcomes {
3491            assert!(matches!(err, GithubError::AuthenticationFailed), "{err:?}");
3492        }
3493        assert_eq!(
3494            client.revalidations_performed(),
3495            1,
3496            "{CALLERS} concurrent 401s must produce ONE attempt, not {CALLERS}"
3497        );
3498
3499        // The same claim, measured from the server rather than from our own
3500        // counter: the mock's `.expect(1)` is verified when the server drops.
3501        let seen = server.received_requests().await.expect("recording is on");
3502        let probes = seen
3503            .iter()
3504            .filter(|r| r.url.path() == "/user/installations")
3505            .count();
3506        assert_eq!(probes, 1, "GitHub itself saw exactly one re-validation");
3507        let attempts = seen
3508            .iter()
3509            .filter(|r| r.url.path() == "/orgs/acme/actions/runners")
3510            .count();
3511        assert_eq!(
3512            attempts,
3513            CALLERS * 2,
3514            "each caller still gets its own single retry"
3515        );
3516    }
3517
3518    // -- the 403 path -------------------------------------------------------
3519
3520    #[tokio::test]
3521    async fn a_403_after_401s_is_a_lockout_and_not_an_authentication_failure() {
3522        let server = MockServer::start().await;
3523        Mock::given(method("GET"))
3524            .and(path("/orgs/acme/actions/runners"))
3525            .respond_with(Script::new(vec![
3526                ResponseTemplate::new(401),
3527                ResponseTemplate::new(403).insert_header("retry-after", "42"),
3528            ]))
3529            .mount(&server)
3530            .await;
3531        Mock::given(method("GET"))
3532            .and(path("/user/installations"))
3533            .respond_with(ResponseTemplate::new(200).set_body_json(installations_body(&[])))
3534            .mount(&server)
3535            .await;
3536
3537        let client = client(&server, Arc::new(TestClock::default()));
3538        let err = client
3539            .send(&ApiRequest::get("/orgs/acme/actions/runners"))
3540            .await
3541            .expect_err("403 after a 401");
3542
3543        match err {
3544            GithubError::AuthenticationLockout { retry_after } => {
3545                assert_eq!(retry_after, Duration::from_secs(42), "honours retry-after");
3546            }
3547            other => panic!("expected a lockout, got {other:?}"),
3548        }
3549        assert!(client.is_locked_out());
3550    }
3551
3552    #[tokio::test]
3553    async fn a_403_with_no_preceding_401_is_a_permissions_answer_not_a_lockout() {
3554        let server = MockServer::start().await;
3555        Mock::given(method("GET"))
3556            .and(path("/orgs/acme/actions/runners"))
3557            .respond_with(
3558                ResponseTemplate::new(403)
3559                    .set_body_json(json!({"message": "Resource not accessible by integration"})),
3560            )
3561            .mount(&server)
3562            .await;
3563
3564        let client = client(&server, Arc::new(TestClock::default()));
3565        let err = client
3566            .send(&ApiRequest::get("/orgs/acme/actions/runners"))
3567            .await
3568            .expect_err("403");
3569
3570        assert!(matches!(err, GithubError::Forbidden { .. }), "{err:?}");
3571        assert!(!err.is_lockout());
3572        assert!(!err.is_authentication());
3573        assert!(!client.is_locked_out(), "a permissions 403 must not latch");
3574    }
3575
3576    #[tokio::test]
3577    async fn a_locked_out_client_issues_no_further_http_until_the_backoff_elapses() {
3578        let server = MockServer::start().await;
3579        Mock::given(method("GET"))
3580            .and(path("/orgs/acme/actions/runners"))
3581            .respond_with(Script::new(vec![
3582                ResponseTemplate::new(401),
3583                ResponseTemplate::new(403).insert_header("retry-after", "60"),
3584                ResponseTemplate::new(200).set_body_json(json!({"total_count": 0})),
3585            ]))
3586            .mount(&server)
3587            .await;
3588        Mock::given(method("GET"))
3589            .and(path("/user/installations"))
3590            .respond_with(ResponseTemplate::new(200).set_body_json(installations_body(&[])))
3591            .mount(&server)
3592            .await;
3593
3594        let clock = Arc::new(TestClock::default());
3595        let client = client(&server, Arc::clone(&clock));
3596        let request = ApiRequest::get("/orgs/acme/actions/runners");
3597
3598        let err = client.send(&request).await.expect_err("locks out");
3599        assert!(err.is_lockout(), "{err:?}");
3600
3601        let after_lockout = server.received_requests().await.unwrap().len();
3602
3603        for _ in 0..3 {
3604            let err = client.send(&request).await.expect_err("still locked out");
3605            assert!(err.is_lockout(), "{err:?}");
3606        }
3607        assert_eq!(
3608            server.received_requests().await.unwrap().len(),
3609            after_lockout,
3610            "a backed-off client must open no sockets at all"
3611        );
3612
3613        clock.advance_secs(61);
3614        assert!(!client.is_locked_out(), "the back-off expires on the clock");
3615        let response = client.send(&request).await.expect("traffic resumes");
3616        assert_eq!(response.status(), StatusCode::OK);
3617        assert_eq!(
3618            server.received_requests().await.unwrap().len(),
3619            after_lockout + 1
3620        );
3621    }
3622
3623    /// `consecutive_unauthorized` is reset only by a successful *caller*
3624    /// response, so a request ending in `404`, `422` or `5xx` leaves it set —
3625    /// and in the agent's long-lived reconciliation loop it stays set for as
3626    /// long as nothing succeeds. Before the fix, the next genuine permissions
3627    /// `403` was therefore reported as `AuthenticationLockout`: sixty seconds of
3628    /// client silence, plus an operator message asserting "the credential itself
3629    /// is not the problem" about a credential that was missing
3630    /// `Administration: write` — the failure `04-subsystem-contracts.md` names
3631    /// as the *expected* one for `generate-jitconfig`.
3632    ///
3633    /// The lockout's real signature is narrower: a `403` on the one retry this
3634    /// client issues after this request's own `401`, or a `403` whose own
3635    /// headers and body say GitHub is continuing a lockout. Neither is "the
3636    /// count is non-zero", which is what a stale `401` leaves behind — so the
3637    /// permissions `403` below is a permissions answer whatever happened minutes
3638    /// ago, and it is the *response*, not the history, that decides.
3639    #[tokio::test]
3640    async fn a_stale_401_does_not_turn_a_later_permissions_403_into_a_lockout() {
3641        let server = MockServer::start().await;
3642        // The first request ends in a 404, which leaves the 401 count set
3643        // because only a 2xx clears it.
3644        Mock::given(method("GET"))
3645            .and(path("/orgs/acme/actions/runners"))
3646            .respond_with(Script::new(vec![
3647                ResponseTemplate::new(401),
3648                ResponseTemplate::new(404).set_body_json(json!({"message": "Not Found"})),
3649            ]))
3650            .mount(&server)
3651            .await;
3652        Mock::given(method("GET"))
3653            .and(path("/user/installations"))
3654            .respond_with(ResponseTemplate::new(200).set_body_json(installations_body(&[])))
3655            .mount(&server)
3656            .await;
3657        // Minutes later, a different call is denied for a missing permission.
3658        Mock::given(method("POST"))
3659            .and(path("/orgs/acme/actions/runners/generate-jitconfig"))
3660            .respond_with(
3661                ResponseTemplate::new(403)
3662                    .set_body_json(json!({"message": "Resource not accessible by integration"})),
3663            )
3664            .mount(&server)
3665            .await;
3666
3667        let client = client(&server, Arc::new(TestClock::default()));
3668        let err = client
3669            .send(&ApiRequest::get("/orgs/acme/actions/runners"))
3670            .await
3671            .expect_err("404");
3672        assert!(
3673            matches!(err, GithubError::Status { status: 404, .. }),
3674            "{err:?}"
3675        );
3676
3677        let err = client
3678            .send(&ApiRequest::new(
3679                Method::POST,
3680                "/orgs/acme/actions/runners/generate-jitconfig",
3681            ))
3682            .await
3683            .expect_err("403");
3684
3685        assert!(
3686            matches!(err, GithubError::Forbidden { .. }),
3687            "a fresh first-attempt 403 is a permissions answer, not a lockout: {err:?}"
3688        );
3689        assert!(!err.is_lockout());
3690        assert!(
3691            !client.is_locked_out(),
3692            "a stale 401 must not be able to silence the client for a minute"
3693        );
3694    }
3695
3696    /// GitHub's own rate limit is not an answer about the credential, and must
3697    /// not be reported as one. `classify` reached the `403` branch before
3698    /// anything looked at the rate-limit headers, so a primary rate limit
3699    /// arriving during a `401` storm was announced as an authentication lockout
3700    /// with the message "the credential itself is not the problem" — about a
3701    /// response that never mentioned the credential.
3702    #[tokio::test]
3703    async fn a_rate_limited_403_is_not_reported_as_an_authentication_lockout() {
3704        let server = MockServer::start().await;
3705        Mock::given(method("GET"))
3706            .and(path("/orgs/acme/actions/runners"))
3707            .respond_with(Script::new(vec![
3708                ResponseTemplate::new(401),
3709                ResponseTemplate::new(403)
3710                    .insert_header("x-ratelimit-remaining", "0")
3711                    .insert_header("x-ratelimit-reset", "1787270460")
3712                    .insert_header("retry-after", "30")
3713                    .set_body_json(json!({"message": "API rate limit exceeded"})),
3714            ]))
3715            .mount(&server)
3716            .await;
3717        Mock::given(method("GET"))
3718            .and(path("/user/installations"))
3719            .respond_with(ResponseTemplate::new(200).set_body_json(installations_body(&[])))
3720            .mount(&server)
3721            .await;
3722
3723        let client = client(&server, Arc::new(TestClock::default()));
3724        let err = client
3725            .send(&ApiRequest::get("/orgs/acme/actions/runners"))
3726            .await
3727            .expect_err("rate limited");
3728
3729        assert!(
3730            matches!(err, GithubError::Forbidden { .. }),
3731            "a rate limit is not an authentication outcome: {err:?}"
3732        );
3733        assert!(!err.is_lockout());
3734        assert!(!err.is_authentication());
3735        assert!(
3736            !client.is_locked_out(),
3737            "a rate limit must not latch this crate's authentication back-off"
3738        );
3739
3740        // And `c3` gets the evidence it needs to apply the policy that is its
3741        // own, without editing this file.
3742        let evidence = err
3743            .rate_limit()
3744            .expect("the headers survived classification");
3745        assert_eq!(evidence.remaining, Some(0));
3746        assert_eq!(evidence.reset_unix_secs, Some(1_787_270_460));
3747        assert_eq!(evidence.retry_after, Some(Duration::from_secs(30)));
3748    }
3749
3750    /// The same claim for the variant `429` lands in.
3751    #[tokio::test]
3752    async fn a_429_carries_its_retry_after_across_the_c2_c3_seam() {
3753        let server = MockServer::start().await;
3754        Mock::given(method("GET"))
3755            .and(path("/orgs/acme/actions/runners"))
3756            .respond_with(
3757                ResponseTemplate::new(429)
3758                    .insert_header("retry-after", "17")
3759                    .insert_header("x-ratelimit-remaining", "0")
3760                    .set_body_json(json!({"message": "You have exceeded a secondary rate limit"})),
3761            )
3762            .mount(&server)
3763            .await;
3764
3765        let client = client(&server, Arc::new(TestClock::default()));
3766        let err = client
3767            .send(&ApiRequest::get("/orgs/acme/actions/runners"))
3768            .await
3769            .expect_err("429");
3770
3771        assert!(
3772            matches!(err, GithubError::Status { status: 429, .. }),
3773            "{err:?}"
3774        );
3775        assert_eq!(
3776            err.retry_after(),
3777            Some(Duration::from_secs(17)),
3778            "destroying this header is what made `c3`'s Definition of Done unmeetable"
3779        );
3780        assert_eq!(
3781            err.headers().and_then(|h| h.get("x-ratelimit-remaining")),
3782            Some(&reqwest::header::HeaderValue::from_static("0"))
3783        );
3784    }
3785
3786    /// A back-off is a safety mechanism, and this one had both failure modes at
3787    /// once: no ceiling, so `Retry-After: 86400` latched a silent twenty-four
3788    /// hour outage; and `TimeDelta::from_std(...).ok()` on a value too large to
3789    /// convert, which yielded `until = None` — *not locked out at all*, the
3790    /// exact inverse of the requirement, reachable by a header alone.
3791    #[tokio::test]
3792    async fn an_extreme_retry_after_is_clamped_and_never_fails_open() {
3793        async fn lockout_for(header: &str) -> (GithubError, bool, Option<Duration>) {
3794            let server = MockServer::start().await;
3795            Mock::given(method("GET"))
3796                .and(path("/orgs/acme/actions/runners"))
3797                .respond_with(Script::new(vec![
3798                    ResponseTemplate::new(401),
3799                    ResponseTemplate::new(403).insert_header("retry-after", header),
3800                ]))
3801                .mount(&server)
3802                .await;
3803            Mock::given(method("GET"))
3804                .and(path("/user/installations"))
3805                .respond_with(ResponseTemplate::new(200).set_body_json(installations_body(&[])))
3806                .mount(&server)
3807                .await;
3808
3809            let client = AuthenticatedClient::new(
3810                Endpoints::for_test_server(&server.uri()).unwrap(),
3811                UserAccessToken::new(SecretString::from(FIXTURE_TOKEN)),
3812                Arc::new(TestClock::default()),
3813            )
3814            .unwrap();
3815            let err = client
3816                .send(&ApiRequest::get("/orgs/acme/actions/runners"))
3817                .await
3818                .expect_err("403 after a 401");
3819            let locked = client.is_locked_out();
3820            let remaining = client.lockout_remaining();
3821            (err, locked, remaining)
3822        }
3823
3824        // A day-long back-off is clamped to the ceiling.
3825        let (err, locked, remaining) = lockout_for("86400").await;
3826        let GithubError::AuthenticationLockout { retry_after } = &err else {
3827            panic!("expected a lockout, got {err:?}");
3828        };
3829        assert_eq!(
3830            *retry_after, MAX_LOCKOUT_BACKOFF,
3831            "an unclamped Retry-After lets a remote party decide how long this product \
3832             stays down"
3833        );
3834        assert!(locked);
3835        assert!(remaining.is_some_and(|r| r <= MAX_LOCKOUT_BACKOFF));
3836
3837        // A value too large for `chrono` must still lock out. Before the fix
3838        // this produced `until = None`: the more extreme the header, the less
3839        // protection it bought.
3840        let (err, locked, remaining) = lockout_for(&u64::MAX.to_string()).await;
3841        assert!(err.is_lockout(), "{err:?}");
3842        assert!(
3843            locked,
3844            "an absurd Retry-After must not mean `not locked out at all` — that fails open"
3845        );
3846        assert!(remaining.is_some_and(|r| r <= MAX_LOCKOUT_BACKOFF));
3847    }
3848
3849    /// The lockout's own contract is "this client issues no HTTP at all", and
3850    /// `revalidate` is HTTP. It documented an `AuthenticationLockout` it could
3851    /// never return, which made the one direct entry point into the probe the
3852    /// single exception to the rule.
3853    #[tokio::test]
3854    async fn a_direct_revalidation_is_refused_while_the_lockout_is_backing_off() {
3855        let server = MockServer::start().await;
3856        Mock::given(method("GET"))
3857            .and(path("/orgs/acme/actions/runners"))
3858            .respond_with(Script::new(vec![
3859                ResponseTemplate::new(401),
3860                ResponseTemplate::new(403).insert_header("retry-after", "60"),
3861            ]))
3862            .mount(&server)
3863            .await;
3864        Mock::given(method("GET"))
3865            .and(path("/user/installations"))
3866            .respond_with(ResponseTemplate::new(200).set_body_json(installations_body(&[])))
3867            .mount(&server)
3868            .await;
3869
3870        let client = client(&server, Arc::new(TestClock::default()));
3871        client
3872            .send(&ApiRequest::get("/orgs/acme/actions/runners"))
3873            .await
3874            .expect_err("locks out");
3875        assert!(client.is_locked_out());
3876
3877        let before = server.received_requests().await.unwrap().len();
3878        let err = client
3879            .revalidate()
3880            .await
3881            .expect_err("the documented lockout error is now reachable");
3882        assert!(err.is_lockout(), "{err:?}");
3883        assert_eq!(
3884            server.received_requests().await.unwrap().len(),
3885            before,
3886            "a locked-out client opens no socket, and the probe is not an exception"
3887        );
3888    }
3889
3890    /// The position rule fixed `classify` and left the same defect one function
3891    /// over, behind a comment asserting it could not happen: "the probe only
3892    /// ever runs after a `401`, so it is always in the retry position". Making
3893    /// [`AuthenticatedClient::revalidate`] public — the previous round's own
3894    /// change — is exactly what made that untrue.
3895    ///
3896    /// The sequence is the agent's, not a contrivance. A request 401s, the probe
3897    /// says the credential is fine, the retry answers `404` — which does *not*
3898    /// reset the counter, by design. Minutes later `f1` renders `auth status`,
3899    /// which probes directly, and the probe meets an ordinary permissions `403`.
3900    /// A stale `401` then latched a sixty-second client-wide lockout and told
3901    /// the operator to wait, when the real answer was a missing grant.
3902    #[tokio::test]
3903    async fn a_directly_requested_probe_does_not_latch_a_lockout_from_a_stale_401() {
3904        let server = MockServer::start().await;
3905        Mock::given(method("GET"))
3906            .and(path("/orgs/acme/actions/runners"))
3907            .respond_with(Script::new(vec![
3908                ResponseTemplate::new(401),
3909                // The retry misses. A `404` leaves `consecutive_unauthorized`
3910                // set, which is the whole premise of the position rule.
3911                ResponseTemplate::new(404).set_body_json(json!({"message": "Not Found"})),
3912            ]))
3913            .mount(&server)
3914            .await;
3915        Mock::given(method("GET"))
3916            .and(path("/user/installations"))
3917            .respond_with(Script::new(vec![
3918                // The probe that accompanies the 401 above.
3919                ResponseTemplate::new(200).set_body_json(installations_body(&[])),
3920                // The direct probe, minutes later: a plain permissions answer,
3921                // with no `retry-after` and a message that names a grant.
3922                ResponseTemplate::new(403)
3923                    .set_body_json(json!({"message": "Resource not accessible by integration"})),
3924            ]))
3925            .mount(&server)
3926            .await;
3927
3928        let clock = Arc::new(TestClock::default());
3929        let client = client(&server, clock.clone());
3930        client
3931            .send(&ApiRequest::get("/orgs/acme/actions/runners"))
3932            .await
3933            .expect_err("the retry 404s");
3934        assert!(
3935            !client.is_locked_out(),
3936            "a 404 on the retry is not a lockout"
3937        );
3938
3939        clock.advance_secs(300);
3940        let outcome = client
3941            .revalidate()
3942            .await
3943            .expect("a direct probe is not a lockout error");
3944
3945        assert_eq!(
3946            outcome,
3947            Revalidation::Unavailable,
3948            "a 403 on the probe teaches this client nothing about the credential"
3949        );
3950        assert!(
3951            !client.is_locked_out(),
3952            "a caller-initiated probe is a *first* attempt, not the retry that follows a 401: \
3953             latching here converts a stale 401 into a 60-second client-wide outage and \
3954             reports a missing permission as `the credential is fine, please wait`"
3955        );
3956        assert_eq!(client.lockout_remaining(), None);
3957    }
3958
3959    /// The square the other three leave empty, and the one where a defect in the
3960    /// composition would hide.
3961    ///
3962    /// Covered elsewhere: a first-attempt continuation through `send`, a
3963    /// first-attempt permissions `403` through `send`, and a direct probe
3964    /// meeting a permissions `403` (immediately above). A direct probe meeting a
3965    /// *continuation-shaped* `403` is the fourth square, and it is the
3966    /// composition point of the two rules that pull in opposite directions —
3967    /// the narrowing to `Attempt::First` that fixed the stale-`401` lockout, and
3968    /// the continuation rule that re-widens `First` on GitHub's own evidence.
3969    ///
3970    /// If the narrowing swallowed the continuation here, every other test in
3971    /// this file would still pass, and `f1`'s `auth status` would poll a
3972    /// credential GitHub had asked it to leave alone — reporting each refusal as
3973    /// a missing grant.
3974    #[tokio::test]
3975    async fn a_directly_requested_probe_latches_a_continuation_shaped_lockout() {
3976        let server = MockServer::start().await;
3977        Mock::given(method("GET"))
3978            .and(path("/user/installations"))
3979            // The continuation's signature: `retry-after`, and a body with no
3980            // message for `error_message` to find.
3981            .respond_with(ResponseTemplate::new(403).insert_header("retry-after", "60"))
3982            .mount(&server)
3983            .await;
3984
3985        let client = client(&server, Arc::new(TestClock::default()));
3986
3987        // No preceding traffic whatsoever: a first attempt in the strongest
3988        // sense, which is exactly the position the narrowed rule refused to
3989        // latch in.
3990        let err = client
3991            .revalidate()
3992            .await
3993            .expect_err("a probe that latches a lockout reports it rather than `Unavailable`");
3994        assert!(
3995            err.is_lockout(),
3996            "`revalidate` latched a client-wide lockout and must say so; answering \
3997             `Ok(Unavailable)` leaves `f1` to discover a 15-minute outage through a separate \
3998             `is_locked_out()` call it has no reason to make: {err:?}"
3999        );
4000        assert!(
4001            client.is_locked_out(),
4002            "a 403 carrying `retry-after` with no message is GitHub continuing a lockout, \
4003             whoever asked for the request that met it"
4004        );
4005        assert_eq!(client.lockout_remaining(), Some(Duration::from_secs(60)));
4006
4007        // And the back-off is real, not just a renamed error.
4008        let before = server.received_requests().await.unwrap().len();
4009        let err = client.revalidate().await.expect_err("still locked out");
4010        assert!(err.is_lockout(), "{err:?}");
4011        assert_eq!(
4012            server.received_requests().await.unwrap().len(),
4013            before,
4014            "latching must actually stop traffic"
4015        );
4016    }
4017
4018    /// [`retry_after`] parses integer seconds only; RFC 9110 §10.2.3 also
4019    /// permits an HTTP-date. Detection used to gate on that parse succeeding, so
4020    /// a date-form `Retry-After` was not recognised as a continuation at all and
4021    /// the hole the continuation rule exists to close reopened for that shape —
4022    /// silently, because the response leaves as an ordinary `Forbidden`.
4023    ///
4024    /// GitHub sends integer seconds in practice. This pins the crate to not
4025    /// depending on that, and records where the two halves part company:
4026    /// presence decides *whether* it is a lockout, the integer parse decides
4027    /// only *how long*, and `latch_lockout` already had a default for the header
4028    /// it could not read.
4029    #[tokio::test]
4030    async fn a_date_form_retry_after_is_still_recognised_as_a_continuation() {
4031        let server = MockServer::start().await;
4032        Mock::given(method("GET"))
4033            .and(path("/user/installations"))
4034            .respond_with(
4035                ResponseTemplate::new(403)
4036                    .insert_header("retry-after", "Wed, 21 Oct 2026 07:28:00 GMT"),
4037            )
4038            .mount(&server)
4039            .await;
4040
4041        let client = client(&server, Arc::new(TestClock::default()));
4042        let err = client
4043            .revalidate()
4044            .await
4045            .expect_err("a date-form `retry-after` is still GitHub asking to be left alone");
4046        assert!(
4047            err.is_lockout(),
4048            "gating detection on an integer parse hands the continuation bug back for every \
4049             lockout GitHub chose to date-stamp: {err:?}"
4050        );
4051        assert_eq!(
4052            client.lockout_remaining(),
4053            Some(DEFAULT_LOCKOUT_BACKOFF),
4054            "the date form is recognised for detection; the duration falls back to the \
4055             default, which is what `latch_lockout` already did with a header it could not \
4056             parse as seconds"
4057        );
4058    }
4059
4060    /// The narrowing that fixed the stale-`401` lockout opened a hole at the
4061    /// other end of the same back-off.
4062    ///
4063    /// While GitHub is still locking the credential out after the back-off
4064    /// elapses, the next request is a *first* attempt by construction — this
4065    /// client's own retry never happened, because the request never reached the
4066    /// wire. So the position rule declined to call it a lockout and `classify`
4067    /// fell through to [`GithubError::Forbidden`], whose documented reading is
4068    /// "the App installation does not grant it". The client then stopped backing
4069    /// off entirely and hammered a credential GitHub had asked it to leave
4070    /// alone, which is the exact inverse of "backs off without retrying".
4071    ///
4072    /// A continuation is distinguishable from a permissions answer without any
4073    /// counter: GitHub sends `retry-after` and no message body for the lockout,
4074    /// and a message and no `retry-after` for a permissions refusal.
4075    #[tokio::test]
4076    async fn a_lockout_outliving_its_backoff_re_latches_instead_of_reporting_a_permissions_answer()
4077    {
4078        let server = MockServer::start().await;
4079        Mock::given(method("GET"))
4080            .and(path("/orgs/acme/actions/runners"))
4081            .respond_with(Script::new(vec![
4082                ResponseTemplate::new(401),
4083                // The retry: the lockout latches here, in the retry position.
4084                ResponseTemplate::new(403).insert_header("retry-after", "60"),
4085                // The continuation, once the back-off has elapsed. Same shape,
4086                // first position.
4087                ResponseTemplate::new(403).insert_header("retry-after", "60"),
4088            ]))
4089            .mount(&server)
4090            .await;
4091        Mock::given(method("GET"))
4092            .and(path("/user/installations"))
4093            .respond_with(ResponseTemplate::new(200).set_body_json(installations_body(&[])))
4094            .mount(&server)
4095            .await;
4096
4097        let clock = Arc::new(TestClock::default());
4098        let client = client(&server, clock.clone());
4099        let err = client
4100            .send(&ApiRequest::get("/orgs/acme/actions/runners"))
4101            .await
4102            .expect_err("403 on the retry");
4103        assert!(err.is_lockout(), "{err:?}");
4104        assert!(client.is_locked_out());
4105
4106        // The back-off elapses with GitHub unchanged.
4107        clock.advance_secs(61);
4108        assert!(!client.is_locked_out(), "the back-off has run out");
4109
4110        let err = client
4111            .send(&ApiRequest::get("/orgs/acme/actions/runners"))
4112            .await
4113            .expect_err("GitHub is still locking the credential out");
4114
4115        assert!(
4116            err.is_lockout(),
4117            "a 403 carrying `retry-after` with no message is GitHub continuing the lockout, \
4118             not the App installation refusing a permission; reporting `Forbidden` here \
4119             tells the operator to fix a grant that is not missing: {err:?}"
4120        );
4121        assert!(
4122            client.is_locked_out(),
4123            "`backs off without retrying` fails for any lockout that outlives one back-off \
4124             if the continuation does not re-latch"
4125        );
4126        let GithubError::AuthenticationLockout { retry_after } = err else {
4127            unreachable!("asserted above")
4128        };
4129        assert_eq!(
4130            retry_after,
4131            Duration::from_secs(60),
4132            "the continuation's own `retry-after` sets the new back-off"
4133        );
4134
4135        // And the next request is suppressed before a socket is opened, which is
4136        // the property the whole back-off exists for.
4137        let before = server.received_requests().await.unwrap().len();
4138        let err = client
4139            .send(&ApiRequest::get("/orgs/acme/actions/runners"))
4140            .await
4141            .expect_err("still locked out");
4142        assert!(err.is_lockout(), "{err:?}");
4143        assert_eq!(
4144            server.received_requests().await.unwrap().len(),
4145            before,
4146            "re-latching must actually stop traffic, not merely rename the error"
4147        );
4148    }
4149
4150    /// A permissions `403` on a first attempt is still a permissions answer, and
4151    /// the continuation rule above must not swallow it. This is the test that
4152    /// keeps that rule from becoming "every 403 is a lockout".
4153    #[tokio::test]
4154    async fn a_first_attempt_permissions_403_is_still_reported_as_forbidden() {
4155        let server = MockServer::start().await;
4156        Mock::given(method("GET"))
4157            .and(path("/orgs/acme/actions/runners"))
4158            .respond_with(
4159                ResponseTemplate::new(403)
4160                    .set_body_json(json!({"message": "Resource not accessible by integration"})),
4161            )
4162            .mount(&server)
4163            .await;
4164
4165        let client = client(&server, Arc::new(TestClock::default()));
4166        let err = client
4167            .send(&ApiRequest::get("/orgs/acme/actions/runners"))
4168            .await
4169            .expect_err("403");
4170
4171        assert!(
4172            matches!(err, GithubError::Forbidden { .. }),
4173            "a message and no `retry-after` is GitHub naming a missing grant: {err:?}"
4174        );
4175        assert!(!client.is_locked_out());
4176    }
4177
4178    /// The `consecutive_unauthorized > 0` conjunct that used to sit alongside
4179    /// the position rule added no signal — `Attempt::Retry` already implies this
4180    /// request's own `401` incremented the counter — and added a fail-open race:
4181    /// any concurrent success `store(0)`s the counter between the `401` and the
4182    /// retry, and a real lockout is then reported as a permissions answer.
4183    ///
4184    /// The race is driven directly rather than by scheduling two requests and
4185    /// hoping: `store(0)` is the *only* thing the concurrent success contributes,
4186    /// so performing it between the `401` and the classification reproduces the
4187    /// race deterministically and on every run.
4188    #[tokio::test]
4189    async fn a_concurrent_success_cannot_downgrade_a_lockout_to_a_permissions_answer() {
4190        let server = MockServer::start().await;
4191        let client = client(&server, Arc::new(TestClock::default()));
4192
4193        // This request's own 401 has landed: the retry position is established.
4194        client
4195            .consecutive_unauthorized
4196            .fetch_add(1, Ordering::SeqCst);
4197        // ... and a request on another task succeeds in the same instant.
4198        client.consecutive_unauthorized.store(0, Ordering::SeqCst);
4199
4200        let mut headers = HeaderMap::new();
4201        headers.insert("retry-after", "60".parse().unwrap());
4202        let lockout = ApiResponse {
4203            status: StatusCode::FORBIDDEN,
4204            headers,
4205            body: Vec::new(),
4206        };
4207
4208        assert!(
4209            client.is_lockout_403(&lockout, Attempt::Retry),
4210            "`Attempt::Retry` already means this request's own 401 incremented the counter, so \
4211             reading the counter again adds no signal and only lets an unrelated success \
4212             downgrade a real lockout to `Forbidden`"
4213        );
4214
4215        // The counter must stay irrelevant in the other direction too: a
4216        // permissions `403` on a first attempt is not a lockout however many
4217        // `401`s are on the count.
4218        client.consecutive_unauthorized.store(7, Ordering::SeqCst);
4219        let permissions = ApiResponse {
4220            status: StatusCode::FORBIDDEN,
4221            headers: HeaderMap::new(),
4222            body: br#"{"message":"Resource not accessible by integration"}"#.to_vec(),
4223        };
4224        assert!(!client.is_lockout_403(&permissions, Attempt::First));
4225    }
4226
4227    // -- installation discovery ---------------------------------------------
4228
4229    #[tokio::test]
4230    async fn discovery_returns_the_reachable_repository_and_organization_set() {
4231        let server = MockServer::start().await;
4232        Mock::given(method("GET"))
4233            .and(path("/user/installations"))
4234            .respond_with(
4235                ResponseTemplate::new(200).set_body_json(installations_body(&[
4236                    (11, "IvanMurzak", "User", "selected"),
4237                    (22, "Tap-Top-Fun", "Organization", "all"),
4238                ])),
4239            )
4240            .mount(&server)
4241            .await;
4242        Mock::given(method("GET"))
4243            .and(path("/user/installations/11/repositories"))
4244            .respond_with(
4245                ResponseTemplate::new(200)
4246                    .set_body_json(repositories_body(&["IvanMurzak/GitHub-Runner-Scaler-UI"])),
4247            )
4248            .mount(&server)
4249            .await;
4250        Mock::given(method("GET"))
4251            .and(path("/user/installations/22/repositories"))
4252            .respond_with(
4253                ResponseTemplate::new(200)
4254                    .set_body_json(repositories_body(&["Tap-Top-Fun/game", "Tap-Top-Fun/site"])),
4255            )
4256            .mount(&server)
4257            .await;
4258
4259        let client = client(&server, Arc::new(TestClock::default()));
4260        let discovery = client.discover_installations(&app()).await.unwrap();
4261
4262        let targets = discovery.targets().expect("installed");
4263        assert_eq!(
4264            targets
4265                .repositories()
4266                .iter()
4267                .map(ToString::to_string)
4268                .collect::<Vec<_>>(),
4269            [
4270                "IvanMurzak/GitHub-Runner-Scaler-UI",
4271                "Tap-Top-Fun/game",
4272                "Tap-Top-Fun/site"
4273            ]
4274        );
4275        assert_eq!(
4276            targets
4277                .organizations()
4278                .iter()
4279                .map(ToString::to_string)
4280                .collect::<Vec<_>>(),
4281            ["Tap-Top-Fun"],
4282            "a User account is not an organization target"
4283        );
4284        assert!(discovery.install_url().is_none());
4285    }
4286
4287    #[tokio::test]
4288    async fn an_over_broad_installation_is_visible_rather_than_assumed() {
4289        let server = MockServer::start().await;
4290        Mock::given(method("GET"))
4291            .and(path("/user/installations"))
4292            .respond_with(
4293                ResponseTemplate::new(200).set_body_json(installations_body(&[
4294                    (11, "IvanMurzak", "User", "selected"),
4295                    (22, "Tap-Top-Fun", "Organization", "all"),
4296                ])),
4297            )
4298            .mount(&server)
4299            .await;
4300        Mock::given(method("GET"))
4301            .and(path("/user/installations/11/repositories"))
4302            .respond_with(ResponseTemplate::new(200).set_body_json(repositories_body(&["a/b"])))
4303            .mount(&server)
4304            .await;
4305        Mock::given(method("GET"))
4306            .and(path("/user/installations/22/repositories"))
4307            .respond_with(ResponseTemplate::new(200).set_body_json(repositories_body(&["c/d"])))
4308            .mount(&server)
4309            .await;
4310
4311        let client = client(&server, Arc::new(TestClock::default()));
4312        let targets = client
4313            .discover_installations(&app())
4314            .await
4315            .unwrap()
4316            .targets()
4317            .cloned()
4318            .expect("installed");
4319
4320        let over_broad = targets.over_broad();
4321        assert_eq!(over_broad.len(), 1);
4322        assert_eq!(over_broad[0].account.login(), "Tap-Top-Fun");
4323        assert!(over_broad[0].is_over_broad());
4324        assert_eq!(
4325            over_broad[0].repository_selection,
4326            RepositorySelection::All,
4327            "`repository_selection: all` reaches repositories created later too"
4328        );
4329        assert!(
4330            targets.installations().iter().any(|i| i
4331                .permissions
4332                .iter()
4333                .any(|(k, v)| k == "administration" && v == "write")),
4334            "the grant GitHub reports is surfaced verbatim, not assumed from the design"
4335        );
4336    }
4337
4338    #[tokio::test]
4339    async fn discovery_returns_the_installation_url_when_the_set_is_empty() {
4340        let server = MockServer::start().await;
4341        Mock::given(method("GET"))
4342            .and(path("/user/installations"))
4343            .respond_with(ResponseTemplate::new(200).set_body_json(installations_body(&[])))
4344            .mount(&server)
4345            .await;
4346
4347        let client = client(&server, Arc::new(TestClock::default()));
4348        let discovery = client.discover_installations(&app()).await.unwrap();
4349
4350        let url = discovery
4351            .install_url()
4352            .expect("an empty set must yield the installation URL");
4353        assert_eq!(url.path(), "/apps/runner-manager/installations/new");
4354        assert!(discovery.targets().is_none());
4355    }
4356
4357    #[tokio::test]
4358    async fn an_installation_that_reaches_no_repository_is_still_not_installed() {
4359        let server = MockServer::start().await;
4360        Mock::given(method("GET"))
4361            .and(path("/user/installations"))
4362            .respond_with(
4363                ResponseTemplate::new(200).set_body_json(installations_body(&[(
4364                    11,
4365                    "IvanMurzak",
4366                    "User",
4367                    "selected",
4368                )])),
4369            )
4370            .mount(&server)
4371            .await;
4372        Mock::given(method("GET"))
4373            .and(path("/user/installations/11/repositories"))
4374            .respond_with(ResponseTemplate::new(200).set_body_json(repositories_body(&[])))
4375            .mount(&server)
4376            .await;
4377
4378        let client = client(&server, Arc::new(TestClock::default()));
4379        let discovery = client.discover_installations(&app()).await.unwrap();
4380        assert!(
4381            discovery.install_url().is_some(),
4382            "a user installation that selected no repository reaches nothing"
4383        );
4384    }
4385
4386    #[tokio::test]
4387    async fn discovery_follows_every_page_rather_than_trusting_the_first() {
4388        let server = MockServer::start().await;
4389        let next = format!("<{}/user/installations?page=2>; rel=\"next\"", server.uri());
4390        Mock::given(method("GET"))
4391            .and(path("/user/installations"))
4392            .respond_with(Script::new(vec![
4393                ResponseTemplate::new(200)
4394                    .set_body_json(installations_body(&[(
4395                        11,
4396                        "one",
4397                        "Organization",
4398                        "selected",
4399                    )]))
4400                    .insert_header("link", next.as_str()),
4401                ResponseTemplate::new(200).set_body_json(installations_body(&[(
4402                    22,
4403                    "two",
4404                    "Organization",
4405                    "selected",
4406                )])),
4407            ]))
4408            .mount(&server)
4409            .await;
4410        Mock::given(method("GET"))
4411            .and(path("/user/installations/11/repositories"))
4412            .respond_with(ResponseTemplate::new(200).set_body_json(repositories_body(&["one/a"])))
4413            .mount(&server)
4414            .await;
4415        Mock::given(method("GET"))
4416            .and(path("/user/installations/22/repositories"))
4417            .respond_with(ResponseTemplate::new(200).set_body_json(repositories_body(&["two/b"])))
4418            .mount(&server)
4419            .await;
4420
4421        let client = client(&server, Arc::new(TestClock::default()));
4422        let targets = client
4423            .discover_installations(&app())
4424            .await
4425            .unwrap()
4426            .targets()
4427            .cloned()
4428            .expect("installed");
4429        assert_eq!(
4430            targets
4431                .organizations()
4432                .iter()
4433                .map(ToString::to_string)
4434                .collect::<Vec<_>>(),
4435            ["one", "two"],
4436            "the second page must not be dropped"
4437        );
4438    }
4439
4440    /// GitHub's published `installation` schema types `account` as **nullable**,
4441    /// and as either a simple-user *or* an enterprise — which carries
4442    /// `slug`/`name` where a user carries `login`. A required `RawAccount` with
4443    /// a required `login` made either shape a hard `response.json()` failure,
4444    /// which takes down all of `discover_installations`, which is all of
4445    /// `auth status`. One unusual installation must not blind the command that
4446    /// exists to show the user what their credential can reach.
4447    #[tokio::test]
4448    async fn an_installation_with_a_null_or_enterprise_account_does_not_fail_the_whole_decode() {
4449        let server = MockServer::start().await;
4450        Mock::given(method("GET"))
4451            .and(path("/user/installations"))
4452            .respond_with(ResponseTemplate::new(200).set_body_json(json!({
4453                "total_count": 3,
4454                "installations": [
4455                    // Nullable, per the published schema.
4456                    { "id": 10, "account": null, "repository_selection": "selected" },
4457                    // An enterprise: no `login` at all.
4458                    {
4459                        "id": 20,
4460                        "account": { "slug": "acme-enterprise", "name": "Acme Inc" },
4461                        "repository_selection": "selected"
4462                    },
4463                    // And an ordinary user alongside them.
4464                    {
4465                        "id": 30,
4466                        "account": { "login": "IvanMurzak", "type": "User" },
4467                        "repository_selection": "selected"
4468                    }
4469                ]
4470            })))
4471            .mount(&server)
4472            .await;
4473        for (id, repo) in [(20_u64, "acme-enterprise/tools"), (30, "IvanMurzak/app")] {
4474            Mock::given(method("GET"))
4475                .and(path(format!("/user/installations/{id}/repositories")))
4476                .respond_with(ResponseTemplate::new(200).set_body_json(repositories_body(&[repo])))
4477                .mount(&server)
4478                .await;
4479        }
4480
4481        let client = client(&server, Arc::new(TestClock::default()));
4482        let targets = client
4483            .discover_installations(&app())
4484            .await
4485            .expect("one odd account must not fail the whole discovery")
4486            .targets()
4487            .cloned()
4488            .expect("installed");
4489
4490        // Membership rather than order: what matters here is that neither
4491        // installation was lost, not how `OwnerRepo` collates.
4492        let reached = targets
4493            .repositories()
4494            .iter()
4495            .map(ToString::to_string)
4496            .collect::<Vec<_>>();
4497        assert!(
4498            reached.contains(&"acme-enterprise/tools".to_string()),
4499            "the enterprise installation is named from `slug` rather than dropped: {reached:?}"
4500        );
4501        assert!(
4502            reached.contains(&"IvanMurzak/app".to_string()),
4503            "the ordinary installation alongside it survives too: {reached:?}"
4504        );
4505        assert_eq!(reached.len(), 2);
4506        assert_eq!(
4507            targets.installations().len(),
4508            2,
4509            "the null account is skipped, and only it"
4510        );
4511        assert_eq!(
4512            targets.skipped(),
4513            1,
4514            "the skip is the right trade, but it must travel with the answer: everything the \
4515             skipped installation reaches is missing from the lists above, and a short list \
4516             reads exactly like a complete one"
4517        );
4518
4519        // The enterprise is labelled an enterprise. It used to fall through to
4520        // `User`, so `auth status` told the operator their enterprise was a
4521        // personal account.
4522        let enterprise = targets
4523            .installations()
4524            .iter()
4525            .find(|i| i.id == 20)
4526            .expect("the enterprise installation survived");
4527        assert_eq!(
4528            enterprise.account,
4529            InstallationAccount::Enterprise("acme-enterprise".to_string()),
4530            "an account with no `login` that names itself through `slug` is an enterprise, \
4531             and calling it a user is a wrong statement about the operator's own account"
4532        );
4533        assert_eq!(enterprise.account.kind(), "enterprise");
4534        assert!(
4535            enterprise.account.organization().is_none(),
4536            "an enterprise is not an organization target: `GET /orgs/{{org}}/actions/runners` \
4537             does not accept one, so contributing nothing to `organizations()` is correct"
4538        );
4539        assert!(
4540            !targets
4541                .organizations()
4542                .iter()
4543                .any(|o| o.as_str() == "acme-enterprise"),
4544            "and it must not be smuggled in as one either"
4545        );
4546    }
4547
4548    /// The skip is right; the verdict flip was not.
4549    ///
4550    /// A null-account installation that is the *only* installation used to
4551    /// collapse to `NotInstalled`, so `auth status` handed an operator who **is**
4552    /// installed the "install the App" URL — a wrong remediation on the only
4553    /// authentication path there is, contradicted by nothing but a `warn!`.
4554    #[tokio::test]
4555    async fn a_credential_whose_only_installation_was_skipped_is_not_reported_as_not_installed() {
4556        let server = MockServer::start().await;
4557        Mock::given(method("GET"))
4558            .and(path("/user/installations"))
4559            .respond_with(ResponseTemplate::new(200).set_body_json(json!({
4560                "total_count": 1,
4561                "installations": [
4562                    { "id": 10, "account": null, "repository_selection": "selected" }
4563                ]
4564            })))
4565            .mount(&server)
4566            .await;
4567
4568        let client = client(&server, Arc::new(TestClock::default()));
4569        let discovery = client.discover_installations(&app()).await.unwrap();
4570
4571        assert_eq!(
4572            discovery,
4573            InstallationDiscovery::Indeterminate { skipped: 1 },
4574            "GitHub reported an installation; this client could not describe it. That is not \
4575             the same answer as `not installed`, and only one of the two is fixed by \
4576             installing the App"
4577        );
4578        assert_eq!(
4579            discovery.install_url(),
4580            None,
4581            "offering the install URL here is the wrong remediation, and putting it one field \
4582             over from the right verdict would just relocate the defect"
4583        );
4584        assert_eq!(discovery.skipped(), 1);
4585        assert!(discovery.targets().is_none());
4586    }
4587
4588    /// The other side of the same rule: with nothing skipped, an empty reach is
4589    /// still an empty reach, and the install URL is still the remediation.
4590    #[tokio::test]
4591    async fn an_empty_reach_with_nothing_skipped_is_still_not_installed() {
4592        let server = MockServer::start().await;
4593        Mock::given(method("GET"))
4594            .and(path("/user/installations"))
4595            .respond_with(ResponseTemplate::new(200).set_body_json(installations_body(&[])))
4596            .mount(&server)
4597            .await;
4598
4599        let client = client(&server, Arc::new(TestClock::default()));
4600        let discovery = client.discover_installations(&app()).await.unwrap();
4601
4602        assert!(discovery.install_url().is_some(), "{discovery:?}");
4603        assert_eq!(discovery.skipped(), 0);
4604    }
4605
4606    /// A `Link: rel="next"` that points back at the page it arrived on is an
4607    /// infinite loop inside the agent's reconciliation loop — the one place in
4608    /// this product that must not be able to wedge. The ceiling is what makes
4609    /// this test terminate at all.
4610    #[tokio::test]
4611    async fn a_self_referential_link_header_stops_at_the_page_ceiling() {
4612        let server = MockServer::start().await;
4613        let self_link = format!("<{}/user/installations?page=2>; rel=\"next\"", server.uri());
4614        Mock::given(method("GET"))
4615            .and(path("/user/installations"))
4616            .respond_with(
4617                ResponseTemplate::new(200)
4618                    .set_body_json(installations_body(&[]))
4619                    .insert_header("link", self_link.as_str()),
4620            )
4621            .mount(&server)
4622            .await;
4623
4624        let client = client(&server, Arc::new(TestClock::default()));
4625        let discovery = client
4626            .discover_installations(&app())
4627            .await
4628            .expect("the ceiling is what makes this return at all");
4629
4630        assert!(
4631            discovery.install_url().is_some(),
4632            "no installation was found"
4633        );
4634        assert_eq!(
4635            server.received_requests().await.unwrap().len(),
4636            MAX_PAGES,
4637            "pagination must stop at the ceiling rather than follow the loop forever"
4638        );
4639    }
4640
4641    #[test]
4642    fn a_link_header_yields_only_the_next_relation() {
4643        let header = "<https://api.github.com/user/installations?page=3>; rel=\"next\", \
4644                      <https://api.github.com/user/installations?page=9>; rel=\"last\"";
4645        assert_eq!(
4646            parse_link_next(header).map(|u| u.to_string()),
4647            Some("https://api.github.com/user/installations?page=3".to_string())
4648        );
4649        assert!(parse_link_next("<https://x/>; rel=\"last\"").is_none());
4650        assert!(parse_link_next("nonsense").is_none());
4651    }
4652
4653    /// A comma is legal inside a URL and GitHub sends such URLs routinely — a
4654    /// runner query carries `labels=self-hosted,windows`. Splitting the header
4655    /// on `,` before recognising `<...>` tore that URL in half, found no
4656    /// relation, and stopped paginating at page 1 while reporting success. That
4657    /// is precisely what `04-subsystem-contracts.md` forbids ("the dashboard
4658    /// must not treat a first page as a complete inventory"), in the one shared
4659    /// reader `c3`'s inventory also goes through.
4660    #[test]
4661    fn a_next_url_containing_a_comma_still_paginates() {
4662        let header = "<https://api.github.com/repos/o/r/actions/runners\
4663                      ?labels=self-hosted,windows&page=2>; rel=\"next\"";
4664        assert_eq!(
4665            parse_link_next(header).map(|u| u.to_string()),
4666            Some(
4667                "https://api.github.com/repos/o/r/actions/runners\
4668                 ?labels=self-hosted,windows&page=2"
4669                    .to_string()
4670            ),
4671            "a comma inside the URL must not end the link-value"
4672        );
4673
4674        // The same URL as the second link-value, so the scan has to walk past a
4675        // comma-bearing target to reach the relation it wants.
4676        let header = "<https://api.github.com/x?a=1,2&page=1>; rel=\"prev\", \
4677                      <https://api.github.com/x?a=1,2&page=3>; rel=\"next\"";
4678        assert_eq!(
4679            parse_link_next(header).map(|u| u.to_string()),
4680            Some("https://api.github.com/x?a=1,2&page=3".to_string())
4681        );
4682    }
4683
4684    /// `rel="next"` is not always the first link-value, and both quoted and
4685    /// unquoted forms are legal.
4686    #[test]
4687    fn the_next_relation_is_found_wherever_it_sits_in_the_header() {
4688        let not_first = "<https://api.github.com/u?page=1>; rel=\"first\", \
4689                         <https://api.github.com/u?page=9>; rel=\"last\", \
4690                         <https://api.github.com/u?page=4>; rel=\"next\"";
4691        assert_eq!(
4692            parse_link_next(not_first).map(|u| u.to_string()),
4693            Some("https://api.github.com/u?page=4".to_string())
4694        );
4695
4696        let unquoted = "<https://api.github.com/u?page=1>; rel=prev, \
4697                        <https://api.github.com/u?page=3>; rel=next";
4698        assert_eq!(
4699            parse_link_next(unquoted).map(|u| u.to_string()),
4700            Some("https://api.github.com/u?page=3".to_string())
4701        );
4702
4703        assert!(
4704            parse_link_next("<https://api.github.com/u?page=2; rel=\"next\"").is_none(),
4705            "an unterminated target is not a link-value"
4706        );
4707    }
4708
4709    /// The free cross-check that would have caught the comma bug on its own.
4710    #[test]
4711    fn a_short_collection_is_measured_against_the_count_github_reported() {
4712        assert_eq!(under_collected(1, Some(2)), Some(2), "page 2 was dropped");
4713        assert_eq!(under_collected(2, Some(2)), None, "complete");
4714        assert_eq!(
4715            under_collected(3, Some(2)),
4716            None,
4717            "a collection that grew between pages is not an under-collection"
4718        );
4719        assert_eq!(under_collected(0, None), None, "no count, no claim");
4720    }
4721
4722    // -- redaction ----------------------------------------------------------
4723
4724    #[test]
4725    fn no_type_in_this_crate_renders_a_secret_through_debug() {
4726        let token = UserAccessToken::new(SecretString::from(FIXTURE_TOKEN));
4727        let rendered = format!("{token:?}");
4728        assert!(!rendered.contains(FIXTURE_TOKEN), "{rendered}");
4729        assert!(rendered.contains("[REDACTED]"));
4730        assert!(
4731            rendered.contains("ghu_"),
4732            "the family prefix is diagnostic and is not the secret"
4733        );
4734
4735        let request = ApiRequest::post_json("/x", &json!({"encoded_jit_config": "SECRETBLOB"}))
4736            .expect("serializes");
4737        let rendered = format!("{request:?}");
4738        assert!(!rendered.contains("SECRETBLOB"), "{rendered}");
4739
4740        let response = ApiResponse {
4741            status: StatusCode::OK,
4742            headers: HeaderMap::new(),
4743            body: b"{\"encoded_jit_config\":\"SECRETBLOB\"}".to_vec(),
4744        };
4745        let rendered = format!("{response:?}");
4746        assert!(!rendered.contains("SECRETBLOB"), "{rendered}");
4747    }
4748
4749    // The Definition of Done's log scan is `tests/no_secret_reaches_the_logs.rs`
4750    // and not a unit test here. See the note at the end of `mod testing` for the
4751    // `tracing` callsite-cache reason it cannot be one.
4752
4753    // -- the crate-shape scans ----------------------------------------------
4754    //
4755    // The three gates below share these helpers on purpose. The previous round
4756    // defined `normalise` twice — once in the scan and once in the meta-test
4757    // that checks it — which left the meta-test structurally unable to notice a
4758    // change to the real one. One definition, used by both, is the only shape
4759    // in which a meta-test proves anything.
4760
4761    /// Spelled in halves so that this file's own source does not trip the scan
4762    /// it runs: normalising `concat!("refresh", "token")` leaves the
4763    /// quote-comma-quote between the halves, so no needle ever appears whole.
4764    // The renewal guard that used to live here is gone, and its absence is the
4765    // point. It forbade this crate from naming a refresh token at all, on the
4766    // reasoning that the published App opts out of user-token expiration "so
4767    // GitHub issues nothing to renew". That reasoning rested on a second claim
4768    // -- that renewing needs a confidential client credential -- which GitHub's
4769    // own documentation contradicts for the device flow, and which was then
4770    // disproved against live GitHub: a refresh exchange with `client_id` alone
4771    // answers `200`.
4772    //
4773    // What the guard below still forbids is the part that was always true and
4774    // is the reason renewal is safe here: no confidential credential in this
4775    // crate. Renewal was added *without* one, so the remaining half of this
4776    // scan is now evidence for the design rather than against it.
4777    const CONFIDENTIAL: &[&str] = &[concat!("client", "secret"), concat!("app", "secret")];
4778
4779    const MANIFEST: (&str, &str) = ("Cargo.toml", include_str!("../Cargo.toml"));
4780
4781    /// Every `.rs` file at or below `src/`, named by its `/`-joined path
4782    /// relative to `src/` — so a nested module is `("rest/runners.rs",
4783    /// include_str!("rest/runners.rs"))`, not just its file name.
4784    ///
4785    /// This list used to *be* the claim "every source file in the crate", and a
4786    /// hard-coded list is not that claim — it is a snapshot of it. `c3` and `c4`
4787    /// are the tasks that will add files to this directory, so the list was
4788    /// guaranteed to go stale on exactly the work that most needed scanning: a
4789    /// new `pagination.rs` holding a confidential credential passed silently.
4790    /// [`the_confidential_credential_scan_covers_every_source_file`] pins this
4791    /// by walking the directory tree, so adding a file — at the top level or in
4792    /// a subdirectory — and not adding it here fails.
4793    const CRATE_SOURCES: &[(&str, &str)] = &[
4794        ("conditional.rs", include_str!("conditional.rs")),
4795        ("demand.rs", include_str!("demand.rs")),
4796        ("device_flow.rs", include_str!("device_flow.rs")),
4797        ("jit.rs", include_str!("jit.rs")),
4798        ("lib.rs", include_str!("lib.rs")),
4799        ("rest.rs", include_str!("rest.rs")),
4800        ("traffic.rs", include_str!("traffic.rs")),
4801    ];
4802
4803    /// The two source files `c2` owns, plus the manifest. The renewal half of
4804    /// the scan stays inside this boundary; see the scan's own documentation.
4805    const SOURCES_OWNED_BY_C2: &[(&str, &str)] = &[
4806        ("device_flow.rs", include_str!("device_flow.rs")),
4807        ("lib.rs", include_str!("lib.rs")),
4808        MANIFEST,
4809    ];
4810
4811    /// Lower-cased with `_` removed, so that one needle catches the snake,
4812    /// camel, Pascal and screaming-snake spellings of an identifier at once.
4813    /// (Those four spellings cannot be written out here: they are exactly what
4814    /// the gate forbids, which is the constraint on documentation this scan
4815    /// imposes and defends below.)
4816    ///
4817    /// # Why `-` is *not* stripped from Rust source
4818    ///
4819    /// It used to be, and that rejected ordinary English. `c3`'s own file opens
4820    /// with a line stating that the gateway holds no such credential, written
4821    /// with the compound adjective English requires — and stripping `-` turned
4822    /// that sentence into the needle, so the gate accused `c3` of naming a
4823    /// confidential credential in the very line that says it holds none. A
4824    /// compound adjective is not an evasion; it is how the language works, and
4825    /// this brief, this crate's documentation and that line all use one.
4826    ///
4827    /// Nothing is lost, because **a Rust identifier cannot contain `-`**.
4828    /// Stripping it never bought identifier coverage: every casing an identifier
4829    /// can actually take is `_`-separated or unseparated, and all of those still
4830    /// collapse onto the needle. What it bought was coverage of a *kebab-case
4831    /// string literal*, and the residual gap is stated plainly rather than
4832    /// papered over: a `.rs` file that wrote this credential's name as a
4833    /// hyphenated string would not be caught here. That gap is narrow on
4834    /// purpose — OAuth 2.0 and GitHub both spell the field `_`-separated, which
4835    /// this catches — and it is the price of a gate that ordinary prose can
4836    /// coexist with. A gate that fires on correct English is not a stricter
4837    /// gate; it is a gate that gets deleted.
4838    ///
4839    /// The alternatives were weighed. Requiring identifier context needs a Rust
4840    /// lexer to tell `a client-secret-free design` from a TOML key, and gets the
4841    /// wrong answer for both string literals and comments. Excluding comment
4842    /// text needs the same lexer to avoid mangling `//` inside a string, and
4843    /// would stop the gate catching a `TODO` comment proposing to read the
4844    /// credential from the environment — which is precisely the drift worth
4845    /// catching early, while it is still a comment. Stripping one character
4846    /// fewer needs neither, which is why it wins.
4847    ///
4848    /// A space is not stripped either, and for the same reason: it is what lets
4849    /// this crate's prose discuss a "client secret" as two words.
4850    fn normalise_source(source: &str) -> String {
4851        source.to_ascii_lowercase().replace('_', "")
4852    }
4853
4854    /// The manifest keeps `-` stripped: TOML keys and crate names are kebab-case
4855    /// by convention, so `-` there is a word separator rather than a hyphen, and
4856    /// a manifest carries no hyphenated English for it to break.
4857    fn normalise_manifest(manifest: &str) -> String {
4858        manifest.to_ascii_lowercase().replace(['_', '-'], "")
4859    }
4860
4861    /// Which normaliser a scanned file gets. The manifest is the only file whose
4862    /// `-` is a separator rather than punctuation.
4863    fn normalise(name: &str, contents: &str) -> String {
4864        if name == MANIFEST.0 {
4865            normalise_manifest(contents)
4866        } else {
4867            normalise_source(contents)
4868        }
4869    }
4870
4871    /// The part of a source file that is not test code.
4872    ///
4873    /// The boundary is the first line that is **exactly** `#[cfg(test)]`, and
4874    /// the word "exactly" is the fix. Splitting on that literal wherever it
4875    /// appeared also split on it in *prose*, and `lib.rs` has carried such a
4876    /// mention since the `testing` module was documented — so the scan below
4877    /// already stopped nine lines early, today, with nothing to say so. A file
4878    /// whose module documentation happened to mention an inline test module
4879    /// would have had its scanned region truncated to a few dozen lines, after
4880    /// which a real `std::fs::write` in non-test code passed silently. That is
4881    /// the same class of defect as the log scan that captured only its own
4882    /// events and the credential scan that claimed a scope it did not have: a
4883    /// gate whose description outran what it did.
4884    fn non_test_prefix(source: &str) -> &str {
4885        let mut offset = 0;
4886        for line in source.split_inclusive('\n') {
4887            if line.trim() == "#[cfg(test)]" {
4888                return &source[..offset];
4889            }
4890            offset += line.len();
4891        }
4892        source
4893    }
4894
4895    /// The Definition of Done's second item, made checkable rather than
4896    /// reviewed: "no renewal token code path exists, and no client secret
4897    /// appears anywhere in the crate **or its configuration**".
4898    ///
4899    /// # Normalised, because a literal scan is evaded by naming
4900    ///
4901    /// This used to be a case-sensitive `contains` over two snake-case
4902    /// spellings, which is a gate that any ordinary Rust or JSON identifier
4903    /// walks straight through: the camel-cased, Pascal-cased and
4904    /// screaming-snake spellings of the very same two identifiers were all
4905    /// invisible to it. None of those is exotic — several are what the
4906    /// surrounding ecosystem actually calls these fields — so evading this gate
4907    /// never had to be deliberate. See [`normalise_source`] for what is
4908    /// collapsed, what is deliberately not, and why.
4909    ///
4910    /// The consequence is that this crate's *prose* may not write those
4911    /// identifiers either, in any casing: it says "renewal token" and "client
4912    /// secret" as separate words, which normalisation preserves and the scan
4913    /// therefore ignores. That is a real constraint on the documentation, and it
4914    /// is the right way round — a gate loosened until the comments compile is
4915    /// not a gate. It is a constraint on *identifier spellings*, though, and
4916    /// never on English: hyphenating a compound adjective is not writing an
4917    /// identifier, and a gate that could not tell those apart is what this round
4918    /// fixed.
4919    ///
4920    /// # Two different scopes, for two different reasons
4921    ///
4922    /// The **renewal** half stays scoped to the two files `c2` owns plus the
4923    /// manifest. A renewal path in `c3`'s or `c4`'s file would be their finding;
4924    /// failing here on their work would be this task reaching across an
4925    /// ownership boundary.
4926    ///
4927    /// The **client secret** half covers every source file in the crate. That is
4928    /// not a boundary crossing but the opposite: a public client cannot hold a
4929    /// client secret at all (D3, `07-security.md`), so one appearing *anywhere*
4930    /// in this crate is a product defect rather than a matter of whose file it
4931    /// is, and `c2` is the designated owner of that clause. "Every source file"
4932    /// is a claim about the directory, so it is checked against the directory —
4933    /// see [`the_confidential_credential_scan_covers_every_source_file`].
4934    #[test]
4935    fn no_confidential_credential_in_this_crate() {
4936        for &(name, source) in CRATE_SOURCES.iter().chain(std::iter::once(&MANIFEST)) {
4937            let haystack = normalise(name, source);
4938            for forbidden in CONFIDENTIAL {
4939                assert!(
4940                    !haystack.contains(forbidden),
4941                    "{name} names {forbidden:?} in some spelling: a public client cannot \
4942                     secure a confidential credential, and this design never tries to (D3)"
4943                );
4944            }
4945        }
4946    }
4947
4948    /// "Every source file in the crate" is a claim about a directory, and the
4949    /// scan above states it as a hard-coded list. A list is a snapshot: the
4950    /// moment `c3` or `c4` adds a file to `src/`, the claim is false and nothing
4951    /// says so. A `src/pagination.rs` holding a confidential credential passed
4952    /// the gate that exists to catch exactly that.
4953    ///
4954    /// Reading the directory here is what turns the claim back into a claim. It
4955    /// cannot be done in the scan itself — `include_str!` needs a literal path
4956    /// at compile time — so the list stays, and this pins it.
4957    ///
4958    /// # Why it walks the tree instead of listing one directory
4959    ///
4960    /// It used to call `read_dir("src")` once and keep the entries ending in
4961    /// `.rs`. That reads like a directory scan and is not one: a subdirectory
4962    /// module — `src/rest/mod.rs`, `src/rest/runners.rs` — arrives as the single
4963    /// entry `rest`, which does not end in `.rs`, so the filter dropped it and
4964    /// took the files underneath with it. The pin went on passing while those
4965    /// files were scanned by nothing at all.
4966    ///
4967    /// That defeated the pin in exactly the case it was written for. `c3` is the
4968    /// REST inventory gateway, a module directory is the ordinary Rust shape for
4969    /// it, and the failure is silent on both sides: the credential scan does not
4970    /// read the file, and the test whose whole job is to notice that reports
4971    /// success.
4972    ///
4973    /// Recursing is the fix, rather than asserting that `src/` holds no
4974    /// subdirectories. The claim being pinned is about *files*, not about
4975    /// layout; banning the directory would fail `c3` for choosing a normal
4976    /// module shape, and a gate that fails correct work is a gate the next round
4977    /// loosens to get its own work compiling — which is how the normalisation
4978    /// half of this same scan was weakened once already.
4979    ///
4980    /// Names are `/`-joined paths relative to `src/`, which is what
4981    /// `include_str!` takes on every platform, so a nested file is listed as
4982    /// `("rest/runners.rs", include_str!("rest/runners.rs"))` and the two sides
4983    /// compare directly.
4984    #[test]
4985    fn the_confidential_credential_scan_covers_every_source_file() {
4986        // Every `.rs` file at or below `dir`, named by its `/`-joined path
4987        // relative to `src/`.
4988        //
4989        // `file_type()` is deliberately not followed through symlinks: a link
4990        // cannot walk this into a cycle, and a symlinked `.rs` file still lands
4991        // in the list through the extension test. A directory is recursed into
4992        // before the extension is considered, so a directory named `foo.rs`
4993        // is walked rather than mistaken for a file.
4994        fn collect(dir: &std::path::Path, prefix: &str, found: &mut Vec<String>) {
4995            for entry in std::fs::read_dir(dir).expect("the source directory is readable") {
4996                let entry = entry.expect("a readable directory entry");
4997                let name = entry.file_name().to_string_lossy().into_owned();
4998                let relative = if prefix.is_empty() {
4999                    name.clone()
5000                } else {
5001                    format!("{prefix}/{name}")
5002                };
5003                if entry.file_type().expect("a readable entry type").is_dir() {
5004                    collect(&entry.path(), &relative, found);
5005                } else if name.ends_with(".rs") {
5006                    found.push(relative);
5007                }
5008            }
5009        }
5010
5011        let mut on_disk = Vec::new();
5012        collect(
5013            std::path::Path::new(concat!(env!("CARGO_MANIFEST_DIR"), "/src")),
5014            "",
5015            &mut on_disk,
5016        );
5017        on_disk.sort();
5018
5019        // Sorted, rather than taken in declaration order. Comparing a sorted
5020        // `on_disk` against an unsorted `scanned` made this assertion depend on
5021        // `CRATE_SOURCES` happening to be declared alphabetically. It is — but
5022        // nothing said so, and the failure that would follow from reordering the
5023        // list is a diff of two lists holding the same names, which reads as a
5024        // coverage gap rather than as the ordering nit it would actually be.
5025        let mut scanned: Vec<String> = CRATE_SOURCES
5026            .iter()
5027            .map(|(name, _)| (*name).to_string())
5028            .collect();
5029        scanned.sort();
5030
5031        assert_eq!(
5032            scanned, on_disk,
5033            "`src/` and the scanned list have diverged. Add the new file to `CRATE_SOURCES` \
5034             with an `include_str!`, naming it by its `/`-joined path relative to `src/`; \
5035             leaving it out means the confidential-credential scan silently stops covering \
5036             `every source file in the crate`, which is the claim it makes."
5037        );
5038    }
5039
5040    /// The scan above, shown to actually catch the spellings it claims to — and
5041    /// to leave alone the ones it claims to leave alone.
5042    ///
5043    /// Without this, "the gate is case-insensitive now" is a comment rather than
5044    /// a fact, and the finding that produced it was precisely a gate whose
5045    /// description outran what it did. It calls [`normalise`], the same function
5046    /// the scan calls, because a meta-test with its own private copy of the
5047    /// thing under test cannot detect a change to it.
5048    #[test]
5049    fn the_confidential_credential_scan_is_not_evaded_by_naming() {
5050        // Assembled at run time rather than written out, for the same reason the
5051        // needles are spelled in halves: a test that contained these spellings
5052        // literally would fail the scan it is checking.
5053        let source_evasions = [
5054            format!("let {}Token = fetch()", "refresh"),
5055            format!("struct {}Token;", "Refresh"),
5056            format!("{}_TOKEN", "REFRESH"),
5057            format!("{}Secret", "client"),
5058            format!("{}_SECRET", "CLIENT"),
5059            format!("{}_secret", "app"),
5060        ];
5061        for evasion in &source_evasions {
5062            let normalised = normalise("lib.rs", evasion);
5063            assert!(
5064                normalised.contains(concat!("refresh", "token"))
5065                    || normalised.contains(concat!("client", "secret"))
5066                    || normalised.contains(concat!("app", "secret")),
5067                "{evasion:?} would walk straight through the scan"
5068            );
5069        }
5070
5071        // The manifest is where kebab-case is a word separator rather than a
5072        // hyphen, so that is where it is still collapsed.
5073        let manifest_evasion = format!("{}-secret = \"...\"", "client");
5074        assert!(
5075            normalise(MANIFEST.0, &manifest_evasion).contains(concat!("client", "secret")),
5076            "a kebab-case TOML key is an identifier, and the manifest normaliser must \
5077             still collapse it"
5078        );
5079
5080        // And the prose the crate legitimately writes must still pass, or the
5081        // gate would be unusable and would be weakened again to make it usable.
5082        for allowed in [
5083            "a public client cannot hold a client secret",
5084            "the published App issues no renewal token",
5085            // The line that fails the old normalisation, quoted from `c3`'s own
5086            // file. It says the *opposite* of what the gate accused it of.
5087            "//! This gateway is deliberately client-secret-free, as D3 requires.",
5088            // The same shape, for the renewal half.
5089            "a refresh-free credential model",
5090        ] {
5091            let normalised = normalise("lib.rs", allowed);
5092            assert!(
5093                !normalised.contains(concat!("client", "secret"))
5094                    && !normalised.contains(concat!("refresh", "token")),
5095                "{allowed:?} is English, not an identifier, and must not trip the scan"
5096            );
5097        }
5098    }
5099
5100    /// The storage boundary, made checkable the same way. `c2` returns the token
5101    /// and never persists it; the machine-scoped store is `d2` and the wiring is
5102    /// `f1`. A dependency on `runner-manager-platform`, or a filesystem write,
5103    /// would silently move that boundary.
5104    #[test]
5105    fn this_crate_persists_nothing_and_does_not_depend_on_the_platform_crate() {
5106        assert!(
5107            !MANIFEST.1.contains("runner-manager-platform"),
5108            "the gateway must be testable with no platform dependency at all"
5109        );
5110
5111        for &(name, source) in SOURCES_OWNED_BY_C2 {
5112            if name == MANIFEST.0 {
5113                continue;
5114            }
5115            // Everything below `#[cfg(test)]` is test code; the boundary is about
5116            // non-test code, and the tests above legitimately read this file.
5117            let non_test = non_test_prefix(source);
5118            // `OpenOptions`, `File::options` and `std::io::Write` are on this
5119            // list because the original four named only the *obvious* ways to
5120            // write a file. A store built with `OpenOptions::new().create(true)`
5121            // would have moved the persistence boundary silently, which is the
5122            // one thing this scan exists to prevent.
5123            for forbidden in [
5124                "std::fs",
5125                "fs::write",
5126                "File::create",
5127                "File::options",
5128                "OpenOptions",
5129                "std::io::Write",
5130                "tokio::fs",
5131            ] {
5132                assert!(
5133                    !non_test.contains(forbidden),
5134                    "{name} performs a filesystem operation ({forbidden:?}) outside its tests"
5135                );
5136            }
5137        }
5138    }
5139
5140    /// The scan above, shown to be looking at what it says it is looking at.
5141    ///
5142    /// `split("#[cfg(test)]")` matched that literal **anywhere**, prose
5143    /// included. One ordinary sentence in a module's documentation truncated the
5144    /// scanned region to whatever preceded it, and every filesystem call after
5145    /// that point became invisible — with the scan still reporting `ok`. This is
5146    /// the third gate in this crate found describing more than it did, so it
5147    /// gets the same treatment as the other two: a synthetic file where the
5148    /// difference is decisive, and an assertion about the real ones.
5149    #[test]
5150    fn the_non_test_boundary_is_a_line_and_not_a_mention() {
5151        // A file shaped like this crate's own: prose that names the attribute,
5152        // then real non-test code, then the actual module.
5153        let file = "//! Test helpers live in an inline #[cfg(test)] module near the bottom.\n\
5154                    \n\
5155                    fn persist() { std::fs::write(\"x\", b\"y\").unwrap(); }\n\
5156                    \n\
5157                    #[cfg(test)]\n\
5158                    mod tests {\n\
5159                        fn helper() { std::fs::write(\"ok-in-tests\", b\"\").unwrap(); }\n\
5160                    }\n";
5161
5162        let non_test = non_test_prefix(file);
5163        assert!(
5164            non_test.contains("fn persist"),
5165            "a prose mention of the attribute truncated the scanned region, and every \
5166             filesystem call below it stopped being scanned — silently:\n{non_test}"
5167        );
5168        assert!(
5169            !non_test.contains("ok-in-tests"),
5170            "the boundary must still exclude the real test module:\n{non_test}"
5171        );
5172
5173        // And on the real files, whose module documentation contains such a
5174        // mention today. `lib.rs` has carried one since `mod testing` was
5175        // written, so this crate was shipping the truncated scan.
5176        for &(name, source) in SOURCES_OWNED_BY_C2 {
5177            if name == MANIFEST.0 {
5178                continue;
5179            }
5180            let expected = source
5181                .lines()
5182                .position(|line| line.trim() == "#[cfg(test)]")
5183                .expect("each source file has an inline test module");
5184            let scanned = non_test_prefix(source).lines().count();
5185            assert_eq!(
5186                scanned, expected,
5187                "{name}: the scanned region ends at line {scanned} but the test module starts \
5188                 at line {expected}. The gap is code that claims to be scanned and is not."
5189            );
5190        }
5191    }
5192
5193    #[derive(Debug)]
5194    struct SpyRenewal {
5195        invocations: std::sync::atomic::AtomicUsize,
5196    }
5197    #[async_trait::async_trait]
5198    impl CredentialRenewal for SpyRenewal {
5199        async fn renew(
5200            &self,
5201            _refresh_token: &SecretString,
5202        ) -> Result<UserAccessToken, RenewalError> {
5203            self.invocations
5204                .fetch_add(1, std::sync::atomic::Ordering::SeqCst);
5205            Ok(UserAccessToken::new(SecretString::from("ghu_renewed")))
5206        }
5207    }
5208
5209    #[tokio::test]
5210    async fn reload_happens_before_renew_to_prevent_cross_process_races() {
5211        let server = MockServer::start().await;
5212        Mock::given(method("GET"))
5213            .and(path("/repos/acme/app"))
5214            .and(header("authorization", "Bearer ghu_initial"))
5215            .respond_with(ResponseTemplate::new(401))
5216            .expect(1)
5217            .mount(&server)
5218            .await;
5219        Mock::given(method("GET"))
5220            .and(path("/repos/acme/app"))
5221            .and(header("authorization", "Bearer ghu_reloaded"))
5222            .respond_with(ResponseTemplate::new(200).set_body_json(serde_json::json!({"id": 1})))
5223            .expect(1)
5224            .mount(&server)
5225            .await;
5226
5227        let renewal = Arc::new(SpyRenewal {
5228            invocations: std::sync::atomic::AtomicUsize::new(0),
5229        });
5230
5231        let document = serde_json::json!({
5232            "access_token": "ghu_initial",
5233            "refresh_token": "ghr_initial",
5234        });
5235        let initial_token =
5236            UserAccessToken::from_stored_document(&SecretString::from(document.to_string()));
5237
5238        let client = AuthenticatedClient::new(
5239            Endpoints::for_test_server(&server.uri()).unwrap(),
5240            initial_token,
5241            Arc::new(TestClock::default()),
5242        )
5243        .unwrap()
5244        .with_credential_source(Arc::new(StoreHolding(Some("ghu_reloaded"))))
5245        .with_renewal(renewal.clone());
5246
5247        let response = client
5248            .get_json::<serde_json::Value>("/repos/acme/app")
5249            .await
5250            .expect("the reloaded token should succeed");
5251        assert_eq!(response["id"], 1);
5252        assert_eq!(
5253            renewal
5254                .invocations
5255                .load(std::sync::atomic::Ordering::SeqCst),
5256            0,
5257            "renew should not be called because reload succeeded"
5258        );
5259    }
5260
5261    #[tokio::test]
5262    async fn proactive_renewal_waits_until_the_access_token_is_near_expiry() {
5263        let renewal = Arc::new(SpyRenewal {
5264            invocations: std::sync::atomic::AtomicUsize::new(0),
5265        });
5266        let document = serde_json::json!({
5267            "access_token": "ghu_initial",
5268            "refresh_token": "ghr_initial",
5269            "access_expires_at": "2026-08-21T02:00:00Z",
5270            "refresh_expires_at": "2027-02-21T00:00:00Z"
5271        });
5272        let clock = Arc::new(TestClock::default());
5273        let client = AuthenticatedClient::new(
5274            Endpoints::production(),
5275            UserAccessToken::from_stored_document(&SecretString::from(document.to_string())),
5276            clock.clone(),
5277        )
5278        .unwrap()
5279        .with_renewal(renewal.clone());
5280
5281        assert!(
5282            !client
5283                .renew_if_expiring_within(Duration::from_secs(30 * 60))
5284                .await
5285        );
5286        assert_eq!(
5287            renewal
5288                .invocations
5289                .load(std::sync::atomic::Ordering::SeqCst),
5290            0
5291        );
5292
5293        clock.advance_secs(91 * 60);
5294        assert!(
5295            client
5296                .renew_if_expiring_within(Duration::from_secs(30 * 60))
5297                .await
5298        );
5299        assert_eq!(
5300            renewal
5301                .invocations
5302                .load(std::sync::atomic::Ordering::SeqCst),
5303            1
5304        );
5305    }
5306
5307    #[tokio::test]
5308    async fn proactive_renewal_repairs_a_pair_with_no_expiry_metadata() {
5309        let renewal = Arc::new(SpyRenewal {
5310            invocations: std::sync::atomic::AtomicUsize::new(0),
5311        });
5312        let document = serde_json::json!({
5313            "access_token": "ghu_initial",
5314            "refresh_token": "ghr_initial"
5315        });
5316        let client = AuthenticatedClient::new(
5317            Endpoints::production(),
5318            UserAccessToken::from_stored_document(&SecretString::from(document.to_string())),
5319            Arc::new(TestClock::default()),
5320        )
5321        .unwrap()
5322        .with_renewal(renewal.clone());
5323
5324        assert!(
5325            client
5326                .renew_if_expiring_within(Duration::from_secs(30 * 60))
5327                .await
5328        );
5329        assert_eq!(
5330            renewal
5331                .invocations
5332                .load(std::sync::atomic::Ordering::SeqCst),
5333            1
5334        );
5335    }
5336}