runner_manager_github/lib.rs
1// owner: c2-device-flow-auth
2//
3// c2 also owns the shared authenticated HTTP client that lives at this crate
4// root; c3 owns `rest`, and c4 owns `demand` and `jit`.
5
6//! The GitHub gateway.
7//!
8//! This crate holds every line of code in the product that talks to GitHub, and
9//! the crate root holds the one client all of it goes through.
10//!
11//! * [`device_flow`] — the OAuth 2.0 Device Authorization Grant, which is the
12//! *only* way this product ever obtains a credential (D3, D16).
13//! * [`AuthenticatedClient`] — the shared `api.github.com` client. Every request
14//! in this crate is built by it, which is what makes "sets
15//! `X-GitHub-Api-Version` and an explicit `Accept`" a property of the design
16//! rather than of each call site, and what lets the authentication-failure
17//! taxonomy be implemented exactly once.
18//! * [`rest`], [`demand`], [`jit`] — typed adapters owned by `c3` and `c4`,
19//! built on [`AuthenticatedClient`].
20//!
21//! # Three properties this crate is required to keep
22//!
23//! **It holds no client secret, and it renews without one.** This paragraph
24//! used to say the opposite — that the App opts out of user-token expiration,
25//! that no renewal token is ever issued, and that renewing would require a
26//! client secret a public client cannot hold. All three were wrong, and a test
27//! in this crate enforced the error by forbidding the word "refresh token".
28//!
29//! The published App has user-token expiration **on**: a device-flow exchange
30//! returns `expires_in: 28800` and a `refresh_token`. GitHub requires the
31//! client secret to refresh *"unless the user access token was generated using
32//! the device flow"*, and every one of this product's is. So the credential
33//! renews itself, no server appears in the design, and the eight-hour life of
34//! an access token is invisible to a daemon that runs for months. See
35//! [`AuthenticatedClient::renew_once`], and
36//! `docs/spikes/token-expiry-and-renewal.md` for the two renewals that
37//! confirmed it on real hosts.
38//!
39//! [`AuthenticatedClient::revalidate`] is what still happens on a `401` for a
40//! credential with no refresh half — every one issued before 0.1.11.
41//!
42//! **It persists nothing.** [`device_flow::DeviceFlow::complete`] *returns* the
43//! token; it never writes it anywhere. The machine-scoped secret store is `d2`
44//! and the wiring is `f1`. That boundary is why this crate has no dependency on
45//! `runner-manager-platform` and performs no filesystem write outside its own
46//! tests — and it is what lets the whole gateway be tested with no platform
47//! dependency at all.
48//!
49//! **It never renders a secret.** The device code, the user access token, and
50//! every header carrying either are absent from `Debug`, from `Display`, from
51//! errors, and from tracing output. Every type here that holds one wraps it in
52//! [`secrecy::SecretString`] *and* implements [`fmt::Debug`] by hand, because a
53//! `#[derive(Debug)]` added later to a struct with a plain `String` field is
54//! precisely how this control is lost. `tests/no_secret_reaches_the_logs.rs`
55//! drives a whole login and an authenticated round trip through a capturing
56//! `tracing` subscriber and fails if any of the three appears.
57//!
58//! That scan is a **separate test binary**, and deliberately so. As a unit test
59//! it silently stopped working: `tracing` caches each callsite's `Interest`
60//! process-wide while `with_default` installs a subscriber on one *thread*, and
61//! run **concurrently** with the crate's other unit tests the scan captured
62//! only its own handful of events — passing with a real device-code leak on the
63//! live path. A binary holding one test has no concurrency to be poisoned by.
64//! The word "concurrently" is load-bearing and was measured;
65//! `tests/no_secret_reaches_the_logs.rs` records the numbers and what they rule
66//! out.
67
68pub mod demand;
69pub mod device_flow;
70pub mod jit;
71pub mod rest;
72
73use std::{
74 fmt,
75 sync::{
76 Arc,
77 atomic::{AtomicU64, Ordering},
78 },
79 time::Duration,
80};
81
82use chrono::{DateTime, Utc};
83use reqwest::{Method, StatusCode};
84use runner_manager_domain::model::{Clock, Org, OwnerRepo, Timestamp};
85
86/// Re-exported because [`GithubError::headers`] and [`ApiResponse::headers`]
87/// return one, and a consumer cannot *name* a type it has no path to.
88///
89/// `a1` owns every manifest in this workspace, so a crate outside this one that
90/// wanted to hold a `HeaderMap` from this seam would otherwise need `reqwest`
91/// added to its own dependencies — turning a `c2` seam into an `a1` change, and
92/// putting `reqwest`'s version in two places at once. [`GithubError::retry_after`]
93/// and [`GithubError::rate_limit`] exist precisely so that the common cases need
94/// no path at all; this is for the ones that do.
95///
96/// It is re-exported under its own name rather than an alias so that the type a
97/// consumer imports is the type the signatures already show.
98pub use reqwest::header::HeaderMap;
99use secrecy::{ExposeSecret, SecretString};
100use serde::{Deserialize, Serialize, de::DeserializeOwned};
101use url::Url;
102
103// ---------------------------------------------------------------------------
104// Constants
105// ---------------------------------------------------------------------------
106
107/// The REST API version every request pins.
108///
109/// `04-subsystem-contracts.md`: "All requests set `X-GitHub-Api-Version` and an
110/// explicit `Accept` header." Both spikes ran against this version.
111pub const GITHUB_API_VERSION: &str = "2022-11-28";
112
113/// The media type every request asks for, stated rather than defaulted.
114pub const GITHUB_ACCEPT: &str = "application/vnd.github+json";
115
116/// Production `api.github.com`.
117pub const GITHUB_API_BASE: &str = "https://api.github.com/";
118
119/// Production `github.com`, which hosts the device-flow endpoints. They are on
120/// the web host, not the API host.
121pub const GITHUB_WEB_BASE: &str = "https://github.com/";
122
123/// The canonical page a user types their code into.
124///
125/// This constant *is* the phishing control (`07-security.md`, threat table): the
126/// tool prints this URL and never proxies, embeds, or imitates the approval
127/// page. [`Endpoints::verification_url`] derives from the configured web base so
128/// a test server can be pointed at, and
129/// [`device_flow::DeviceAuthorization::verification_uri`] is checked against it
130/// so a response that tries to send the user somewhere else is rejected rather
131/// than displayed.
132pub const DEVICE_VERIFICATION_PATH: &str = "login/device";
133
134/// What a `401` re-validates the held credential against.
135///
136/// `GET /user/installations` rather than `GET /user`, because it is the call the
137/// D18 spike actually made with a user-to-server token and observed `200` from
138/// (`docs/spikes/d18-org-jit-verification.md`, "The permission that authorized
139/// it"), and because a successful re-validation then carries the same answer
140/// [`AuthenticatedClient::discover_installations`] needs.
141pub const REVALIDATION_PATH: &str = "/user/installations";
142
143/// How long a lockout backs off for when GitHub sends no `retry-after`.
144///
145/// `03-control-flows.md` flow 4.3 requires a back-off but names no duration.
146/// Sixty seconds is GitHub's own documented floor for its secondary rate limits.
147pub const DEFAULT_LOCKOUT_BACKOFF: Duration = Duration::from_secs(60);
148
149/// The longest a lockout may silence this client, whatever `Retry-After` said.
150///
151/// A back-off is a *safety* mechanism, and an unclamped one is a denial of
152/// service with extra steps: `Retry-After: 86400` would latch a silent
153/// twenty-four-hour outage of the agent's reconciliation loop, clearable only by
154/// [`AuthenticatedClient::clear_lockout`]. Fifteen minutes is far longer than any
155/// back-off GitHub documents for the authentication lockout this latches on, and
156/// short enough that a hostile or simply wrong header cannot take the product
157/// down for a shift. Honouring a header without a ceiling is trusting a remote
158/// party with the product's availability.
159pub const MAX_LOCKOUT_BACKOFF: Duration = Duration::from_secs(15 * 60);
160
161/// The most pages either pagination loop follows before giving up.
162///
163/// A `Link: rel="next"` that points back at the page it arrived on — a proxy
164/// rewriting the header, or a bug at the other end — is an infinite loop inside
165/// the agent's reconciliation loop, which is the one place in this product that
166/// must not be able to wedge. At `per_page=100` this ceiling is ten thousand
167/// installations or repositories, past any real account by orders of magnitude,
168/// so it bounds the pathological case without truncating a legitimate one.
169pub const MAX_PAGES: usize = 100;
170
171/// Per-request ceiling, so one wedged connection cannot stall the agent's
172/// reconciliation loop forever.
173pub const DEFAULT_REQUEST_TIMEOUT: Duration = Duration::from_secs(30);
174
175/// The `User-Agent` GitHub requires on every API request.
176pub const USER_AGENT: &str = concat!("runner-manager/", env!("CARGO_PKG_VERSION"));
177
178// ---------------------------------------------------------------------------
179// The published App
180// ---------------------------------------------------------------------------
181
182/// The published GitHub App this product authenticates as (D3, D16).
183///
184/// Both fields are **public by design**. `07-security.md`'s credential inventory
185/// lists the `client_id` as "Not secret … may appear in logs and documentation",
186/// which is exactly what makes the device flow serverless: a public client
187/// cannot secure a client secret, and this design never tries to.
188///
189/// The concrete values are *not* compiled in here. Registering and publishing
190/// the App is Phase 0 of `06-migration-rollout.md` and has not happened, so
191/// there is no honest value to write; `f1` supplies both when it wires the CLI.
192/// Committing a placeholder that looked real would be worse than requiring the
193/// caller to pass one.
194#[derive(Debug, Clone, PartialEq, Eq)]
195pub struct AppRegistration {
196 client_id: String,
197 slug: String,
198}
199
200impl AppRegistration {
201 /// # Errors
202 /// An empty `client_id` or an empty `slug`.
203 pub fn new(client_id: impl Into<String>, slug: impl Into<String>) -> Result<Self, ConfigError> {
204 let client_id = client_id.into();
205 let slug = slug.into();
206 if client_id.trim().is_empty() {
207 return Err(ConfigError::Empty { what: "client_id" });
208 }
209 if slug.trim().is_empty() {
210 return Err(ConfigError::Empty { what: "app slug" });
211 }
212 Ok(Self { client_id, slug })
213 }
214
215 #[must_use]
216 pub fn client_id(&self) -> &str {
217 &self.client_id
218 }
219
220 #[must_use]
221 pub fn slug(&self) -> &str {
222 &self.slug
223 }
224
225 /// The canonical URL a user with no installation must visit.
226 ///
227 /// `03-control-flows.md` flow 1.1: "If the published App is not yet installed
228 /// on any repository, it prints the installation URL."
229 ///
230 /// # Panics
231 /// Never, for a registration built through [`AppRegistration::new`]: the slug
232 /// is non-empty and is percent-encoded into the path.
233 #[must_use]
234 pub fn install_url(&self, endpoints: &Endpoints) -> Url {
235 endpoints
236 .web_base
237 .join("apps/")
238 .and_then(|u| u.join(&format!("{}/", encode_path_segment(&self.slug))))
239 .and_then(|u| u.join("installations/new"))
240 .expect("a non-empty encoded slug always joins onto the web base")
241 }
242}
243
244fn encode_path_segment(raw: &str) -> String {
245 raw.chars()
246 .map(|c| {
247 if c.is_ascii_alphanumeric() || matches!(c, '-' | '_' | '.' | '~') {
248 c.to_string()
249 } else {
250 let mut buf = [0_u8; 4];
251 c.encode_utf8(&mut buf)
252 .as_bytes()
253 .iter()
254 .map(|b| format!("%{b:02X}"))
255 .collect()
256 }
257 })
258 .collect()
259}
260
261/// Where GitHub is.
262///
263/// Two bases rather than one, because the device grant lives on `github.com`
264/// while every API call lives on `api.github.com`. Tests point both at one
265/// `wiremock` server; the paths do not collide.
266#[derive(Debug, Clone, PartialEq, Eq)]
267pub struct Endpoints {
268 api_base: Url,
269 web_base: Url,
270}
271
272impl Endpoints {
273 /// Production GitHub.
274 ///
275 /// # Panics
276 /// Never: both constants are parsed at every call and are valid URLs.
277 #[must_use]
278 pub fn production() -> Self {
279 Self {
280 api_base: Url::parse(GITHUB_API_BASE).expect("GITHUB_API_BASE is a valid URL"),
281 web_base: Url::parse(GITHUB_WEB_BASE).expect("GITHUB_WEB_BASE is a valid URL"),
282 }
283 }
284
285 /// Both bases are normalised to end in `/` so that relative joins keep the
286 /// whole base path instead of replacing its last segment.
287 #[must_use]
288 pub fn new(api_base: Url, web_base: Url) -> Self {
289 Self {
290 api_base: with_trailing_slash(api_base),
291 web_base: with_trailing_slash(web_base),
292 }
293 }
294
295 /// Point every endpoint at one test server.
296 ///
297 /// # Errors
298 /// `root` not being a parseable absolute URL.
299 pub fn for_test_server(root: &str) -> Result<Self, ConfigError> {
300 let root = Url::parse(root).map_err(|_| ConfigError::Empty {
301 what: "test server URL",
302 })?;
303 Ok(Self::new(root.clone(), root))
304 }
305
306 #[must_use]
307 pub fn api_base(&self) -> &Url {
308 &self.api_base
309 }
310
311 #[must_use]
312 pub fn web_base(&self) -> &Url {
313 &self.web_base
314 }
315
316 /// # Panics
317 /// Never: the path is a constant and the base ends in `/`.
318 #[must_use]
319 pub fn device_code_url(&self) -> Url {
320 self.web_base
321 .join("login/device/code")
322 .expect("a constant path joins onto a normalised base")
323 }
324
325 /// # Panics
326 /// Never: the path is a constant and the base ends in `/`.
327 #[must_use]
328 pub fn access_token_url(&self) -> Url {
329 self.web_base
330 .join("login/oauth/access_token")
331 .expect("a constant path joins onto a normalised base")
332 }
333
334 /// The canonical page the user code is typed into, and the only device-flow
335 /// URL this product ever prints.
336 ///
337 /// # Panics
338 /// Never: the path is a constant and the base ends in `/`.
339 #[must_use]
340 pub fn verification_url(&self) -> Url {
341 self.web_base
342 .join(DEVICE_VERIFICATION_PATH)
343 .expect("a constant path joins onto a normalised base")
344 }
345}
346
347impl Default for Endpoints {
348 fn default() -> Self {
349 Self::production()
350 }
351}
352
353fn with_trailing_slash(mut url: Url) -> Url {
354 if !url.path().ends_with('/') {
355 let path = format!("{}/", url.path());
356 url.set_path(&path);
357 }
358 url
359}
360
361/// A configuration value this crate refuses to start with.
362#[derive(Debug, Clone, PartialEq, Eq, thiserror::Error)]
363pub enum ConfigError {
364 #[error("{what} must not be empty")]
365 Empty { what: &'static str },
366}
367
368// ---------------------------------------------------------------------------
369// The credential
370// ---------------------------------------------------------------------------
371
372/// A user access token obtained from the device flow.
373///
374/// **Non-expiring, and non-renewable.** The published App opts out of
375/// user-token expiration, so GitHub issues no renewal token alongside this one
376/// and there is nothing to renew — see the crate documentation. The token is
377/// invalidated only by the user uninstalling the App or revoking the
378/// authorization at GitHub.
379///
380/// `Debug` is written by hand. Deriving it here would put the token into every
381/// `tracing` field, every `unwrap()` panic message, and every `anyhow` chain
382/// that ever carries one, which is the exact leak `07-security.md` gates on.
383#[derive(Clone)]
384pub struct UserAccessToken {
385 token: SecretString,
386 token_type: String,
387 scope: Option<String>,
388 /// The renewal half, when the App issues one.
389 ///
390 /// # Why this is an `Option` rather than a second type
391 ///
392 /// Whether a credential can renew itself is a setting on the *App*, not a
393 /// property of this build: an App with user-token expiration off returns an
394 /// access token and nothing else, and one with it on returns a pair. Both
395 /// shapes reach this type, and a host holding either must keep working --
396 /// otherwise flipping that setting would strand every installation that had
397 /// not upgraded, which for a published product is an outage nobody asked
398 /// for.
399 ///
400 /// `None` is therefore not a defect. It is the credential this product held
401 /// for its whole life until now: non-expiring, unrenewable, replaced only
402 /// by an interactive `auth login`.
403 renewal: Option<Renewal>,
404}
405
406/// What a token needs in order to replace itself without a person.
407#[derive(Clone)]
408pub struct Renewal {
409 refresh_token: SecretString,
410 /// When the access token stops being accepted, if it was stated.
411 pub access_expires_at: Option<DateTime<Utc>>,
412 /// When the *refresh* token stops working. Past this, only an interactive
413 /// sign-in helps -- and it is six months from the last renewal, not from
414 /// the first sign-in, so a host that runs at all never reaches it.
415 pub refresh_expires_at: Option<DateTime<Utc>>,
416}
417
418impl Renewal {
419 /// The refresh token itself. Kept behind a method for the same reason the
420 /// access token is: it is the more dangerous half of the pair, because it
421 /// mints access tokens indefinitely.
422 #[must_use]
423 pub fn refresh_token(&self) -> &SecretString {
424 &self.refresh_token
425 }
426}
427
428impl fmt::Debug for Renewal {
429 /// Written by hand, like [`UserAccessToken`]'s, and for a stronger reason:
430 /// a leaked refresh token does not expire in eight hours.
431 fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
432 f.debug_struct("Renewal")
433 .field("refresh_token", &"[redacted]")
434 .field("access_expires_at", &self.access_expires_at)
435 .field("refresh_expires_at", &self.refresh_expires_at)
436 .finish()
437 }
438}
439
440impl UserAccessToken {
441 #[must_use]
442 pub fn new(token: SecretString) -> Self {
443 Self {
444 token,
445 token_type: "bearer".to_string(),
446 scope: None,
447 renewal: None,
448 }
449 }
450
451 /// The whole credential as the token endpoint returned it. `pub(crate)`
452 /// because [`device_flow`] is the only thing in the product entitled to mint
453 /// one — every other path receives a token rather than constructing it.
454 pub(crate) fn from_parts(
455 token: SecretString,
456 token_type: String,
457 scope: Option<String>,
458 ) -> Self {
459 Self {
460 token,
461 token_type,
462 scope,
463 renewal: None,
464 }
465 }
466
467 /// Attach the renewal half, if the App issued one.
468 ///
469 /// The two durations are seconds-from-now as GitHub states them, turned
470 /// into instants here so that nothing downstream has to remember when
471 /// "now" was.
472 #[must_use]
473 pub(crate) fn with_renewal(
474 mut self,
475 refresh_token: Option<SecretString>,
476 access_expires_in: Option<u64>,
477 refresh_expires_in: Option<u64>,
478 ) -> Self {
479 self.renewal = refresh_token.map(|refresh_token| {
480 let at = |secs: Option<u64>| {
481 secs.and_then(|s| i64::try_from(s).ok())
482 .and_then(|s| Utc::now().checked_add_signed(chrono::TimeDelta::seconds(s)))
483 };
484 Renewal {
485 refresh_token,
486 access_expires_at: at(access_expires_in),
487 refresh_expires_at: at(refresh_expires_in),
488 }
489 });
490 self
491 }
492
493 /// The renewal half, when there is one.
494 #[must_use]
495 pub fn renewal(&self) -> Option<&Renewal> {
496 self.renewal.as_ref()
497 }
498
499 /// Rebuild the credential `d2` handed back, for `f1`.
500 #[must_use]
501 pub fn from_stored(token: SecretString) -> Self {
502 Self::from_stored_document(&token)
503 }
504
505 /// Reads whichever of the two stored shapes is there.
506 ///
507 /// # Why the store holds a document now, and why the old shape still loads
508 ///
509 /// A renewable credential is three values -- access token, refresh token,
510 /// and when each stops working -- where there used to be one string. The
511 /// secret store takes one opaque value per host, so the document goes
512 /// inside it rather than the store growing a schema: no platform change, no
513 /// migration step, and the same DPAPI blob or keychain item as before.
514 ///
515 /// **A value that is not this document is a bare access token**, which is
516 /// what every host stored until now. That is not a fallback for tidiness:
517 /// upgrading must not log anybody out, and the App's expiration setting can
518 /// be turned on -- or back off -- without stranding hosts that are mid-way
519 /// through either. A token has no internal structure to confuse with JSON,
520 /// so the discrimination is unambiguous.
521 #[must_use]
522 pub fn from_stored_document(stored: &SecretString) -> Self {
523 #[derive(Deserialize)]
524 struct Document {
525 access_token: String,
526 #[serde(default)]
527 refresh_token: Option<String>,
528 #[serde(default)]
529 access_expires_at: Option<DateTime<Utc>>,
530 #[serde(default)]
531 refresh_expires_at: Option<DateTime<Utc>>,
532 }
533
534 match serde_json::from_str::<Document>(stored.expose_secret()) {
535 Ok(document) => Self {
536 token: SecretString::from(document.access_token),
537 token_type: "bearer".to_string(),
538 scope: None,
539 renewal: document.refresh_token.map(|refresh_token| Renewal {
540 refresh_token: SecretString::from(refresh_token),
541 access_expires_at: document.access_expires_at,
542 refresh_expires_at: document.refresh_expires_at,
543 }),
544 },
545 Err(_) => Self::new(stored.clone()),
546 }
547 }
548
549 /// The value to hand the secret store.
550 ///
551 /// Always the document, even for a credential with no renewal half: one
552 /// shape written means one shape to reason about, and reading still accepts
553 /// the bare token that older versions wrote.
554 #[must_use]
555 pub fn to_stored_document(&self) -> SecretString {
556 #[derive(Serialize)]
557 struct Document<'a> {
558 access_token: &'a str,
559 #[serde(skip_serializing_if = "Option::is_none")]
560 refresh_token: Option<&'a str>,
561 #[serde(skip_serializing_if = "Option::is_none")]
562 access_expires_at: Option<DateTime<Utc>>,
563 #[serde(skip_serializing_if = "Option::is_none")]
564 refresh_expires_at: Option<DateTime<Utc>>,
565 }
566
567 let document = Document {
568 access_token: self.token.expose_secret(),
569 refresh_token: self
570 .renewal
571 .as_ref()
572 .map(|r| r.refresh_token.expose_secret()),
573 access_expires_at: self.renewal.as_ref().and_then(|r| r.access_expires_at),
574 refresh_expires_at: self.renewal.as_ref().and_then(|r| r.refresh_expires_at),
575 };
576 // Serialising a struct of `&str` cannot fail; the fallback keeps the
577 // access token usable rather than inventing an error path nobody can
578 // act on.
579 SecretString::from(
580 serde_json::to_string(&document)
581 .unwrap_or_else(|_| self.token.expose_secret().to_string()),
582 )
583 }
584
585 /// The token itself. Every call site of this is a place a secret can escape,
586 /// so there are deliberately few: the `Authorization` header, and `d2`'s
587 /// store call.
588 #[must_use]
589 pub fn secret(&self) -> &SecretString {
590 &self.token
591 }
592
593 #[must_use]
594 pub fn token_type(&self) -> &str {
595 &self.token_type
596 }
597
598 #[must_use]
599 pub fn scope(&self) -> Option<&str> {
600 self.scope.as_deref()
601 }
602
603 /// The token's four-character family prefix — `ghu_` for an App
604 /// user-to-server token — and nothing else.
605 ///
606 /// This exists so diagnostics can answer "did the device flow return the
607 /// kind of token we expected?" without exposing the token. The D17 spike
608 /// asserted exactly this and no more.
609 #[must_use]
610 pub fn family(&self) -> &str {
611 let raw = self.token.expose_secret();
612 match raw.find('_') {
613 Some(idx) if idx < 8 => &raw[..=idx],
614 _ => "",
615 }
616 }
617
618 /// `true` for the `ghu_` family the published App issues.
619 #[must_use]
620 pub fn is_user_to_server(&self) -> bool {
621 self.family() == "ghu_"
622 }
623}
624
625impl fmt::Debug for UserAccessToken {
626 fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
627 f.debug_struct("UserAccessToken")
628 .field("token", &"[REDACTED]")
629 .field("family", &self.family())
630 .finish_non_exhaustive()
631 }
632}
633
634impl PartialEq for UserAccessToken {
635 /// Equality exists so [`device_flow::PollOutcome`] can carry a token and
636 /// still be compared in a test. Production code never compares two
637 /// credentials, and this is not a constant-time comparison.
638 fn eq(&self, other: &Self) -> bool {
639 self.token.expose_secret() == other.token.expose_secret()
640 && self.token_type == other.token_type
641 && self.scope == other.scope
642 }
643}
644
645impl Eq for UserAccessToken {}
646
647// ---------------------------------------------------------------------------
648// Errors
649// ---------------------------------------------------------------------------
650
651/// Everything [`AuthenticatedClient`] can fail with.
652///
653/// The first three variants are the taxonomy `03-control-flows.md` flow 4.3
654/// requires, and they are separate variants because `c3` and `f1` both act on
655/// the distinction: [`GithubError::AuthenticationFailed`] moves a policy to
656/// `authentication_failed` and tells the operator to run `auth login`,
657/// [`GithubError::AuthenticationLockout`] must *wait* and tell the operator
658/// nothing is wrong with their credential, and [`GithubError::Forbidden`] is a
659/// permissions answer that re-authenticating will not change.
660///
661/// # Why the failing variants carry response headers
662///
663/// Rate-limit *policy* is `c3`'s and is deliberately not implemented here. But a
664/// policy needs evidence, and the evidence — `retry-after`,
665/// `x-ratelimit-remaining`, `x-ratelimit-reset` — only exists on the response
666/// that failed. An error taxonomy that dropped those headers would leave `c3`
667/// with no way to honour a `429` except by editing this file, which is exactly
668/// the conflict the `c2`/`c3` ownership split exists to prevent. So
669/// [`GithubError::Status`] and [`GithubError::Forbidden`] carry the headers
670/// verbatim and interpret none of them; see [`GithubError::headers`].
671#[derive(Debug, thiserror::Error)]
672pub enum GithubError {
673 /// GitHub rejected the credential and a single re-validation confirmed it.
674 /// Terminal: only an interactive `auth login` clears this.
675 #[error(
676 "GitHub rejected the stored credential; run `runner-manager auth login` to sign in again"
677 )]
678 AuthenticationFailed,
679
680 /// GitHub answered `403` after `401`s — its temporary authentication
681 /// lockout, not a permissions change. Back off; do not re-authenticate and
682 /// do not retry.
683 #[error(
684 "GitHub has temporarily locked out authentication for this credential; \
685 back off for {}s and do not retry — the credential itself is not the problem",
686 retry_after.as_secs()
687 )]
688 AuthenticationLockout { retry_after: Duration },
689
690 /// A `403` that is not the lockout: a permissions answer, or GitHub's own
691 /// rate limit. `c3` tells the two apart from `headers`; this crate does not,
692 /// because which of them is worth retrying is rate-limit policy.
693 #[error(
694 "GitHub denied {method} {path}: the App installation does not grant it{}",
695 message.as_deref().map(|m| format!(" ({m})")).unwrap_or_default()
696 )]
697 Forbidden {
698 method: String,
699 path: String,
700 message: Option<String>,
701 /// The response headers, verbatim and uninterpreted.
702 headers: Box<HeaderMap>,
703 },
704
705 #[error(
706 "GitHub returned {status} for {method} {path}{}",
707 message.as_deref().map(|m| format!(": {m}")).unwrap_or_default()
708 )]
709 Status {
710 status: u16,
711 method: String,
712 path: String,
713 message: Option<String>,
714 /// The response headers, verbatim and uninterpreted. A `429` reaches
715 /// `c3` through this variant, and its `retry-after` survives with it.
716 headers: Box<HeaderMap>,
717 },
718
719 /// The request never got an answer. The URL is stripped from the source
720 /// error before it is stored: a device-flow URL never carries a secret, but
721 /// stripping it costs nothing and removes a whole class of future leak.
722 #[error("GitHub was unreachable")]
723 Transport(#[source] reqwest::Error),
724
725 #[error("a {what} response from GitHub could not be decoded as {expected}")]
726 Decode {
727 what: &'static str,
728 expected: &'static str,
729 #[source]
730 source: serde_json::Error,
731 },
732
733 #[error("GitHub returned {value:?} for {what}, which this client cannot use")]
734 Malformed { what: &'static str, value: String },
735
736 #[error(transparent)]
737 Config(#[from] ConfigError),
738}
739
740impl GithubError {
741 /// `true` for the two authentication outcomes, which callers handle
742 /// differently from every other failure.
743 #[must_use]
744 pub fn is_authentication(&self) -> bool {
745 matches!(
746 self,
747 Self::AuthenticationFailed | Self::AuthenticationLockout { .. }
748 )
749 }
750
751 /// `true` only for the lockout, which is the one authentication outcome that
752 /// resolves by waiting rather than by signing in again.
753 #[must_use]
754 pub fn is_lockout(&self) -> bool {
755 matches!(self, Self::AuthenticationLockout { .. })
756 }
757
758 /// The failing response's headers, for the variants that have them.
759 ///
760 /// This is the whole of `c2`'s contribution to rate limiting: it hands `c3`
761 /// the evidence and stops there. Nothing in this crate reads
762 /// `x-ratelimit-remaining` to decide anything.
763 #[must_use]
764 pub fn headers(&self) -> Option<&HeaderMap> {
765 match self {
766 Self::Status { headers, .. } | Self::Forbidden { headers, .. } => Some(headers),
767 _ => None,
768 }
769 }
770
771 /// The failing response's `retry-after`, in seconds, if it sent one.
772 ///
773 /// Reading a documented header is evidence, not policy: what to *do* with a
774 /// `retry-after` — wait, shed load, surface it to an operator — is `c3`'s.
775 #[must_use]
776 pub fn retry_after(&self) -> Option<Duration> {
777 self.headers().and_then(retry_after)
778 }
779
780 /// The `x-ratelimit-remaining` / `x-ratelimit-reset` pair, when present.
781 ///
782 /// Returned as the raw numbers GitHub sent. `reset` is a Unix timestamp in
783 /// seconds, which is what the header carries; it is deliberately not turned
784 /// into a [`Timestamp`] here, because comparing it against a clock is the
785 /// first step of a policy decision and that decision is `c3`'s.
786 #[must_use]
787 pub fn rate_limit(&self) -> Option<RateLimitEvidence> {
788 let headers = self.headers()?;
789 let read = |name: &str| {
790 headers
791 .get(name)
792 .and_then(|v| v.to_str().ok())
793 .and_then(|v| v.trim().parse::<u64>().ok())
794 };
795 let remaining = read("x-ratelimit-remaining");
796 let reset = read("x-ratelimit-reset");
797 if remaining.is_none() && reset.is_none() {
798 return None;
799 }
800 Some(RateLimitEvidence {
801 remaining,
802 reset_unix_secs: reset,
803 retry_after: self.retry_after(),
804 })
805 }
806}
807
808/// What GitHub said about its own rate limit on a response that failed.
809///
810/// Evidence, carried across the `c2`/`c3` seam. Every field is what the wire
811/// said, and none of them has been interpreted.
812#[derive(Debug, Clone, Copy, PartialEq, Eq)]
813pub struct RateLimitEvidence {
814 /// `x-ratelimit-remaining`. Zero is GitHub's primary rate limit.
815 pub remaining: Option<u64>,
816 /// `x-ratelimit-reset`, a Unix timestamp in seconds.
817 pub reset_unix_secs: Option<u64>,
818 /// `retry-after`, which secondary rate limits send instead.
819 pub retry_after: Option<Duration>,
820}
821
822fn transport(err: reqwest::Error) -> GithubError {
823 GithubError::Transport(err.without_url())
824}
825
826// ---------------------------------------------------------------------------
827// Requests and responses
828// ---------------------------------------------------------------------------
829
830/// One `api.github.com` request, before authentication headers are applied.
831///
832/// `Debug` is written by hand and never renders the body: `c4` posts
833/// `generate-jitconfig` requests through this type, and a JIT configuration is
834/// a sensitive short-lived value (`07-security.md`, credential inventory).
835#[derive(Clone)]
836pub struct ApiRequest {
837 method: Method,
838 /// Either a path relative to the API base, or an absolute URL — which is
839 /// what a `Link: rel="next"` page is.
840 path: String,
841 query: Vec<(String, String)>,
842 body: Option<serde_json::Value>,
843}
844
845impl ApiRequest {
846 #[must_use]
847 pub fn get(path: impl Into<String>) -> Self {
848 Self::new(Method::GET, path)
849 }
850
851 #[must_use]
852 pub fn delete(path: impl Into<String>) -> Self {
853 Self::new(Method::DELETE, path)
854 }
855
856 #[must_use]
857 pub fn new(method: Method, path: impl Into<String>) -> Self {
858 Self {
859 method,
860 path: path.into(),
861 query: Vec::new(),
862 body: None,
863 }
864 }
865
866 /// # Errors
867 /// `body` failing to serialize.
868 pub fn post_json<T: Serialize>(path: impl Into<String>, body: &T) -> Result<Self, GithubError> {
869 let value = serde_json::to_value(body).map_err(|source| GithubError::Decode {
870 what: "request",
871 expected: "JSON",
872 source,
873 })?;
874 Ok(Self {
875 method: Method::POST,
876 path: path.into(),
877 query: Vec::new(),
878 body: Some(value),
879 })
880 }
881
882 #[must_use]
883 pub fn query(mut self, key: impl Into<String>, value: impl fmt::Display) -> Self {
884 self.query.push((key.into(), value.to_string()));
885 self
886 }
887
888 #[must_use]
889 pub fn method(&self) -> &Method {
890 &self.method
891 }
892
893 #[must_use]
894 pub fn path(&self) -> &str {
895 &self.path
896 }
897}
898
899impl fmt::Debug for ApiRequest {
900 fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
901 f.debug_struct("ApiRequest")
902 .field("method", &self.method.as_str())
903 .field("path", &self.path)
904 .field(
905 "query_keys",
906 &self.query.iter().map(|(k, _)| k).collect::<Vec<_>>(),
907 )
908 .field(
909 "body",
910 &self.body.as_ref().map_or("none", |_| "[REDACTED JSON]"),
911 )
912 .finish()
913 }
914}
915
916/// One buffered `api.github.com` response.
917///
918/// Buffered rather than streamed because every API response this crate reads is
919/// small JSON. The one large download in the product — the runner package — is
920/// `e2`'s and uses its own streaming client.
921///
922/// `Debug` renders the status and the body's *length*, never the body: a
923/// `generate-jitconfig` response body is an encoded JIT configuration.
924#[derive(Clone)]
925pub struct ApiResponse {
926 status: StatusCode,
927 headers: HeaderMap,
928 body: Vec<u8>,
929}
930
931impl ApiResponse {
932 #[must_use]
933 pub fn status(&self) -> StatusCode {
934 self.status
935 }
936
937 #[must_use]
938 pub fn headers(&self) -> &HeaderMap {
939 &self.headers
940 }
941
942 #[must_use]
943 pub fn header(&self, name: &str) -> Option<&str> {
944 self.headers.get(name).and_then(|v| v.to_str().ok())
945 }
946
947 /// # Errors
948 /// A body that is not the expected JSON shape.
949 pub fn json<T: DeserializeOwned>(&self) -> Result<T, GithubError> {
950 serde_json::from_slice(&self.body).map_err(|source| GithubError::Decode {
951 what: "response",
952 expected: std::any::type_name::<T>(),
953 source,
954 })
955 }
956
957 /// The next page of a paginated collection, from the `Link` header.
958 ///
959 /// `04-subsystem-contracts.md`: "Pagination is mandatory; the dashboard must
960 /// not treat a first page as a complete inventory." It lives on the shared
961 /// response type so that `c3`'s inventory and this module's installation
962 /// discovery cannot disagree about how a `Link` header is read.
963 #[must_use]
964 pub fn next_page(&self) -> Option<Url> {
965 self.header("link").and_then(parse_link_next)
966 }
967}
968
969impl fmt::Debug for ApiResponse {
970 fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
971 f.debug_struct("ApiResponse")
972 .field("status", &self.status.as_u16())
973 .field("body_bytes", &self.body.len())
974 .finish_non_exhaustive()
975 }
976}
977
978/// The `rel="next"` target of an RFC 8288 `Link` header, or `None`.
979///
980/// # Why this scans rather than splits
981///
982/// A comma separates one link-value from the next, but a comma is also a legal
983/// character *inside* a URL, and GitHub sends such URLs routinely — a runner
984/// query carries `labels=self-hosted,windows`. Splitting the whole header on `,`
985/// first tears that URL in half, neither half parses as `<...>`, and the
986/// relation is silently lost. The caller then treats page 1 as the whole
987/// inventory, which is the specific outcome `04-subsystem-contracts.md` forbids:
988/// "the dashboard must not treat a first page as a complete inventory".
989///
990/// So the target is located by its `<`…`>` delimiters, and only a comma that
991/// actually begins the next link-value — one followed by optional whitespace and
992/// `<` — ends the parameter section.
993fn parse_link_next(link: &str) -> Option<Url> {
994 let mut rest = link;
995 while let Some(open) = rest.find('<') {
996 let after_open = &rest[open + 1..];
997 let Some(close) = after_open.find('>') else {
998 // An unterminated `<` cannot be a link-value; nothing after it is
999 // interpretable either.
1000 return None;
1001 };
1002 let target = after_open[..close].trim();
1003 let tail = &after_open[close + 1..];
1004
1005 // The parameters run to the start of the next link-value.
1006 let cut = tail
1007 .match_indices(',')
1008 .find(|(i, _)| tail[i + 1..].trim_start().starts_with('<'))
1009 .map_or(tail.len(), |(i, _)| i);
1010 let (params, next) = tail.split_at(cut);
1011
1012 let is_next = params.split(';').any(|param| {
1013 let param = param.trim().replace(['"', '\''], "");
1014 param.eq_ignore_ascii_case("rel=next")
1015 });
1016 if is_next {
1017 return Url::parse(target).ok();
1018 }
1019 rest = next.strip_prefix(',').unwrap_or(next);
1020 }
1021 None
1022}
1023
1024#[derive(Debug, Deserialize)]
1025struct ErrorEnvelope {
1026 message: Option<String>,
1027}
1028
1029/// GitHub's own message for a failure, and never the raw body.
1030fn error_message(body: &[u8]) -> Option<String> {
1031 serde_json::from_slice::<ErrorEnvelope>(body)
1032 .ok()
1033 .and_then(|e| e.message)
1034 .filter(|m| !m.is_empty())
1035}
1036
1037fn retry_after(headers: &HeaderMap) -> Option<Duration> {
1038 headers
1039 .get("retry-after")
1040 .and_then(|v| v.to_str().ok())
1041 .and_then(|v| v.trim().parse::<u64>().ok())
1042 .map(Duration::from_secs)
1043}
1044
1045// ---------------------------------------------------------------------------
1046// Sleeping
1047// ---------------------------------------------------------------------------
1048
1049/// The one way anything in this crate waits.
1050///
1051/// A port rather than a direct `tokio::time::sleep`, for the same reason the
1052/// domain has a `Clock`: the device flow's `slow_down` handling is a *timing*
1053/// behaviour, and a timing behaviour tested by actually waiting is either
1054/// untested or slow. A test substitutes a sleeper that records the requested
1055/// durations and returns immediately, which turns "`slow_down` increases the
1056/// poll interval" into an equality assertion on a `Vec<Duration>` rather than a
1057/// stopwatch reading.
1058#[async_trait::async_trait]
1059pub trait Sleeper: Send + Sync + fmt::Debug {
1060 async fn sleep(&self, duration: Duration);
1061}
1062
1063/// The production adapter.
1064#[derive(Debug, Clone, Copy, Default)]
1065pub struct TokioSleeper;
1066
1067#[async_trait::async_trait]
1068impl Sleeper for TokioSleeper {
1069 async fn sleep(&self, duration: Duration) {
1070 tokio::time::sleep(duration).await;
1071 }
1072}
1073
1074// ---------------------------------------------------------------------------
1075// The shared authenticated client
1076// ---------------------------------------------------------------------------
1077
1078/// What a single re-validation of the held credential concluded.
1079#[derive(Debug, Clone, Copy, PartialEq, Eq)]
1080pub enum Revalidation {
1081 /// GitHub still accepts the credential, so the `401` was about the request
1082 /// rather than about the token. One retry is warranted.
1083 Valid,
1084 /// GitHub rejects the credential. Terminal; only `auth login` clears it.
1085 Rejected,
1086 /// The probe itself could not be completed — GitHub was unreachable, or
1087 /// answered something neither `2xx` nor `401`. Nothing was learned, so the
1088 /// caller still gets its one retry.
1089 Unavailable,
1090}
1091
1092#[derive(Debug)]
1093struct LockoutState {
1094 until: Option<Timestamp>,
1095 backoff: Duration,
1096}
1097
1098/// The one client every `api.github.com` request in this crate goes through.
1099///
1100/// It exists to make four things structural rather than remembered:
1101///
1102/// 1. `X-GitHub-Api-Version`, `Accept`, `User-Agent`, and `Authorization` are
1103/// set on every request because they are set *here*.
1104/// 2. The `401` / `403` taxonomy of `03-control-flows.md` flow 4.3 is
1105/// implemented once. `c3` and `f1` both branch on the distinction, and two
1106/// implementations of it would eventually disagree.
1107/// 3. A `401` storm produces **one** credential re-validation, not one per
1108/// caller — see [`AuthenticatedClient::revalidate`].
1109/// 4. A lockout stops traffic. Once GitHub answers `403` after `401`s, this
1110/// client issues no further HTTP at all until the back-off elapses.
1111pub struct AuthenticatedClient {
1112 http: reqwest::Client,
1113 endpoints: Endpoints,
1114 /// Swappable, because a renewable credential replaces itself while this
1115 /// client is in use. A `Mutex` rather than a lock-free cell: it is read
1116 /// once per request and written once every eight hours, so contention is
1117 /// not the concern -- being obviously correct is.
1118 credential: std::sync::Mutex<UserAccessToken>,
1119 /// Where to re-read the credential when a `401` outlives renewal.
1120 ///
1121 /// `None` for a short-lived client, which is every one that is not the
1122 /// daemon's: a command that runs for a second cannot be outlived by a
1123 /// sign-in.
1124 source: Option<Arc<dyn CredentialSource>>,
1125 /// How a credential replaces itself, when it can.
1126 ///
1127 /// `None` for a client whose credential has no renewal half, which is every
1128 /// client until an App turns user-token expiration on, and for the paths
1129 /// that hold a token for one call and never outlive its eight hours.
1130 renewal: Option<Arc<dyn CredentialRenewal>>,
1131 clock: Arc<dyn Clock>,
1132
1133 /// Bumped once per completed re-validation. A caller that took a `401`
1134 /// samples this *before* queuing on the gate; if it changed while the caller
1135 /// waited, some other caller already did the work and this one must not
1136 /// repeat it. This is the whole single-flight mechanism.
1137 revalidation_generation: AtomicU64,
1138 revalidation_gate: tokio::sync::Mutex<()>,
1139 last_revalidation: std::sync::Mutex<Revalidation>,
1140 revalidations_performed: AtomicU64,
1141
1142 /// `401`s seen since the last successful caller response.
1143 ///
1144 /// **Nothing in production reads this.** It is incremented on every `401`
1145 /// and cleared by a successful caller response, and that is the whole of
1146 /// what it does. It used to be documented as the lockout's test — "a `403`
1147 /// while this is non-zero is a lockout" — and
1148 /// [`AuthenticatedClient::is_lockout_403`] stopped consulting it when that
1149 /// rule was replaced by position plus GitHub's own evidence:
1150 /// [`Attempt::Retry`] already implies this request's own `401` incremented
1151 /// it moments ago, so reading it added no signal, and it did add a race that
1152 /// failed open — a concurrent success clearing the count downgraded a real
1153 /// lockout to a permissions answer.
1154 ///
1155 /// The field is kept on purpose, and only the tests read it: they drive it
1156 /// to values that *would* change the answer if it were still consulted, and
1157 /// assert that the answer does not change. Deleting it would delete the
1158 /// ability to make that assertion, which is the only thing standing between
1159 /// the conjunct and its reintroduction. See
1160 /// `a_concurrent_success_cannot_downgrade_a_lockout_to_a_permissions_answer`.
1161 consecutive_unauthorized: AtomicU64,
1162 lockout: std::sync::Mutex<LockoutState>,
1163}
1164
1165impl fmt::Debug for AuthenticatedClient {
1166 fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
1167 // `try_lock`, not `is_locked_out()`. `std::sync::Mutex` is not
1168 // reentrant, and `latch_lockout` holds this one; the moment anybody adds
1169 // a `tracing` call inside that function — which is a natural thing to
1170 // want there — rendering the client would deadlock the whole agent. A
1171 // `Debug` impl must never be able to block, so it reports what it can
1172 // see and says so when it cannot.
1173 let locked_out = match self.lockout.try_lock() {
1174 Ok(state) => {
1175 if state.until.is_some_and(|until| self.clock.now() < until) {
1176 "yes"
1177 } else {
1178 "no"
1179 }
1180 }
1181 Err(_) => "unknown (the lockout state is being updated)",
1182 };
1183 f.debug_struct("AuthenticatedClient")
1184 .field("api_base", &self.endpoints.api_base.as_str())
1185 .field(
1186 "credential",
1187 &self
1188 .credential
1189 .try_lock()
1190 .map_or("[in use]", |_| "[redacted]"),
1191 )
1192 .field(
1193 "revalidations_performed",
1194 &self.revalidations_performed.load(Ordering::Relaxed),
1195 )
1196 .field("locked_out", &locked_out)
1197 .finish_non_exhaustive()
1198 }
1199}
1200
1201/// How a credential replaces itself.
1202///
1203/// # Why this is a port rather than a method
1204///
1205/// Renewal is two acts that must happen in one order: exchange the refresh
1206/// token with GitHub, then **persist the new pair before anything uses it**.
1207/// GitHub rotates on use -- the old pair dies the instant the new one is
1208/// issued -- so a response that is used but not stored leaves the host holding
1209/// a credential it will forget, and the one it forgot is already dead. There
1210/// is no retry: a spent refresh token answers `incorrect_client_credentials`,
1211/// a message about the client id and secret that is about neither.
1212///
1213/// The exchange belongs to `c2` and the store belongs to `d2`, and this crate
1214/// owns neither. So the ordering lives with whoever implements this, in one
1215/// place, rather than being a rule each caller has to remember.
1216#[async_trait::async_trait]
1217pub trait CredentialRenewal: fmt::Debug + Send + Sync {
1218 /// Exchange `refresh_token` for a fresh pair and persist it.
1219 ///
1220 /// # Errors
1221 /// Any failure; the caller treats every one the same way, by keeping the
1222 /// credential it has and letting the next `401` try again.
1223 async fn renew(&self, refresh_token: &SecretString) -> Result<UserAccessToken, String>;
1224}
1225
1226/// Where a client can go to find out that the stored credential changed under
1227/// it.
1228///
1229/// # Why a long-running client needs this
1230///
1231/// A daemon reads the store once, at startup, and holds the result for as long
1232/// as it runs. That was invisible while the only way to change the store was to
1233/// stop the daemon — but `auth login` does not stop anything, so a host whose
1234/// credential died before its daemon started stays dead through every sign-in
1235/// meant to fix it. The operator does the right thing, watches it not work, and
1236/// has nothing to tell them why.
1237///
1238/// Watched on 2026-08-29: a Windows daemon started at `04:08Z` holding an
1239/// already-expired token, a sign-in at `10:43Z` that wrote a good pair, and 180
1240/// `unauthorized` events an hour for 28 hours without a single minute's pause
1241/// across the sign-in. See `docs/spikes/token-expiry-and-renewal.md`.
1242///
1243/// # Why it is not a file watch
1244///
1245/// This is consulted on `401` and nowhere else, so a store that never changes
1246/// costs nothing and a daemon that is working never reads the disk. It also
1247/// covers the case a watch would miss on macOS, where the credential lives in a
1248/// keychain rather than at a path.
1249pub trait CredentialSource: fmt::Debug + Send + Sync {
1250 /// The credential the store holds *now*, or `None` if it cannot be read.
1251 ///
1252 /// Infallible by design: every failure — missing, unreadable, corrupt —
1253 /// means the same thing to the caller, which is that there is nothing new
1254 /// to try and the `401` stands.
1255 fn reload(&self) -> Option<UserAccessToken>;
1256}
1257
1258impl AuthenticatedClient {
1259 /// # Errors
1260 /// The HTTP client failing to build — a TLS backend that will not
1261 /// initialise, in practice.
1262 pub fn new(
1263 endpoints: Endpoints,
1264 credential: UserAccessToken,
1265 clock: Arc<dyn Clock>,
1266 ) -> Result<Self, GithubError> {
1267 let http = reqwest::Client::builder()
1268 .timeout(DEFAULT_REQUEST_TIMEOUT)
1269 .build()
1270 .map_err(transport)?;
1271 Ok(Self::with_http_client(http, endpoints, credential, clock))
1272 }
1273
1274 #[must_use]
1275 pub fn with_http_client(
1276 http: reqwest::Client,
1277 endpoints: Endpoints,
1278 credential: UserAccessToken,
1279 clock: Arc<dyn Clock>,
1280 ) -> Self {
1281 Self {
1282 http,
1283 endpoints,
1284 credential: std::sync::Mutex::new(credential),
1285 source: None,
1286 renewal: None,
1287 clock,
1288 revalidation_generation: AtomicU64::new(0),
1289 revalidation_gate: tokio::sync::Mutex::new(()),
1290 last_revalidation: std::sync::Mutex::new(Revalidation::Valid),
1291 revalidations_performed: AtomicU64::new(0),
1292 consecutive_unauthorized: AtomicU64::new(0),
1293 lockout: std::sync::Mutex::new(LockoutState {
1294 until: None,
1295 backoff: DEFAULT_LOCKOUT_BACKOFF,
1296 }),
1297 }
1298 }
1299
1300 #[must_use]
1301 pub fn endpoints(&self) -> &Endpoints {
1302 &self.endpoints
1303 }
1304
1305 /// How many credential re-validations this client has performed.
1306 ///
1307 /// Public because it is the observable the single-flight requirement is
1308 /// stated in terms of: "concurrent callers hitting `401` together produce
1309 /// **one** attempt, not N".
1310 #[must_use]
1311 pub fn revalidations_performed(&self) -> u64 {
1312 self.revalidations_performed.load(Ordering::SeqCst)
1313 }
1314
1315 /// `true` while a lockout back-off is still running, during which this
1316 /// client issues no HTTP at all.
1317 ///
1318 /// # Panics
1319 /// If a previous holder panicked while the lockout lock was held.
1320 #[must_use]
1321 pub fn is_locked_out(&self) -> bool {
1322 self.lockout_remaining().is_some()
1323 }
1324
1325 /// How much of the lockout back-off is left, or `None` when not locked out.
1326 ///
1327 /// # Panics
1328 /// If a previous holder panicked while the lockout lock was held.
1329 #[must_use]
1330 pub fn lockout_remaining(&self) -> Option<Duration> {
1331 let state = self.lockout.lock().expect("lockout lock poisoned");
1332 let until = state.until?;
1333 let now = self.clock.now();
1334 if now >= until {
1335 return None;
1336 }
1337 (until - now).to_std().ok()
1338 }
1339
1340 /// Clear a lockout early. `f1` does not need this — the back-off expires on
1341 /// its own against the clock — but a successful interactive `auth login`
1342 /// legitimately invalidates the whole lockout premise.
1343 ///
1344 /// # Panics
1345 /// If a previous holder panicked while the lockout lock was held.
1346 pub fn clear_lockout(&self) {
1347 self.lockout.lock().expect("lockout lock poisoned").until = None;
1348 self.consecutive_unauthorized.store(0, Ordering::SeqCst);
1349 }
1350
1351 /// Send one request, applying the authentication taxonomy.
1352 ///
1353 /// On `401` this performs a single-flight credential re-validation and then
1354 /// **one** retry — never more, and never a token renewal, because there is
1355 /// nothing to renew (see [`AuthenticatedClient::revalidate`]).
1356 ///
1357 /// # Errors
1358 /// Every variant of [`GithubError`].
1359 pub async fn send(&self, request: &ApiRequest) -> Result<ApiResponse, GithubError> {
1360 if let Some(remaining) = self.lockout_remaining() {
1361 // "backs off without further attempts": no socket is opened at all.
1362 tracing::debug!(
1363 method = request.method.as_str(),
1364 path = %request.path,
1365 remaining_secs = remaining.as_secs(),
1366 "suppressed a request: GitHub authentication lockout is still backing off"
1367 );
1368 return Err(GithubError::AuthenticationLockout {
1369 retry_after: remaining,
1370 });
1371 }
1372
1373 let first = self.send_raw(request).await?;
1374 match self.classify(request, &first, Attempt::First) {
1375 Classified::Ok => Ok(first),
1376 Classified::Unauthorized => self.revalidate_and_retry_once(request).await,
1377 Classified::Error(err) => Err(err),
1378 }
1379 }
1380
1381 /// Deserialize a `GET` in one step.
1382 ///
1383 /// # Errors
1384 /// Every variant of [`GithubError`].
1385 pub async fn get_json<T: DeserializeOwned>(&self, path: &str) -> Result<T, GithubError> {
1386 self.send(&ApiRequest::get(path)).await?.json()
1387 }
1388
1389 /// The access token to send, as a string, for exactly one request.
1390 ///
1391 /// Cloned out of the lock rather than borrowed through it: the value is
1392 /// about to go into a header, and holding the lock across the request would
1393 /// serialise every call in the process behind one mutex.
1394 fn bearer(&self) -> String {
1395 self.credential
1396 .lock()
1397 .unwrap_or_else(std::sync::PoisonError::into_inner)
1398 .secret()
1399 .expose_secret()
1400 .to_string()
1401 }
1402
1403 /// Attach a way for this client's credential to replace itself.
1404 #[must_use]
1405 pub fn with_renewal(mut self, renewal: Arc<dyn CredentialRenewal>) -> Self {
1406 self.renewal = Some(renewal);
1407 self
1408 }
1409
1410 /// Attach the store this client's credential came from, so a `401` it
1411 /// cannot renew its way out of can still notice a sign-in that already
1412 /// happened.
1413 #[must_use]
1414 pub fn with_credential_source(mut self, source: Arc<dyn CredentialSource>) -> Self {
1415 self.source = Some(source);
1416 self
1417 }
1418
1419 /// Replace the credential with whatever `produce` finds, once, however many
1420 /// callers asked at the same moment.
1421 ///
1422 /// Answers whether the credential in hand is now a *different* one, which
1423 /// is the caller's cue to retry. `false` means there was nothing new, and
1424 /// the `401` stands.
1425 ///
1426 /// # Why both ways in share this
1427 ///
1428 /// [`Self::renew_once`] and [`Self::reload_once`] ask one question —
1429 /// *can this `401` be retried with something else* — and differ only in
1430 /// where the something else comes from. Written separately they were two
1431 /// copies of the same `SeqCst` generation protocol, and the copies had
1432 /// already drifted: the equality check below existed in one of them and not
1433 /// the other, so a renewal that handed back an identical token reported a
1434 /// change that had not happened.
1435 ///
1436 /// # Why the comparison, and not just a swap
1437 ///
1438 /// A swap on every `401` would answer `true` forever and turn the one retry
1439 /// into an endless pair of requests against a credential that is genuinely
1440 /// dead. Only a different token is evidence that retrying might go
1441 /// differently.
1442 async fn swap_credential_once<F>(&self, produce: F, note: &'static str) -> bool
1443 where
1444 F: AsyncFnOnce(&Self) -> Option<UserAccessToken>,
1445 {
1446 let before = self.revalidation_generation.load(Ordering::SeqCst);
1447 let _gate = self.revalidation_gate.lock().await;
1448 if self.revalidation_generation.load(Ordering::SeqCst) != before {
1449 // Somebody swapped while this caller queued, so there is already
1450 // something new to retry with. Producing again would be wasted at
1451 // best and, for a renewal, would burn the pair they just minted.
1452 return true;
1453 }
1454
1455 let Some(fresh) = produce(self).await else {
1456 return false;
1457 };
1458 {
1459 let mut guard = self
1460 .credential
1461 .lock()
1462 .unwrap_or_else(std::sync::PoisonError::into_inner);
1463 if guard.secret().expose_secret() == fresh.secret().expose_secret() {
1464 return false;
1465 }
1466 *guard = fresh;
1467 }
1468 self.revalidation_generation.fetch_add(1, Ordering::SeqCst);
1469 tracing::info!("{note}");
1470 true
1471 }
1472
1473 /// Pick up a credential somebody else stored.
1474 ///
1475 /// `false` when there is no source, the store cannot be read, or what it
1476 /// holds is the token that just failed.
1477 async fn reload_once(&self) -> bool {
1478 let Some(source) = self.source.clone() else {
1479 return false;
1480 };
1481 self.swap_credential_once(
1482 async |_| source.reload(),
1483 "the stored credential changed and was picked up without a restart",
1484 )
1485 .await
1486 }
1487
1488 /// Spend the refresh half for a fresh pair.
1489 ///
1490 /// # Ordering
1491 ///
1492 /// The implementation of [`CredentialRenewal::renew`] persists before
1493 /// returning, so by the time the swap happens the new pair is already
1494 /// durable. A crash between the two loses nothing: the store holds the pair
1495 /// that works, and the next start reads it.
1496 async fn renew_once(&self) -> bool {
1497 let Some(renewal) = self.renewal.clone() else {
1498 return false;
1499 };
1500 self.swap_credential_once(
1501 async |client: &Self| {
1502 let refresh = {
1503 let guard = client
1504 .credential
1505 .lock()
1506 .unwrap_or_else(std::sync::PoisonError::into_inner);
1507 guard.renewal().map(|r| r.refresh_token().clone())
1508 }?;
1509 match renewal.renew(&refresh).await {
1510 Ok(fresh) => Some(fresh),
1511 Err(error) => {
1512 tracing::warn!(
1513 %error,
1514 "the user access token could not be renewed; an interactive \
1515 sign-in may be required"
1516 );
1517 None
1518 }
1519 }
1520 },
1521 "the user access token was renewed",
1522 )
1523 .await
1524 }
1525
1526 /// Proactively replace a renewable credential when either half is close
1527 /// to expiry.
1528 ///
1529 /// Ordinary request handling also renews after a `401`, but a daemon with
1530 /// no GitHub work can make no request for months. In that state the refresh
1531 /// token itself eventually expires and an interactive sign-in is the only
1532 /// recovery. A long-running owner calls this periodically so every
1533 /// successful rotation extends the credential even while it is otherwise
1534 /// idle.
1535 ///
1536 /// The stored credential is checked first for the same reason as the `401`
1537 /// path: another process may already have rotated the pair, and spending
1538 /// its predecessor after that would be a refresh-token replay.
1539 ///
1540 /// Returns `true` when a different credential was loaded or minted. A
1541 /// credential without a refresh half is left untouched. Missing expiry
1542 /// metadata is treated as due once so that the exchange can repair an old
1543 /// stored document.
1544 pub async fn renew_if_expiring_within(&self, window: Duration) -> bool {
1545 let due = {
1546 let credential = self
1547 .credential
1548 .lock()
1549 .unwrap_or_else(std::sync::PoisonError::into_inner);
1550 let Some(renewal) = credential.renewal() else {
1551 return false;
1552 };
1553 let Some(deadline) = self.clock.now().checked_add_signed(
1554 chrono::TimeDelta::from_std(window).unwrap_or(chrono::TimeDelta::MAX),
1555 ) else {
1556 return true;
1557 };
1558 renewal
1559 .access_expires_at
1560 .is_none_or(|expires| expires <= deadline)
1561 || renewal
1562 .refresh_expires_at
1563 .is_some_and(|expires| expires <= deadline)
1564 };
1565
1566 due && (self.reload_once().await || self.renew_once().await)
1567 }
1568
1569 /// Serialize, `POST`, and deserialize in one step.
1570 ///
1571 /// # Errors
1572 /// Every variant of [`GithubError`].
1573 pub async fn post_json<B: Serialize, T: DeserializeOwned>(
1574 &self,
1575 path: &str,
1576 body: &B,
1577 ) -> Result<T, GithubError> {
1578 self.send(&ApiRequest::post_json(path, body)?).await?.json()
1579 }
1580
1581 /// Re-validate the credential once, no matter how many callers ask at once.
1582 ///
1583 /// # This is not a token renewal — renewal is [`Self::renew_once`]
1584 ///
1585 /// `03-control-flows.md` flow 4.3 says a `401` "triggers one refresh under
1586 /// a single-flight mutex, then one retry", and that is implemented
1587 /// literally: one attempt shared by every concurrent caller, then one retry
1588 /// each.
1589 ///
1590 /// This method used to claim the word "refresh" could not be meant, because
1591 /// renewing needs a client secret and the App issues no renewal token. Both
1592 /// halves were false; see this module's header. Renewal exists, it runs
1593 /// first in [`Self::revalidate_and_retry_once`], and what is left here is
1594 /// the path for a credential that has no refresh half to spend — one stored
1595 /// before 0.1.11, or issued while the App had expiration switched off.
1596 ///
1597 /// So the single thing that happens under the mutex is a re-validation of
1598 /// the credential already held: one `GET /user/installations` with the same
1599 /// token, asking GitHub whether it still accepts it. A
1600 /// [`Revalidation::Rejected`] answer is terminal
1601 /// [`GithubError::AuthenticationFailed`] requiring an interactive
1602 /// `auth login`; [`Revalidation::Valid`] and [`Revalidation::Unavailable`]
1603 /// both spend the one retry.
1604 ///
1605 /// # Position, and what it no longer decides on its own
1606 ///
1607 /// This method is the caller's own probe: it is a **first** attempt by
1608 /// construction, whatever happened on some other request minutes ago, so it
1609 /// passes [`Attempt::First`] down.
1610 ///
1611 /// That used to settle the matter. This heading read "why this entry point
1612 /// may not latch a lockout", and the text said the probe it drives *cannot*
1613 /// latch — an accurate description of the position rule as it then stood,
1614 /// and a false statement about the product. A lockout that outlives one
1615 /// back-off continues on a **first** attempt by construction, because this
1616 /// client's retry never happened: the request never reached the wire.
1617 /// Refusing to latch there stopped the back-off entirely and hammered a
1618 /// credential GitHub had asked to be left alone.
1619 ///
1620 /// So position alone no longer decides.
1621 /// [`AuthenticatedClient::is_lockout_403`] reads GitHub's own evidence in
1622 /// the first position instead: a first attempt latches when, and only when,
1623 /// the response carries `retry-after` and no parseable GitHub message body.
1624 /// A permissions refusal names what is not accessible, so it still does not
1625 /// latch, which is what keeps [`GithubError::Forbidden`] reachable from
1626 /// here.
1627 ///
1628 /// `revalidate_and_retry_once` uses the private
1629 /// [`AuthenticatedClient::revalidate_after_unauthorized`] instead, which is
1630 /// in the retry position, where any `403` that is not a rate limit is the
1631 /// lockout regardless of what the body says.
1632 ///
1633 /// # This call can latch a lockout, and then it says so
1634 ///
1635 /// Because a first attempt can latch, this call can leave the whole client
1636 /// backed off for up to [`MAX_LOCKOUT_BACKOFF`]. It reports that as
1637 /// [`GithubError::AuthenticationLockout`] rather than answering
1638 /// `Ok(`[`Revalidation::Unavailable`]`)` and leaving the caller to discover
1639 /// it through a separate [`AuthenticatedClient::is_locked_out`] call. An
1640 /// `auth status` that printed "could not determine" while the client it had
1641 /// just silenced sat mute for fifteen minutes would be reporting the wrong
1642 /// event, and reporting it as the milder one.
1643 ///
1644 /// # Errors
1645 /// [`GithubError::AuthenticationLockout`] if this client is already backing
1646 /// off when the call arrives, **or** if this call's own probe latches one.
1647 ///
1648 /// A credential GitHub has rejected outright is *not* an error here: that
1649 /// comes back as `Ok(`[`Revalidation::Rejected`]`)`, and what to do about it
1650 /// — prompt for `auth login` — is the caller's decision, not this method's.
1651 ///
1652 /// # Panics
1653 /// If a previous holder panicked while the re-validation result lock was
1654 /// held.
1655 pub async fn revalidate(&self) -> Result<Revalidation, GithubError> {
1656 self.revalidate_from(Attempt::First).await
1657 }
1658
1659 /// The re-validation that `send` runs between a `401` and its one retry.
1660 ///
1661 /// Identical to [`AuthenticatedClient::revalidate`] except for position:
1662 /// this one *is* the retry path, so a `403` on its probe is the lockout and
1663 /// is latched.
1664 async fn revalidate_after_unauthorized(&self) -> Result<Revalidation, GithubError> {
1665 self.revalidate_from(Attempt::Retry).await
1666 }
1667
1668 async fn revalidate_from(&self, attempt: Attempt) -> Result<Revalidation, GithubError> {
1669 // "A lockout stops traffic. This client issues no further HTTP at all
1670 // until the back-off elapses" is a property of the client, not of
1671 // `send`, and the probe is HTTP like any other. `send` has already
1672 // returned by the time it calls in here, so this guard only bites a
1673 // caller that probes on its own — and one that did would otherwise be
1674 // the single exception to the rule, which is how a rule stops holding.
1675 if let Some(retry_after) = self.lockout_remaining() {
1676 return Err(GithubError::AuthenticationLockout { retry_after });
1677 }
1678
1679 // Sample before queuing. If this changes while we wait for the gate,
1680 // someone else's re-validation covers us and we must not repeat it.
1681 let sampled = self.revalidation_generation.load(Ordering::SeqCst);
1682 let _guard = self.revalidation_gate.lock().await;
1683 let outcome = if self.revalidation_generation.load(Ordering::SeqCst) != sampled {
1684 let shared = *self
1685 .last_revalidation
1686 .lock()
1687 .expect("re-validation lock poisoned");
1688 tracing::debug!(
1689 outcome = ?shared,
1690 "reused an in-flight credential re-validation instead of starting another"
1691 );
1692 shared
1693 } else {
1694 self.revalidations_performed.fetch_add(1, Ordering::SeqCst);
1695 let fresh = self.probe_credential(attempt).await;
1696 *self
1697 .last_revalidation
1698 .lock()
1699 .expect("re-validation lock poisoned") = fresh;
1700 self.revalidation_generation.fetch_add(1, Ordering::SeqCst);
1701 tracing::info!(
1702 outcome = ?fresh,
1703 "re-validated the stored credential; it carries no refresh half to renew"
1704 );
1705 fresh
1706 };
1707
1708 // The probe is HTTP, and since the continuation rule re-widened
1709 // `Attempt::First`, HTTP from *this* entry point can latch a lockout.
1710 // `probe_credential` reports a `403` as `Unavailable` whether or not it
1711 // latched one, and `Unavailable` on its own reads as "this taught us
1712 // nothing about the credential" — so without this check `revalidate`
1713 // answers `Ok(Unavailable)` having just silenced the entire client for
1714 // up to `MAX_LOCKOUT_BACKOFF`, and the only way for `f1` to find that
1715 // out is a separate `is_locked_out()` call it has no reason to make.
1716 //
1717 // `revalidate_and_retry_once` has always re-checked after its own probe.
1718 // Doing it here instead makes the two entry points agree: before, being
1719 // told about the lockout depended on *who latched it* — the guard above
1720 // reports one latched by someone else's traffic, this reports one
1721 // latched by the caller's own probe — and that distinction is invisible
1722 // from outside and actionable by nobody.
1723 //
1724 // This changes what is reported, not when a lockout latches:
1725 // `latch_lockout` is reached on exactly the paths it was before.
1726 //
1727 // It also covers the shared branch above, which the retry path's own
1728 // check never could. A second caller arriving while the first one's
1729 // probe is in flight takes the cached `Unavailable` without probing at
1730 // all, and is just as silenced by the lockout that probe latched.
1731 if let Some(retry_after) = self.lockout_remaining() {
1732 return Err(GithubError::AuthenticationLockout { retry_after });
1733 }
1734 Ok(outcome)
1735 }
1736
1737 /// One `GET /user/installations` with the credential already held, asking
1738 /// GitHub whether it still accepts it.
1739 ///
1740 /// `attempt` is the *caller's* position, not the probe's own. A probe driven
1741 /// by `send`'s `401` handling is part of that request's retry; a probe a
1742 /// caller asked for through [`AuthenticatedClient::revalidate`] is a first
1743 /// attempt.
1744 ///
1745 /// Either may latch a lockout, on different evidence. This used to say "only
1746 /// the former may", which was the position rule before the continuation rule
1747 /// re-widened `Attempt::First`: a retry `403` that is not a rate limit is
1748 /// the lockout outright, and a first-attempt `403` is the lockout when
1749 /// GitHub's own evidence says so — `retry-after` present, no parseable
1750 /// message. See [`AuthenticatedClient::is_lockout_403`], which both this and
1751 /// `classify` go through, so the rule is stated once instead of twice.
1752 async fn probe_credential(&self, attempt: Attempt) -> Revalidation {
1753 let probe = ApiRequest::get(REVALIDATION_PATH).query("per_page", 1);
1754 match self.send_raw(&probe).await {
1755 // Deliberately does **not** reset `consecutive_unauthorized`. The
1756 // probe is this client's own diagnostic, not the caller's traffic,
1757 // and a successful probe is exactly the state a lockout arrives in:
1758 // GitHub still accepts the credential, and answers the *next* real
1759 // request with `403`. Only a successful caller request clears the
1760 // count, in `classify`.
1761 //
1762 // The reason used to be given as "resetting here would erase the
1763 // evidence the `403` is classified against", and that stopped being
1764 // true when `is_lockout_403` stopped consulting the count. There is
1765 // no such evidence to erase now — nothing in production reads the
1766 // field. The line stays because the field's one remaining job is to
1767 // let the tests prove it is *not* consulted, and a probe that
1768 // quietly rewrote it would make those tests assert against a counter
1769 // value no caller path actually produces.
1770 Ok(response) if response.status.is_success() => Revalidation::Valid,
1771 Ok(response) if response.status == StatusCode::UNAUTHORIZED => {
1772 self.consecutive_unauthorized.fetch_add(1, Ordering::SeqCst);
1773 Revalidation::Rejected
1774 }
1775 Ok(response) if response.status == StatusCode::FORBIDDEN => {
1776 // A `403` on a probe that *is* this request's retry is the
1777 // lockout outright. A `403` on a probe a caller asked for
1778 // directly is the lockout only when the response itself says so
1779 // — `retry-after` present, no parseable message — because that
1780 // is what a lockout continuing past one back-off looks like, and
1781 // it necessarily arrives in the first position.
1782 //
1783 // The comment that used to sit here claimed "the probe only ever
1784 // runs after a `401`, so it is always in the retry position",
1785 // and publishing `revalidate` is what made that untrue. Its
1786 // replacement then said a direct probe "is not" the lockout,
1787 // which the continuation rule in turn made untrue. Both were
1788 // position asserted as a conclusion; the position now arrives as
1789 // an argument and the conclusion is drawn in one place, by
1790 // `is_lockout_403`.
1791 if self.is_lockout_403(&response, attempt) {
1792 self.latch_lockout(&response.headers);
1793 }
1794 Revalidation::Unavailable
1795 }
1796 Ok(_) | Err(_) => Revalidation::Unavailable,
1797 }
1798 }
1799
1800 async fn revalidate_and_retry_once(
1801 &self,
1802 request: &ApiRequest,
1803 ) -> Result<ApiResponse, GithubError> {
1804 // ------------------------------------------------------------------
1805 // RENEWAL COMES FIRST, AND IT DID NOT USED TO EXIST.
1806 // ------------------------------------------------------------------
1807 // This method's documentation once said a `401` could only ever be
1808 // re-validated, never renewed, because renewing needs a confidential
1809 // client credential and a published binary cannot carry one. The first
1810 // half of that is wrong: GitHub requires one *"unless the user access
1811 // token was generated using the device flow"*, and this product's
1812 // always are. Verified against live GitHub before this was written.
1813 //
1814 // So a credential that carries a refresh token replaces itself here,
1815 // silently, and the caller's request is retried with the new one. That
1816 // is what lets an eight-hour token serve a daemon that runs for months
1817 // -- and what lets two machines hold their own credentials at once,
1818 // because each renews its own pair instead of re-authorising and
1819 // revoking the other's.
1820 //
1821 // A credential with no refresh token falls through to exactly the
1822 // behaviour that was here before.
1823 //
1824 // ------------------------------------------------------------------
1825 // THEN THE STORE, FOR THE 401 RENEWAL CANNOT ANSWER.
1826 // ------------------------------------------------------------------
1827 // Renewal covers a token that expired under a daemon holding a
1828 // *renewable* pair. It cannot cover a daemon that started holding a
1829 // dead bare token, because there is no refresh half to spend -- and
1830 // that daemon will not recover on its own no matter how many times an
1831 // operator runs `auth login`, because it never looks at the store
1832 // again. Consulting it here is what makes the obvious remedy work.
1833 //
1834 // First, not second: if multiple processes (e.g., daemon and TUI) share the
1835 // same token, they will both get 401 at the same time. The first one to renew
1836 // will write the new token to disk. If the second process tries to renew with
1837 // its in-memory refresh token *before* reloading, GitHub will detect a refresh
1838 // token replay and instantly revoke the entire token chain. So we must always
1839 // check disk for a newer token first.
1840 //
1841 // To further prevent a race condition if two processes hit 401 at the exact
1842 // same millisecond, we introduce a pseudo-random jitter based on the OS
1843 // process ID. This ensures one process wakes up first, finishes the renewal,
1844 // and writes to disk, so the second process sees the new file during its
1845 // `reload_once()`. All threads in the *same* process compute the exact same
1846 // jitter, preserving their ability to coalesce behind `revalidation_gate`.
1847 let jitter = (std::process::id() % 1500) as u64 + 50;
1848 tokio::time::sleep(std::time::Duration::from_millis(jitter)).await;
1849
1850 if self.reload_once().await || self.renew_once().await {
1851 let second = self.send_raw(request).await?;
1852 return match self.classify(request, &second, Attempt::Retry) {
1853 Classified::Ok => Ok(second),
1854 // The renewed credential was rejected too. Nothing here can
1855 // help: this is a sign-in, not a token, that has gone.
1856 Classified::Unauthorized => Err(GithubError::AuthenticationFailed),
1857 Classified::Error(err) => Err(err),
1858 };
1859 }
1860 match self.revalidate_after_unauthorized().await? {
1861 Revalidation::Rejected => {
1862 tracing::warn!(
1863 method = request.method.as_str(),
1864 path = %request.path,
1865 "GitHub rejected the stored credential; re-authentication is required"
1866 );
1867 Err(GithubError::AuthenticationFailed)
1868 }
1869 Revalidation::Valid | Revalidation::Unavailable => {
1870 // `revalidate_from` now converts a lockout that its own probe
1871 // latched, so this no longer catches that case. It stays for the
1872 // one it still catches: a *concurrent* request latching between
1873 // that check and this one. Sending the retry into a live lockout
1874 // is the thing the back-off exists to prevent, and this is the
1875 // last point at which it can be declined.
1876 if let Some(remaining) = self.lockout_remaining() {
1877 return Err(GithubError::AuthenticationLockout {
1878 retry_after: remaining,
1879 });
1880 }
1881 let second = self.send_raw(request).await?;
1882 match self.classify(request, &second, Attempt::Retry) {
1883 Classified::Ok => Ok(second),
1884 // The one retry is spent. A second `401` is terminal.
1885 Classified::Unauthorized => Err(GithubError::AuthenticationFailed),
1886 Classified::Error(err) => Err(err),
1887 }
1888 }
1889 }
1890 }
1891
1892 fn classify(
1893 &self,
1894 request: &ApiRequest,
1895 response: &ApiResponse,
1896 attempt: Attempt,
1897 ) -> Classified {
1898 let status = response.status;
1899 if status.is_success() {
1900 self.consecutive_unauthorized.store(0, Ordering::SeqCst);
1901 return Classified::Ok;
1902 }
1903 if status == StatusCode::UNAUTHORIZED {
1904 self.consecutive_unauthorized.fetch_add(1, Ordering::SeqCst);
1905 return Classified::Unauthorized;
1906 }
1907 if status == StatusCode::FORBIDDEN && self.is_lockout_403(response, attempt) {
1908 let backoff = self.latch_lockout(&response.headers);
1909 tracing::warn!(
1910 method = request.method.as_str(),
1911 path = %request.path,
1912 backoff_secs = backoff.as_secs(),
1913 "GitHub answered 403 after 401s: temporary authentication lockout, backing off"
1914 );
1915 return Classified::Error(GithubError::AuthenticationLockout {
1916 retry_after: backoff,
1917 });
1918 }
1919 let headers = Box::new(response.headers.clone());
1920 let message = error_message(&response.body);
1921 if status == StatusCode::FORBIDDEN {
1922 return Classified::Error(GithubError::Forbidden {
1923 method: request.method.as_str().to_string(),
1924 path: request.path.clone(),
1925 message,
1926 headers,
1927 });
1928 }
1929 Classified::Error(GithubError::Status {
1930 status: status.as_u16(),
1931 method: request.method.as_str().to_string(),
1932 path: request.path.clone(),
1933 message,
1934 headers,
1935 })
1936 }
1937
1938 /// Whether a `403` is GitHub's temporary *authentication* lockout, as
1939 /// opposed to a permissions answer or a rate limit.
1940 ///
1941 /// # It must not be a rate limit
1942 ///
1943 /// `classify` used to reach the `403` branch before anything looked at the
1944 /// rate-limit headers, so a primary rate limit arriving during a `401` storm
1945 /// was reported as `AuthenticationLockout` — telling the operator "the
1946 /// credential itself is not the problem" about a response that never
1947 /// mentioned the credential. Recognising GitHub's own rate-limit evidence is
1948 /// not rate-limit *policy*; it is declining to make an assertion the
1949 /// evidence contradicts. What to do about the rate limit stays `c3`'s, which
1950 /// is why this only changes which variant carries the headers onward.
1951 ///
1952 /// # Then one of two positions, and the second one is a fix for the first
1953 ///
1954 /// **The retry.** `consecutive_unauthorized` counts `401`s since the last
1955 /// successful caller response and — correctly — does not decay: a request
1956 /// that ends in `404`, `422` or `500` leaves it set. In the agent's
1957 /// long-lived reconciliation loop that meant a single `401` from minutes ago
1958 /// converted the *next* genuine permissions `403` into a fake lockout: sixty
1959 /// seconds of silence plus an operator message insisting the credential is
1960 /// fine, when in truth `generate-jitconfig` was missing
1961 /// `Administration: write`. The lockout's signature is narrower than "a
1962 /// `403` while the count is set" — it is a `403` on the one retry this
1963 /// client itself issued after this request's own `401`.
1964 ///
1965 /// The count is deliberately *not* consulted. [`Attempt::Retry`] already
1966 /// means this request's own `401` incremented it moments ago, so reading it
1967 /// adds no signal — and does add a race that fails open: any concurrent
1968 /// request succeeding between the `401` and the retry `store(0)`s the
1969 /// counter, and a real lockout is then reported as a plain permissions
1970 /// refusal. A conjunct that can only ever weaken a safety check is worse
1971 /// than no conjunct.
1972 ///
1973 /// **The continuation.** Narrowing to the retry position opened a hole at
1974 /// the far end of the same back-off. When the back-off elapses and GitHub is
1975 /// still locking the credential out, the next request is a *first* attempt
1976 /// by construction — this client's retry never happened, because the request
1977 /// never reached the wire. The position rule then declined to call it a
1978 /// lockout, `classify` fell through to [`GithubError::Forbidden`] — whose
1979 /// documented reading is "the App installation does not grant it" — and the
1980 /// client **stopped backing off entirely**, hammering a credential GitHub
1981 /// had asked it to leave alone. That is the exact inverse of the
1982 /// Definition of Done's "backs off without retrying", and it failed for
1983 /// every lockout outliving one back-off.
1984 ///
1985 /// No counter is needed for that case either, because the response says so
1986 /// itself. GitHub's lockout carries `retry-after` and no parseable message;
1987 /// a permissions refusal carries a message naming what is not accessible and
1988 /// no `retry-after`. Requiring **both** halves of that signature is what
1989 /// keeps this from degenerating into "every `403` is a lockout": a
1990 /// permissions answer has a message, so it never matches, and a secondary
1991 /// rate limit has both a message and `retry-after`, so `is_rate_limited`
1992 /// takes it first.
1993 ///
1994 /// "No parseable message" is deliberately wider than "an empty body", which
1995 /// is how this used to be stated. See [`is_lockout_continuation`] for what
1996 /// else falls into it — a proxy's HTML error page most notably — and for why
1997 /// the resulting false positives are accepted rather than tightened away.
1998 ///
1999 /// This also settles a standing worry about [`MAX_LOCKOUT_BACKOFF`]. With
2000 /// the continuation recognised, the ceiling no longer decides whether the
2001 /// product ever gives up — it only decides how often it re-asks. A lockout
2002 /// longer than the ceiling now re-latches instead of being reported as a
2003 /// permissions failure, so the value is a polling interval rather than a
2004 /// deadline.
2005 fn is_lockout_403(&self, response: &ApiResponse, attempt: Attempt) -> bool {
2006 if is_rate_limited(response) {
2007 return false;
2008 }
2009 match attempt {
2010 Attempt::Retry => true,
2011 Attempt::First => is_lockout_continuation(response),
2012 }
2013 }
2014
2015 fn latch_lockout(&self, headers: &HeaderMap) -> Duration {
2016 // Clamp before latching. An unclamped `Retry-After` is a remote party
2017 // deciding how long this product stays down.
2018 let requested = retry_after(headers).unwrap_or(DEFAULT_LOCKOUT_BACKOFF);
2019 let clamped = requested.min(MAX_LOCKOUT_BACKOFF);
2020
2021 // A span too large for `chrono` must fall back to the default, never to
2022 // `None`: the old code's `.ok()` turned an absurd `Retry-After` into "no
2023 // lockout at all", which fails *open* — the exact inverse of what a
2024 // back-off is for, and reachable by a header alone. The clamp above
2025 // already makes this branch unreachable; it stays because the invariant
2026 // it protects ("latching always latches") is worth more than the line.
2027 let delta = chrono::TimeDelta::from_std(clamped).unwrap_or_else(|_| {
2028 chrono::TimeDelta::from_std(DEFAULT_LOCKOUT_BACKOFF)
2029 .expect("sixty seconds is a representable span")
2030 });
2031 // No third clamp. `clamped` is already `<= MAX_LOCKOUT_BACKOFF`, and
2032 // `TimeDelta` round-trips it exactly, so re-clamping here was dead twice
2033 // over — it could only ever re-apply a bound already applied, and the
2034 // fallback it guarded is `DEFAULT_LOCKOUT_BACKOFF`, which is smaller
2035 // than the ceiling by construction.
2036 let backoff = delta.to_std().unwrap_or(DEFAULT_LOCKOUT_BACKOFF);
2037
2038 let mut state = self.lockout.lock().expect("lockout lock poisoned");
2039 state.backoff = backoff;
2040 state.until = Some(self.clock.now() + delta);
2041 backoff
2042 }
2043
2044 /// One HTTP round trip with the standard headers applied and no
2045 /// interpretation of the result.
2046 async fn send_raw(&self, request: &ApiRequest) -> Result<ApiResponse, GithubError> {
2047 let url = self.resolve(&request.path)?;
2048 let mut builder = self
2049 .http
2050 .request(request.method.clone(), url)
2051 .header(reqwest::header::ACCEPT, GITHUB_ACCEPT)
2052 .header(reqwest::header::USER_AGENT, USER_AGENT)
2053 .header("X-GitHub-Api-Version", GITHUB_API_VERSION)
2054 // The only place the token is ever written onto the wire. It is
2055 // never logged, and `reqwest` does not render headers in its errors.
2056 .header(
2057 reqwest::header::AUTHORIZATION,
2058 format!("Bearer {}", self.bearer()),
2059 );
2060 if !request.query.is_empty() {
2061 builder = builder.query(&request.query);
2062 }
2063 if let Some(body) = &request.body {
2064 builder = builder.json(body);
2065 }
2066
2067 let response = builder.send().await.map_err(transport)?;
2068 let status = response.status();
2069 let headers = response.headers().clone();
2070 let body = response.bytes().await.map_err(transport)?.to_vec();
2071
2072 tracing::debug!(
2073 method = request.method.as_str(),
2074 path = %request.path,
2075 status = status.as_u16(),
2076 body_bytes = body.len(),
2077 "github api request"
2078 );
2079
2080 Ok(ApiResponse {
2081 status,
2082 headers,
2083 body,
2084 })
2085 }
2086
2087 fn resolve(&self, path: &str) -> Result<Url, GithubError> {
2088 if path.starts_with("http://") || path.starts_with("https://") {
2089 return Url::parse(path).map_err(|_| GithubError::Malformed {
2090 what: "an absolute request URL",
2091 value: path.to_string(),
2092 });
2093 }
2094 self.endpoints
2095 .api_base
2096 .join(path.trim_start_matches('/'))
2097 .map_err(|_| GithubError::Malformed {
2098 what: "a request path",
2099 value: path.to_string(),
2100 })
2101 }
2102}
2103
2104enum Classified {
2105 Ok,
2106 Unauthorized,
2107 Error(GithubError),
2108}
2109
2110/// Which of a request's at-most-two attempts produced a response.
2111///
2112/// The authentication lockout is defined by *position*, not just by status: it
2113/// is what GitHub answers the retry that follows a `401`. Passing this in makes
2114/// that explicit at both call sites instead of inferring it from a counter that
2115/// outlives the request.
2116#[derive(Debug, Clone, Copy, PartialEq, Eq)]
2117enum Attempt {
2118 First,
2119 Retry,
2120}
2121
2122/// Whether GitHub attributed a failing response to its own rate limit.
2123///
2124/// Reading the evidence, and nothing else — see [`GithubError`]'s note on why
2125/// the headers travel with the error. `c3` decides what to do about it.
2126fn is_rate_limited(response: &ApiResponse) -> bool {
2127 if response.status == StatusCode::TOO_MANY_REQUESTS {
2128 return true;
2129 }
2130 // The primary rate limit's documented signature.
2131 if response
2132 .header("x-ratelimit-remaining")
2133 .is_some_and(|v| v.trim() == "0")
2134 {
2135 return true;
2136 }
2137 // A secondary rate limit sends `retry-after` — but so does the
2138 // authentication lockout, so that header alone cannot tell them apart.
2139 // GitHub's own message ("You have exceeded a secondary rate limit") can.
2140 error_message(&response.body).is_some_and(|m| m.to_ascii_lowercase().contains("rate limit"))
2141}
2142
2143/// Whether a `403` on a *first* attempt is GitHub continuing an authentication
2144/// lockout that outlived this client's back-off.
2145///
2146/// The two halves are both required, and both are GitHub's own evidence rather
2147/// than this client's memory:
2148///
2149/// * **`retry-after` is present.** GitHub sends it when it wants to be left
2150/// alone. A permissions refusal never does — there is nothing to wait for.
2151///
2152/// The header's *presence* is what is tested, not whether it parses, and that
2153/// is a fix rather than laziness. [`retry_after`] reads **integer seconds
2154/// only**, while RFC 9110 §10.2.3 also permits an HTTP-date. Gating detection
2155/// on `retry_after(..).is_some()` meant a date-form header was not recognised
2156/// as a continuation at all, and the bug this function exists to fix came
2157/// straight back for that shape — silently, since the response still looks
2158/// like an ordinary [`GithubError::Forbidden`] on the way out.
2159///
2160/// How long to wait stays a separate question, still answered by the integer
2161/// parse: [`AuthenticatedClient::latch_lockout`] already falls back to
2162/// [`DEFAULT_LOCKOUT_BACKOFF`] for a header it cannot read, so a date-form
2163/// header now latches sixty seconds instead of latching nothing. GitHub sends
2164/// integer seconds in practice; the point is not to depend on that.
2165/// * **The body carries no parseable GitHub message.** A permissions refusal
2166/// always names what is not accessible ("Resource not accessible by
2167/// integration"); the lockout's does not. This is the half that stops the rule
2168/// from swallowing [`GithubError::Forbidden`] entirely.
2169///
2170/// "No parseable GitHub message" is wider than "the body is empty", which is
2171/// how this used to be written, and the difference is worth stating because it
2172/// is what the code actually tests. [`error_message`] returns `None` for *any*
2173/// body that is not JSON carrying a non-empty `message`: an HTML error page
2174/// from a proxy or a CDN, a JSON body carrying only `documentation_url`, plain
2175/// text, a truncated response. Proxies routinely send `Retry-After` too, so a
2176/// `403` that never came from GitHub at all can read as an authentication
2177/// lockout here.
2178///
2179/// That is accepted rather than missed. The cost is bounded — the client
2180/// waits, clamped by [`MAX_LOCKOUT_BACKOFF`], then re-asks — and the direction
2181/// is the safe one: treating a strange `403` as "wait" costs latency, while
2182/// treating a real lockout as a permissions answer costs the back-off
2183/// entirely and tells the operator to fix a grant that is not missing.
2184/// Tightening it would mean asserting the body *is* GitHub's, which is
2185/// precisely the assertion an intercepting proxy makes false.
2186///
2187/// Callers reach this through [`AuthenticatedClient::is_lockout_403`], which
2188/// rules out a rate limit first — a secondary rate limit carries `retry-after`
2189/// *and* a message, so it fails this test on the second half anyway, but the
2190/// ordering makes the precedence explicit rather than incidental.
2191fn is_lockout_continuation(response: &ApiResponse) -> bool {
2192 response.headers.contains_key("retry-after") && error_message(&response.body).is_none()
2193}
2194
2195// ---------------------------------------------------------------------------
2196// Installation discovery
2197// ---------------------------------------------------------------------------
2198
2199/// Whether an installation can reach every repository on its account, or only
2200/// the ones the user picked.
2201#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
2202#[serde(rename_all = "snake_case")]
2203pub enum RepositorySelection {
2204 /// Every repository on the account, including ones created later.
2205 All,
2206 /// Only the repositories the user chose at install time.
2207 Selected,
2208}
2209
2210impl RepositorySelection {
2211 /// `07-security.md`: "`auth status` shows which repositories the token can
2212 /// reach, so an over-broad installation is visible rather than assumed."
2213 /// This is the flag that makes it visible.
2214 #[must_use]
2215 pub fn is_over_broad(self) -> bool {
2216 matches!(self, Self::All)
2217 }
2218}
2219
2220/// Whose account an installation sits on.
2221#[derive(Debug, Clone, PartialEq, Eq)]
2222pub enum InstallationAccount {
2223 User(String),
2224 Organization(Org),
2225 /// An enterprise account.
2226 ///
2227 /// It is its own variant rather than a [`InstallationAccount::User`]
2228 /// because it is not one, and `auth status` says out loud whose account
2229 /// each installation sits on. Everything GitHub reports without
2230 /// `type: "Organization"` used to fall into `User`, so an enterprise was
2231 /// labelled a user — a wrong statement about the operator's own account, on
2232 /// the one screen that exists to tell them what their credential reaches.
2233 ///
2234 /// It contributes nothing to [`ReachableTargets::organizations`], and that
2235 /// is correct rather than a second bug: an enterprise is not an
2236 /// organization, and `GET /orgs/{org}/actions/runners` does not accept one.
2237 /// The distinction is only visible now because the label is.
2238 Enterprise(String),
2239}
2240
2241impl InstallationAccount {
2242 #[must_use]
2243 pub fn login(&self) -> &str {
2244 match self {
2245 Self::User(login) | Self::Enterprise(login) => login,
2246 Self::Organization(org) => org.as_str(),
2247 }
2248 }
2249
2250 /// The organization, when the account is one. An organization account is a
2251 /// reachable *target* in its own right (D18): a policy may scale for the
2252 /// whole organization.
2253 #[must_use]
2254 pub fn organization(&self) -> Option<&Org> {
2255 match self {
2256 Self::Organization(org) => Some(org),
2257 Self::User(_) | Self::Enterprise(_) => None,
2258 }
2259 }
2260
2261 /// What to call this account in `auth status`. `f1` renders it; nothing in
2262 /// this crate branches on it.
2263 #[must_use]
2264 pub fn kind(&self) -> &'static str {
2265 match self {
2266 Self::User(_) => "user",
2267 Self::Organization(_) => "organization",
2268 Self::Enterprise(_) => "enterprise",
2269 }
2270 }
2271}
2272
2273impl fmt::Display for InstallationAccount {
2274 fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
2275 f.write_str(self.login())
2276 }
2277}
2278
2279/// One installation of the published App, and what it can actually reach.
2280#[derive(Debug, Clone, PartialEq, Eq)]
2281pub struct Installation {
2282 pub id: u64,
2283 pub account: InstallationAccount,
2284 pub repository_selection: RepositorySelection,
2285 pub repositories: Vec<OwnerRepo>,
2286 /// The permissions GitHub reports for this installation, as
2287 /// `name -> level`. Surfaced verbatim so `auth status` can show a grant the
2288 /// user did not expect rather than assert the published set was applied.
2289 pub permissions: Vec<(String, String)>,
2290}
2291
2292impl Installation {
2293 #[must_use]
2294 pub fn is_over_broad(&self) -> bool {
2295 self.repository_selection.is_over_broad()
2296 }
2297}
2298
2299/// Everything the stored credential can reach.
2300#[derive(Debug, Clone, PartialEq, Eq)]
2301pub struct ReachableTargets {
2302 installations: Vec<Installation>,
2303 skipped: usize,
2304}
2305
2306impl ReachableTargets {
2307 #[must_use]
2308 pub fn installations(&self) -> &[Installation] {
2309 &self.installations
2310 }
2311
2312 /// How many installations GitHub reported that this client could not
2313 /// describe, and therefore left out of everything above.
2314 ///
2315 /// Non-zero means this report is **incomplete**, not merely small: whatever
2316 /// those installations reach is absent from
2317 /// [`ReachableTargets::repositories`] and
2318 /// [`ReachableTargets::organizations`]. `auth status` should say so, because
2319 /// the alternative is an operator reading a short list as a complete one.
2320 #[must_use]
2321 pub fn skipped(&self) -> usize {
2322 self.skipped
2323 }
2324
2325 /// Every repository the credential can reach, sorted and de-duplicated.
2326 #[must_use]
2327 pub fn repositories(&self) -> Vec<OwnerRepo> {
2328 let mut all: Vec<OwnerRepo> = self
2329 .installations
2330 .iter()
2331 .flat_map(|i| i.repositories.iter().cloned())
2332 .collect();
2333 all.sort();
2334 all.dedup();
2335 all
2336 }
2337
2338 /// Every organization the App is installed on, sorted and de-duplicated.
2339 #[must_use]
2340 pub fn organizations(&self) -> Vec<Org> {
2341 let mut all: Vec<Org> = self
2342 .installations
2343 .iter()
2344 .filter_map(|i| i.account.organization().cloned())
2345 .collect();
2346 all.sort();
2347 all.dedup();
2348 all
2349 }
2350
2351 /// The installations that hold `repository_selection: all`.
2352 #[must_use]
2353 pub fn over_broad(&self) -> Vec<&Installation> {
2354 self.installations
2355 .iter()
2356 .filter(|i| i.is_over_broad())
2357 .collect()
2358 }
2359
2360 #[must_use]
2361 pub fn is_empty(&self) -> bool {
2362 self.repositories().is_empty() && self.organizations().is_empty()
2363 }
2364}
2365
2366/// What `auth status` and `auth login` show after a successful sign-in.
2367#[derive(Debug, Clone, PartialEq, Eq)]
2368pub enum InstallationDiscovery {
2369 /// The credential is valid, GitHub reported nothing this client could not
2370 /// describe, and still nothing is reachable: the App is installed nowhere,
2371 /// or on nothing. `03-control-flows.md` flow 1.1 requires the installation
2372 /// URL here, and the URL is the remediation.
2373 NotInstalled { install_url: Url },
2374 /// Nothing is reachable, but at least one installation was **skipped**, so
2375 /// this client cannot tell "not installed" from "installed on something it
2376 /// could not describe".
2377 ///
2378 /// # Why this variant exists at all
2379 ///
2380 /// Skipping an unnameable installation is the right trade — one odd
2381 /// installation must not take down `auth status` for every other one — but
2382 /// it was made silently, and the silence flipped a verdict. An account this
2383 /// client cannot name, on the *only* installation the credential has, used
2384 /// to collapse to [`InstallationDiscovery::NotInstalled`], and `auth status`
2385 /// then handed an already-installed operator the "install the App" URL. That
2386 /// is a wrong remediation on the only authentication path there is,
2387 /// contradicted by nothing louder than a `warn!` in a log the operator is
2388 /// not reading.
2389 ///
2390 /// So the skip stays and the verdict does not flip. There is deliberately no
2391 /// `install_url` here: the whole point is that this client does not know
2392 /// whether installing is the remedy, and offering the URL anyway would put
2393 /// the wrong answer back one field over. `f1` says "1 installation could not
2394 /// be described" and stops there, which is true.
2395 Indeterminate { skipped: usize },
2396 /// The credential reaches at least one repository or organization. It may
2397 /// still be an incomplete picture — see [`ReachableTargets::skipped`].
2398 Installed(ReachableTargets),
2399}
2400
2401impl InstallationDiscovery {
2402 #[must_use]
2403 pub fn targets(&self) -> Option<&ReachableTargets> {
2404 match self {
2405 Self::Installed(t) => Some(t),
2406 Self::NotInstalled { .. } | Self::Indeterminate { .. } => None,
2407 }
2408 }
2409
2410 /// The installation URL, and *only* when installing is actually the
2411 /// remediation. See [`InstallationDiscovery::Indeterminate`].
2412 #[must_use]
2413 pub fn install_url(&self) -> Option<&Url> {
2414 match self {
2415 Self::NotInstalled { install_url } => Some(install_url),
2416 Self::Installed(_) | Self::Indeterminate { .. } => None,
2417 }
2418 }
2419
2420 /// How many installations GitHub reported that this client could not
2421 /// describe, whichever verdict was reached. One call for `f1`, so that
2422 /// "this report is incomplete" does not depend on which variant it landed
2423 /// in.
2424 #[must_use]
2425 pub fn skipped(&self) -> usize {
2426 match self {
2427 Self::NotInstalled { .. } => 0,
2428 Self::Indeterminate { skipped } => *skipped,
2429 Self::Installed(targets) => targets.skipped(),
2430 }
2431 }
2432}
2433
2434#[derive(Debug, Deserialize)]
2435struct InstallationsPage {
2436 /// GitHub reports the size of the whole collection on every page. Decoding
2437 /// it costs nothing and turns silent under-collection into a visible
2438 /// warning — see [`under_collected`].
2439 #[serde(default)]
2440 total_count: Option<u64>,
2441 #[serde(default)]
2442 installations: Vec<RawInstallation>,
2443}
2444
2445#[derive(Debug, Deserialize)]
2446struct RawInstallation {
2447 id: u64,
2448 /// **Nullable.** GitHub's published `installation` schema types `account` as
2449 /// nullable, so a required field here would fail the *whole* decode — and
2450 /// with it all of `discover_installations`, which is all of `auth status` —
2451 /// over one installation whose account this client did not need to name.
2452 #[serde(default)]
2453 account: Option<RawAccount>,
2454 #[serde(default)]
2455 repository_selection: Option<String>,
2456 #[serde(default)]
2457 permissions: std::collections::BTreeMap<String, String>,
2458}
2459
2460/// An installation's account, which is *not* always a simple user.
2461///
2462/// GitHub's schema makes `account` either a simple-user or an enterprise, and an
2463/// enterprise carries `slug` and `name` where a user carries `login`. Requiring
2464/// `login` therefore made an enterprise installation a hard decode failure of
2465/// the entire response. All three are optional here and
2466/// [`RawAccount::display_login`] takes the first usable one.
2467#[derive(Debug, Deserialize)]
2468struct RawAccount {
2469 #[serde(default)]
2470 login: Option<String>,
2471 /// An enterprise account's stable identifier.
2472 #[serde(default)]
2473 slug: Option<String>,
2474 /// An enterprise account's display name, the last resort.
2475 #[serde(default)]
2476 name: Option<String>,
2477 #[serde(rename = "type", default)]
2478 account_type: Option<String>,
2479}
2480
2481impl RawAccount {
2482 fn display_login(&self) -> Option<&str> {
2483 [
2484 self.login.as_deref(),
2485 self.slug.as_deref(),
2486 self.name.as_deref(),
2487 ]
2488 .into_iter()
2489 .flatten()
2490 .find(|value| !value.is_empty())
2491 }
2492
2493 /// An account with no `login` that still names itself is an enterprise:
2494 /// `slug`/`name` is the enterprise shape, and every simple-user and
2495 /// organization account carries `login`.
2496 fn is_enterprise_shaped(&self) -> bool {
2497 self.login.as_deref().is_none_or(str::is_empty)
2498 && (self.slug.as_deref().is_some_and(|s| !s.is_empty())
2499 || self.name.as_deref().is_some_and(|s| !s.is_empty()))
2500 }
2501}
2502
2503#[derive(Debug, Deserialize)]
2504struct RepositoriesPage {
2505 #[serde(default)]
2506 total_count: Option<u64>,
2507 #[serde(default)]
2508 repositories: Vec<RawRepository>,
2509}
2510
2511#[derive(Debug, Deserialize)]
2512struct RawRepository {
2513 full_name: String,
2514}
2515
2516/// How many items a paginated collection said it had, when that is more than
2517/// arrived.
2518///
2519/// This is the cheapest possible check and it is worth more than it looks. The
2520/// `Link`-header parser used to lose the relation whenever a page URL contained
2521/// a comma, which stopped pagination at page 1 — and *nothing* noticed, because
2522/// a short answer and a complete answer are the same shape. Cross-checking the
2523/// count GitHub itself reported turns that class of bug from a wrong answer into
2524/// a logged warning. A collection larger than `total_count` is not reported:
2525/// GitHub can legitimately grow a collection between pages.
2526fn under_collected(collected: usize, total_count: Option<u64>) -> Option<u64> {
2527 let total = total_count?;
2528 (total > collected as u64).then_some(total)
2529}
2530
2531impl AuthenticatedClient {
2532 /// Which repositories and organizations the stored credential can actually
2533 /// reach.
2534 ///
2535 /// Two calls, both paginated: `GET /user/installations`, then
2536 /// `GET /user/installations/{id}/repositories` per installation. The shapes
2537 /// are the ones the D18 spike observed live
2538 /// (`docs/spikes/d18-org-jit-verification.md`, "The permission that
2539 /// authorized it").
2540 ///
2541 /// An installation is reported even when it is broader than the user
2542 /// expected — [`Installation::is_over_broad`] — because `07-security.md`
2543 /// requires that an over-broad installation be *visible* rather than
2544 /// assumed. Nothing here narrows or hides one.
2545 ///
2546 /// # Errors
2547 /// Every variant of [`GithubError`]. A `401` here goes through the same
2548 /// single-flight re-validation as any other request.
2549 pub async fn discover_installations(
2550 &self,
2551 app: &AppRegistration,
2552 ) -> Result<InstallationDiscovery, GithubError> {
2553 let mut installations = Vec::new();
2554 let mut skipped = 0_usize;
2555 for raw in self.all_installations().await? {
2556 // A null or nameless account is skipped rather than fatal. GitHub
2557 // types this field as nullable, and one unnameable installation must
2558 // not take down `auth status` for every other one — but it is also
2559 // not something to swallow quietly, because the repositories behind
2560 // it are then absent from the reported reach. The count is what
2561 // carries that out of here; a `warn!` alone let the skip change the
2562 // verdict with nothing to say so.
2563 let Some(login) = raw.account.as_ref().and_then(RawAccount::display_login) else {
2564 skipped += 1;
2565 tracing::warn!(
2566 installation_id = raw.id,
2567 "skipping an installation GitHub reported with no nameable account; \
2568 anything it reaches is missing from this report"
2569 );
2570 continue;
2571 };
2572 let account_type = raw.account.as_ref().and_then(|a| a.account_type.as_deref());
2573 let account = match account_type {
2574 Some("Organization") => {
2575 InstallationAccount::Organization(Org::new(login).map_err(|_| {
2576 GithubError::Malformed {
2577 what: "an installation account login",
2578 value: login.to_string(),
2579 }
2580 })?)
2581 }
2582 Some("Enterprise") => InstallationAccount::Enterprise(login.to_string()),
2583 // An enterprise is also reported with no `type` at all, carrying
2584 // `slug`/`name` where a user carries `login` — which is the
2585 // shape D18 observed and the shape `display_login` exists for.
2586 // Recognising it by that shape is what stops it being labelled a
2587 // user by default.
2588 _ if raw
2589 .account
2590 .as_ref()
2591 .is_some_and(RawAccount::is_enterprise_shaped) =>
2592 {
2593 InstallationAccount::Enterprise(login.to_string())
2594 }
2595 _ => InstallationAccount::User(login.to_string()),
2596 };
2597 let repository_selection = match raw.repository_selection.as_deref() {
2598 Some("all") => RepositorySelection::All,
2599 _ => RepositorySelection::Selected,
2600 };
2601 installations.push(Installation {
2602 id: raw.id,
2603 account,
2604 repository_selection,
2605 repositories: self.installation_repositories(raw.id).await?,
2606 permissions: raw.permissions.into_iter().collect(),
2607 });
2608 }
2609
2610 let targets = ReachableTargets {
2611 installations,
2612 skipped,
2613 };
2614 if targets.is_empty() {
2615 // "Nothing reachable" and "nothing this client could describe" are
2616 // different answers, and only the first one is fixed by installing
2617 // the App. Reporting them as the same answer is how an
2618 // already-installed operator was handed an install URL.
2619 if skipped > 0 {
2620 tracing::warn!(
2621 skipped,
2622 "every installation GitHub reported was skipped; whether the App is \
2623 installed cannot be determined from this credential"
2624 );
2625 return Ok(InstallationDiscovery::Indeterminate { skipped });
2626 }
2627 let install_url = app.install_url(&self.endpoints);
2628 tracing::info!(
2629 install_url = %install_url,
2630 "the published App is not installed on anything this credential can reach"
2631 );
2632 return Ok(InstallationDiscovery::NotInstalled { install_url });
2633 }
2634 tracing::info!(
2635 repositories = targets.repositories().len(),
2636 organizations = targets.organizations().len(),
2637 over_broad = targets.over_broad().len(),
2638 skipped,
2639 "discovered the targets this credential can reach"
2640 );
2641 Ok(InstallationDiscovery::Installed(targets))
2642 }
2643
2644 async fn all_installations(&self) -> Result<Vec<RawInstallation>, GithubError> {
2645 let mut out = Vec::new();
2646 let mut total_count = None;
2647 let mut next = Some(ApiRequest::get("/user/installations").query("per_page", 100));
2648 let mut pages = 0_usize;
2649 while let Some(request) = next.take() {
2650 let response = self.send(&request).await?;
2651 let page: InstallationsPage = response.json()?;
2652 total_count = page.total_count.or(total_count);
2653 out.extend(page.installations);
2654
2655 pages += 1;
2656 if pages >= MAX_PAGES {
2657 tracing::warn!(
2658 pages,
2659 collected = out.len(),
2660 "stopped following installation pages at the ceiling; a `Link: rel=next` \
2661 that never ends would otherwise loop forever"
2662 );
2663 break;
2664 }
2665 next = response
2666 .next_page()
2667 .map(|url| ApiRequest::get(url.as_str()));
2668 }
2669 if let Some(expected) = under_collected(out.len(), total_count) {
2670 tracing::warn!(
2671 expected,
2672 collected = out.len(),
2673 "GitHub reported more installations than pagination collected; the reported \
2674 reach is incomplete"
2675 );
2676 }
2677 Ok(out)
2678 }
2679
2680 async fn installation_repositories(&self, id: u64) -> Result<Vec<OwnerRepo>, GithubError> {
2681 let mut out = Vec::new();
2682 let mut total_count = None;
2683 let mut next = Some(
2684 ApiRequest::get(format!("/user/installations/{id}/repositories"))
2685 .query("per_page", 100),
2686 );
2687 let mut pages = 0_usize;
2688 while let Some(request) = next.take() {
2689 let response = self.send(&request).await?;
2690 let page: RepositoriesPage = response.json()?;
2691 total_count = page.total_count.or(total_count);
2692 for repo in page.repositories {
2693 out.push(OwnerRepo::parse(&repo.full_name).map_err(|_| {
2694 GithubError::Malformed {
2695 what: "a repository full_name",
2696 value: repo.full_name.clone(),
2697 }
2698 })?);
2699 }
2700
2701 pages += 1;
2702 if pages >= MAX_PAGES {
2703 tracing::warn!(
2704 installation_id = id,
2705 pages,
2706 collected = out.len(),
2707 "stopped following repository pages at the ceiling; a `Link: rel=next` \
2708 that never ends would otherwise loop forever"
2709 );
2710 break;
2711 }
2712 next = response
2713 .next_page()
2714 .map(|url| ApiRequest::get(url.as_str()));
2715 }
2716 if let Some(expected) = under_collected(out.len(), total_count) {
2717 tracing::warn!(
2718 installation_id = id,
2719 expected,
2720 collected = out.len(),
2721 "GitHub reported more repositories than pagination collected; this \
2722 installation's reach is under-reported"
2723 );
2724 }
2725 Ok(out)
2726 }
2727}
2728
2729/// Test support shared by this file and [`device_flow`].
2730///
2731/// It lives inline rather than in `src/testing.rs` on purpose. `a1` laid out
2732/// this crate's five source files — `lib.rs`, `device_flow.rs`, `rest.rs`,
2733/// `demand.rs`, `jit.rs` — and owns every manifest; `c3` and `c4` are working in
2734/// the same directory in parallel, and a new file there is a merge conflict
2735/// waiting to happen for no benefit. An inline `#[cfg(test)]` module is
2736/// reachable as `crate::testing` from every module in the crate and adds nothing
2737/// to a release build.
2738///
2739/// It does not live in `runner-manager-testkit` either, and that one is
2740/// mechanical: `testkit` depends on `runner-manager-github`, so a unit test
2741/// inside this crate that used a `testkit` helper would link a *second* instance
2742/// of this library and the two instances' types would not unify — the same
2743/// hazard `testkit`'s own crate documentation records for `domain`.
2744#[cfg(test)]
2745pub(crate) mod testing {
2746 use super::*;
2747 use serde_json::{Value, json};
2748 use std::sync::{Mutex, atomic::AtomicUsize};
2749 use wiremock::{Request, Respond, ResponseTemplate};
2750
2751 /// Shaped like a real `ghu_` token, and unmistakably not one.
2752 pub const FIXTURE_TOKEN: &str = "ghu_fixtureTOKENnotARealCredential00";
2753 /// Shaped like a real device code, and unmistakably not one.
2754 pub const FIXTURE_DEVICE_CODE: &str = "fixture-device-code-0e37a9c1b4d84f2a";
2755 /// The example user code from RFC 8628.
2756 pub const FIXTURE_USER_CODE: &str = "WDJB-MJHT";
2757
2758 /// A clock the test moves.
2759 ///
2760 /// Deliberately not `runner_manager_testkit::clock::FakeClock`; see this
2761 /// module's documentation for why a `testkit` import is not available here.
2762 #[derive(Debug)]
2763 pub struct TestClock {
2764 now: Mutex<Timestamp>,
2765 }
2766
2767 impl TestClock {
2768 /// # Panics
2769 /// If a previous holder panicked while the lock was held.
2770 pub fn advance_secs(&self, secs: i64) {
2771 let mut now = self.now.lock().expect("TestClock lock poisoned");
2772 *now += chrono::TimeDelta::seconds(secs);
2773 }
2774 }
2775
2776 impl Default for TestClock {
2777 fn default() -> Self {
2778 // 2026-08-21T00:00:00Z, the date this taskflow's decisions were
2779 // locked — the same epoch `testkit`'s clock starts at.
2780 Self {
2781 now: Mutex::new(
2782 chrono::DateTime::from_timestamp(1_787_270_400, 0).expect("a valid instant"),
2783 ),
2784 }
2785 }
2786 }
2787
2788 impl Clock for TestClock {
2789 fn now(&self) -> Timestamp {
2790 *self.now.lock().expect("TestClock lock poisoned")
2791 }
2792 }
2793
2794 /// A sleeper that records what it was asked to wait and returns at once.
2795 ///
2796 /// This is what turns "`slow_down` demonstrably increases the poll interval"
2797 /// into an equality assertion on a `Vec<Duration>`.
2798 #[derive(Debug, Default)]
2799 pub struct RecordingSleeper {
2800 recorded: Mutex<Vec<Duration>>,
2801 }
2802
2803 impl RecordingSleeper {
2804 /// # Panics
2805 /// If a previous holder panicked while the lock was held.
2806 pub fn recorded(&self) -> Vec<Duration> {
2807 self.recorded.lock().expect("sleeper lock poisoned").clone()
2808 }
2809 }
2810
2811 #[async_trait::async_trait]
2812 impl Sleeper for RecordingSleeper {
2813 async fn sleep(&self, duration: Duration) {
2814 self.recorded
2815 .lock()
2816 .expect("sleeper lock poisoned")
2817 .push(duration);
2818 }
2819 }
2820
2821 /// Answers from a fixed script, one entry per call, repeating the last.
2822 pub struct Script {
2823 responses: Vec<ResponseTemplate>,
2824 calls: AtomicUsize,
2825 }
2826
2827 impl Script {
2828 #[must_use]
2829 pub fn new(responses: Vec<ResponseTemplate>) -> Self {
2830 assert!(
2831 !responses.is_empty(),
2832 "a script needs at least one response"
2833 );
2834 Self {
2835 responses,
2836 calls: AtomicUsize::new(0),
2837 }
2838 }
2839 }
2840
2841 impl Respond for Script {
2842 fn respond(&self, _: &Request) -> ResponseTemplate {
2843 let i = self.calls.fetch_add(1, Ordering::SeqCst);
2844 self.responses[i.min(self.responses.len() - 1)].clone()
2845 }
2846 }
2847
2848 /// `POST https://github.com/login/device/code` → `200`, in the shape both
2849 /// spikes observed (`docs/spikes/d17-spike.ps1`).
2850 #[must_use]
2851 pub fn device_code_body(server_uri: &str, interval: u64, expires_in: u64) -> Value {
2852 json!({
2853 "device_code": FIXTURE_DEVICE_CODE,
2854 "user_code": FIXTURE_USER_CODE,
2855 "verification_uri": format!("{server_uri}/login/device"),
2856 "expires_in": expires_in,
2857 "interval": interval
2858 })
2859 }
2860
2861 /// `POST .../login/oauth/access_token` → `200` with an `error` field, which
2862 /// is how GitHub answers every state in the matrix.
2863 #[must_use]
2864 pub fn error_body(code: &str, interval: Option<u64>) -> Value {
2865 let mut body = json!({
2866 "error": code,
2867 "error_description": "see the OAuth 2.0 Device Authorization Grant",
2868 "error_uri": "https://docs.github.com/developers/apps/authorizing-oauth-apps"
2869 });
2870 if let Some(interval) = interval {
2871 body["interval"] = json!(interval);
2872 }
2873 body
2874 }
2875
2876 /// `POST .../login/oauth/access_token` → `200` with an approved token.
2877 #[must_use]
2878 pub fn token_body() -> Value {
2879 json!({ "access_token": FIXTURE_TOKEN, "token_type": "bearer", "scope": "" })
2880 }
2881
2882 /// `GET /user/installations` → `200`. The permission set is the one D18 read
2883 /// back from the live installation.
2884 #[must_use]
2885 pub fn installations_body(entries: &[(u64, &str, &str, &str)]) -> Value {
2886 let installations: Vec<Value> = entries
2887 .iter()
2888 .map(|(id, login, account_type, selection)| {
2889 json!({
2890 "id": id,
2891 "account": { "login": login, "type": account_type },
2892 "repository_selection": selection,
2893 "permissions": {
2894 "actions": "read",
2895 "administration": "write",
2896 "metadata": "read",
2897 "organization_self_hosted_runners": "write"
2898 }
2899 })
2900 })
2901 .collect();
2902 json!({ "total_count": installations.len(), "installations": installations })
2903 }
2904
2905 /// `GET /user/installations/{id}/repositories` → `200`.
2906 #[must_use]
2907 pub fn repositories_body(full_names: &[&str]) -> Value {
2908 let repositories: Vec<Value> = full_names
2909 .iter()
2910 .map(|full_name| json!({ "full_name": full_name }))
2911 .collect();
2912 json!({ "total_count": repositories.len(), "repositories": repositories })
2913 }
2914
2915 // The `tracing` capture subscriber that used to live here now lives in
2916 // `tests/no_secret_reaches_the_logs.rs`, and the move is the point rather
2917 // than tidying. `tracing` caches a callsite's `Interest` process-wide while
2918 // `with_default` installs a subscriber only on the calling *thread*, so a
2919 // scan running alongside the crate's other unit tests captured nothing but
2920 // its own handful of events and passed with a real device-code leak in the
2921 // flow. A scan that is the only test in its process has no concurrent
2922 // thread to be poisoned by, and no `#[cfg(test)]` module here can offer
2923 // that guarantee.
2924 //
2925 // The blinding is a *concurrency* effect and not a permanent
2926 // first-registration one — see that file's header for the measurement that
2927 // separates the two. The distinction matters here because only the
2928 // concurrency reading implies what this comment concludes: that one test
2929 // per process is the fix.
2930}
2931
2932#[cfg(test)]
2933mod tests {
2934
2935 /// Upgrading must not log anybody out, and the App's expiration setting must
2936 /// be safe to turn on -- or back off -- with hosts mid-way through either.
2937 #[test]
2938 fn both_stored_shapes_load_and_a_pair_survives_a_round_trip() {
2939 // What every host stored before renewal existed: a bare token.
2940 let legacy = UserAccessToken::from_stored_document(&SecretString::from("ghu_legacy123"));
2941 assert_eq!(legacy.secret().expose_secret(), "ghu_legacy123");
2942 assert!(
2943 legacy.renewal().is_none(),
2944 "a bare token has no renewal half, and inventing one would make the client try to refresh a credential the App never issued a refresh token for"
2945 );
2946
2947 // A pair, written and read back.
2948 let pair = UserAccessToken::new(SecretString::from("ghu_new")).with_renewal(
2949 Some(SecretString::from("ghr_new")),
2950 Some(28_800),
2951 Some(15_897_600),
2952 );
2953 let stored = pair.to_stored_document();
2954 let read = UserAccessToken::from_stored_document(&stored);
2955 assert_eq!(read.secret().expose_secret(), "ghu_new");
2956 let renewal = read.renewal().expect("the pair survives the round trip");
2957 assert_eq!(renewal.refresh_token().expose_secret(), "ghr_new");
2958 assert!(renewal.access_expires_at.is_some());
2959 assert!(renewal.refresh_expires_at.is_some());
2960
2961 // A credential with no renewal still writes the document shape, and
2962 // still reads back as having none.
2963 let bare_round_trip = UserAccessToken::from_stored_document(&legacy.to_stored_document());
2964 assert_eq!(bare_round_trip.secret().expose_secret(), "ghu_legacy123");
2965 assert!(bare_round_trip.renewal().is_none());
2966 }
2967
2968 /// The refresh token is the more dangerous half -- it mints access tokens
2969 /// for six months -- so it must not reach a log through `Debug`.
2970 #[test]
2971 fn a_refresh_token_never_appears_in_debug_output() {
2972 let pair = UserAccessToken::new(SecretString::from("ghu_x")).with_renewal(
2973 Some(SecretString::from("ghr_SUPERSECRET")),
2974 Some(1),
2975 Some(2),
2976 );
2977 let rendered = format!("{:?}", pair.renewal().expect("a renewal"));
2978 assert!(!rendered.contains("ghr_SUPERSECRET"), "{rendered}");
2979 assert!(rendered.contains("redacted"), "{rendered}");
2980 }
2981 use super::*;
2982 use crate::testing::{FIXTURE_TOKEN, Script, TestClock, installations_body, repositories_body};
2983 use serde_json::json;
2984 use wiremock::{
2985 Mock, MockServer, ResponseTemplate,
2986 matchers::{header, method, path},
2987 };
2988
2989 fn client(server: &MockServer, clock: Arc<TestClock>) -> AuthenticatedClient {
2990 AuthenticatedClient::new(
2991 Endpoints::for_test_server(&server.uri()).unwrap(),
2992 UserAccessToken::new(SecretString::from(FIXTURE_TOKEN)),
2993 clock,
2994 )
2995 .unwrap()
2996 }
2997
2998 fn app() -> AppRegistration {
2999 AppRegistration::new("Iv23liTESTCLIENTID", "runner-manager").unwrap()
3000 }
3001
3002 // -- picking up a credential somebody else stored -------------------------
3003
3004 /// A [`CredentialSource`] over a fixed answer, which is what a store looks
3005 /// like from here.
3006 #[derive(Debug)]
3007 struct StoreHolding(Option<&'static str>);
3008
3009 impl CredentialSource for StoreHolding {
3010 fn reload(&self) -> Option<UserAccessToken> {
3011 self.0
3012 .map(|token| UserAccessToken::new(SecretString::from(token)))
3013 }
3014 }
3015
3016 /// The 28-hour failure, as a test: a daemon holding a dead bare token, an
3017 /// operator who signs in, and nothing that tells the daemon.
3018 ///
3019 /// Bare on purpose. A pair would renew and never reach the store at all,
3020 /// which is why renewal alone did not cover this.
3021 #[tokio::test]
3022 async fn a_daemon_picks_up_a_sign_in_that_happened_after_it_started() {
3023 let server = MockServer::start().await;
3024 Mock::given(method("GET"))
3025 .and(path("/repos/acme/app"))
3026 .and(header("authorization", "Bearer ghu_dead"))
3027 .respond_with(ResponseTemplate::new(401))
3028 .expect(1)
3029 .mount(&server)
3030 .await;
3031 Mock::given(method("GET"))
3032 .and(path("/repos/acme/app"))
3033 .and(header("authorization", "Bearer ghu_freshly_signed_in"))
3034 .respond_with(ResponseTemplate::new(200).set_body_json(json!({"id": 1})))
3035 .expect(1)
3036 .mount(&server)
3037 .await;
3038
3039 let client = AuthenticatedClient::new(
3040 Endpoints::for_test_server(&server.uri()).unwrap(),
3041 UserAccessToken::new(SecretString::from("ghu_dead")),
3042 Arc::new(TestClock::default()),
3043 )
3044 .unwrap()
3045 .with_credential_source(Arc::new(StoreHolding(Some("ghu_freshly_signed_in"))));
3046
3047 client
3048 .send(&ApiRequest::get("/repos/acme/app"))
3049 .await
3050 .expect(
3051 "the 401 is retried with what the store holds now, without anybody \n restarting the daemon",
3052 );
3053 }
3054
3055 /// The other half, and the reason for the comparison in `reload_once`: a
3056 /// store that still holds the token that just failed is not news.
3057 ///
3058 /// Without the check, every `401` would answer "something changed, retry"
3059 /// and a genuinely revoked credential would spend two requests per poll
3060 /// forever instead of being reported.
3061 #[tokio::test]
3062 async fn a_store_holding_the_same_dead_token_is_not_worth_a_retry() {
3063 let server = MockServer::start().await;
3064 Mock::given(method("GET"))
3065 .and(path("/repos/acme/app"))
3066 .respond_with(ResponseTemplate::new(401))
3067 .expect(1)
3068 .mount(&server)
3069 .await;
3070 // The re-validation probe that runs once reload declines.
3071 Mock::given(method("GET"))
3072 .and(path("/user/installations"))
3073 .respond_with(ResponseTemplate::new(401))
3074 .expect(1)
3075 .mount(&server)
3076 .await;
3077
3078 let client = AuthenticatedClient::new(
3079 Endpoints::for_test_server(&server.uri()).unwrap(),
3080 UserAccessToken::new(SecretString::from("ghu_revoked")),
3081 Arc::new(TestClock::default()),
3082 )
3083 .unwrap()
3084 .with_credential_source(Arc::new(StoreHolding(Some("ghu_revoked"))));
3085
3086 let failure = client
3087 .send(&ApiRequest::get("/repos/acme/app"))
3088 .await
3089 .expect_err("a revoked credential is still revoked when the store agrees");
3090 assert!(
3091 matches!(failure, GithubError::AuthenticationFailed),
3092 "{failure:?}"
3093 );
3094 }
3095
3096 /// An unreadable store leaves the `401` exactly where it was, rather than
3097 /// turning a rejection into a different kind of error.
3098 #[tokio::test]
3099 async fn an_unreadable_store_changes_nothing_about_the_rejection() {
3100 let server = MockServer::start().await;
3101 Mock::given(method("GET"))
3102 .and(path("/repos/acme/app"))
3103 .respond_with(ResponseTemplate::new(401))
3104 .expect(1)
3105 .mount(&server)
3106 .await;
3107 Mock::given(method("GET"))
3108 .and(path("/user/installations"))
3109 .respond_with(ResponseTemplate::new(401))
3110 .expect(1)
3111 .mount(&server)
3112 .await;
3113
3114 let client = AuthenticatedClient::new(
3115 Endpoints::for_test_server(&server.uri()).unwrap(),
3116 UserAccessToken::new(SecretString::from("ghu_revoked")),
3117 Arc::new(TestClock::default()),
3118 )
3119 .unwrap()
3120 .with_credential_source(Arc::new(StoreHolding(None)));
3121
3122 let failure = client
3123 .send(&ApiRequest::get("/repos/acme/app"))
3124 .await
3125 .expect_err("nothing to pick up means the rejection stands");
3126 assert!(
3127 matches!(failure, GithubError::AuthenticationFailed),
3128 "{failure:?}"
3129 );
3130 }
3131
3132 // -- headers ------------------------------------------------------------
3133
3134 #[tokio::test]
3135 async fn every_request_states_its_api_version_and_accept_header() {
3136 let server = MockServer::start().await;
3137 Mock::given(method("GET"))
3138 .and(path("/user/installations"))
3139 .and(header("x-github-api-version", GITHUB_API_VERSION))
3140 .and(header("accept", GITHUB_ACCEPT))
3141 .and(header("authorization", format!("Bearer {FIXTURE_TOKEN}")))
3142 .and(header("user-agent", USER_AGENT))
3143 .respond_with(ResponseTemplate::new(200).set_body_json(installations_body(&[])))
3144 .expect(1)
3145 .mount(&server)
3146 .await;
3147
3148 let client = client(&server, Arc::new(TestClock::default()));
3149 client
3150 .send(&ApiRequest::get("/user/installations"))
3151 .await
3152 .expect("the mock only matches when all four headers are present");
3153 }
3154
3155 // -- the 401 path -------------------------------------------------------
3156
3157 #[tokio::test]
3158 async fn a_401_revalidates_once_and_retries_once_then_succeeds() {
3159 let server = MockServer::start().await;
3160 Mock::given(method("GET"))
3161 .and(path("/orgs/acme/actions/runners"))
3162 .respond_with(Script::new(vec![
3163 ResponseTemplate::new(401).set_body_json(json!({"message": "Bad credentials"})),
3164 ResponseTemplate::new(200).set_body_json(json!({"total_count": 0})),
3165 ]))
3166 .expect(2)
3167 .mount(&server)
3168 .await;
3169 Mock::given(method("GET"))
3170 .and(path("/user/installations"))
3171 .respond_with(ResponseTemplate::new(200).set_body_json(installations_body(&[])))
3172 .expect(1)
3173 .mount(&server)
3174 .await;
3175
3176 let client = client(&server, Arc::new(TestClock::default()));
3177 let response = client
3178 .send(&ApiRequest::get("/orgs/acme/actions/runners"))
3179 .await
3180 .expect("the retry succeeds");
3181
3182 assert_eq!(response.status(), StatusCode::OK);
3183 assert_eq!(
3184 client.revalidations_performed(),
3185 1,
3186 "one 401 must produce exactly one re-validation"
3187 );
3188 }
3189
3190 #[tokio::test]
3191 async fn a_second_401_after_the_retry_is_terminal_authentication_failure() {
3192 let server = MockServer::start().await;
3193 Mock::given(method("GET"))
3194 .and(path("/orgs/acme/actions/runners"))
3195 .respond_with(ResponseTemplate::new(401))
3196 .expect(2)
3197 .mount(&server)
3198 .await;
3199 Mock::given(method("GET"))
3200 .and(path("/user/installations"))
3201 .respond_with(ResponseTemplate::new(200).set_body_json(installations_body(&[])))
3202 .mount(&server)
3203 .await;
3204
3205 let client = client(&server, Arc::new(TestClock::default()));
3206 let err = client
3207 .send(&ApiRequest::get("/orgs/acme/actions/runners"))
3208 .await
3209 .expect_err("two 401s is terminal");
3210
3211 assert!(matches!(err, GithubError::AuthenticationFailed), "{err:?}");
3212 assert!(err.is_authentication());
3213 assert!(!err.is_lockout());
3214 }
3215
3216 #[tokio::test]
3217 async fn a_rejected_revalidation_fails_without_spending_the_retry() {
3218 let server = MockServer::start().await;
3219 Mock::given(method("GET"))
3220 .and(path("/orgs/acme/actions/runners"))
3221 .respond_with(ResponseTemplate::new(401))
3222 // Exactly one: a credential GitHub has confirmed dead must not be
3223 // used for a retry.
3224 .expect(1)
3225 .mount(&server)
3226 .await;
3227 Mock::given(method("GET"))
3228 .and(path("/user/installations"))
3229 .respond_with(ResponseTemplate::new(401))
3230 .expect(1)
3231 .mount(&server)
3232 .await;
3233
3234 let client = client(&server, Arc::new(TestClock::default()));
3235 let err = client
3236 .send(&ApiRequest::get("/orgs/acme/actions/runners"))
3237 .await
3238 .expect_err("the credential is dead");
3239 assert!(matches!(err, GithubError::AuthenticationFailed), "{err:?}");
3240 }
3241
3242 /// The Definition of Done's concurrency claim, tested with real concurrent
3243 /// callers on a multi-threaded runtime rather than by reasoning about the
3244 /// mutex.
3245 ///
3246 /// Two things make the assertion deterministic rather than lucky. A barrier
3247 /// releases all eight callers into `send` together, so all eight take their
3248 /// `401` before any of them reaches the gate; and the re-validation endpoint
3249 /// is delayed, so the first caller still holds the gate while the other
3250 /// seven sample the generation counter. Without the delay a caller could
3251 /// legitimately arrive after the first re-validation completed, which is a
3252 /// *new* `401` storm and correctly earns its own attempt.
3253 #[tokio::test(flavor = "multi_thread", worker_threads = 8)]
3254 async fn eight_concurrent_401s_produce_one_revalidation_not_eight() {
3255 const CALLERS: usize = 8;
3256
3257 let server = MockServer::start().await;
3258 Mock::given(method("GET"))
3259 .and(path("/orgs/acme/actions/runners"))
3260 .respond_with(ResponseTemplate::new(401))
3261 .mount(&server)
3262 .await;
3263 Mock::given(method("GET"))
3264 .and(path("/user/installations"))
3265 .respond_with(
3266 ResponseTemplate::new(200)
3267 .set_body_json(installations_body(&[]))
3268 .set_delay(Duration::from_millis(250)),
3269 )
3270 .expect(1)
3271 .mount(&server)
3272 .await;
3273
3274 let client = Arc::new(client(&server, Arc::new(TestClock::default())));
3275 let barrier = Arc::new(tokio::sync::Barrier::new(CALLERS));
3276 let mut tasks = Vec::new();
3277 for _ in 0..CALLERS {
3278 let client = Arc::clone(&client);
3279 let barrier = Arc::clone(&barrier);
3280 tasks.push(tokio::spawn(async move {
3281 barrier.wait().await;
3282 client
3283 .send(&ApiRequest::get("/orgs/acme/actions/runners"))
3284 .await
3285 .expect_err("every caller sees a dead endpoint")
3286 }));
3287 }
3288
3289 let mut outcomes = Vec::new();
3290 for task in tasks {
3291 outcomes.push(task.await.expect("no caller panicked"));
3292 }
3293
3294 assert_eq!(outcomes.len(), CALLERS);
3295 for err in &outcomes {
3296 assert!(matches!(err, GithubError::AuthenticationFailed), "{err:?}");
3297 }
3298 assert_eq!(
3299 client.revalidations_performed(),
3300 1,
3301 "{CALLERS} concurrent 401s must produce ONE attempt, not {CALLERS}"
3302 );
3303
3304 // The same claim, measured from the server rather than from our own
3305 // counter: the mock's `.expect(1)` is verified when the server drops.
3306 let seen = server.received_requests().await.expect("recording is on");
3307 let probes = seen
3308 .iter()
3309 .filter(|r| r.url.path() == "/user/installations")
3310 .count();
3311 assert_eq!(probes, 1, "GitHub itself saw exactly one re-validation");
3312 let attempts = seen
3313 .iter()
3314 .filter(|r| r.url.path() == "/orgs/acme/actions/runners")
3315 .count();
3316 assert_eq!(
3317 attempts,
3318 CALLERS * 2,
3319 "each caller still gets its own single retry"
3320 );
3321 }
3322
3323 // -- the 403 path -------------------------------------------------------
3324
3325 #[tokio::test]
3326 async fn a_403_after_401s_is_a_lockout_and_not_an_authentication_failure() {
3327 let server = MockServer::start().await;
3328 Mock::given(method("GET"))
3329 .and(path("/orgs/acme/actions/runners"))
3330 .respond_with(Script::new(vec![
3331 ResponseTemplate::new(401),
3332 ResponseTemplate::new(403).insert_header("retry-after", "42"),
3333 ]))
3334 .mount(&server)
3335 .await;
3336 Mock::given(method("GET"))
3337 .and(path("/user/installations"))
3338 .respond_with(ResponseTemplate::new(200).set_body_json(installations_body(&[])))
3339 .mount(&server)
3340 .await;
3341
3342 let client = client(&server, Arc::new(TestClock::default()));
3343 let err = client
3344 .send(&ApiRequest::get("/orgs/acme/actions/runners"))
3345 .await
3346 .expect_err("403 after a 401");
3347
3348 match err {
3349 GithubError::AuthenticationLockout { retry_after } => {
3350 assert_eq!(retry_after, Duration::from_secs(42), "honours retry-after");
3351 }
3352 other => panic!("expected a lockout, got {other:?}"),
3353 }
3354 assert!(client.is_locked_out());
3355 }
3356
3357 #[tokio::test]
3358 async fn a_403_with_no_preceding_401_is_a_permissions_answer_not_a_lockout() {
3359 let server = MockServer::start().await;
3360 Mock::given(method("GET"))
3361 .and(path("/orgs/acme/actions/runners"))
3362 .respond_with(
3363 ResponseTemplate::new(403)
3364 .set_body_json(json!({"message": "Resource not accessible by integration"})),
3365 )
3366 .mount(&server)
3367 .await;
3368
3369 let client = client(&server, Arc::new(TestClock::default()));
3370 let err = client
3371 .send(&ApiRequest::get("/orgs/acme/actions/runners"))
3372 .await
3373 .expect_err("403");
3374
3375 assert!(matches!(err, GithubError::Forbidden { .. }), "{err:?}");
3376 assert!(!err.is_lockout());
3377 assert!(!err.is_authentication());
3378 assert!(!client.is_locked_out(), "a permissions 403 must not latch");
3379 }
3380
3381 #[tokio::test]
3382 async fn a_locked_out_client_issues_no_further_http_until_the_backoff_elapses() {
3383 let server = MockServer::start().await;
3384 Mock::given(method("GET"))
3385 .and(path("/orgs/acme/actions/runners"))
3386 .respond_with(Script::new(vec![
3387 ResponseTemplate::new(401),
3388 ResponseTemplate::new(403).insert_header("retry-after", "60"),
3389 ResponseTemplate::new(200).set_body_json(json!({"total_count": 0})),
3390 ]))
3391 .mount(&server)
3392 .await;
3393 Mock::given(method("GET"))
3394 .and(path("/user/installations"))
3395 .respond_with(ResponseTemplate::new(200).set_body_json(installations_body(&[])))
3396 .mount(&server)
3397 .await;
3398
3399 let clock = Arc::new(TestClock::default());
3400 let client = client(&server, Arc::clone(&clock));
3401 let request = ApiRequest::get("/orgs/acme/actions/runners");
3402
3403 let err = client.send(&request).await.expect_err("locks out");
3404 assert!(err.is_lockout(), "{err:?}");
3405
3406 let after_lockout = server.received_requests().await.unwrap().len();
3407
3408 for _ in 0..3 {
3409 let err = client.send(&request).await.expect_err("still locked out");
3410 assert!(err.is_lockout(), "{err:?}");
3411 }
3412 assert_eq!(
3413 server.received_requests().await.unwrap().len(),
3414 after_lockout,
3415 "a backed-off client must open no sockets at all"
3416 );
3417
3418 clock.advance_secs(61);
3419 assert!(!client.is_locked_out(), "the back-off expires on the clock");
3420 let response = client.send(&request).await.expect("traffic resumes");
3421 assert_eq!(response.status(), StatusCode::OK);
3422 assert_eq!(
3423 server.received_requests().await.unwrap().len(),
3424 after_lockout + 1
3425 );
3426 }
3427
3428 /// `consecutive_unauthorized` is reset only by a successful *caller*
3429 /// response, so a request ending in `404`, `422` or `5xx` leaves it set —
3430 /// and in the agent's long-lived reconciliation loop it stays set for as
3431 /// long as nothing succeeds. Before the fix, the next genuine permissions
3432 /// `403` was therefore reported as `AuthenticationLockout`: sixty seconds of
3433 /// client silence, plus an operator message asserting "the credential itself
3434 /// is not the problem" about a credential that was missing
3435 /// `Administration: write` — the failure `04-subsystem-contracts.md` names
3436 /// as the *expected* one for `generate-jitconfig`.
3437 ///
3438 /// The lockout's real signature is narrower: a `403` on the one retry this
3439 /// client issues after this request's own `401`, or a `403` whose own
3440 /// headers and body say GitHub is continuing a lockout. Neither is "the
3441 /// count is non-zero", which is what a stale `401` leaves behind — so the
3442 /// permissions `403` below is a permissions answer whatever happened minutes
3443 /// ago, and it is the *response*, not the history, that decides.
3444 #[tokio::test]
3445 async fn a_stale_401_does_not_turn_a_later_permissions_403_into_a_lockout() {
3446 let server = MockServer::start().await;
3447 // The first request ends in a 404, which leaves the 401 count set
3448 // because only a 2xx clears it.
3449 Mock::given(method("GET"))
3450 .and(path("/orgs/acme/actions/runners"))
3451 .respond_with(Script::new(vec![
3452 ResponseTemplate::new(401),
3453 ResponseTemplate::new(404).set_body_json(json!({"message": "Not Found"})),
3454 ]))
3455 .mount(&server)
3456 .await;
3457 Mock::given(method("GET"))
3458 .and(path("/user/installations"))
3459 .respond_with(ResponseTemplate::new(200).set_body_json(installations_body(&[])))
3460 .mount(&server)
3461 .await;
3462 // Minutes later, a different call is denied for a missing permission.
3463 Mock::given(method("POST"))
3464 .and(path("/orgs/acme/actions/runners/generate-jitconfig"))
3465 .respond_with(
3466 ResponseTemplate::new(403)
3467 .set_body_json(json!({"message": "Resource not accessible by integration"})),
3468 )
3469 .mount(&server)
3470 .await;
3471
3472 let client = client(&server, Arc::new(TestClock::default()));
3473 let err = client
3474 .send(&ApiRequest::get("/orgs/acme/actions/runners"))
3475 .await
3476 .expect_err("404");
3477 assert!(
3478 matches!(err, GithubError::Status { status: 404, .. }),
3479 "{err:?}"
3480 );
3481
3482 let err = client
3483 .send(&ApiRequest::new(
3484 Method::POST,
3485 "/orgs/acme/actions/runners/generate-jitconfig",
3486 ))
3487 .await
3488 .expect_err("403");
3489
3490 assert!(
3491 matches!(err, GithubError::Forbidden { .. }),
3492 "a fresh first-attempt 403 is a permissions answer, not a lockout: {err:?}"
3493 );
3494 assert!(!err.is_lockout());
3495 assert!(
3496 !client.is_locked_out(),
3497 "a stale 401 must not be able to silence the client for a minute"
3498 );
3499 }
3500
3501 /// GitHub's own rate limit is not an answer about the credential, and must
3502 /// not be reported as one. `classify` reached the `403` branch before
3503 /// anything looked at the rate-limit headers, so a primary rate limit
3504 /// arriving during a `401` storm was announced as an authentication lockout
3505 /// with the message "the credential itself is not the problem" — about a
3506 /// response that never mentioned the credential.
3507 #[tokio::test]
3508 async fn a_rate_limited_403_is_not_reported_as_an_authentication_lockout() {
3509 let server = MockServer::start().await;
3510 Mock::given(method("GET"))
3511 .and(path("/orgs/acme/actions/runners"))
3512 .respond_with(Script::new(vec![
3513 ResponseTemplate::new(401),
3514 ResponseTemplate::new(403)
3515 .insert_header("x-ratelimit-remaining", "0")
3516 .insert_header("x-ratelimit-reset", "1787270460")
3517 .insert_header("retry-after", "30")
3518 .set_body_json(json!({"message": "API rate limit exceeded"})),
3519 ]))
3520 .mount(&server)
3521 .await;
3522 Mock::given(method("GET"))
3523 .and(path("/user/installations"))
3524 .respond_with(ResponseTemplate::new(200).set_body_json(installations_body(&[])))
3525 .mount(&server)
3526 .await;
3527
3528 let client = client(&server, Arc::new(TestClock::default()));
3529 let err = client
3530 .send(&ApiRequest::get("/orgs/acme/actions/runners"))
3531 .await
3532 .expect_err("rate limited");
3533
3534 assert!(
3535 matches!(err, GithubError::Forbidden { .. }),
3536 "a rate limit is not an authentication outcome: {err:?}"
3537 );
3538 assert!(!err.is_lockout());
3539 assert!(!err.is_authentication());
3540 assert!(
3541 !client.is_locked_out(),
3542 "a rate limit must not latch this crate's authentication back-off"
3543 );
3544
3545 // And `c3` gets the evidence it needs to apply the policy that is its
3546 // own, without editing this file.
3547 let evidence = err
3548 .rate_limit()
3549 .expect("the headers survived classification");
3550 assert_eq!(evidence.remaining, Some(0));
3551 assert_eq!(evidence.reset_unix_secs, Some(1_787_270_460));
3552 assert_eq!(evidence.retry_after, Some(Duration::from_secs(30)));
3553 }
3554
3555 /// The same claim for the variant `429` lands in.
3556 #[tokio::test]
3557 async fn a_429_carries_its_retry_after_across_the_c2_c3_seam() {
3558 let server = MockServer::start().await;
3559 Mock::given(method("GET"))
3560 .and(path("/orgs/acme/actions/runners"))
3561 .respond_with(
3562 ResponseTemplate::new(429)
3563 .insert_header("retry-after", "17")
3564 .insert_header("x-ratelimit-remaining", "0")
3565 .set_body_json(json!({"message": "You have exceeded a secondary rate limit"})),
3566 )
3567 .mount(&server)
3568 .await;
3569
3570 let client = client(&server, Arc::new(TestClock::default()));
3571 let err = client
3572 .send(&ApiRequest::get("/orgs/acme/actions/runners"))
3573 .await
3574 .expect_err("429");
3575
3576 assert!(
3577 matches!(err, GithubError::Status { status: 429, .. }),
3578 "{err:?}"
3579 );
3580 assert_eq!(
3581 err.retry_after(),
3582 Some(Duration::from_secs(17)),
3583 "destroying this header is what made `c3`'s Definition of Done unmeetable"
3584 );
3585 assert_eq!(
3586 err.headers().and_then(|h| h.get("x-ratelimit-remaining")),
3587 Some(&reqwest::header::HeaderValue::from_static("0"))
3588 );
3589 }
3590
3591 /// A back-off is a safety mechanism, and this one had both failure modes at
3592 /// once: no ceiling, so `Retry-After: 86400` latched a silent twenty-four
3593 /// hour outage; and `TimeDelta::from_std(...).ok()` on a value too large to
3594 /// convert, which yielded `until = None` — *not locked out at all*, the
3595 /// exact inverse of the requirement, reachable by a header alone.
3596 #[tokio::test]
3597 async fn an_extreme_retry_after_is_clamped_and_never_fails_open() {
3598 async fn lockout_for(header: &str) -> (GithubError, bool, Option<Duration>) {
3599 let server = MockServer::start().await;
3600 Mock::given(method("GET"))
3601 .and(path("/orgs/acme/actions/runners"))
3602 .respond_with(Script::new(vec![
3603 ResponseTemplate::new(401),
3604 ResponseTemplate::new(403).insert_header("retry-after", header),
3605 ]))
3606 .mount(&server)
3607 .await;
3608 Mock::given(method("GET"))
3609 .and(path("/user/installations"))
3610 .respond_with(ResponseTemplate::new(200).set_body_json(installations_body(&[])))
3611 .mount(&server)
3612 .await;
3613
3614 let client = AuthenticatedClient::new(
3615 Endpoints::for_test_server(&server.uri()).unwrap(),
3616 UserAccessToken::new(SecretString::from(FIXTURE_TOKEN)),
3617 Arc::new(TestClock::default()),
3618 )
3619 .unwrap();
3620 let err = client
3621 .send(&ApiRequest::get("/orgs/acme/actions/runners"))
3622 .await
3623 .expect_err("403 after a 401");
3624 let locked = client.is_locked_out();
3625 let remaining = client.lockout_remaining();
3626 (err, locked, remaining)
3627 }
3628
3629 // A day-long back-off is clamped to the ceiling.
3630 let (err, locked, remaining) = lockout_for("86400").await;
3631 let GithubError::AuthenticationLockout { retry_after } = &err else {
3632 panic!("expected a lockout, got {err:?}");
3633 };
3634 assert_eq!(
3635 *retry_after, MAX_LOCKOUT_BACKOFF,
3636 "an unclamped Retry-After lets a remote party decide how long this product \
3637 stays down"
3638 );
3639 assert!(locked);
3640 assert!(remaining.is_some_and(|r| r <= MAX_LOCKOUT_BACKOFF));
3641
3642 // A value too large for `chrono` must still lock out. Before the fix
3643 // this produced `until = None`: the more extreme the header, the less
3644 // protection it bought.
3645 let (err, locked, remaining) = lockout_for(&u64::MAX.to_string()).await;
3646 assert!(err.is_lockout(), "{err:?}");
3647 assert!(
3648 locked,
3649 "an absurd Retry-After must not mean `not locked out at all` — that fails open"
3650 );
3651 assert!(remaining.is_some_and(|r| r <= MAX_LOCKOUT_BACKOFF));
3652 }
3653
3654 /// The lockout's own contract is "this client issues no HTTP at all", and
3655 /// `revalidate` is HTTP. It documented an `AuthenticationLockout` it could
3656 /// never return, which made the one direct entry point into the probe the
3657 /// single exception to the rule.
3658 #[tokio::test]
3659 async fn a_direct_revalidation_is_refused_while_the_lockout_is_backing_off() {
3660 let server = MockServer::start().await;
3661 Mock::given(method("GET"))
3662 .and(path("/orgs/acme/actions/runners"))
3663 .respond_with(Script::new(vec![
3664 ResponseTemplate::new(401),
3665 ResponseTemplate::new(403).insert_header("retry-after", "60"),
3666 ]))
3667 .mount(&server)
3668 .await;
3669 Mock::given(method("GET"))
3670 .and(path("/user/installations"))
3671 .respond_with(ResponseTemplate::new(200).set_body_json(installations_body(&[])))
3672 .mount(&server)
3673 .await;
3674
3675 let client = client(&server, Arc::new(TestClock::default()));
3676 client
3677 .send(&ApiRequest::get("/orgs/acme/actions/runners"))
3678 .await
3679 .expect_err("locks out");
3680 assert!(client.is_locked_out());
3681
3682 let before = server.received_requests().await.unwrap().len();
3683 let err = client
3684 .revalidate()
3685 .await
3686 .expect_err("the documented lockout error is now reachable");
3687 assert!(err.is_lockout(), "{err:?}");
3688 assert_eq!(
3689 server.received_requests().await.unwrap().len(),
3690 before,
3691 "a locked-out client opens no socket, and the probe is not an exception"
3692 );
3693 }
3694
3695 /// The position rule fixed `classify` and left the same defect one function
3696 /// over, behind a comment asserting it could not happen: "the probe only
3697 /// ever runs after a `401`, so it is always in the retry position". Making
3698 /// [`AuthenticatedClient::revalidate`] public — the previous round's own
3699 /// change — is exactly what made that untrue.
3700 ///
3701 /// The sequence is the agent's, not a contrivance. A request 401s, the probe
3702 /// says the credential is fine, the retry answers `404` — which does *not*
3703 /// reset the counter, by design. Minutes later `f1` renders `auth status`,
3704 /// which probes directly, and the probe meets an ordinary permissions `403`.
3705 /// A stale `401` then latched a sixty-second client-wide lockout and told
3706 /// the operator to wait, when the real answer was a missing grant.
3707 #[tokio::test]
3708 async fn a_directly_requested_probe_does_not_latch_a_lockout_from_a_stale_401() {
3709 let server = MockServer::start().await;
3710 Mock::given(method("GET"))
3711 .and(path("/orgs/acme/actions/runners"))
3712 .respond_with(Script::new(vec![
3713 ResponseTemplate::new(401),
3714 // The retry misses. A `404` leaves `consecutive_unauthorized`
3715 // set, which is the whole premise of the position rule.
3716 ResponseTemplate::new(404).set_body_json(json!({"message": "Not Found"})),
3717 ]))
3718 .mount(&server)
3719 .await;
3720 Mock::given(method("GET"))
3721 .and(path("/user/installations"))
3722 .respond_with(Script::new(vec![
3723 // The probe that accompanies the 401 above.
3724 ResponseTemplate::new(200).set_body_json(installations_body(&[])),
3725 // The direct probe, minutes later: a plain permissions answer,
3726 // with no `retry-after` and a message that names a grant.
3727 ResponseTemplate::new(403)
3728 .set_body_json(json!({"message": "Resource not accessible by integration"})),
3729 ]))
3730 .mount(&server)
3731 .await;
3732
3733 let clock = Arc::new(TestClock::default());
3734 let client = client(&server, clock.clone());
3735 client
3736 .send(&ApiRequest::get("/orgs/acme/actions/runners"))
3737 .await
3738 .expect_err("the retry 404s");
3739 assert!(
3740 !client.is_locked_out(),
3741 "a 404 on the retry is not a lockout"
3742 );
3743
3744 clock.advance_secs(300);
3745 let outcome = client
3746 .revalidate()
3747 .await
3748 .expect("a direct probe is not a lockout error");
3749
3750 assert_eq!(
3751 outcome,
3752 Revalidation::Unavailable,
3753 "a 403 on the probe teaches this client nothing about the credential"
3754 );
3755 assert!(
3756 !client.is_locked_out(),
3757 "a caller-initiated probe is a *first* attempt, not the retry that follows a 401: \
3758 latching here converts a stale 401 into a 60-second client-wide outage and \
3759 reports a missing permission as `the credential is fine, please wait`"
3760 );
3761 assert_eq!(client.lockout_remaining(), None);
3762 }
3763
3764 /// The square the other three leave empty, and the one where a defect in the
3765 /// composition would hide.
3766 ///
3767 /// Covered elsewhere: a first-attempt continuation through `send`, a
3768 /// first-attempt permissions `403` through `send`, and a direct probe
3769 /// meeting a permissions `403` (immediately above). A direct probe meeting a
3770 /// *continuation-shaped* `403` is the fourth square, and it is the
3771 /// composition point of the two rules that pull in opposite directions —
3772 /// the narrowing to `Attempt::First` that fixed the stale-`401` lockout, and
3773 /// the continuation rule that re-widens `First` on GitHub's own evidence.
3774 ///
3775 /// If the narrowing swallowed the continuation here, every other test in
3776 /// this file would still pass, and `f1`'s `auth status` would poll a
3777 /// credential GitHub had asked it to leave alone — reporting each refusal as
3778 /// a missing grant.
3779 #[tokio::test]
3780 async fn a_directly_requested_probe_latches_a_continuation_shaped_lockout() {
3781 let server = MockServer::start().await;
3782 Mock::given(method("GET"))
3783 .and(path("/user/installations"))
3784 // The continuation's signature: `retry-after`, and a body with no
3785 // message for `error_message` to find.
3786 .respond_with(ResponseTemplate::new(403).insert_header("retry-after", "60"))
3787 .mount(&server)
3788 .await;
3789
3790 let client = client(&server, Arc::new(TestClock::default()));
3791
3792 // No preceding traffic whatsoever: a first attempt in the strongest
3793 // sense, which is exactly the position the narrowed rule refused to
3794 // latch in.
3795 let err = client
3796 .revalidate()
3797 .await
3798 .expect_err("a probe that latches a lockout reports it rather than `Unavailable`");
3799 assert!(
3800 err.is_lockout(),
3801 "`revalidate` latched a client-wide lockout and must say so; answering \
3802 `Ok(Unavailable)` leaves `f1` to discover a 15-minute outage through a separate \
3803 `is_locked_out()` call it has no reason to make: {err:?}"
3804 );
3805 assert!(
3806 client.is_locked_out(),
3807 "a 403 carrying `retry-after` with no message is GitHub continuing a lockout, \
3808 whoever asked for the request that met it"
3809 );
3810 assert_eq!(client.lockout_remaining(), Some(Duration::from_secs(60)));
3811
3812 // And the back-off is real, not just a renamed error.
3813 let before = server.received_requests().await.unwrap().len();
3814 let err = client.revalidate().await.expect_err("still locked out");
3815 assert!(err.is_lockout(), "{err:?}");
3816 assert_eq!(
3817 server.received_requests().await.unwrap().len(),
3818 before,
3819 "latching must actually stop traffic"
3820 );
3821 }
3822
3823 /// [`retry_after`] parses integer seconds only; RFC 9110 §10.2.3 also
3824 /// permits an HTTP-date. Detection used to gate on that parse succeeding, so
3825 /// a date-form `Retry-After` was not recognised as a continuation at all and
3826 /// the hole the continuation rule exists to close reopened for that shape —
3827 /// silently, because the response leaves as an ordinary `Forbidden`.
3828 ///
3829 /// GitHub sends integer seconds in practice. This pins the crate to not
3830 /// depending on that, and records where the two halves part company:
3831 /// presence decides *whether* it is a lockout, the integer parse decides
3832 /// only *how long*, and `latch_lockout` already had a default for the header
3833 /// it could not read.
3834 #[tokio::test]
3835 async fn a_date_form_retry_after_is_still_recognised_as_a_continuation() {
3836 let server = MockServer::start().await;
3837 Mock::given(method("GET"))
3838 .and(path("/user/installations"))
3839 .respond_with(
3840 ResponseTemplate::new(403)
3841 .insert_header("retry-after", "Wed, 21 Oct 2026 07:28:00 GMT"),
3842 )
3843 .mount(&server)
3844 .await;
3845
3846 let client = client(&server, Arc::new(TestClock::default()));
3847 let err = client
3848 .revalidate()
3849 .await
3850 .expect_err("a date-form `retry-after` is still GitHub asking to be left alone");
3851 assert!(
3852 err.is_lockout(),
3853 "gating detection on an integer parse hands the continuation bug back for every \
3854 lockout GitHub chose to date-stamp: {err:?}"
3855 );
3856 assert_eq!(
3857 client.lockout_remaining(),
3858 Some(DEFAULT_LOCKOUT_BACKOFF),
3859 "the date form is recognised for detection; the duration falls back to the \
3860 default, which is what `latch_lockout` already did with a header it could not \
3861 parse as seconds"
3862 );
3863 }
3864
3865 /// The narrowing that fixed the stale-`401` lockout opened a hole at the
3866 /// other end of the same back-off.
3867 ///
3868 /// While GitHub is still locking the credential out after the back-off
3869 /// elapses, the next request is a *first* attempt by construction — this
3870 /// client's own retry never happened, because the request never reached the
3871 /// wire. So the position rule declined to call it a lockout and `classify`
3872 /// fell through to [`GithubError::Forbidden`], whose documented reading is
3873 /// "the App installation does not grant it". The client then stopped backing
3874 /// off entirely and hammered a credential GitHub had asked it to leave
3875 /// alone, which is the exact inverse of "backs off without retrying".
3876 ///
3877 /// A continuation is distinguishable from a permissions answer without any
3878 /// counter: GitHub sends `retry-after` and no message body for the lockout,
3879 /// and a message and no `retry-after` for a permissions refusal.
3880 #[tokio::test]
3881 async fn a_lockout_outliving_its_backoff_re_latches_instead_of_reporting_a_permissions_answer()
3882 {
3883 let server = MockServer::start().await;
3884 Mock::given(method("GET"))
3885 .and(path("/orgs/acme/actions/runners"))
3886 .respond_with(Script::new(vec![
3887 ResponseTemplate::new(401),
3888 // The retry: the lockout latches here, in the retry position.
3889 ResponseTemplate::new(403).insert_header("retry-after", "60"),
3890 // The continuation, once the back-off has elapsed. Same shape,
3891 // first position.
3892 ResponseTemplate::new(403).insert_header("retry-after", "60"),
3893 ]))
3894 .mount(&server)
3895 .await;
3896 Mock::given(method("GET"))
3897 .and(path("/user/installations"))
3898 .respond_with(ResponseTemplate::new(200).set_body_json(installations_body(&[])))
3899 .mount(&server)
3900 .await;
3901
3902 let clock = Arc::new(TestClock::default());
3903 let client = client(&server, clock.clone());
3904 let err = client
3905 .send(&ApiRequest::get("/orgs/acme/actions/runners"))
3906 .await
3907 .expect_err("403 on the retry");
3908 assert!(err.is_lockout(), "{err:?}");
3909 assert!(client.is_locked_out());
3910
3911 // The back-off elapses with GitHub unchanged.
3912 clock.advance_secs(61);
3913 assert!(!client.is_locked_out(), "the back-off has run out");
3914
3915 let err = client
3916 .send(&ApiRequest::get("/orgs/acme/actions/runners"))
3917 .await
3918 .expect_err("GitHub is still locking the credential out");
3919
3920 assert!(
3921 err.is_lockout(),
3922 "a 403 carrying `retry-after` with no message is GitHub continuing the lockout, \
3923 not the App installation refusing a permission; reporting `Forbidden` here \
3924 tells the operator to fix a grant that is not missing: {err:?}"
3925 );
3926 assert!(
3927 client.is_locked_out(),
3928 "`backs off without retrying` fails for any lockout that outlives one back-off \
3929 if the continuation does not re-latch"
3930 );
3931 let GithubError::AuthenticationLockout { retry_after } = err else {
3932 unreachable!("asserted above")
3933 };
3934 assert_eq!(
3935 retry_after,
3936 Duration::from_secs(60),
3937 "the continuation's own `retry-after` sets the new back-off"
3938 );
3939
3940 // And the next request is suppressed before a socket is opened, which is
3941 // the property the whole back-off exists for.
3942 let before = server.received_requests().await.unwrap().len();
3943 let err = client
3944 .send(&ApiRequest::get("/orgs/acme/actions/runners"))
3945 .await
3946 .expect_err("still locked out");
3947 assert!(err.is_lockout(), "{err:?}");
3948 assert_eq!(
3949 server.received_requests().await.unwrap().len(),
3950 before,
3951 "re-latching must actually stop traffic, not merely rename the error"
3952 );
3953 }
3954
3955 /// A permissions `403` on a first attempt is still a permissions answer, and
3956 /// the continuation rule above must not swallow it. This is the test that
3957 /// keeps that rule from becoming "every 403 is a lockout".
3958 #[tokio::test]
3959 async fn a_first_attempt_permissions_403_is_still_reported_as_forbidden() {
3960 let server = MockServer::start().await;
3961 Mock::given(method("GET"))
3962 .and(path("/orgs/acme/actions/runners"))
3963 .respond_with(
3964 ResponseTemplate::new(403)
3965 .set_body_json(json!({"message": "Resource not accessible by integration"})),
3966 )
3967 .mount(&server)
3968 .await;
3969
3970 let client = client(&server, Arc::new(TestClock::default()));
3971 let err = client
3972 .send(&ApiRequest::get("/orgs/acme/actions/runners"))
3973 .await
3974 .expect_err("403");
3975
3976 assert!(
3977 matches!(err, GithubError::Forbidden { .. }),
3978 "a message and no `retry-after` is GitHub naming a missing grant: {err:?}"
3979 );
3980 assert!(!client.is_locked_out());
3981 }
3982
3983 /// The `consecutive_unauthorized > 0` conjunct that used to sit alongside
3984 /// the position rule added no signal — `Attempt::Retry` already implies this
3985 /// request's own `401` incremented the counter — and added a fail-open race:
3986 /// any concurrent success `store(0)`s the counter between the `401` and the
3987 /// retry, and a real lockout is then reported as a permissions answer.
3988 ///
3989 /// The race is driven directly rather than by scheduling two requests and
3990 /// hoping: `store(0)` is the *only* thing the concurrent success contributes,
3991 /// so performing it between the `401` and the classification reproduces the
3992 /// race deterministically and on every run.
3993 #[tokio::test]
3994 async fn a_concurrent_success_cannot_downgrade_a_lockout_to_a_permissions_answer() {
3995 let server = MockServer::start().await;
3996 let client = client(&server, Arc::new(TestClock::default()));
3997
3998 // This request's own 401 has landed: the retry position is established.
3999 client
4000 .consecutive_unauthorized
4001 .fetch_add(1, Ordering::SeqCst);
4002 // ... and a request on another task succeeds in the same instant.
4003 client.consecutive_unauthorized.store(0, Ordering::SeqCst);
4004
4005 let mut headers = HeaderMap::new();
4006 headers.insert("retry-after", "60".parse().unwrap());
4007 let lockout = ApiResponse {
4008 status: StatusCode::FORBIDDEN,
4009 headers,
4010 body: Vec::new(),
4011 };
4012
4013 assert!(
4014 client.is_lockout_403(&lockout, Attempt::Retry),
4015 "`Attempt::Retry` already means this request's own 401 incremented the counter, so \
4016 reading the counter again adds no signal and only lets an unrelated success \
4017 downgrade a real lockout to `Forbidden`"
4018 );
4019
4020 // The counter must stay irrelevant in the other direction too: a
4021 // permissions `403` on a first attempt is not a lockout however many
4022 // `401`s are on the count.
4023 client.consecutive_unauthorized.store(7, Ordering::SeqCst);
4024 let permissions = ApiResponse {
4025 status: StatusCode::FORBIDDEN,
4026 headers: HeaderMap::new(),
4027 body: br#"{"message":"Resource not accessible by integration"}"#.to_vec(),
4028 };
4029 assert!(!client.is_lockout_403(&permissions, Attempt::First));
4030 }
4031
4032 // -- installation discovery ---------------------------------------------
4033
4034 #[tokio::test]
4035 async fn discovery_returns_the_reachable_repository_and_organization_set() {
4036 let server = MockServer::start().await;
4037 Mock::given(method("GET"))
4038 .and(path("/user/installations"))
4039 .respond_with(
4040 ResponseTemplate::new(200).set_body_json(installations_body(&[
4041 (11, "IvanMurzak", "User", "selected"),
4042 (22, "Tap-Top-Fun", "Organization", "all"),
4043 ])),
4044 )
4045 .mount(&server)
4046 .await;
4047 Mock::given(method("GET"))
4048 .and(path("/user/installations/11/repositories"))
4049 .respond_with(
4050 ResponseTemplate::new(200)
4051 .set_body_json(repositories_body(&["IvanMurzak/GitHub-Runner-Scaler-UI"])),
4052 )
4053 .mount(&server)
4054 .await;
4055 Mock::given(method("GET"))
4056 .and(path("/user/installations/22/repositories"))
4057 .respond_with(
4058 ResponseTemplate::new(200)
4059 .set_body_json(repositories_body(&["Tap-Top-Fun/game", "Tap-Top-Fun/site"])),
4060 )
4061 .mount(&server)
4062 .await;
4063
4064 let client = client(&server, Arc::new(TestClock::default()));
4065 let discovery = client.discover_installations(&app()).await.unwrap();
4066
4067 let targets = discovery.targets().expect("installed");
4068 assert_eq!(
4069 targets
4070 .repositories()
4071 .iter()
4072 .map(ToString::to_string)
4073 .collect::<Vec<_>>(),
4074 [
4075 "IvanMurzak/GitHub-Runner-Scaler-UI",
4076 "Tap-Top-Fun/game",
4077 "Tap-Top-Fun/site"
4078 ]
4079 );
4080 assert_eq!(
4081 targets
4082 .organizations()
4083 .iter()
4084 .map(ToString::to_string)
4085 .collect::<Vec<_>>(),
4086 ["Tap-Top-Fun"],
4087 "a User account is not an organization target"
4088 );
4089 assert!(discovery.install_url().is_none());
4090 }
4091
4092 #[tokio::test]
4093 async fn an_over_broad_installation_is_visible_rather_than_assumed() {
4094 let server = MockServer::start().await;
4095 Mock::given(method("GET"))
4096 .and(path("/user/installations"))
4097 .respond_with(
4098 ResponseTemplate::new(200).set_body_json(installations_body(&[
4099 (11, "IvanMurzak", "User", "selected"),
4100 (22, "Tap-Top-Fun", "Organization", "all"),
4101 ])),
4102 )
4103 .mount(&server)
4104 .await;
4105 Mock::given(method("GET"))
4106 .and(path("/user/installations/11/repositories"))
4107 .respond_with(ResponseTemplate::new(200).set_body_json(repositories_body(&["a/b"])))
4108 .mount(&server)
4109 .await;
4110 Mock::given(method("GET"))
4111 .and(path("/user/installations/22/repositories"))
4112 .respond_with(ResponseTemplate::new(200).set_body_json(repositories_body(&["c/d"])))
4113 .mount(&server)
4114 .await;
4115
4116 let client = client(&server, Arc::new(TestClock::default()));
4117 let targets = client
4118 .discover_installations(&app())
4119 .await
4120 .unwrap()
4121 .targets()
4122 .cloned()
4123 .expect("installed");
4124
4125 let over_broad = targets.over_broad();
4126 assert_eq!(over_broad.len(), 1);
4127 assert_eq!(over_broad[0].account.login(), "Tap-Top-Fun");
4128 assert!(over_broad[0].is_over_broad());
4129 assert_eq!(
4130 over_broad[0].repository_selection,
4131 RepositorySelection::All,
4132 "`repository_selection: all` reaches repositories created later too"
4133 );
4134 assert!(
4135 targets.installations().iter().any(|i| i
4136 .permissions
4137 .iter()
4138 .any(|(k, v)| k == "administration" && v == "write")),
4139 "the grant GitHub reports is surfaced verbatim, not assumed from the design"
4140 );
4141 }
4142
4143 #[tokio::test]
4144 async fn discovery_returns_the_installation_url_when_the_set_is_empty() {
4145 let server = MockServer::start().await;
4146 Mock::given(method("GET"))
4147 .and(path("/user/installations"))
4148 .respond_with(ResponseTemplate::new(200).set_body_json(installations_body(&[])))
4149 .mount(&server)
4150 .await;
4151
4152 let client = client(&server, Arc::new(TestClock::default()));
4153 let discovery = client.discover_installations(&app()).await.unwrap();
4154
4155 let url = discovery
4156 .install_url()
4157 .expect("an empty set must yield the installation URL");
4158 assert_eq!(url.path(), "/apps/runner-manager/installations/new");
4159 assert!(discovery.targets().is_none());
4160 }
4161
4162 #[tokio::test]
4163 async fn an_installation_that_reaches_no_repository_is_still_not_installed() {
4164 let server = MockServer::start().await;
4165 Mock::given(method("GET"))
4166 .and(path("/user/installations"))
4167 .respond_with(
4168 ResponseTemplate::new(200).set_body_json(installations_body(&[(
4169 11,
4170 "IvanMurzak",
4171 "User",
4172 "selected",
4173 )])),
4174 )
4175 .mount(&server)
4176 .await;
4177 Mock::given(method("GET"))
4178 .and(path("/user/installations/11/repositories"))
4179 .respond_with(ResponseTemplate::new(200).set_body_json(repositories_body(&[])))
4180 .mount(&server)
4181 .await;
4182
4183 let client = client(&server, Arc::new(TestClock::default()));
4184 let discovery = client.discover_installations(&app()).await.unwrap();
4185 assert!(
4186 discovery.install_url().is_some(),
4187 "a user installation that selected no repository reaches nothing"
4188 );
4189 }
4190
4191 #[tokio::test]
4192 async fn discovery_follows_every_page_rather_than_trusting_the_first() {
4193 let server = MockServer::start().await;
4194 let next = format!("<{}/user/installations?page=2>; rel=\"next\"", server.uri());
4195 Mock::given(method("GET"))
4196 .and(path("/user/installations"))
4197 .respond_with(Script::new(vec![
4198 ResponseTemplate::new(200)
4199 .set_body_json(installations_body(&[(
4200 11,
4201 "one",
4202 "Organization",
4203 "selected",
4204 )]))
4205 .insert_header("link", next.as_str()),
4206 ResponseTemplate::new(200).set_body_json(installations_body(&[(
4207 22,
4208 "two",
4209 "Organization",
4210 "selected",
4211 )])),
4212 ]))
4213 .mount(&server)
4214 .await;
4215 Mock::given(method("GET"))
4216 .and(path("/user/installations/11/repositories"))
4217 .respond_with(ResponseTemplate::new(200).set_body_json(repositories_body(&["one/a"])))
4218 .mount(&server)
4219 .await;
4220 Mock::given(method("GET"))
4221 .and(path("/user/installations/22/repositories"))
4222 .respond_with(ResponseTemplate::new(200).set_body_json(repositories_body(&["two/b"])))
4223 .mount(&server)
4224 .await;
4225
4226 let client = client(&server, Arc::new(TestClock::default()));
4227 let targets = client
4228 .discover_installations(&app())
4229 .await
4230 .unwrap()
4231 .targets()
4232 .cloned()
4233 .expect("installed");
4234 assert_eq!(
4235 targets
4236 .organizations()
4237 .iter()
4238 .map(ToString::to_string)
4239 .collect::<Vec<_>>(),
4240 ["one", "two"],
4241 "the second page must not be dropped"
4242 );
4243 }
4244
4245 /// GitHub's published `installation` schema types `account` as **nullable**,
4246 /// and as either a simple-user *or* an enterprise — which carries
4247 /// `slug`/`name` where a user carries `login`. A required `RawAccount` with
4248 /// a required `login` made either shape a hard `response.json()` failure,
4249 /// which takes down all of `discover_installations`, which is all of
4250 /// `auth status`. One unusual installation must not blind the command that
4251 /// exists to show the user what their credential can reach.
4252 #[tokio::test]
4253 async fn an_installation_with_a_null_or_enterprise_account_does_not_fail_the_whole_decode() {
4254 let server = MockServer::start().await;
4255 Mock::given(method("GET"))
4256 .and(path("/user/installations"))
4257 .respond_with(ResponseTemplate::new(200).set_body_json(json!({
4258 "total_count": 3,
4259 "installations": [
4260 // Nullable, per the published schema.
4261 { "id": 10, "account": null, "repository_selection": "selected" },
4262 // An enterprise: no `login` at all.
4263 {
4264 "id": 20,
4265 "account": { "slug": "acme-enterprise", "name": "Acme Inc" },
4266 "repository_selection": "selected"
4267 },
4268 // And an ordinary user alongside them.
4269 {
4270 "id": 30,
4271 "account": { "login": "IvanMurzak", "type": "User" },
4272 "repository_selection": "selected"
4273 }
4274 ]
4275 })))
4276 .mount(&server)
4277 .await;
4278 for (id, repo) in [(20_u64, "acme-enterprise/tools"), (30, "IvanMurzak/app")] {
4279 Mock::given(method("GET"))
4280 .and(path(format!("/user/installations/{id}/repositories")))
4281 .respond_with(ResponseTemplate::new(200).set_body_json(repositories_body(&[repo])))
4282 .mount(&server)
4283 .await;
4284 }
4285
4286 let client = client(&server, Arc::new(TestClock::default()));
4287 let targets = client
4288 .discover_installations(&app())
4289 .await
4290 .expect("one odd account must not fail the whole discovery")
4291 .targets()
4292 .cloned()
4293 .expect("installed");
4294
4295 // Membership rather than order: what matters here is that neither
4296 // installation was lost, not how `OwnerRepo` collates.
4297 let reached = targets
4298 .repositories()
4299 .iter()
4300 .map(ToString::to_string)
4301 .collect::<Vec<_>>();
4302 assert!(
4303 reached.contains(&"acme-enterprise/tools".to_string()),
4304 "the enterprise installation is named from `slug` rather than dropped: {reached:?}"
4305 );
4306 assert!(
4307 reached.contains(&"IvanMurzak/app".to_string()),
4308 "the ordinary installation alongside it survives too: {reached:?}"
4309 );
4310 assert_eq!(reached.len(), 2);
4311 assert_eq!(
4312 targets.installations().len(),
4313 2,
4314 "the null account is skipped, and only it"
4315 );
4316 assert_eq!(
4317 targets.skipped(),
4318 1,
4319 "the skip is the right trade, but it must travel with the answer: everything the \
4320 skipped installation reaches is missing from the lists above, and a short list \
4321 reads exactly like a complete one"
4322 );
4323
4324 // The enterprise is labelled an enterprise. It used to fall through to
4325 // `User`, so `auth status` told the operator their enterprise was a
4326 // personal account.
4327 let enterprise = targets
4328 .installations()
4329 .iter()
4330 .find(|i| i.id == 20)
4331 .expect("the enterprise installation survived");
4332 assert_eq!(
4333 enterprise.account,
4334 InstallationAccount::Enterprise("acme-enterprise".to_string()),
4335 "an account with no `login` that names itself through `slug` is an enterprise, \
4336 and calling it a user is a wrong statement about the operator's own account"
4337 );
4338 assert_eq!(enterprise.account.kind(), "enterprise");
4339 assert!(
4340 enterprise.account.organization().is_none(),
4341 "an enterprise is not an organization target: `GET /orgs/{{org}}/actions/runners` \
4342 does not accept one, so contributing nothing to `organizations()` is correct"
4343 );
4344 assert!(
4345 !targets
4346 .organizations()
4347 .iter()
4348 .any(|o| o.as_str() == "acme-enterprise"),
4349 "and it must not be smuggled in as one either"
4350 );
4351 }
4352
4353 /// The skip is right; the verdict flip was not.
4354 ///
4355 /// A null-account installation that is the *only* installation used to
4356 /// collapse to `NotInstalled`, so `auth status` handed an operator who **is**
4357 /// installed the "install the App" URL — a wrong remediation on the only
4358 /// authentication path there is, contradicted by nothing but a `warn!`.
4359 #[tokio::test]
4360 async fn a_credential_whose_only_installation_was_skipped_is_not_reported_as_not_installed() {
4361 let server = MockServer::start().await;
4362 Mock::given(method("GET"))
4363 .and(path("/user/installations"))
4364 .respond_with(ResponseTemplate::new(200).set_body_json(json!({
4365 "total_count": 1,
4366 "installations": [
4367 { "id": 10, "account": null, "repository_selection": "selected" }
4368 ]
4369 })))
4370 .mount(&server)
4371 .await;
4372
4373 let client = client(&server, Arc::new(TestClock::default()));
4374 let discovery = client.discover_installations(&app()).await.unwrap();
4375
4376 assert_eq!(
4377 discovery,
4378 InstallationDiscovery::Indeterminate { skipped: 1 },
4379 "GitHub reported an installation; this client could not describe it. That is not \
4380 the same answer as `not installed`, and only one of the two is fixed by \
4381 installing the App"
4382 );
4383 assert_eq!(
4384 discovery.install_url(),
4385 None,
4386 "offering the install URL here is the wrong remediation, and putting it one field \
4387 over from the right verdict would just relocate the defect"
4388 );
4389 assert_eq!(discovery.skipped(), 1);
4390 assert!(discovery.targets().is_none());
4391 }
4392
4393 /// The other side of the same rule: with nothing skipped, an empty reach is
4394 /// still an empty reach, and the install URL is still the remediation.
4395 #[tokio::test]
4396 async fn an_empty_reach_with_nothing_skipped_is_still_not_installed() {
4397 let server = MockServer::start().await;
4398 Mock::given(method("GET"))
4399 .and(path("/user/installations"))
4400 .respond_with(ResponseTemplate::new(200).set_body_json(installations_body(&[])))
4401 .mount(&server)
4402 .await;
4403
4404 let client = client(&server, Arc::new(TestClock::default()));
4405 let discovery = client.discover_installations(&app()).await.unwrap();
4406
4407 assert!(discovery.install_url().is_some(), "{discovery:?}");
4408 assert_eq!(discovery.skipped(), 0);
4409 }
4410
4411 /// A `Link: rel="next"` that points back at the page it arrived on is an
4412 /// infinite loop inside the agent's reconciliation loop — the one place in
4413 /// this product that must not be able to wedge. The ceiling is what makes
4414 /// this test terminate at all.
4415 #[tokio::test]
4416 async fn a_self_referential_link_header_stops_at_the_page_ceiling() {
4417 let server = MockServer::start().await;
4418 let self_link = format!("<{}/user/installations?page=2>; rel=\"next\"", server.uri());
4419 Mock::given(method("GET"))
4420 .and(path("/user/installations"))
4421 .respond_with(
4422 ResponseTemplate::new(200)
4423 .set_body_json(installations_body(&[]))
4424 .insert_header("link", self_link.as_str()),
4425 )
4426 .mount(&server)
4427 .await;
4428
4429 let client = client(&server, Arc::new(TestClock::default()));
4430 let discovery = client
4431 .discover_installations(&app())
4432 .await
4433 .expect("the ceiling is what makes this return at all");
4434
4435 assert!(
4436 discovery.install_url().is_some(),
4437 "no installation was found"
4438 );
4439 assert_eq!(
4440 server.received_requests().await.unwrap().len(),
4441 MAX_PAGES,
4442 "pagination must stop at the ceiling rather than follow the loop forever"
4443 );
4444 }
4445
4446 #[test]
4447 fn a_link_header_yields_only_the_next_relation() {
4448 let header = "<https://api.github.com/user/installations?page=3>; rel=\"next\", \
4449 <https://api.github.com/user/installations?page=9>; rel=\"last\"";
4450 assert_eq!(
4451 parse_link_next(header).map(|u| u.to_string()),
4452 Some("https://api.github.com/user/installations?page=3".to_string())
4453 );
4454 assert!(parse_link_next("<https://x/>; rel=\"last\"").is_none());
4455 assert!(parse_link_next("nonsense").is_none());
4456 }
4457
4458 /// A comma is legal inside a URL and GitHub sends such URLs routinely — a
4459 /// runner query carries `labels=self-hosted,windows`. Splitting the header
4460 /// on `,` before recognising `<...>` tore that URL in half, found no
4461 /// relation, and stopped paginating at page 1 while reporting success. That
4462 /// is precisely what `04-subsystem-contracts.md` forbids ("the dashboard
4463 /// must not treat a first page as a complete inventory"), in the one shared
4464 /// reader `c3`'s inventory also goes through.
4465 #[test]
4466 fn a_next_url_containing_a_comma_still_paginates() {
4467 let header = "<https://api.github.com/repos/o/r/actions/runners\
4468 ?labels=self-hosted,windows&page=2>; rel=\"next\"";
4469 assert_eq!(
4470 parse_link_next(header).map(|u| u.to_string()),
4471 Some(
4472 "https://api.github.com/repos/o/r/actions/runners\
4473 ?labels=self-hosted,windows&page=2"
4474 .to_string()
4475 ),
4476 "a comma inside the URL must not end the link-value"
4477 );
4478
4479 // The same URL as the second link-value, so the scan has to walk past a
4480 // comma-bearing target to reach the relation it wants.
4481 let header = "<https://api.github.com/x?a=1,2&page=1>; rel=\"prev\", \
4482 <https://api.github.com/x?a=1,2&page=3>; rel=\"next\"";
4483 assert_eq!(
4484 parse_link_next(header).map(|u| u.to_string()),
4485 Some("https://api.github.com/x?a=1,2&page=3".to_string())
4486 );
4487 }
4488
4489 /// `rel="next"` is not always the first link-value, and both quoted and
4490 /// unquoted forms are legal.
4491 #[test]
4492 fn the_next_relation_is_found_wherever_it_sits_in_the_header() {
4493 let not_first = "<https://api.github.com/u?page=1>; rel=\"first\", \
4494 <https://api.github.com/u?page=9>; rel=\"last\", \
4495 <https://api.github.com/u?page=4>; rel=\"next\"";
4496 assert_eq!(
4497 parse_link_next(not_first).map(|u| u.to_string()),
4498 Some("https://api.github.com/u?page=4".to_string())
4499 );
4500
4501 let unquoted = "<https://api.github.com/u?page=1>; rel=prev, \
4502 <https://api.github.com/u?page=3>; rel=next";
4503 assert_eq!(
4504 parse_link_next(unquoted).map(|u| u.to_string()),
4505 Some("https://api.github.com/u?page=3".to_string())
4506 );
4507
4508 assert!(
4509 parse_link_next("<https://api.github.com/u?page=2; rel=\"next\"").is_none(),
4510 "an unterminated target is not a link-value"
4511 );
4512 }
4513
4514 /// The free cross-check that would have caught the comma bug on its own.
4515 #[test]
4516 fn a_short_collection_is_measured_against_the_count_github_reported() {
4517 assert_eq!(under_collected(1, Some(2)), Some(2), "page 2 was dropped");
4518 assert_eq!(under_collected(2, Some(2)), None, "complete");
4519 assert_eq!(
4520 under_collected(3, Some(2)),
4521 None,
4522 "a collection that grew between pages is not an under-collection"
4523 );
4524 assert_eq!(under_collected(0, None), None, "no count, no claim");
4525 }
4526
4527 // -- redaction ----------------------------------------------------------
4528
4529 #[test]
4530 fn no_type_in_this_crate_renders_a_secret_through_debug() {
4531 let token = UserAccessToken::new(SecretString::from(FIXTURE_TOKEN));
4532 let rendered = format!("{token:?}");
4533 assert!(!rendered.contains(FIXTURE_TOKEN), "{rendered}");
4534 assert!(rendered.contains("[REDACTED]"));
4535 assert!(
4536 rendered.contains("ghu_"),
4537 "the family prefix is diagnostic and is not the secret"
4538 );
4539
4540 let request = ApiRequest::post_json("/x", &json!({"encoded_jit_config": "SECRETBLOB"}))
4541 .expect("serializes");
4542 let rendered = format!("{request:?}");
4543 assert!(!rendered.contains("SECRETBLOB"), "{rendered}");
4544
4545 let response = ApiResponse {
4546 status: StatusCode::OK,
4547 headers: HeaderMap::new(),
4548 body: b"{\"encoded_jit_config\":\"SECRETBLOB\"}".to_vec(),
4549 };
4550 let rendered = format!("{response:?}");
4551 assert!(!rendered.contains("SECRETBLOB"), "{rendered}");
4552 }
4553
4554 // The Definition of Done's log scan is `tests/no_secret_reaches_the_logs.rs`
4555 // and not a unit test here. See the note at the end of `mod testing` for the
4556 // `tracing` callsite-cache reason it cannot be one.
4557
4558 // -- the crate-shape scans ----------------------------------------------
4559 //
4560 // The three gates below share these helpers on purpose. The previous round
4561 // defined `normalise` twice — once in the scan and once in the meta-test
4562 // that checks it — which left the meta-test structurally unable to notice a
4563 // change to the real one. One definition, used by both, is the only shape
4564 // in which a meta-test proves anything.
4565
4566 /// Spelled in halves so that this file's own source does not trip the scan
4567 /// it runs: normalising `concat!("refresh", "token")` leaves the
4568 /// quote-comma-quote between the halves, so no needle ever appears whole.
4569 // The renewal guard that used to live here is gone, and its absence is the
4570 // point. It forbade this crate from naming a refresh token at all, on the
4571 // reasoning that the published App opts out of user-token expiration "so
4572 // GitHub issues nothing to renew". That reasoning rested on a second claim
4573 // -- that renewing needs a confidential client credential -- which GitHub's
4574 // own documentation contradicts for the device flow, and which was then
4575 // disproved against live GitHub: a refresh exchange with `client_id` alone
4576 // answers `200`.
4577 //
4578 // What the guard below still forbids is the part that was always true and
4579 // is the reason renewal is safe here: no confidential credential in this
4580 // crate. Renewal was added *without* one, so the remaining half of this
4581 // scan is now evidence for the design rather than against it.
4582 const CONFIDENTIAL: &[&str] = &[concat!("client", "secret"), concat!("app", "secret")];
4583
4584 const MANIFEST: (&str, &str) = ("Cargo.toml", include_str!("../Cargo.toml"));
4585
4586 /// Every `.rs` file at or below `src/`, named by its `/`-joined path
4587 /// relative to `src/` — so a nested module is `("rest/runners.rs",
4588 /// include_str!("rest/runners.rs"))`, not just its file name.
4589 ///
4590 /// This list used to *be* the claim "every source file in the crate", and a
4591 /// hard-coded list is not that claim — it is a snapshot of it. `c3` and `c4`
4592 /// are the tasks that will add files to this directory, so the list was
4593 /// guaranteed to go stale on exactly the work that most needed scanning: a
4594 /// new `pagination.rs` holding a confidential credential passed silently.
4595 /// [`the_confidential_credential_scan_covers_every_source_file`] pins this
4596 /// by walking the directory tree, so adding a file — at the top level or in
4597 /// a subdirectory — and not adding it here fails.
4598 const CRATE_SOURCES: &[(&str, &str)] = &[
4599 ("demand.rs", include_str!("demand.rs")),
4600 ("device_flow.rs", include_str!("device_flow.rs")),
4601 ("jit.rs", include_str!("jit.rs")),
4602 ("lib.rs", include_str!("lib.rs")),
4603 ("rest.rs", include_str!("rest.rs")),
4604 ];
4605
4606 /// The two source files `c2` owns, plus the manifest. The renewal half of
4607 /// the scan stays inside this boundary; see the scan's own documentation.
4608 const SOURCES_OWNED_BY_C2: &[(&str, &str)] = &[
4609 ("device_flow.rs", include_str!("device_flow.rs")),
4610 ("lib.rs", include_str!("lib.rs")),
4611 MANIFEST,
4612 ];
4613
4614 /// Lower-cased with `_` removed, so that one needle catches the snake,
4615 /// camel, Pascal and screaming-snake spellings of an identifier at once.
4616 /// (Those four spellings cannot be written out here: they are exactly what
4617 /// the gate forbids, which is the constraint on documentation this scan
4618 /// imposes and defends below.)
4619 ///
4620 /// # Why `-` is *not* stripped from Rust source
4621 ///
4622 /// It used to be, and that rejected ordinary English. `c3`'s own file opens
4623 /// with a line stating that the gateway holds no such credential, written
4624 /// with the compound adjective English requires — and stripping `-` turned
4625 /// that sentence into the needle, so the gate accused `c3` of naming a
4626 /// confidential credential in the very line that says it holds none. A
4627 /// compound adjective is not an evasion; it is how the language works, and
4628 /// this brief, this crate's documentation and that line all use one.
4629 ///
4630 /// Nothing is lost, because **a Rust identifier cannot contain `-`**.
4631 /// Stripping it never bought identifier coverage: every casing an identifier
4632 /// can actually take is `_`-separated or unseparated, and all of those still
4633 /// collapse onto the needle. What it bought was coverage of a *kebab-case
4634 /// string literal*, and the residual gap is stated plainly rather than
4635 /// papered over: a `.rs` file that wrote this credential's name as a
4636 /// hyphenated string would not be caught here. That gap is narrow on
4637 /// purpose — OAuth 2.0 and GitHub both spell the field `_`-separated, which
4638 /// this catches — and it is the price of a gate that ordinary prose can
4639 /// coexist with. A gate that fires on correct English is not a stricter
4640 /// gate; it is a gate that gets deleted.
4641 ///
4642 /// The alternatives were weighed. Requiring identifier context needs a Rust
4643 /// lexer to tell `a client-secret-free design` from a TOML key, and gets the
4644 /// wrong answer for both string literals and comments. Excluding comment
4645 /// text needs the same lexer to avoid mangling `//` inside a string, and
4646 /// would stop the gate catching a `TODO` comment proposing to read the
4647 /// credential from the environment — which is precisely the drift worth
4648 /// catching early, while it is still a comment. Stripping one character
4649 /// fewer needs neither, which is why it wins.
4650 ///
4651 /// A space is not stripped either, and for the same reason: it is what lets
4652 /// this crate's prose discuss a "client secret" as two words.
4653 fn normalise_source(source: &str) -> String {
4654 source.to_ascii_lowercase().replace('_', "")
4655 }
4656
4657 /// The manifest keeps `-` stripped: TOML keys and crate names are kebab-case
4658 /// by convention, so `-` there is a word separator rather than a hyphen, and
4659 /// a manifest carries no hyphenated English for it to break.
4660 fn normalise_manifest(manifest: &str) -> String {
4661 manifest.to_ascii_lowercase().replace(['_', '-'], "")
4662 }
4663
4664 /// Which normaliser a scanned file gets. The manifest is the only file whose
4665 /// `-` is a separator rather than punctuation.
4666 fn normalise(name: &str, contents: &str) -> String {
4667 if name == MANIFEST.0 {
4668 normalise_manifest(contents)
4669 } else {
4670 normalise_source(contents)
4671 }
4672 }
4673
4674 /// The part of a source file that is not test code.
4675 ///
4676 /// The boundary is the first line that is **exactly** `#[cfg(test)]`, and
4677 /// the word "exactly" is the fix. Splitting on that literal wherever it
4678 /// appeared also split on it in *prose*, and `lib.rs` has carried such a
4679 /// mention since the `testing` module was documented — so the scan below
4680 /// already stopped nine lines early, today, with nothing to say so. A file
4681 /// whose module documentation happened to mention an inline test module
4682 /// would have had its scanned region truncated to a few dozen lines, after
4683 /// which a real `std::fs::write` in non-test code passed silently. That is
4684 /// the same class of defect as the log scan that captured only its own
4685 /// events and the credential scan that claimed a scope it did not have: a
4686 /// gate whose description outran what it did.
4687 fn non_test_prefix(source: &str) -> &str {
4688 let mut offset = 0;
4689 for line in source.split_inclusive('\n') {
4690 if line.trim() == "#[cfg(test)]" {
4691 return &source[..offset];
4692 }
4693 offset += line.len();
4694 }
4695 source
4696 }
4697
4698 /// The Definition of Done's second item, made checkable rather than
4699 /// reviewed: "no renewal token code path exists, and no client secret
4700 /// appears anywhere in the crate **or its configuration**".
4701 ///
4702 /// # Normalised, because a literal scan is evaded by naming
4703 ///
4704 /// This used to be a case-sensitive `contains` over two snake-case
4705 /// spellings, which is a gate that any ordinary Rust or JSON identifier
4706 /// walks straight through: the camel-cased, Pascal-cased and
4707 /// screaming-snake spellings of the very same two identifiers were all
4708 /// invisible to it. None of those is exotic — several are what the
4709 /// surrounding ecosystem actually calls these fields — so evading this gate
4710 /// never had to be deliberate. See [`normalise_source`] for what is
4711 /// collapsed, what is deliberately not, and why.
4712 ///
4713 /// The consequence is that this crate's *prose* may not write those
4714 /// identifiers either, in any casing: it says "renewal token" and "client
4715 /// secret" as separate words, which normalisation preserves and the scan
4716 /// therefore ignores. That is a real constraint on the documentation, and it
4717 /// is the right way round — a gate loosened until the comments compile is
4718 /// not a gate. It is a constraint on *identifier spellings*, though, and
4719 /// never on English: hyphenating a compound adjective is not writing an
4720 /// identifier, and a gate that could not tell those apart is what this round
4721 /// fixed.
4722 ///
4723 /// # Two different scopes, for two different reasons
4724 ///
4725 /// The **renewal** half stays scoped to the two files `c2` owns plus the
4726 /// manifest. A renewal path in `c3`'s or `c4`'s file would be their finding;
4727 /// failing here on their work would be this task reaching across an
4728 /// ownership boundary.
4729 ///
4730 /// The **client secret** half covers every source file in the crate. That is
4731 /// not a boundary crossing but the opposite: a public client cannot hold a
4732 /// client secret at all (D3, `07-security.md`), so one appearing *anywhere*
4733 /// in this crate is a product defect rather than a matter of whose file it
4734 /// is, and `c2` is the designated owner of that clause. "Every source file"
4735 /// is a claim about the directory, so it is checked against the directory —
4736 /// see [`the_confidential_credential_scan_covers_every_source_file`].
4737 #[test]
4738 fn no_confidential_credential_in_this_crate() {
4739 for &(name, source) in CRATE_SOURCES.iter().chain(std::iter::once(&MANIFEST)) {
4740 let haystack = normalise(name, source);
4741 for forbidden in CONFIDENTIAL {
4742 assert!(
4743 !haystack.contains(forbidden),
4744 "{name} names {forbidden:?} in some spelling: a public client cannot \
4745 secure a confidential credential, and this design never tries to (D3)"
4746 );
4747 }
4748 }
4749 }
4750
4751 /// "Every source file in the crate" is a claim about a directory, and the
4752 /// scan above states it as a hard-coded list. A list is a snapshot: the
4753 /// moment `c3` or `c4` adds a file to `src/`, the claim is false and nothing
4754 /// says so. A `src/pagination.rs` holding a confidential credential passed
4755 /// the gate that exists to catch exactly that.
4756 ///
4757 /// Reading the directory here is what turns the claim back into a claim. It
4758 /// cannot be done in the scan itself — `include_str!` needs a literal path
4759 /// at compile time — so the list stays, and this pins it.
4760 ///
4761 /// # Why it walks the tree instead of listing one directory
4762 ///
4763 /// It used to call `read_dir("src")` once and keep the entries ending in
4764 /// `.rs`. That reads like a directory scan and is not one: a subdirectory
4765 /// module — `src/rest/mod.rs`, `src/rest/runners.rs` — arrives as the single
4766 /// entry `rest`, which does not end in `.rs`, so the filter dropped it and
4767 /// took the files underneath with it. The pin went on passing while those
4768 /// files were scanned by nothing at all.
4769 ///
4770 /// That defeated the pin in exactly the case it was written for. `c3` is the
4771 /// REST inventory gateway, a module directory is the ordinary Rust shape for
4772 /// it, and the failure is silent on both sides: the credential scan does not
4773 /// read the file, and the test whose whole job is to notice that reports
4774 /// success.
4775 ///
4776 /// Recursing is the fix, rather than asserting that `src/` holds no
4777 /// subdirectories. The claim being pinned is about *files*, not about
4778 /// layout; banning the directory would fail `c3` for choosing a normal
4779 /// module shape, and a gate that fails correct work is a gate the next round
4780 /// loosens to get its own work compiling — which is how the normalisation
4781 /// half of this same scan was weakened once already.
4782 ///
4783 /// Names are `/`-joined paths relative to `src/`, which is what
4784 /// `include_str!` takes on every platform, so a nested file is listed as
4785 /// `("rest/runners.rs", include_str!("rest/runners.rs"))` and the two sides
4786 /// compare directly.
4787 #[test]
4788 fn the_confidential_credential_scan_covers_every_source_file() {
4789 // Every `.rs` file at or below `dir`, named by its `/`-joined path
4790 // relative to `src/`.
4791 //
4792 // `file_type()` is deliberately not followed through symlinks: a link
4793 // cannot walk this into a cycle, and a symlinked `.rs` file still lands
4794 // in the list through the extension test. A directory is recursed into
4795 // before the extension is considered, so a directory named `foo.rs`
4796 // is walked rather than mistaken for a file.
4797 fn collect(dir: &std::path::Path, prefix: &str, found: &mut Vec<String>) {
4798 for entry in std::fs::read_dir(dir).expect("the source directory is readable") {
4799 let entry = entry.expect("a readable directory entry");
4800 let name = entry.file_name().to_string_lossy().into_owned();
4801 let relative = if prefix.is_empty() {
4802 name.clone()
4803 } else {
4804 format!("{prefix}/{name}")
4805 };
4806 if entry.file_type().expect("a readable entry type").is_dir() {
4807 collect(&entry.path(), &relative, found);
4808 } else if name.ends_with(".rs") {
4809 found.push(relative);
4810 }
4811 }
4812 }
4813
4814 let mut on_disk = Vec::new();
4815 collect(
4816 std::path::Path::new(concat!(env!("CARGO_MANIFEST_DIR"), "/src")),
4817 "",
4818 &mut on_disk,
4819 );
4820 on_disk.sort();
4821
4822 // Sorted, rather than taken in declaration order. Comparing a sorted
4823 // `on_disk` against an unsorted `scanned` made this assertion depend on
4824 // `CRATE_SOURCES` happening to be declared alphabetically. It is — but
4825 // nothing said so, and the failure that would follow from reordering the
4826 // list is a diff of two lists holding the same names, which reads as a
4827 // coverage gap rather than as the ordering nit it would actually be.
4828 let mut scanned: Vec<String> = CRATE_SOURCES
4829 .iter()
4830 .map(|(name, _)| (*name).to_string())
4831 .collect();
4832 scanned.sort();
4833
4834 assert_eq!(
4835 scanned, on_disk,
4836 "`src/` and the scanned list have diverged. Add the new file to `CRATE_SOURCES` \
4837 with an `include_str!`, naming it by its `/`-joined path relative to `src/`; \
4838 leaving it out means the confidential-credential scan silently stops covering \
4839 `every source file in the crate`, which is the claim it makes."
4840 );
4841 }
4842
4843 /// The scan above, shown to actually catch the spellings it claims to — and
4844 /// to leave alone the ones it claims to leave alone.
4845 ///
4846 /// Without this, "the gate is case-insensitive now" is a comment rather than
4847 /// a fact, and the finding that produced it was precisely a gate whose
4848 /// description outran what it did. It calls [`normalise`], the same function
4849 /// the scan calls, because a meta-test with its own private copy of the
4850 /// thing under test cannot detect a change to it.
4851 #[test]
4852 fn the_confidential_credential_scan_is_not_evaded_by_naming() {
4853 // Assembled at run time rather than written out, for the same reason the
4854 // needles are spelled in halves: a test that contained these spellings
4855 // literally would fail the scan it is checking.
4856 let source_evasions = [
4857 format!("let {}Token = fetch()", "refresh"),
4858 format!("struct {}Token;", "Refresh"),
4859 format!("{}_TOKEN", "REFRESH"),
4860 format!("{}Secret", "client"),
4861 format!("{}_SECRET", "CLIENT"),
4862 format!("{}_secret", "app"),
4863 ];
4864 for evasion in &source_evasions {
4865 let normalised = normalise("lib.rs", evasion);
4866 assert!(
4867 normalised.contains(concat!("refresh", "token"))
4868 || normalised.contains(concat!("client", "secret"))
4869 || normalised.contains(concat!("app", "secret")),
4870 "{evasion:?} would walk straight through the scan"
4871 );
4872 }
4873
4874 // The manifest is where kebab-case is a word separator rather than a
4875 // hyphen, so that is where it is still collapsed.
4876 let manifest_evasion = format!("{}-secret = \"...\"", "client");
4877 assert!(
4878 normalise(MANIFEST.0, &manifest_evasion).contains(concat!("client", "secret")),
4879 "a kebab-case TOML key is an identifier, and the manifest normaliser must \
4880 still collapse it"
4881 );
4882
4883 // And the prose the crate legitimately writes must still pass, or the
4884 // gate would be unusable and would be weakened again to make it usable.
4885 for allowed in [
4886 "a public client cannot hold a client secret",
4887 "the published App issues no renewal token",
4888 // The line that fails the old normalisation, quoted from `c3`'s own
4889 // file. It says the *opposite* of what the gate accused it of.
4890 "//! This gateway is deliberately client-secret-free, as D3 requires.",
4891 // The same shape, for the renewal half.
4892 "a refresh-free credential model",
4893 ] {
4894 let normalised = normalise("lib.rs", allowed);
4895 assert!(
4896 !normalised.contains(concat!("client", "secret"))
4897 && !normalised.contains(concat!("refresh", "token")),
4898 "{allowed:?} is English, not an identifier, and must not trip the scan"
4899 );
4900 }
4901 }
4902
4903 /// The storage boundary, made checkable the same way. `c2` returns the token
4904 /// and never persists it; the machine-scoped store is `d2` and the wiring is
4905 /// `f1`. A dependency on `runner-manager-platform`, or a filesystem write,
4906 /// would silently move that boundary.
4907 #[test]
4908 fn this_crate_persists_nothing_and_does_not_depend_on_the_platform_crate() {
4909 assert!(
4910 !MANIFEST.1.contains("runner-manager-platform"),
4911 "the gateway must be testable with no platform dependency at all"
4912 );
4913
4914 for &(name, source) in SOURCES_OWNED_BY_C2 {
4915 if name == MANIFEST.0 {
4916 continue;
4917 }
4918 // Everything below `#[cfg(test)]` is test code; the boundary is about
4919 // non-test code, and the tests above legitimately read this file.
4920 let non_test = non_test_prefix(source);
4921 // `OpenOptions`, `File::options` and `std::io::Write` are on this
4922 // list because the original four named only the *obvious* ways to
4923 // write a file. A store built with `OpenOptions::new().create(true)`
4924 // would have moved the persistence boundary silently, which is the
4925 // one thing this scan exists to prevent.
4926 for forbidden in [
4927 "std::fs",
4928 "fs::write",
4929 "File::create",
4930 "File::options",
4931 "OpenOptions",
4932 "std::io::Write",
4933 "tokio::fs",
4934 ] {
4935 assert!(
4936 !non_test.contains(forbidden),
4937 "{name} performs a filesystem operation ({forbidden:?}) outside its tests"
4938 );
4939 }
4940 }
4941 }
4942
4943 /// The scan above, shown to be looking at what it says it is looking at.
4944 ///
4945 /// `split("#[cfg(test)]")` matched that literal **anywhere**, prose
4946 /// included. One ordinary sentence in a module's documentation truncated the
4947 /// scanned region to whatever preceded it, and every filesystem call after
4948 /// that point became invisible — with the scan still reporting `ok`. This is
4949 /// the third gate in this crate found describing more than it did, so it
4950 /// gets the same treatment as the other two: a synthetic file where the
4951 /// difference is decisive, and an assertion about the real ones.
4952 #[test]
4953 fn the_non_test_boundary_is_a_line_and_not_a_mention() {
4954 // A file shaped like this crate's own: prose that names the attribute,
4955 // then real non-test code, then the actual module.
4956 let file = "//! Test helpers live in an inline #[cfg(test)] module near the bottom.\n\
4957 \n\
4958 fn persist() { std::fs::write(\"x\", b\"y\").unwrap(); }\n\
4959 \n\
4960 #[cfg(test)]\n\
4961 mod tests {\n\
4962 fn helper() { std::fs::write(\"ok-in-tests\", b\"\").unwrap(); }\n\
4963 }\n";
4964
4965 let non_test = non_test_prefix(file);
4966 assert!(
4967 non_test.contains("fn persist"),
4968 "a prose mention of the attribute truncated the scanned region, and every \
4969 filesystem call below it stopped being scanned — silently:\n{non_test}"
4970 );
4971 assert!(
4972 !non_test.contains("ok-in-tests"),
4973 "the boundary must still exclude the real test module:\n{non_test}"
4974 );
4975
4976 // And on the real files, whose module documentation contains such a
4977 // mention today. `lib.rs` has carried one since `mod testing` was
4978 // written, so this crate was shipping the truncated scan.
4979 for &(name, source) in SOURCES_OWNED_BY_C2 {
4980 if name == MANIFEST.0 {
4981 continue;
4982 }
4983 let expected = source
4984 .lines()
4985 .position(|line| line.trim() == "#[cfg(test)]")
4986 .expect("each source file has an inline test module");
4987 let scanned = non_test_prefix(source).lines().count();
4988 assert_eq!(
4989 scanned, expected,
4990 "{name}: the scanned region ends at line {scanned} but the test module starts \
4991 at line {expected}. The gap is code that claims to be scanned and is not."
4992 );
4993 }
4994 }
4995
4996 #[derive(Debug)]
4997 struct SpyRenewal {
4998 invocations: std::sync::atomic::AtomicUsize,
4999 }
5000 #[async_trait::async_trait]
5001 impl CredentialRenewal for SpyRenewal {
5002 async fn renew(&self, _refresh_token: &SecretString) -> Result<UserAccessToken, String> {
5003 self.invocations
5004 .fetch_add(1, std::sync::atomic::Ordering::SeqCst);
5005 Ok(UserAccessToken::new(SecretString::from("ghu_renewed")))
5006 }
5007 }
5008
5009 #[tokio::test]
5010 async fn reload_happens_before_renew_to_prevent_cross_process_races() {
5011 let server = MockServer::start().await;
5012 Mock::given(method("GET"))
5013 .and(path("/repos/acme/app"))
5014 .and(header("authorization", "Bearer ghu_initial"))
5015 .respond_with(ResponseTemplate::new(401))
5016 .expect(1)
5017 .mount(&server)
5018 .await;
5019 Mock::given(method("GET"))
5020 .and(path("/repos/acme/app"))
5021 .and(header("authorization", "Bearer ghu_reloaded"))
5022 .respond_with(ResponseTemplate::new(200).set_body_json(serde_json::json!({"id": 1})))
5023 .expect(1)
5024 .mount(&server)
5025 .await;
5026
5027 let renewal = Arc::new(SpyRenewal {
5028 invocations: std::sync::atomic::AtomicUsize::new(0),
5029 });
5030
5031 let document = serde_json::json!({
5032 "access_token": "ghu_initial",
5033 "refresh_token": "ghr_initial",
5034 });
5035 let initial_token =
5036 UserAccessToken::from_stored_document(&SecretString::from(document.to_string()));
5037
5038 let client = AuthenticatedClient::new(
5039 Endpoints::for_test_server(&server.uri()).unwrap(),
5040 initial_token,
5041 Arc::new(TestClock::default()),
5042 )
5043 .unwrap()
5044 .with_credential_source(Arc::new(StoreHolding(Some("ghu_reloaded"))))
5045 .with_renewal(renewal.clone());
5046
5047 let response = client
5048 .get_json::<serde_json::Value>("/repos/acme/app")
5049 .await
5050 .expect("the reloaded token should succeed");
5051 assert_eq!(response["id"], 1);
5052 assert_eq!(
5053 renewal
5054 .invocations
5055 .load(std::sync::atomic::Ordering::SeqCst),
5056 0,
5057 "renew should not be called because reload succeeded"
5058 );
5059 }
5060
5061 #[tokio::test]
5062 async fn proactive_renewal_waits_until_the_access_token_is_near_expiry() {
5063 let renewal = Arc::new(SpyRenewal {
5064 invocations: std::sync::atomic::AtomicUsize::new(0),
5065 });
5066 let document = serde_json::json!({
5067 "access_token": "ghu_initial",
5068 "refresh_token": "ghr_initial",
5069 "access_expires_at": "2026-08-21T02:00:00Z",
5070 "refresh_expires_at": "2027-02-21T00:00:00Z"
5071 });
5072 let clock = Arc::new(TestClock::default());
5073 let client = AuthenticatedClient::new(
5074 Endpoints::production(),
5075 UserAccessToken::from_stored_document(&SecretString::from(document.to_string())),
5076 clock.clone(),
5077 )
5078 .unwrap()
5079 .with_renewal(renewal.clone());
5080
5081 assert!(
5082 !client
5083 .renew_if_expiring_within(Duration::from_secs(30 * 60))
5084 .await
5085 );
5086 assert_eq!(
5087 renewal
5088 .invocations
5089 .load(std::sync::atomic::Ordering::SeqCst),
5090 0
5091 );
5092
5093 clock.advance_secs(91 * 60);
5094 assert!(
5095 client
5096 .renew_if_expiring_within(Duration::from_secs(30 * 60))
5097 .await
5098 );
5099 assert_eq!(
5100 renewal
5101 .invocations
5102 .load(std::sync::atomic::Ordering::SeqCst),
5103 1
5104 );
5105 }
5106
5107 #[tokio::test]
5108 async fn proactive_renewal_repairs_a_pair_with_no_expiry_metadata() {
5109 let renewal = Arc::new(SpyRenewal {
5110 invocations: std::sync::atomic::AtomicUsize::new(0),
5111 });
5112 let document = serde_json::json!({
5113 "access_token": "ghu_initial",
5114 "refresh_token": "ghr_initial"
5115 });
5116 let client = AuthenticatedClient::new(
5117 Endpoints::production(),
5118 UserAccessToken::from_stored_document(&SecretString::from(document.to_string())),
5119 Arc::new(TestClock::default()),
5120 )
5121 .unwrap()
5122 .with_renewal(renewal.clone());
5123
5124 assert!(
5125 client
5126 .renew_if_expiring_within(Duration::from_secs(30 * 60))
5127 .await
5128 );
5129 assert_eq!(
5130 renewal
5131 .invocations
5132 .load(std::sync::atomic::Ordering::SeqCst),
5133 1
5134 );
5135 }
5136}