runner_manager_github/lib.rs
1// owner: c2-device-flow-auth
2//
3// c2 also owns the shared authenticated HTTP client that lives at this crate
4// root; c3 owns `rest`, and c4 owns `demand` and `jit`.
5
6//! The GitHub gateway.
7//!
8//! This crate holds every line of code in the product that talks to GitHub, and
9//! the crate root holds the one client all of it goes through.
10//!
11//! * [`device_flow`] — the OAuth 2.0 Device Authorization Grant, which is the
12//! *only* way this product ever obtains a credential (D3, D16).
13//! * [`AuthenticatedClient`] — the shared `api.github.com` client. Every request
14//! in this crate is built by it, which is what makes "sets
15//! `X-GitHub-Api-Version` and an explicit `Accept`" a property of the design
16//! rather than of each call site, and what lets the authentication-failure
17//! taxonomy be implemented exactly once.
18//! * [`rest`], [`demand`], [`jit`] — typed adapters owned by `c3` and `c4`,
19//! built on [`AuthenticatedClient`].
20//!
21//! # Three properties this crate is required to keep
22//!
23//! **It holds no client secret, and it renews without one.** This paragraph
24//! used to say the opposite — that the App opts out of user-token expiration,
25//! that no renewal token is ever issued, and that renewing would require a
26//! client secret a public client cannot hold. All three were wrong, and a test
27//! in this crate enforced the error by forbidding the word "refresh token".
28//!
29//! The published App has user-token expiration **on**: a device-flow exchange
30//! returns `expires_in: 28800` and a `refresh_token`. GitHub requires the
31//! client secret to refresh *"unless the user access token was generated using
32//! the device flow"*, and every one of this product's is. So the credential
33//! renews itself, no server appears in the design, and the eight-hour life of
34//! an access token is invisible to a daemon that runs for months. See
35//! [`AuthenticatedClient::renew_once`], and
36//! `docs/spikes/token-expiry-and-renewal.md` for the two renewals that
37//! confirmed it on real hosts.
38//!
39//! [`AuthenticatedClient::revalidate`] is what still happens on a `401` for a
40//! credential with no refresh half — every one issued before 0.1.11.
41//!
42//! **It persists nothing.** [`device_flow::DeviceFlow::complete`] *returns* the
43//! token; it never writes it anywhere. The machine-scoped secret store is `d2`
44//! and the wiring is `f1`. That boundary is why this crate has no dependency on
45//! `runner-manager-platform` and performs no filesystem write outside its own
46//! tests — and it is what lets the whole gateway be tested with no platform
47//! dependency at all.
48//!
49//! **It never renders a secret.** The device code, the user access token, and
50//! every header carrying either are absent from `Debug`, from `Display`, from
51//! errors, and from tracing output. Every type here that holds one wraps it in
52//! [`secrecy::SecretString`] *and* implements [`fmt::Debug`] by hand, because a
53//! `#[derive(Debug)]` added later to a struct with a plain `String` field is
54//! precisely how this control is lost. `tests/no_secret_reaches_the_logs.rs`
55//! drives a whole login and an authenticated round trip through a capturing
56//! `tracing` subscriber and fails if any of the three appears.
57//!
58//! That scan is a **separate test binary**, and deliberately so. As a unit test
59//! it silently stopped working: `tracing` caches each callsite's `Interest`
60//! process-wide while `with_default` installs a subscriber on one *thread*, and
61//! run **concurrently** with the crate's other unit tests the scan captured
62//! only its own handful of events — passing with a real device-code leak on the
63//! live path. A binary holding one test has no concurrency to be poisoned by.
64//! The word "concurrently" is load-bearing and was measured;
65//! `tests/no_secret_reaches_the_logs.rs` records the numbers and what they rule
66//! out.
67
68pub mod conditional;
69pub mod demand;
70pub mod device_flow;
71pub mod jit;
72pub mod rest;
73pub mod traffic;
74
75use std::{
76 fmt,
77 sync::{
78 Arc,
79 atomic::{AtomicU64, Ordering},
80 },
81 time::Duration,
82};
83
84use chrono::{DateTime, Utc};
85use reqwest::{Method, StatusCode};
86use runner_manager_domain::model::{Clock, Org, OwnerRepo, Timestamp};
87
88/// Re-exported because [`GithubError::headers`] and [`ApiResponse::headers`]
89/// return one, and a consumer cannot *name* a type it has no path to.
90///
91/// `a1` owns every manifest in this workspace, so a crate outside this one that
92/// wanted to hold a `HeaderMap` from this seam would otherwise need `reqwest`
93/// added to its own dependencies — turning a `c2` seam into an `a1` change, and
94/// putting `reqwest`'s version in two places at once. [`GithubError::retry_after`]
95/// and [`GithubError::rate_limit`] exist precisely so that the common cases need
96/// no path at all; this is for the ones that do.
97///
98/// It is re-exported under its own name rather than an alias so that the type a
99/// consumer imports is the type the signatures already show.
100pub use reqwest::header::HeaderMap;
101use secrecy::{ExposeSecret, SecretString};
102use serde::{Deserialize, Serialize, de::DeserializeOwned};
103use url::Url;
104
105// ---------------------------------------------------------------------------
106// Constants
107// ---------------------------------------------------------------------------
108
109/// The REST API version every request pins.
110///
111/// `04-subsystem-contracts.md`: "All requests set `X-GitHub-Api-Version` and an
112/// explicit `Accept` header." Both spikes ran against this version.
113pub const GITHUB_API_VERSION: &str = "2022-11-28";
114
115/// The media type every request asks for, stated rather than defaulted.
116pub const GITHUB_ACCEPT: &str = "application/vnd.github+json";
117
118/// Production `api.github.com`.
119pub const GITHUB_API_BASE: &str = "https://api.github.com/";
120
121/// Production `github.com`, which hosts the device-flow endpoints. They are on
122/// the web host, not the API host.
123pub const GITHUB_WEB_BASE: &str = "https://github.com/";
124
125/// The canonical page a user types their code into.
126///
127/// This constant *is* the phishing control (`07-security.md`, threat table): the
128/// tool prints this URL and never proxies, embeds, or imitates the approval
129/// page. [`Endpoints::verification_url`] derives from the configured web base so
130/// a test server can be pointed at, and
131/// [`device_flow::DeviceAuthorization::verification_uri`] is checked against it
132/// so a response that tries to send the user somewhere else is rejected rather
133/// than displayed.
134pub const DEVICE_VERIFICATION_PATH: &str = "login/device";
135
136/// What a `401` re-validates the held credential against.
137///
138/// `GET /user/installations` rather than `GET /user`, because it is the call the
139/// D18 spike actually made with a user-to-server token and observed `200` from
140/// (`docs/spikes/d18-org-jit-verification.md`, "The permission that authorized
141/// it"), and because a successful re-validation then carries the same answer
142/// [`AuthenticatedClient::discover_installations`] needs.
143pub const REVALIDATION_PATH: &str = "/user/installations";
144
145/// How long a lockout backs off for when GitHub sends no `retry-after`.
146///
147/// `03-control-flows.md` flow 4.3 requires a back-off but names no duration.
148/// Sixty seconds is GitHub's own documented floor for its secondary rate limits.
149pub const DEFAULT_LOCKOUT_BACKOFF: Duration = Duration::from_secs(60);
150
151/// The longest a lockout may silence this client, whatever `Retry-After` said.
152///
153/// A back-off is a *safety* mechanism, and an unclamped one is a denial of
154/// service with extra steps: `Retry-After: 86400` would latch a silent
155/// twenty-four-hour outage of the agent's reconciliation loop, clearable only by
156/// [`AuthenticatedClient::clear_lockout`]. Fifteen minutes is far longer than any
157/// back-off GitHub documents for the authentication lockout this latches on, and
158/// short enough that a hostile or simply wrong header cannot take the product
159/// down for a shift. Honouring a header without a ceiling is trusting a remote
160/// party with the product's availability.
161pub const MAX_LOCKOUT_BACKOFF: Duration = Duration::from_secs(15 * 60);
162
163/// The most pages either pagination loop follows before giving up.
164///
165/// A `Link: rel="next"` that points back at the page it arrived on — a proxy
166/// rewriting the header, or a bug at the other end — is an infinite loop inside
167/// the agent's reconciliation loop, which is the one place in this product that
168/// must not be able to wedge. At `per_page=100` this ceiling is ten thousand
169/// installations or repositories, past any real account by orders of magnitude,
170/// so it bounds the pathological case without truncating a legitimate one.
171pub const MAX_PAGES: usize = 100;
172
173/// Per-request ceiling, so one wedged connection cannot stall the agent's
174/// reconciliation loop forever.
175pub const DEFAULT_REQUEST_TIMEOUT: Duration = Duration::from_secs(30);
176
177/// The `User-Agent` GitHub requires on every API request.
178pub const USER_AGENT: &str = concat!("runner-manager/", env!("CARGO_PKG_VERSION"));
179
180// ---------------------------------------------------------------------------
181// The published App
182// ---------------------------------------------------------------------------
183
184/// The published GitHub App this product authenticates as (D3, D16).
185///
186/// Both fields are **public by design**. `07-security.md`'s credential inventory
187/// lists the `client_id` as "Not secret … may appear in logs and documentation",
188/// which is exactly what makes the device flow serverless: a public client
189/// cannot secure a client secret, and this design never tries to.
190///
191/// The concrete values are *not* compiled in here. Registering and publishing
192/// the App is Phase 0 of `06-migration-rollout.md` and has not happened, so
193/// there is no honest value to write; `f1` supplies both when it wires the CLI.
194/// Committing a placeholder that looked real would be worse than requiring the
195/// caller to pass one.
196#[derive(Debug, Clone, PartialEq, Eq)]
197pub struct AppRegistration {
198 client_id: String,
199 slug: String,
200}
201
202impl AppRegistration {
203 /// # Errors
204 /// An empty `client_id` or an empty `slug`.
205 pub fn new(client_id: impl Into<String>, slug: impl Into<String>) -> Result<Self, ConfigError> {
206 let client_id = client_id.into();
207 let slug = slug.into();
208 if client_id.trim().is_empty() {
209 return Err(ConfigError::Empty { what: "client_id" });
210 }
211 if slug.trim().is_empty() {
212 return Err(ConfigError::Empty { what: "app slug" });
213 }
214 Ok(Self { client_id, slug })
215 }
216
217 #[must_use]
218 pub fn client_id(&self) -> &str {
219 &self.client_id
220 }
221
222 #[must_use]
223 pub fn slug(&self) -> &str {
224 &self.slug
225 }
226
227 /// The canonical URL a user with no installation must visit.
228 ///
229 /// `03-control-flows.md` flow 1.1: "If the published App is not yet installed
230 /// on any repository, it prints the installation URL."
231 ///
232 /// # Panics
233 /// Never, for a registration built through [`AppRegistration::new`]: the slug
234 /// is non-empty and is percent-encoded into the path.
235 #[must_use]
236 pub fn install_url(&self, endpoints: &Endpoints) -> Url {
237 endpoints
238 .web_base
239 .join("apps/")
240 .and_then(|u| u.join(&format!("{}/", encode_path_segment(&self.slug))))
241 .and_then(|u| u.join("installations/new"))
242 .expect("a non-empty encoded slug always joins onto the web base")
243 }
244}
245
246fn encode_path_segment(raw: &str) -> String {
247 raw.chars()
248 .map(|c| {
249 if c.is_ascii_alphanumeric() || matches!(c, '-' | '_' | '.' | '~') {
250 c.to_string()
251 } else {
252 let mut buf = [0_u8; 4];
253 c.encode_utf8(&mut buf)
254 .as_bytes()
255 .iter()
256 .map(|b| format!("%{b:02X}"))
257 .collect()
258 }
259 })
260 .collect()
261}
262
263/// Where GitHub is.
264///
265/// Two bases rather than one, because the device grant lives on `github.com`
266/// while every API call lives on `api.github.com`. Tests point both at one
267/// `wiremock` server; the paths do not collide.
268#[derive(Debug, Clone, PartialEq, Eq)]
269pub struct Endpoints {
270 api_base: Url,
271 web_base: Url,
272}
273
274impl Endpoints {
275 /// Production GitHub.
276 ///
277 /// # Panics
278 /// Never: both constants are parsed at every call and are valid URLs.
279 #[must_use]
280 pub fn production() -> Self {
281 Self {
282 api_base: Url::parse(GITHUB_API_BASE).expect("GITHUB_API_BASE is a valid URL"),
283 web_base: Url::parse(GITHUB_WEB_BASE).expect("GITHUB_WEB_BASE is a valid URL"),
284 }
285 }
286
287 /// Both bases are normalised to end in `/` so that relative joins keep the
288 /// whole base path instead of replacing its last segment.
289 #[must_use]
290 pub fn new(api_base: Url, web_base: Url) -> Self {
291 Self {
292 api_base: with_trailing_slash(api_base),
293 web_base: with_trailing_slash(web_base),
294 }
295 }
296
297 /// Point every endpoint at one test server.
298 ///
299 /// # Errors
300 /// `root` not being a parseable absolute URL.
301 pub fn for_test_server(root: &str) -> Result<Self, ConfigError> {
302 let root = Url::parse(root).map_err(|_| ConfigError::Empty {
303 what: "test server URL",
304 })?;
305 Ok(Self::new(root.clone(), root))
306 }
307
308 #[must_use]
309 pub fn api_base(&self) -> &Url {
310 &self.api_base
311 }
312
313 #[must_use]
314 pub fn web_base(&self) -> &Url {
315 &self.web_base
316 }
317
318 /// # Panics
319 /// Never: the path is a constant and the base ends in `/`.
320 #[must_use]
321 pub fn device_code_url(&self) -> Url {
322 self.web_base
323 .join("login/device/code")
324 .expect("a constant path joins onto a normalised base")
325 }
326
327 /// # Panics
328 /// Never: the path is a constant and the base ends in `/`.
329 #[must_use]
330 pub fn access_token_url(&self) -> Url {
331 self.web_base
332 .join("login/oauth/access_token")
333 .expect("a constant path joins onto a normalised base")
334 }
335
336 /// The canonical page the user code is typed into, and the only device-flow
337 /// URL this product ever prints.
338 ///
339 /// # Panics
340 /// Never: the path is a constant and the base ends in `/`.
341 #[must_use]
342 pub fn verification_url(&self) -> Url {
343 self.web_base
344 .join(DEVICE_VERIFICATION_PATH)
345 .expect("a constant path joins onto a normalised base")
346 }
347}
348
349impl Default for Endpoints {
350 fn default() -> Self {
351 Self::production()
352 }
353}
354
355fn with_trailing_slash(mut url: Url) -> Url {
356 if !url.path().ends_with('/') {
357 let path = format!("{}/", url.path());
358 url.set_path(&path);
359 }
360 url
361}
362
363/// A configuration value this crate refuses to start with.
364#[derive(Debug, Clone, PartialEq, Eq, thiserror::Error)]
365pub enum ConfigError {
366 #[error("{what} must not be empty")]
367 Empty { what: &'static str },
368}
369
370// ---------------------------------------------------------------------------
371// The credential
372// ---------------------------------------------------------------------------
373
374/// A user access token obtained from the device flow.
375///
376/// **Non-expiring, and non-renewable.** The published App opts out of
377/// user-token expiration, so GitHub issues no renewal token alongside this one
378/// and there is nothing to renew — see the crate documentation. The token is
379/// invalidated only by the user uninstalling the App or revoking the
380/// authorization at GitHub.
381///
382/// `Debug` is written by hand. Deriving it here would put the token into every
383/// `tracing` field, every `unwrap()` panic message, and every `anyhow` chain
384/// that ever carries one, which is the exact leak `07-security.md` gates on.
385#[derive(Clone)]
386pub struct UserAccessToken {
387 token: SecretString,
388 token_type: String,
389 scope: Option<String>,
390 /// The renewal half, when the App issues one.
391 ///
392 /// # Why this is an `Option` rather than a second type
393 ///
394 /// Whether a credential can renew itself is a setting on the *App*, not a
395 /// property of this build: an App with user-token expiration off returns an
396 /// access token and nothing else, and one with it on returns a pair. Both
397 /// shapes reach this type, and a host holding either must keep working --
398 /// otherwise flipping that setting would strand every installation that had
399 /// not upgraded, which for a published product is an outage nobody asked
400 /// for.
401 ///
402 /// `None` is therefore not a defect. It is the credential this product held
403 /// for its whole life until now: non-expiring, unrenewable, replaced only
404 /// by an interactive `auth login`.
405 renewal: Option<Renewal>,
406}
407
408/// What a token needs in order to replace itself without a person.
409#[derive(Clone)]
410pub struct Renewal {
411 refresh_token: SecretString,
412 /// When the access token stops being accepted, if it was stated.
413 pub access_expires_at: Option<DateTime<Utc>>,
414 /// When the *refresh* token stops working. Past this, only an interactive
415 /// sign-in helps -- and it is six months from the last renewal, not from
416 /// the first sign-in, so a host that runs at all never reaches it.
417 pub refresh_expires_at: Option<DateTime<Utc>>,
418}
419
420impl Renewal {
421 /// The refresh token itself. Kept behind a method for the same reason the
422 /// access token is: it is the more dangerous half of the pair, because it
423 /// mints access tokens indefinitely.
424 #[must_use]
425 pub fn refresh_token(&self) -> &SecretString {
426 &self.refresh_token
427 }
428}
429
430impl fmt::Debug for Renewal {
431 /// Written by hand, like [`UserAccessToken`]'s, and for a stronger reason:
432 /// a leaked refresh token does not expire in eight hours.
433 fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
434 f.debug_struct("Renewal")
435 .field("refresh_token", &"[redacted]")
436 .field("access_expires_at", &self.access_expires_at)
437 .field("refresh_expires_at", &self.refresh_expires_at)
438 .finish()
439 }
440}
441
442impl UserAccessToken {
443 #[must_use]
444 pub fn new(token: SecretString) -> Self {
445 Self {
446 token,
447 token_type: "bearer".to_string(),
448 scope: None,
449 renewal: None,
450 }
451 }
452
453 /// The whole credential as the token endpoint returned it. `pub(crate)`
454 /// because [`device_flow`] is the only thing in the product entitled to mint
455 /// one — every other path receives a token rather than constructing it.
456 pub(crate) fn from_parts(
457 token: SecretString,
458 token_type: String,
459 scope: Option<String>,
460 ) -> Self {
461 Self {
462 token,
463 token_type,
464 scope,
465 renewal: None,
466 }
467 }
468
469 /// Attach the renewal half, if the App issued one.
470 ///
471 /// The two durations are seconds-from-now as GitHub states them, turned
472 /// into instants here so that nothing downstream has to remember when
473 /// "now" was.
474 #[must_use]
475 pub(crate) fn with_renewal(
476 mut self,
477 refresh_token: Option<SecretString>,
478 access_expires_in: Option<u64>,
479 refresh_expires_in: Option<u64>,
480 ) -> Self {
481 self.renewal = refresh_token.map(|refresh_token| {
482 let at = |secs: Option<u64>| {
483 secs.and_then(|s| i64::try_from(s).ok())
484 .and_then(|s| Utc::now().checked_add_signed(chrono::TimeDelta::seconds(s)))
485 };
486 Renewal {
487 refresh_token,
488 access_expires_at: at(access_expires_in),
489 refresh_expires_at: at(refresh_expires_in),
490 }
491 });
492 self
493 }
494
495 /// The renewal half, when there is one.
496 #[must_use]
497 pub fn renewal(&self) -> Option<&Renewal> {
498 self.renewal.as_ref()
499 }
500
501 /// Rebuild the credential `d2` handed back, for `f1`.
502 #[must_use]
503 pub fn from_stored(token: SecretString) -> Self {
504 Self::from_stored_document(&token)
505 }
506
507 /// Reads whichever of the two stored shapes is there.
508 ///
509 /// # Why the store holds a document now, and why the old shape still loads
510 ///
511 /// A renewable credential is three values -- access token, refresh token,
512 /// and when each stops working -- where there used to be one string. The
513 /// secret store takes one opaque value per host, so the document goes
514 /// inside it rather than the store growing a schema: no platform change, no
515 /// migration step, and the same DPAPI blob or keychain item as before.
516 ///
517 /// **A value that is not this document is a bare access token**, which is
518 /// what every host stored until now. That is not a fallback for tidiness:
519 /// upgrading must not log anybody out, and the App's expiration setting can
520 /// be turned on -- or back off -- without stranding hosts that are mid-way
521 /// through either. A token has no internal structure to confuse with JSON,
522 /// so the discrimination is unambiguous.
523 #[must_use]
524 pub fn from_stored_document(stored: &SecretString) -> Self {
525 #[derive(Deserialize)]
526 struct Document {
527 access_token: String,
528 #[serde(default)]
529 refresh_token: Option<String>,
530 #[serde(default)]
531 access_expires_at: Option<DateTime<Utc>>,
532 #[serde(default)]
533 refresh_expires_at: Option<DateTime<Utc>>,
534 }
535
536 match serde_json::from_str::<Document>(stored.expose_secret()) {
537 Ok(document) => Self {
538 token: SecretString::from(document.access_token),
539 token_type: "bearer".to_string(),
540 scope: None,
541 renewal: document.refresh_token.map(|refresh_token| Renewal {
542 refresh_token: SecretString::from(refresh_token),
543 access_expires_at: document.access_expires_at,
544 refresh_expires_at: document.refresh_expires_at,
545 }),
546 },
547 Err(_) => Self::new(stored.clone()),
548 }
549 }
550
551 /// The value to hand the secret store.
552 ///
553 /// Always the document, even for a credential with no renewal half: one
554 /// shape written means one shape to reason about, and reading still accepts
555 /// the bare token that older versions wrote.
556 #[must_use]
557 pub fn to_stored_document(&self) -> SecretString {
558 #[derive(Serialize)]
559 struct Document<'a> {
560 access_token: &'a str,
561 #[serde(skip_serializing_if = "Option::is_none")]
562 refresh_token: Option<&'a str>,
563 #[serde(skip_serializing_if = "Option::is_none")]
564 access_expires_at: Option<DateTime<Utc>>,
565 #[serde(skip_serializing_if = "Option::is_none")]
566 refresh_expires_at: Option<DateTime<Utc>>,
567 }
568
569 let document = Document {
570 access_token: self.token.expose_secret(),
571 refresh_token: self
572 .renewal
573 .as_ref()
574 .map(|r| r.refresh_token.expose_secret()),
575 access_expires_at: self.renewal.as_ref().and_then(|r| r.access_expires_at),
576 refresh_expires_at: self.renewal.as_ref().and_then(|r| r.refresh_expires_at),
577 };
578 // Serialising a struct of `&str` cannot fail; the fallback keeps the
579 // access token usable rather than inventing an error path nobody can
580 // act on.
581 SecretString::from(
582 serde_json::to_string(&document)
583 .unwrap_or_else(|_| self.token.expose_secret().to_string()),
584 )
585 }
586
587 /// The token itself. Every call site of this is a place a secret can escape,
588 /// so there are deliberately few: the `Authorization` header, and `d2`'s
589 /// store call.
590 #[must_use]
591 pub fn secret(&self) -> &SecretString {
592 &self.token
593 }
594
595 #[must_use]
596 pub fn token_type(&self) -> &str {
597 &self.token_type
598 }
599
600 #[must_use]
601 pub fn scope(&self) -> Option<&str> {
602 self.scope.as_deref()
603 }
604
605 /// The token's four-character family prefix — `ghu_` for an App
606 /// user-to-server token — and nothing else.
607 ///
608 /// This exists so diagnostics can answer "did the device flow return the
609 /// kind of token we expected?" without exposing the token. The D17 spike
610 /// asserted exactly this and no more.
611 #[must_use]
612 pub fn family(&self) -> &str {
613 let raw = self.token.expose_secret();
614 match raw.find('_') {
615 Some(idx) if idx < 8 => &raw[..=idx],
616 _ => "",
617 }
618 }
619
620 /// `true` for the `ghu_` family the published App issues.
621 #[must_use]
622 pub fn is_user_to_server(&self) -> bool {
623 self.family() == "ghu_"
624 }
625}
626
627impl fmt::Debug for UserAccessToken {
628 fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
629 f.debug_struct("UserAccessToken")
630 .field("token", &"[REDACTED]")
631 .field("family", &self.family())
632 .finish_non_exhaustive()
633 }
634}
635
636impl PartialEq for UserAccessToken {
637 /// Equality exists so [`device_flow::PollOutcome`] can carry a token and
638 /// still be compared in a test. Production code never compares two
639 /// credentials, and this is not a constant-time comparison.
640 fn eq(&self, other: &Self) -> bool {
641 self.token.expose_secret() == other.token.expose_secret()
642 && self.token_type == other.token_type
643 && self.scope == other.scope
644 }
645}
646
647impl Eq for UserAccessToken {}
648
649// ---------------------------------------------------------------------------
650// Errors
651// ---------------------------------------------------------------------------
652
653/// Everything [`AuthenticatedClient`] can fail with.
654///
655/// The first three variants are the taxonomy `03-control-flows.md` flow 4.3
656/// requires, and they are separate variants because `c3` and `f1` both act on
657/// the distinction: [`GithubError::AuthenticationFailed`] moves a policy to
658/// `authentication_failed` and tells the operator to run `auth login`,
659/// [`GithubError::AuthenticationLockout`] must *wait* and tell the operator
660/// nothing is wrong with their credential, and [`GithubError::Forbidden`] is a
661/// permissions answer that re-authenticating will not change.
662///
663/// # Why the failing variants carry response headers
664///
665/// Rate-limit *policy* is `c3`'s and is deliberately not implemented here. But a
666/// policy needs evidence, and the evidence — `retry-after`,
667/// `x-ratelimit-remaining`, `x-ratelimit-reset` — only exists on the response
668/// that failed. An error taxonomy that dropped those headers would leave `c3`
669/// with no way to honour a `429` except by editing this file, which is exactly
670/// the conflict the `c2`/`c3` ownership split exists to prevent. So
671/// [`GithubError::Status`] and [`GithubError::Forbidden`] carry the headers
672/// verbatim and interpret none of them; see [`GithubError::headers`].
673#[derive(Debug, thiserror::Error)]
674pub enum GithubError {
675 /// GitHub rejected the credential and a single re-validation confirmed it.
676 /// Terminal: only an interactive `auth login` clears this.
677 #[error(
678 "GitHub rejected the stored credential; run `runner-manager auth login` to sign in again"
679 )]
680 AuthenticationFailed,
681
682 /// GitHub answered `403` after `401`s — its temporary authentication
683 /// lockout, not a permissions change. Back off; do not re-authenticate and
684 /// do not retry.
685 #[error(
686 "GitHub has temporarily locked out authentication for this credential; \
687 back off for {}s and do not retry — the credential itself is not the problem",
688 retry_after.as_secs()
689 )]
690 AuthenticationLockout { retry_after: Duration },
691
692 /// A `403` that is not the lockout: a permissions answer, or GitHub's own
693 /// rate limit. `c3` tells the two apart from `headers`; this crate does not,
694 /// because which of them is worth retrying is rate-limit policy.
695 #[error(
696 "GitHub denied {method} {path}: the App installation does not grant it{}",
697 message.as_deref().map(|m| format!(" ({m})")).unwrap_or_default()
698 )]
699 Forbidden {
700 method: String,
701 path: String,
702 message: Option<String>,
703 /// The response headers, verbatim and uninterpreted.
704 headers: Box<HeaderMap>,
705 },
706
707 #[error(
708 "GitHub returned {status} for {method} {path}{}",
709 message.as_deref().map(|m| format!(": {m}")).unwrap_or_default()
710 )]
711 Status {
712 status: u16,
713 method: String,
714 path: String,
715 message: Option<String>,
716 /// The response headers, verbatim and uninterpreted. A `429` reaches
717 /// `c3` through this variant, and its `retry-after` survives with it.
718 headers: Box<HeaderMap>,
719 },
720
721 /// The request never got an answer. The URL is stripped from the source
722 /// error before it is stored: a device-flow URL never carries a secret, but
723 /// stripping it costs nothing and removes a whole class of future leak.
724 #[error("GitHub was unreachable")]
725 Transport(#[source] reqwest::Error),
726
727 #[error("a {what} response from GitHub could not be decoded as {expected}")]
728 Decode {
729 what: &'static str,
730 expected: &'static str,
731 #[source]
732 source: serde_json::Error,
733 },
734
735 #[error("GitHub returned {value:?} for {what}, which this client cannot use")]
736 Malformed { what: &'static str, value: String },
737
738 #[error(transparent)]
739 Config(#[from] ConfigError),
740}
741
742impl GithubError {
743 /// `true` for the two authentication outcomes, which callers handle
744 /// differently from every other failure.
745 #[must_use]
746 pub fn is_authentication(&self) -> bool {
747 matches!(
748 self,
749 Self::AuthenticationFailed | Self::AuthenticationLockout { .. }
750 )
751 }
752
753 /// `true` only for the lockout, which is the one authentication outcome that
754 /// resolves by waiting rather than by signing in again.
755 #[must_use]
756 pub fn is_lockout(&self) -> bool {
757 matches!(self, Self::AuthenticationLockout { .. })
758 }
759
760 /// The failing response's headers, for the variants that have them.
761 ///
762 /// This is the whole of `c2`'s contribution to rate limiting: it hands `c3`
763 /// the evidence and stops there. Nothing in this crate reads
764 /// `x-ratelimit-remaining` to decide anything.
765 #[must_use]
766 pub fn headers(&self) -> Option<&HeaderMap> {
767 match self {
768 Self::Status { headers, .. } | Self::Forbidden { headers, .. } => Some(headers),
769 _ => None,
770 }
771 }
772
773 /// The failing response's `retry-after`, in seconds, if it sent one.
774 ///
775 /// Reading a documented header is evidence, not policy: what to *do* with a
776 /// `retry-after` — wait, shed load, surface it to an operator — is `c3`'s.
777 #[must_use]
778 pub fn retry_after(&self) -> Option<Duration> {
779 self.headers().and_then(retry_after)
780 }
781
782 /// The `x-ratelimit-remaining` / `x-ratelimit-reset` pair, when present.
783 ///
784 /// Returned as the raw numbers GitHub sent. `reset` is a Unix timestamp in
785 /// seconds, which is what the header carries; it is deliberately not turned
786 /// into a [`Timestamp`] here, because comparing it against a clock is the
787 /// first step of a policy decision and that decision is `c3`'s.
788 #[must_use]
789 pub fn rate_limit(&self) -> Option<RateLimitEvidence> {
790 let headers = self.headers()?;
791 let read = |name: &str| {
792 headers
793 .get(name)
794 .and_then(|v| v.to_str().ok())
795 .and_then(|v| v.trim().parse::<u64>().ok())
796 };
797 let remaining = read("x-ratelimit-remaining");
798 let reset = read("x-ratelimit-reset");
799 if remaining.is_none() && reset.is_none() {
800 return None;
801 }
802 Some(RateLimitEvidence {
803 remaining,
804 reset_unix_secs: reset,
805 retry_after: self.retry_after(),
806 })
807 }
808}
809
810/// What GitHub said about its own rate limit on a response that failed.
811///
812/// Evidence, carried across the `c2`/`c3` seam. Every field is what the wire
813/// said, and none of them has been interpreted.
814#[derive(Debug, Clone, Copy, PartialEq, Eq)]
815pub struct RateLimitEvidence {
816 /// `x-ratelimit-remaining`. Zero is GitHub's primary rate limit.
817 pub remaining: Option<u64>,
818 /// `x-ratelimit-reset`, a Unix timestamp in seconds.
819 pub reset_unix_secs: Option<u64>,
820 /// `retry-after`, which secondary rate limits send instead.
821 pub retry_after: Option<Duration>,
822}
823
824fn transport(err: reqwest::Error) -> GithubError {
825 GithubError::Transport(err.without_url())
826}
827
828// ---------------------------------------------------------------------------
829// Requests and responses
830// ---------------------------------------------------------------------------
831
832/// One `api.github.com` request, before authentication headers are applied.
833///
834/// `Debug` is written by hand and never renders the body: `c4` posts
835/// `generate-jitconfig` requests through this type, and a JIT configuration is
836/// a sensitive short-lived value (`07-security.md`, credential inventory).
837#[derive(Clone)]
838pub struct ApiRequest {
839 method: Method,
840 /// Either a path relative to the API base, or an absolute URL — which is
841 /// what a `Link: rel="next"` page is.
842 path: String,
843 query: Vec<(String, String)>,
844 body: Option<serde_json::Value>,
845 /// What to send as `If-None-Match` / `If-Modified-Since`, when this is a
846 /// conditional `GET`. See [`crate::conditional`].
847 validators: Option<Validators>,
848}
849
850impl ApiRequest {
851 #[must_use]
852 pub fn get(path: impl Into<String>) -> Self {
853 Self::new(Method::GET, path)
854 }
855
856 #[must_use]
857 pub fn delete(path: impl Into<String>) -> Self {
858 Self::new(Method::DELETE, path)
859 }
860
861 #[must_use]
862 pub fn new(method: Method, path: impl Into<String>) -> Self {
863 Self {
864 method,
865 path: path.into(),
866 query: Vec::new(),
867 body: None,
868 validators: None,
869 }
870 }
871
872 /// # Errors
873 /// `body` failing to serialize.
874 pub fn post_json<T: Serialize>(path: impl Into<String>, body: &T) -> Result<Self, GithubError> {
875 let value = serde_json::to_value(body).map_err(|source| GithubError::Decode {
876 what: "request",
877 expected: "JSON",
878 source,
879 })?;
880 Ok(Self {
881 method: Method::POST,
882 path: path.into(),
883 query: Vec::new(),
884 body: Some(value),
885 validators: None,
886 })
887 }
888
889 #[must_use]
890 pub fn query(mut self, key: impl Into<String>, value: impl fmt::Display) -> Self {
891 self.query.push((key.into(), value.to_string()));
892 self
893 }
894
895 /// Make this a conditional request: GitHub answers `304 Not Modified`,
896 /// with no body, while the resource still matches `validators`.
897 ///
898 /// Only meaningful on a `GET`. GitHub: "Conditional requests for unsafe
899 /// methods, such as `POST`, `PUT`, `PATCH`, and `DELETE` are not supported
900 /// unless otherwise noted in the documentation."
901 #[must_use]
902 pub fn conditional(mut self, validators: Validators) -> Self {
903 self.validators = (!validators.is_empty()).then_some(validators);
904 self
905 }
906
907 /// Whether this request carries validators, and so may be answered `304`.
908 #[must_use]
909 pub const fn is_conditional(&self) -> bool {
910 self.validators.is_some()
911 }
912
913 /// The resource this request reads: method, path, and query in the order
914 /// it was added. An `ETag` is scoped to exactly this — GitHub: "Use the
915 /// same parameters every time you poll the same data. A different page
916 /// size, page number, or filter produces a different response with a
917 /// different `etag`."
918 #[must_use]
919 pub fn cache_key(&self) -> String {
920 let mut key = format!("{} {}", self.method.as_str(), self.path);
921 for (index, (name, value)) in self.query.iter().enumerate() {
922 key.push(if index == 0 { '?' } else { '&' });
923 key.push_str(name);
924 key.push('=');
925 key.push_str(value);
926 }
927 key
928 }
929
930 #[must_use]
931 pub fn method(&self) -> &Method {
932 &self.method
933 }
934
935 #[must_use]
936 pub fn path(&self) -> &str {
937 &self.path
938 }
939}
940
941impl fmt::Debug for ApiRequest {
942 fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
943 f.debug_struct("ApiRequest")
944 .field("method", &self.method.as_str())
945 .field("path", &self.path)
946 .field(
947 "query_keys",
948 &self.query.iter().map(|(k, _)| k).collect::<Vec<_>>(),
949 )
950 .field(
951 "body",
952 &self.body.as_ref().map_or("none", |_| "[REDACTED JSON]"),
953 )
954 .field("conditional", &self.validators.is_some())
955 .finish()
956 }
957}
958
959/// A response's cache validators: its `ETag` and `Last-Modified`.
960///
961/// Sent back verbatim, a weak `W/"…"` tag included, because that is what
962/// GitHub compares against. Measured on `api.github.com` before this was
963/// written: the workflow-run, job and runner listings each return a weak
964/// `ETag`, and an `If-None-Match` carrying it is answered `304` for as long as
965/// the listing is unchanged.
966#[derive(Debug, Clone, Default, PartialEq, Eq)]
967pub struct Validators {
968 pub etag: Option<String>,
969 pub last_modified: Option<String>,
970}
971
972impl Validators {
973 /// The validators a response carries, if any.
974 #[must_use]
975 pub fn of(response: &ApiResponse) -> Self {
976 let read = |name: &str| {
977 response
978 .header(name)
979 .map(str::trim)
980 .filter(|value| !value.is_empty())
981 .map(str::to_string)
982 };
983 Self {
984 etag: read("etag"),
985 last_modified: read("last-modified"),
986 }
987 }
988
989 #[must_use]
990 pub const fn is_empty(&self) -> bool {
991 self.etag.is_none() && self.last_modified.is_none()
992 }
993}
994
995/// One buffered `api.github.com` response.
996///
997/// Buffered rather than streamed because every API response this crate reads is
998/// small JSON. The one large download in the product — the runner package — is
999/// `e2`'s and uses its own streaming client.
1000///
1001/// `Debug` renders the status and the body's *length*, never the body: a
1002/// `generate-jitconfig` response body is an encoded JIT configuration.
1003#[derive(Clone)]
1004pub struct ApiResponse {
1005 status: StatusCode,
1006 headers: HeaderMap,
1007 body: Vec<u8>,
1008}
1009
1010impl ApiResponse {
1011 #[must_use]
1012 pub fn status(&self) -> StatusCode {
1013 self.status
1014 }
1015
1016 /// `304 Not Modified`: the answer to a conditional request whose resource
1017 /// has not changed. It has no body; [`crate::conditional`] substitutes the
1018 /// cached one.
1019 #[must_use]
1020 pub fn is_not_modified(&self) -> bool {
1021 self.status == StatusCode::NOT_MODIFIED
1022 }
1023
1024 #[must_use]
1025 pub fn headers(&self) -> &HeaderMap {
1026 &self.headers
1027 }
1028
1029 #[must_use]
1030 pub fn header(&self, name: &str) -> Option<&str> {
1031 self.headers.get(name).and_then(|v| v.to_str().ok())
1032 }
1033
1034 /// # Errors
1035 /// A body that is not the expected JSON shape.
1036 pub fn json<T: DeserializeOwned>(&self) -> Result<T, GithubError> {
1037 serde_json::from_slice(&self.body).map_err(|source| GithubError::Decode {
1038 what: "response",
1039 expected: std::any::type_name::<T>(),
1040 source,
1041 })
1042 }
1043
1044 /// The next page of a paginated collection, from the `Link` header.
1045 ///
1046 /// `04-subsystem-contracts.md`: "Pagination is mandatory; the dashboard must
1047 /// not treat a first page as a complete inventory." It lives on the shared
1048 /// response type so that `c3`'s inventory and this module's installation
1049 /// discovery cannot disagree about how a `Link` header is read.
1050 #[must_use]
1051 pub fn next_page(&self) -> Option<Url> {
1052 self.header("link").and_then(parse_link_next)
1053 }
1054}
1055
1056impl fmt::Debug for ApiResponse {
1057 fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
1058 f.debug_struct("ApiResponse")
1059 .field("status", &self.status.as_u16())
1060 .field("body_bytes", &self.body.len())
1061 .finish_non_exhaustive()
1062 }
1063}
1064
1065/// The `rel="next"` target of an RFC 8288 `Link` header, or `None`.
1066///
1067/// # Why this scans rather than splits
1068///
1069/// A comma separates one link-value from the next, but a comma is also a legal
1070/// character *inside* a URL, and GitHub sends such URLs routinely — a runner
1071/// query carries `labels=self-hosted,windows`. Splitting the whole header on `,`
1072/// first tears that URL in half, neither half parses as `<...>`, and the
1073/// relation is silently lost. The caller then treats page 1 as the whole
1074/// inventory, which is the specific outcome `04-subsystem-contracts.md` forbids:
1075/// "the dashboard must not treat a first page as a complete inventory".
1076///
1077/// So the target is located by its `<`…`>` delimiters, and only a comma that
1078/// actually begins the next link-value — one followed by optional whitespace and
1079/// `<` — ends the parameter section.
1080fn parse_link_next(link: &str) -> Option<Url> {
1081 let mut rest = link;
1082 while let Some(open) = rest.find('<') {
1083 let after_open = &rest[open + 1..];
1084 let Some(close) = after_open.find('>') else {
1085 // An unterminated `<` cannot be a link-value; nothing after it is
1086 // interpretable either.
1087 return None;
1088 };
1089 let target = after_open[..close].trim();
1090 let tail = &after_open[close + 1..];
1091
1092 // The parameters run to the start of the next link-value.
1093 let cut = tail
1094 .match_indices(',')
1095 .find(|(i, _)| tail[i + 1..].trim_start().starts_with('<'))
1096 .map_or(tail.len(), |(i, _)| i);
1097 let (params, next) = tail.split_at(cut);
1098
1099 let is_next = params.split(';').any(|param| {
1100 let param = param.trim().replace(['"', '\''], "");
1101 param.eq_ignore_ascii_case("rel=next")
1102 });
1103 if is_next {
1104 return Url::parse(target).ok();
1105 }
1106 rest = next.strip_prefix(',').unwrap_or(next);
1107 }
1108 None
1109}
1110
1111#[derive(Debug, Deserialize)]
1112struct ErrorEnvelope {
1113 message: Option<String>,
1114}
1115
1116/// GitHub's own message for a failure, and never the raw body.
1117fn error_message(body: &[u8]) -> Option<String> {
1118 serde_json::from_slice::<ErrorEnvelope>(body)
1119 .ok()
1120 .and_then(|e| e.message)
1121 .filter(|m| !m.is_empty())
1122}
1123
1124fn retry_after(headers: &HeaderMap) -> Option<Duration> {
1125 headers
1126 .get("retry-after")
1127 .and_then(|v| v.to_str().ok())
1128 .and_then(|v| v.trim().parse::<u64>().ok())
1129 .map(Duration::from_secs)
1130}
1131
1132// ---------------------------------------------------------------------------
1133// Sleeping
1134// ---------------------------------------------------------------------------
1135
1136/// The one way anything in this crate waits.
1137///
1138/// A port rather than a direct `tokio::time::sleep`, for the same reason the
1139/// domain has a `Clock`: the device flow's `slow_down` handling is a *timing*
1140/// behaviour, and a timing behaviour tested by actually waiting is either
1141/// untested or slow. A test substitutes a sleeper that records the requested
1142/// durations and returns immediately, which turns "`slow_down` increases the
1143/// poll interval" into an equality assertion on a `Vec<Duration>` rather than a
1144/// stopwatch reading.
1145#[async_trait::async_trait]
1146pub trait Sleeper: Send + Sync + fmt::Debug {
1147 async fn sleep(&self, duration: Duration);
1148}
1149
1150/// The production adapter.
1151#[derive(Debug, Clone, Copy, Default)]
1152pub struct TokioSleeper;
1153
1154#[async_trait::async_trait]
1155impl Sleeper for TokioSleeper {
1156 async fn sleep(&self, duration: Duration) {
1157 tokio::time::sleep(duration).await;
1158 }
1159}
1160
1161// ---------------------------------------------------------------------------
1162// The shared authenticated client
1163// ---------------------------------------------------------------------------
1164
1165/// What a single re-validation of the held credential concluded.
1166#[derive(Debug, Clone, Copy, PartialEq, Eq)]
1167pub enum Revalidation {
1168 /// GitHub still accepts the credential, so the `401` was about the request
1169 /// rather than about the token. One retry is warranted.
1170 Valid,
1171 /// GitHub rejects the credential. Terminal; only `auth login` clears it.
1172 Rejected,
1173 /// The probe itself could not be completed — GitHub was unreachable, or
1174 /// answered something neither `2xx` nor `401`. Nothing was learned, so the
1175 /// caller still gets its one retry.
1176 Unavailable,
1177}
1178
1179#[derive(Debug)]
1180struct LockoutState {
1181 until: Option<Timestamp>,
1182 backoff: Duration,
1183}
1184
1185/// The one client every `api.github.com` request in this crate goes through.
1186///
1187/// It exists to make four things structural rather than remembered:
1188///
1189/// 1. `X-GitHub-Api-Version`, `Accept`, `User-Agent`, and `Authorization` are
1190/// set on every request because they are set *here*.
1191/// 2. The `401` / `403` taxonomy of `03-control-flows.md` flow 4.3 is
1192/// implemented once. `c3` and `f1` both branch on the distinction, and two
1193/// implementations of it would eventually disagree.
1194/// 3. A `401` storm produces **one** credential re-validation, not one per
1195/// caller — see [`AuthenticatedClient::revalidate`].
1196/// 4. A lockout stops traffic. Once GitHub answers `403` after `401`s, this
1197/// client issues no further HTTP at all until the back-off elapses.
1198pub struct AuthenticatedClient {
1199 http: reqwest::Client,
1200 endpoints: Endpoints,
1201 /// Swappable, because a renewable credential replaces itself while this
1202 /// client is in use. A `Mutex` rather than a lock-free cell: it is read
1203 /// once per request and written once every eight hours, so contention is
1204 /// not the concern -- being obviously correct is.
1205 credential: std::sync::Mutex<UserAccessToken>,
1206 /// Where to re-read the credential when a `401` outlives renewal.
1207 ///
1208 /// `None` for a short-lived client, which is every one that is not the
1209 /// daemon's: a command that runs for a second cannot be outlived by a
1210 /// sign-in.
1211 source: Option<Arc<dyn CredentialSource>>,
1212 /// How a credential replaces itself, when it can.
1213 ///
1214 /// `None` for a client whose credential has no renewal half, which is every
1215 /// client until an App turns user-token expiration on, and for the paths
1216 /// that hold a token for one call and never outlive its eight hours.
1217 renewal: Option<Arc<dyn CredentialRenewal>>,
1218 clock: Arc<dyn Clock>,
1219
1220 /// Bumped once per completed re-validation. A caller that took a `401`
1221 /// samples this *before* queuing on the gate; if it changed while the caller
1222 /// waited, some other caller already did the work and this one must not
1223 /// repeat it. This is the whole single-flight mechanism.
1224 revalidation_generation: AtomicU64,
1225 revalidation_gate: tokio::sync::Mutex<()>,
1226 last_revalidation: std::sync::Mutex<Revalidation>,
1227 revalidations_performed: AtomicU64,
1228
1229 /// `401`s seen since the last successful caller response.
1230 ///
1231 /// **Nothing in production reads this.** It is incremented on every `401`
1232 /// and cleared by a successful caller response, and that is the whole of
1233 /// what it does. It used to be documented as the lockout's test — "a `403`
1234 /// while this is non-zero is a lockout" — and
1235 /// [`AuthenticatedClient::is_lockout_403`] stopped consulting it when that
1236 /// rule was replaced by position plus GitHub's own evidence:
1237 /// [`Attempt::Retry`] already implies this request's own `401` incremented
1238 /// it moments ago, so reading it added no signal, and it did add a race that
1239 /// failed open — a concurrent success clearing the count downgraded a real
1240 /// lockout to a permissions answer.
1241 ///
1242 /// The field is kept on purpose, and only the tests read it: they drive it
1243 /// to values that *would* change the answer if it were still consulted, and
1244 /// assert that the answer does not change. Deleting it would delete the
1245 /// ability to make that assertion, which is the only thing standing between
1246 /// the conjunct and its reintroduction. See
1247 /// `a_concurrent_success_cannot_downgrade_a_lockout_to_a_permissions_answer`.
1248 consecutive_unauthorized: AtomicU64,
1249 lockout: std::sync::Mutex<LockoutState>,
1250 /// Every response this client received, counted. Observation only:
1251 /// nothing in this client decides anything from it. See
1252 /// [`crate::traffic`].
1253 traffic: traffic::ApiTraffic,
1254}
1255
1256impl fmt::Debug for AuthenticatedClient {
1257 fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
1258 // `try_lock`, not `is_locked_out()`. `std::sync::Mutex` is not
1259 // reentrant, and `latch_lockout` holds this one; the moment anybody adds
1260 // a `tracing` call inside that function — which is a natural thing to
1261 // want there — rendering the client would deadlock the whole agent. A
1262 // `Debug` impl must never be able to block, so it reports what it can
1263 // see and says so when it cannot.
1264 let locked_out = match self.lockout.try_lock() {
1265 Ok(state) => {
1266 if state.until.is_some_and(|until| self.clock.now() < until) {
1267 "yes"
1268 } else {
1269 "no"
1270 }
1271 }
1272 Err(_) => "unknown (the lockout state is being updated)",
1273 };
1274 f.debug_struct("AuthenticatedClient")
1275 .field("api_base", &self.endpoints.api_base.as_str())
1276 .field(
1277 "credential",
1278 &self
1279 .credential
1280 .try_lock()
1281 .map_or("[in use]", |_| "[redacted]"),
1282 )
1283 .field(
1284 "revalidations_performed",
1285 &self.revalidations_performed.load(Ordering::Relaxed),
1286 )
1287 .field("locked_out", &locked_out)
1288 .finish_non_exhaustive()
1289 }
1290}
1291
1292/// How a credential replaces itself.
1293///
1294/// # Why this is a port rather than a method
1295///
1296/// Renewal is two acts that must happen in one order: exchange the refresh
1297/// token with GitHub, then **persist the new pair before anything uses it**.
1298/// GitHub rotates on use -- the old pair dies the instant the new one is
1299/// issued -- so a response that is used but not stored leaves the host holding
1300/// a credential it will forget, and the one it forgot is already dead. There
1301/// is no retry: a spent refresh token answers `incorrect_client_credentials`,
1302/// a message about the client id and secret that is about neither.
1303///
1304/// The exchange belongs to `c2` and the store belongs to `d2`, and this crate
1305/// owns neither. So the ordering lives with whoever implements this, in one
1306/// place, rather than being a rule each caller has to remember.
1307#[async_trait::async_trait]
1308pub trait CredentialRenewal: fmt::Debug + Send + Sync {
1309 /// Exchange `refresh_token` for a fresh pair and persist it.
1310 ///
1311 /// # Errors
1312 /// Any failure; the caller treats every one the same way, by keeping the
1313 /// credential it has and letting the next `401` try again. The variant
1314 /// decides only how loudly that is reported; see [`RenewalError`].
1315 async fn renew(&self, refresh_token: &SecretString) -> Result<UserAccessToken, RenewalError>;
1316}
1317
1318/// Why [`CredentialRenewal::renew`] handed back no fresh pair.
1319#[derive(Debug, thiserror::Error)]
1320pub enum RenewalError {
1321 /// This process chose not to spend the refresh token, and GitHub was not
1322 /// asked.
1323 ///
1324 /// The pair is intact, so this is not a credential problem: a process that
1325 /// cannot store what the exchange returns leaves renewal to one that can --
1326 /// on a boot-mode host, the service. Reported quietly, because it recurs
1327 /// on every attempt for as long as the renewal window is open and nothing
1328 /// about it needs an operator.
1329 #[error("{0}")]
1330 Declined(String),
1331 /// The renewal was attempted and did not produce a stored pair.
1332 #[error("{0}")]
1333 Failed(String),
1334}
1335
1336impl From<String> for RenewalError {
1337 fn from(detail: String) -> Self {
1338 Self::Failed(detail)
1339 }
1340}
1341
1342/// Where a client can go to find out that the stored credential changed under
1343/// it.
1344///
1345/// # Why a long-running client needs this
1346///
1347/// A daemon reads the store once, at startup, and holds the result for as long
1348/// as it runs. That was invisible while the only way to change the store was to
1349/// stop the daemon — but `auth login` does not stop anything, so a host whose
1350/// credential died before its daemon started stays dead through every sign-in
1351/// meant to fix it. The operator does the right thing, watches it not work, and
1352/// has nothing to tell them why.
1353///
1354/// Watched on 2026-08-29: a Windows daemon started at `04:08Z` holding an
1355/// already-expired token, a sign-in at `10:43Z` that wrote a good pair, and 180
1356/// `unauthorized` events an hour for 28 hours without a single minute's pause
1357/// across the sign-in. See `docs/spikes/token-expiry-and-renewal.md`.
1358///
1359/// # Why it is not a file watch
1360///
1361/// This is consulted on `401` and nowhere else, so a store that never changes
1362/// costs nothing and a daemon that is working never reads the disk. It also
1363/// covers the case a watch would miss on macOS, where the credential lives in a
1364/// keychain rather than at a path.
1365pub trait CredentialSource: fmt::Debug + Send + Sync {
1366 /// The credential the store holds *now*, or `None` if it cannot be read.
1367 ///
1368 /// Infallible by design: every failure — missing, unreadable, corrupt —
1369 /// means the same thing to the caller, which is that there is nothing new
1370 /// to try and the `401` stands.
1371 fn reload(&self) -> Option<UserAccessToken>;
1372}
1373
1374impl AuthenticatedClient {
1375 /// # Errors
1376 /// The HTTP client failing to build — a TLS backend that will not
1377 /// initialise, in practice.
1378 pub fn new(
1379 endpoints: Endpoints,
1380 credential: UserAccessToken,
1381 clock: Arc<dyn Clock>,
1382 ) -> Result<Self, GithubError> {
1383 let http = reqwest::Client::builder()
1384 .timeout(DEFAULT_REQUEST_TIMEOUT)
1385 .build()
1386 .map_err(transport)?;
1387 Ok(Self::with_http_client(http, endpoints, credential, clock))
1388 }
1389
1390 #[must_use]
1391 pub fn with_http_client(
1392 http: reqwest::Client,
1393 endpoints: Endpoints,
1394 credential: UserAccessToken,
1395 clock: Arc<dyn Clock>,
1396 ) -> Self {
1397 Self {
1398 http,
1399 endpoints,
1400 credential: std::sync::Mutex::new(credential),
1401 source: None,
1402 renewal: None,
1403 clock,
1404 revalidation_generation: AtomicU64::new(0),
1405 revalidation_gate: tokio::sync::Mutex::new(()),
1406 last_revalidation: std::sync::Mutex::new(Revalidation::Valid),
1407 revalidations_performed: AtomicU64::new(0),
1408 consecutive_unauthorized: AtomicU64::new(0),
1409 lockout: std::sync::Mutex::new(LockoutState {
1410 until: None,
1411 backoff: DEFAULT_LOCKOUT_BACKOFF,
1412 }),
1413 traffic: traffic::ApiTraffic::new(),
1414 }
1415 }
1416
1417 #[must_use]
1418 pub fn endpoints(&self) -> &Endpoints {
1419 &self.endpoints
1420 }
1421
1422 /// What this client has received from GitHub, by class, and the newest
1423 /// reading of the account's hourly quota.
1424 #[must_use]
1425 pub const fn traffic(&self) -> &traffic::ApiTraffic {
1426 &self.traffic
1427 }
1428
1429 /// How many credential re-validations this client has performed.
1430 ///
1431 /// Public because it is the observable the single-flight requirement is
1432 /// stated in terms of: "concurrent callers hitting `401` together produce
1433 /// **one** attempt, not N".
1434 #[must_use]
1435 pub fn revalidations_performed(&self) -> u64 {
1436 self.revalidations_performed.load(Ordering::SeqCst)
1437 }
1438
1439 /// `true` while a lockout back-off is still running, during which this
1440 /// client issues no HTTP at all.
1441 ///
1442 /// # Panics
1443 /// If a previous holder panicked while the lockout lock was held.
1444 #[must_use]
1445 pub fn is_locked_out(&self) -> bool {
1446 self.lockout_remaining().is_some()
1447 }
1448
1449 /// How much of the lockout back-off is left, or `None` when not locked out.
1450 ///
1451 /// # Panics
1452 /// If a previous holder panicked while the lockout lock was held.
1453 #[must_use]
1454 pub fn lockout_remaining(&self) -> Option<Duration> {
1455 let state = self.lockout.lock().expect("lockout lock poisoned");
1456 let until = state.until?;
1457 let now = self.clock.now();
1458 if now >= until {
1459 return None;
1460 }
1461 (until - now).to_std().ok()
1462 }
1463
1464 /// Clear a lockout early. `f1` does not need this — the back-off expires on
1465 /// its own against the clock — but a successful interactive `auth login`
1466 /// legitimately invalidates the whole lockout premise.
1467 ///
1468 /// # Panics
1469 /// If a previous holder panicked while the lockout lock was held.
1470 pub fn clear_lockout(&self) {
1471 self.lockout.lock().expect("lockout lock poisoned").until = None;
1472 self.consecutive_unauthorized.store(0, Ordering::SeqCst);
1473 }
1474
1475 /// Send one request, applying the authentication taxonomy.
1476 ///
1477 /// On `401` this performs a single-flight credential re-validation and then
1478 /// **one** retry — never more, and never a token renewal, because there is
1479 /// nothing to renew (see [`AuthenticatedClient::revalidate`]).
1480 ///
1481 /// # Errors
1482 /// Every variant of [`GithubError`].
1483 pub async fn send(&self, request: &ApiRequest) -> Result<ApiResponse, GithubError> {
1484 if let Some(remaining) = self.lockout_remaining() {
1485 // "backs off without further attempts": no socket is opened at all.
1486 tracing::debug!(
1487 method = request.method.as_str(),
1488 path = %request.path,
1489 remaining_secs = remaining.as_secs(),
1490 "suppressed a request: GitHub authentication lockout is still backing off"
1491 );
1492 return Err(GithubError::AuthenticationLockout {
1493 retry_after: remaining,
1494 });
1495 }
1496
1497 let first = self.send_raw(request).await?;
1498 match self.classify(request, &first, Attempt::First) {
1499 Classified::Ok => Ok(first),
1500 Classified::Unauthorized => self.revalidate_and_retry_once(request).await,
1501 Classified::Error(err) => Err(err),
1502 }
1503 }
1504
1505 /// Deserialize a `GET` in one step.
1506 ///
1507 /// # Errors
1508 /// Every variant of [`GithubError`].
1509 pub async fn get_json<T: DeserializeOwned>(&self, path: &str) -> Result<T, GithubError> {
1510 self.send(&ApiRequest::get(path)).await?.json()
1511 }
1512
1513 /// The access token to send, as a string, for exactly one request.
1514 ///
1515 /// Cloned out of the lock rather than borrowed through it: the value is
1516 /// about to go into a header, and holding the lock across the request would
1517 /// serialise every call in the process behind one mutex.
1518 fn bearer(&self) -> String {
1519 self.credential
1520 .lock()
1521 .unwrap_or_else(std::sync::PoisonError::into_inner)
1522 .secret()
1523 .expose_secret()
1524 .to_string()
1525 }
1526
1527 /// Attach a way for this client's credential to replace itself.
1528 #[must_use]
1529 pub fn with_renewal(mut self, renewal: Arc<dyn CredentialRenewal>) -> Self {
1530 self.renewal = Some(renewal);
1531 self
1532 }
1533
1534 /// Attach the store this client's credential came from, so a `401` it
1535 /// cannot renew its way out of can still notice a sign-in that already
1536 /// happened.
1537 #[must_use]
1538 pub fn with_credential_source(mut self, source: Arc<dyn CredentialSource>) -> Self {
1539 self.source = Some(source);
1540 self
1541 }
1542
1543 /// Replace the credential with whatever `produce` finds, once, however many
1544 /// callers asked at the same moment.
1545 ///
1546 /// Answers whether the credential in hand is now a *different* one, which
1547 /// is the caller's cue to retry. `false` means there was nothing new, and
1548 /// the `401` stands.
1549 ///
1550 /// # Why both ways in share this
1551 ///
1552 /// [`Self::renew_once`] and [`Self::reload_once`] ask one question —
1553 /// *can this `401` be retried with something else* — and differ only in
1554 /// where the something else comes from. Written separately they were two
1555 /// copies of the same `SeqCst` generation protocol, and the copies had
1556 /// already drifted: the equality check below existed in one of them and not
1557 /// the other, so a renewal that handed back an identical token reported a
1558 /// change that had not happened.
1559 ///
1560 /// # Why the comparison, and not just a swap
1561 ///
1562 /// A swap on every `401` would answer `true` forever and turn the one retry
1563 /// into an endless pair of requests against a credential that is genuinely
1564 /// dead. Only a different token is evidence that retrying might go
1565 /// differently.
1566 async fn swap_credential_once<F>(&self, produce: F, note: &'static str) -> bool
1567 where
1568 F: AsyncFnOnce(&Self) -> Option<UserAccessToken>,
1569 {
1570 let before = self.revalidation_generation.load(Ordering::SeqCst);
1571 let _gate = self.revalidation_gate.lock().await;
1572 if self.revalidation_generation.load(Ordering::SeqCst) != before {
1573 // Somebody swapped while this caller queued, so there is already
1574 // something new to retry with. Producing again would be wasted at
1575 // best and, for a renewal, would burn the pair they just minted.
1576 return true;
1577 }
1578
1579 let Some(fresh) = produce(self).await else {
1580 return false;
1581 };
1582 {
1583 let mut guard = self
1584 .credential
1585 .lock()
1586 .unwrap_or_else(std::sync::PoisonError::into_inner);
1587 if guard.secret().expose_secret() == fresh.secret().expose_secret() {
1588 return false;
1589 }
1590 *guard = fresh;
1591 }
1592 self.revalidation_generation.fetch_add(1, Ordering::SeqCst);
1593 tracing::info!("{note}");
1594 true
1595 }
1596
1597 /// Pick up a credential somebody else stored.
1598 ///
1599 /// `false` when there is no source, the store cannot be read, or what it
1600 /// holds is the token that just failed.
1601 async fn reload_once(&self) -> bool {
1602 let Some(source) = self.source.clone() else {
1603 return false;
1604 };
1605 self.swap_credential_once(
1606 async |_| source.reload(),
1607 "the stored credential changed and was picked up without a restart",
1608 )
1609 .await
1610 }
1611
1612 /// Spend the refresh half for a fresh pair.
1613 ///
1614 /// # Ordering
1615 ///
1616 /// The implementation of [`CredentialRenewal::renew`] persists before
1617 /// returning, so by the time the swap happens the new pair is already
1618 /// durable. A crash between the two loses nothing: the store holds the pair
1619 /// that works, and the next start reads it.
1620 async fn renew_once(&self) -> bool {
1621 let Some(renewal) = self.renewal.clone() else {
1622 return false;
1623 };
1624 self.swap_credential_once(
1625 async |client: &Self| {
1626 let refresh = {
1627 let guard = client
1628 .credential
1629 .lock()
1630 .unwrap_or_else(std::sync::PoisonError::into_inner);
1631 guard.renewal().map(|r| r.refresh_token().clone())
1632 }?;
1633 match renewal.renew(&refresh).await {
1634 Ok(fresh) => Some(fresh),
1635 Err(RenewalError::Declined(reason)) => {
1636 tracing::debug!(
1637 %reason,
1638 "left the user access token for a process that can store its renewal"
1639 );
1640 None
1641 }
1642 Err(RenewalError::Failed(error)) => {
1643 tracing::warn!(
1644 %error,
1645 "the user access token could not be renewed; an interactive \
1646 sign-in may be required"
1647 );
1648 None
1649 }
1650 }
1651 },
1652 "the user access token was renewed",
1653 )
1654 .await
1655 }
1656
1657 /// Proactively replace a renewable credential when either half is close
1658 /// to expiry.
1659 ///
1660 /// Ordinary request handling also renews after a `401`, but a daemon with
1661 /// no GitHub work can make no request for months. In that state the refresh
1662 /// token itself eventually expires and an interactive sign-in is the only
1663 /// recovery. A long-running owner calls this periodically so every
1664 /// successful rotation extends the credential even while it is otherwise
1665 /// idle.
1666 ///
1667 /// The stored credential is checked first for the same reason as the `401`
1668 /// path: another process may already have rotated the pair, and spending
1669 /// its predecessor after that would be a refresh-token replay.
1670 ///
1671 /// Returns `true` when a different credential was loaded or minted. A
1672 /// credential without a refresh half is left untouched. Missing expiry
1673 /// metadata is treated as due once so that the exchange can repair an old
1674 /// stored document.
1675 pub async fn renew_if_expiring_within(&self, window: Duration) -> bool {
1676 let due = {
1677 let credential = self
1678 .credential
1679 .lock()
1680 .unwrap_or_else(std::sync::PoisonError::into_inner);
1681 let Some(renewal) = credential.renewal() else {
1682 return false;
1683 };
1684 let Some(deadline) = self.clock.now().checked_add_signed(
1685 chrono::TimeDelta::from_std(window).unwrap_or(chrono::TimeDelta::MAX),
1686 ) else {
1687 return true;
1688 };
1689 renewal
1690 .access_expires_at
1691 .is_none_or(|expires| expires <= deadline)
1692 || renewal
1693 .refresh_expires_at
1694 .is_some_and(|expires| expires <= deadline)
1695 };
1696
1697 due && (self.reload_once().await || self.renew_once().await)
1698 }
1699
1700 /// Serialize, `POST`, and deserialize in one step.
1701 ///
1702 /// # Errors
1703 /// Every variant of [`GithubError`].
1704 pub async fn post_json<B: Serialize, T: DeserializeOwned>(
1705 &self,
1706 path: &str,
1707 body: &B,
1708 ) -> Result<T, GithubError> {
1709 self.send(&ApiRequest::post_json(path, body)?).await?.json()
1710 }
1711
1712 /// Re-validate the credential once, no matter how many callers ask at once.
1713 ///
1714 /// # This is not a token renewal — renewal is [`Self::renew_once`]
1715 ///
1716 /// `03-control-flows.md` flow 4.3 says a `401` "triggers one refresh under
1717 /// a single-flight mutex, then one retry", and that is implemented
1718 /// literally: one attempt shared by every concurrent caller, then one retry
1719 /// each.
1720 ///
1721 /// This method used to claim the word "refresh" could not be meant, because
1722 /// renewing needs a client secret and the App issues no renewal token. Both
1723 /// halves were false; see this module's header. Renewal exists, it runs
1724 /// first in [`Self::revalidate_and_retry_once`], and what is left here is
1725 /// the path for a credential that has no refresh half to spend — one stored
1726 /// before 0.1.11, or issued while the App had expiration switched off.
1727 ///
1728 /// So the single thing that happens under the mutex is a re-validation of
1729 /// the credential already held: one `GET /user/installations` with the same
1730 /// token, asking GitHub whether it still accepts it. A
1731 /// [`Revalidation::Rejected`] answer is terminal
1732 /// [`GithubError::AuthenticationFailed`] requiring an interactive
1733 /// `auth login`; [`Revalidation::Valid`] and [`Revalidation::Unavailable`]
1734 /// both spend the one retry.
1735 ///
1736 /// # Position, and what it no longer decides on its own
1737 ///
1738 /// This method is the caller's own probe: it is a **first** attempt by
1739 /// construction, whatever happened on some other request minutes ago, so it
1740 /// passes [`Attempt::First`] down.
1741 ///
1742 /// That used to settle the matter. This heading read "why this entry point
1743 /// may not latch a lockout", and the text said the probe it drives *cannot*
1744 /// latch — an accurate description of the position rule as it then stood,
1745 /// and a false statement about the product. A lockout that outlives one
1746 /// back-off continues on a **first** attempt by construction, because this
1747 /// client's retry never happened: the request never reached the wire.
1748 /// Refusing to latch there stopped the back-off entirely and hammered a
1749 /// credential GitHub had asked to be left alone.
1750 ///
1751 /// So position alone no longer decides.
1752 /// [`AuthenticatedClient::is_lockout_403`] reads GitHub's own evidence in
1753 /// the first position instead: a first attempt latches when, and only when,
1754 /// the response carries `retry-after` and no parseable GitHub message body.
1755 /// A permissions refusal names what is not accessible, so it still does not
1756 /// latch, which is what keeps [`GithubError::Forbidden`] reachable from
1757 /// here.
1758 ///
1759 /// `revalidate_and_retry_once` uses the private
1760 /// [`AuthenticatedClient::revalidate_after_unauthorized`] instead, which is
1761 /// in the retry position, where any `403` that is not a rate limit is the
1762 /// lockout regardless of what the body says.
1763 ///
1764 /// # This call can latch a lockout, and then it says so
1765 ///
1766 /// Because a first attempt can latch, this call can leave the whole client
1767 /// backed off for up to [`MAX_LOCKOUT_BACKOFF`]. It reports that as
1768 /// [`GithubError::AuthenticationLockout`] rather than answering
1769 /// `Ok(`[`Revalidation::Unavailable`]`)` and leaving the caller to discover
1770 /// it through a separate [`AuthenticatedClient::is_locked_out`] call. An
1771 /// `auth status` that printed "could not determine" while the client it had
1772 /// just silenced sat mute for fifteen minutes would be reporting the wrong
1773 /// event, and reporting it as the milder one.
1774 ///
1775 /// # Errors
1776 /// [`GithubError::AuthenticationLockout`] if this client is already backing
1777 /// off when the call arrives, **or** if this call's own probe latches one.
1778 ///
1779 /// A credential GitHub has rejected outright is *not* an error here: that
1780 /// comes back as `Ok(`[`Revalidation::Rejected`]`)`, and what to do about it
1781 /// — prompt for `auth login` — is the caller's decision, not this method's.
1782 ///
1783 /// # Panics
1784 /// If a previous holder panicked while the re-validation result lock was
1785 /// held.
1786 pub async fn revalidate(&self) -> Result<Revalidation, GithubError> {
1787 self.revalidate_from(Attempt::First).await
1788 }
1789
1790 /// The re-validation that `send` runs between a `401` and its one retry.
1791 ///
1792 /// Identical to [`AuthenticatedClient::revalidate`] except for position:
1793 /// this one *is* the retry path, so a `403` on its probe is the lockout and
1794 /// is latched.
1795 async fn revalidate_after_unauthorized(&self) -> Result<Revalidation, GithubError> {
1796 self.revalidate_from(Attempt::Retry).await
1797 }
1798
1799 async fn revalidate_from(&self, attempt: Attempt) -> Result<Revalidation, GithubError> {
1800 // "A lockout stops traffic. This client issues no further HTTP at all
1801 // until the back-off elapses" is a property of the client, not of
1802 // `send`, and the probe is HTTP like any other. `send` has already
1803 // returned by the time it calls in here, so this guard only bites a
1804 // caller that probes on its own — and one that did would otherwise be
1805 // the single exception to the rule, which is how a rule stops holding.
1806 if let Some(retry_after) = self.lockout_remaining() {
1807 return Err(GithubError::AuthenticationLockout { retry_after });
1808 }
1809
1810 // Sample before queuing. If this changes while we wait for the gate,
1811 // someone else's re-validation covers us and we must not repeat it.
1812 let sampled = self.revalidation_generation.load(Ordering::SeqCst);
1813 let _guard = self.revalidation_gate.lock().await;
1814 let outcome = if self.revalidation_generation.load(Ordering::SeqCst) != sampled {
1815 let shared = *self
1816 .last_revalidation
1817 .lock()
1818 .expect("re-validation lock poisoned");
1819 tracing::debug!(
1820 outcome = ?shared,
1821 "reused an in-flight credential re-validation instead of starting another"
1822 );
1823 shared
1824 } else {
1825 self.revalidations_performed.fetch_add(1, Ordering::SeqCst);
1826 let fresh = self.probe_credential(attempt).await;
1827 *self
1828 .last_revalidation
1829 .lock()
1830 .expect("re-validation lock poisoned") = fresh;
1831 self.revalidation_generation.fetch_add(1, Ordering::SeqCst);
1832 tracing::info!(
1833 outcome = ?fresh,
1834 "re-validated the stored credential; it carries no refresh half to renew"
1835 );
1836 fresh
1837 };
1838
1839 // The probe is HTTP, and since the continuation rule re-widened
1840 // `Attempt::First`, HTTP from *this* entry point can latch a lockout.
1841 // `probe_credential` reports a `403` as `Unavailable` whether or not it
1842 // latched one, and `Unavailable` on its own reads as "this taught us
1843 // nothing about the credential" — so without this check `revalidate`
1844 // answers `Ok(Unavailable)` having just silenced the entire client for
1845 // up to `MAX_LOCKOUT_BACKOFF`, and the only way for `f1` to find that
1846 // out is a separate `is_locked_out()` call it has no reason to make.
1847 //
1848 // `revalidate_and_retry_once` has always re-checked after its own probe.
1849 // Doing it here instead makes the two entry points agree: before, being
1850 // told about the lockout depended on *who latched it* — the guard above
1851 // reports one latched by someone else's traffic, this reports one
1852 // latched by the caller's own probe — and that distinction is invisible
1853 // from outside and actionable by nobody.
1854 //
1855 // This changes what is reported, not when a lockout latches:
1856 // `latch_lockout` is reached on exactly the paths it was before.
1857 //
1858 // It also covers the shared branch above, which the retry path's own
1859 // check never could. A second caller arriving while the first one's
1860 // probe is in flight takes the cached `Unavailable` without probing at
1861 // all, and is just as silenced by the lockout that probe latched.
1862 if let Some(retry_after) = self.lockout_remaining() {
1863 return Err(GithubError::AuthenticationLockout { retry_after });
1864 }
1865 Ok(outcome)
1866 }
1867
1868 /// One `GET /user/installations` with the credential already held, asking
1869 /// GitHub whether it still accepts it.
1870 ///
1871 /// `attempt` is the *caller's* position, not the probe's own. A probe driven
1872 /// by `send`'s `401` handling is part of that request's retry; a probe a
1873 /// caller asked for through [`AuthenticatedClient::revalidate`] is a first
1874 /// attempt.
1875 ///
1876 /// Either may latch a lockout, on different evidence. This used to say "only
1877 /// the former may", which was the position rule before the continuation rule
1878 /// re-widened `Attempt::First`: a retry `403` that is not a rate limit is
1879 /// the lockout outright, and a first-attempt `403` is the lockout when
1880 /// GitHub's own evidence says so — `retry-after` present, no parseable
1881 /// message. See [`AuthenticatedClient::is_lockout_403`], which both this and
1882 /// `classify` go through, so the rule is stated once instead of twice.
1883 async fn probe_credential(&self, attempt: Attempt) -> Revalidation {
1884 let probe = ApiRequest::get(REVALIDATION_PATH).query("per_page", 1);
1885 match self.send_raw(&probe).await {
1886 // Deliberately does **not** reset `consecutive_unauthorized`. The
1887 // probe is this client's own diagnostic, not the caller's traffic,
1888 // and a successful probe is exactly the state a lockout arrives in:
1889 // GitHub still accepts the credential, and answers the *next* real
1890 // request with `403`. Only a successful caller request clears the
1891 // count, in `classify`.
1892 //
1893 // The reason used to be given as "resetting here would erase the
1894 // evidence the `403` is classified against", and that stopped being
1895 // true when `is_lockout_403` stopped consulting the count. There is
1896 // no such evidence to erase now — nothing in production reads the
1897 // field. The line stays because the field's one remaining job is to
1898 // let the tests prove it is *not* consulted, and a probe that
1899 // quietly rewrote it would make those tests assert against a counter
1900 // value no caller path actually produces.
1901 Ok(response) if response.status.is_success() => Revalidation::Valid,
1902 Ok(response) if response.status == StatusCode::UNAUTHORIZED => {
1903 self.consecutive_unauthorized.fetch_add(1, Ordering::SeqCst);
1904 Revalidation::Rejected
1905 }
1906 Ok(response) if response.status == StatusCode::FORBIDDEN => {
1907 // A `403` on a probe that *is* this request's retry is the
1908 // lockout outright. A `403` on a probe a caller asked for
1909 // directly is the lockout only when the response itself says so
1910 // — `retry-after` present, no parseable message — because that
1911 // is what a lockout continuing past one back-off looks like, and
1912 // it necessarily arrives in the first position.
1913 //
1914 // The comment that used to sit here claimed "the probe only ever
1915 // runs after a `401`, so it is always in the retry position",
1916 // and publishing `revalidate` is what made that untrue. Its
1917 // replacement then said a direct probe "is not" the lockout,
1918 // which the continuation rule in turn made untrue. Both were
1919 // position asserted as a conclusion; the position now arrives as
1920 // an argument and the conclusion is drawn in one place, by
1921 // `is_lockout_403`.
1922 if self.is_lockout_403(&response, attempt) {
1923 self.latch_lockout(&response.headers);
1924 }
1925 Revalidation::Unavailable
1926 }
1927 Ok(_) | Err(_) => Revalidation::Unavailable,
1928 }
1929 }
1930
1931 async fn revalidate_and_retry_once(
1932 &self,
1933 request: &ApiRequest,
1934 ) -> Result<ApiResponse, GithubError> {
1935 // ------------------------------------------------------------------
1936 // RENEWAL COMES FIRST, AND IT DID NOT USED TO EXIST.
1937 // ------------------------------------------------------------------
1938 // This method's documentation once said a `401` could only ever be
1939 // re-validated, never renewed, because renewing needs a confidential
1940 // client credential and a published binary cannot carry one. The first
1941 // half of that is wrong: GitHub requires one *"unless the user access
1942 // token was generated using the device flow"*, and this product's
1943 // always are. Verified against live GitHub before this was written.
1944 //
1945 // So a credential that carries a refresh token replaces itself here,
1946 // silently, and the caller's request is retried with the new one. That
1947 // is what lets an eight-hour token serve a daemon that runs for months
1948 // -- and what lets two machines hold their own credentials at once,
1949 // because each renews its own pair instead of re-authorising and
1950 // revoking the other's.
1951 //
1952 // A credential with no refresh token falls through to exactly the
1953 // behaviour that was here before.
1954 //
1955 // ------------------------------------------------------------------
1956 // THEN THE STORE, FOR THE 401 RENEWAL CANNOT ANSWER.
1957 // ------------------------------------------------------------------
1958 // Renewal covers a token that expired under a daemon holding a
1959 // *renewable* pair. It cannot cover a daemon that started holding a
1960 // dead bare token, because there is no refresh half to spend -- and
1961 // that daemon will not recover on its own no matter how many times an
1962 // operator runs `auth login`, because it never looks at the store
1963 // again. Consulting it here is what makes the obvious remedy work.
1964 //
1965 // First, not second: if multiple processes (e.g., daemon and TUI) share the
1966 // same token, they will both get 401 at the same time. The first one to renew
1967 // will write the new token to disk. If the second process tries to renew with
1968 // its in-memory refresh token *before* reloading, GitHub will detect a refresh
1969 // token replay and instantly revoke the entire token chain. So we must always
1970 // check disk for a newer token first.
1971 //
1972 // To further prevent a race condition if two processes hit 401 at the exact
1973 // same millisecond, we introduce a pseudo-random jitter based on the OS
1974 // process ID. This ensures one process wakes up first, finishes the renewal,
1975 // and writes to disk, so the second process sees the new file during its
1976 // `reload_once()`. All threads in the *same* process compute the exact same
1977 // jitter, preserving their ability to coalesce behind `revalidation_gate`.
1978 let jitter = (std::process::id() % 1500) as u64 + 50;
1979 tokio::time::sleep(std::time::Duration::from_millis(jitter)).await;
1980
1981 if self.reload_once().await || self.renew_once().await {
1982 let second = self.send_raw(request).await?;
1983 return match self.classify(request, &second, Attempt::Retry) {
1984 Classified::Ok => Ok(second),
1985 // The renewed credential was rejected too. Nothing here can
1986 // help: this is a sign-in, not a token, that has gone.
1987 Classified::Unauthorized => Err(GithubError::AuthenticationFailed),
1988 Classified::Error(err) => Err(err),
1989 };
1990 }
1991 match self.revalidate_after_unauthorized().await? {
1992 Revalidation::Rejected => {
1993 tracing::warn!(
1994 method = request.method.as_str(),
1995 path = %request.path,
1996 "GitHub rejected the stored credential; re-authentication is required"
1997 );
1998 Err(GithubError::AuthenticationFailed)
1999 }
2000 Revalidation::Valid | Revalidation::Unavailable => {
2001 // `revalidate_from` now converts a lockout that its own probe
2002 // latched, so this no longer catches that case. It stays for the
2003 // one it still catches: a *concurrent* request latching between
2004 // that check and this one. Sending the retry into a live lockout
2005 // is the thing the back-off exists to prevent, and this is the
2006 // last point at which it can be declined.
2007 if let Some(remaining) = self.lockout_remaining() {
2008 return Err(GithubError::AuthenticationLockout {
2009 retry_after: remaining,
2010 });
2011 }
2012 let second = self.send_raw(request).await?;
2013 match self.classify(request, &second, Attempt::Retry) {
2014 Classified::Ok => Ok(second),
2015 // The one retry is spent. A second `401` is terminal.
2016 Classified::Unauthorized => Err(GithubError::AuthenticationFailed),
2017 Classified::Error(err) => Err(err),
2018 }
2019 }
2020 }
2021 }
2022
2023 fn classify(
2024 &self,
2025 request: &ApiRequest,
2026 response: &ApiResponse,
2027 attempt: Attempt,
2028 ) -> Classified {
2029 let status = response.status;
2030 // A `304` is a success only as the answer to a conditional request:
2031 // it means "what you hold is still current". Unasked for, it is not an
2032 // answer this client can hand on, and it stays an error.
2033 if status.is_success() || (status == StatusCode::NOT_MODIFIED && request.is_conditional()) {
2034 self.consecutive_unauthorized.store(0, Ordering::SeqCst);
2035 return Classified::Ok;
2036 }
2037 if status == StatusCode::UNAUTHORIZED {
2038 self.consecutive_unauthorized.fetch_add(1, Ordering::SeqCst);
2039 return Classified::Unauthorized;
2040 }
2041 if status == StatusCode::FORBIDDEN && self.is_lockout_403(response, attempt) {
2042 let backoff = self.latch_lockout(&response.headers);
2043 tracing::warn!(
2044 method = request.method.as_str(),
2045 path = %request.path,
2046 backoff_secs = backoff.as_secs(),
2047 "GitHub answered 403 after 401s: temporary authentication lockout, backing off"
2048 );
2049 return Classified::Error(GithubError::AuthenticationLockout {
2050 retry_after: backoff,
2051 });
2052 }
2053 let headers = Box::new(response.headers.clone());
2054 let message = error_message(&response.body);
2055 if status == StatusCode::FORBIDDEN {
2056 return Classified::Error(GithubError::Forbidden {
2057 method: request.method.as_str().to_string(),
2058 path: request.path.clone(),
2059 message,
2060 headers,
2061 });
2062 }
2063 Classified::Error(GithubError::Status {
2064 status: status.as_u16(),
2065 method: request.method.as_str().to_string(),
2066 path: request.path.clone(),
2067 message,
2068 headers,
2069 })
2070 }
2071
2072 /// Whether a `403` is GitHub's temporary *authentication* lockout, as
2073 /// opposed to a permissions answer or a rate limit.
2074 ///
2075 /// # It must not be a rate limit
2076 ///
2077 /// `classify` used to reach the `403` branch before anything looked at the
2078 /// rate-limit headers, so a primary rate limit arriving during a `401` storm
2079 /// was reported as `AuthenticationLockout` — telling the operator "the
2080 /// credential itself is not the problem" about a response that never
2081 /// mentioned the credential. Recognising GitHub's own rate-limit evidence is
2082 /// not rate-limit *policy*; it is declining to make an assertion the
2083 /// evidence contradicts. What to do about the rate limit stays `c3`'s, which
2084 /// is why this only changes which variant carries the headers onward.
2085 ///
2086 /// # Then one of two positions, and the second one is a fix for the first
2087 ///
2088 /// **The retry.** `consecutive_unauthorized` counts `401`s since the last
2089 /// successful caller response and — correctly — does not decay: a request
2090 /// that ends in `404`, `422` or `500` leaves it set. In the agent's
2091 /// long-lived reconciliation loop that meant a single `401` from minutes ago
2092 /// converted the *next* genuine permissions `403` into a fake lockout: sixty
2093 /// seconds of silence plus an operator message insisting the credential is
2094 /// fine, when in truth `generate-jitconfig` was missing
2095 /// `Administration: write`. The lockout's signature is narrower than "a
2096 /// `403` while the count is set" — it is a `403` on the one retry this
2097 /// client itself issued after this request's own `401`.
2098 ///
2099 /// The count is deliberately *not* consulted. [`Attempt::Retry`] already
2100 /// means this request's own `401` incremented it moments ago, so reading it
2101 /// adds no signal — and does add a race that fails open: any concurrent
2102 /// request succeeding between the `401` and the retry `store(0)`s the
2103 /// counter, and a real lockout is then reported as a plain permissions
2104 /// refusal. A conjunct that can only ever weaken a safety check is worse
2105 /// than no conjunct.
2106 ///
2107 /// **The continuation.** Narrowing to the retry position opened a hole at
2108 /// the far end of the same back-off. When the back-off elapses and GitHub is
2109 /// still locking the credential out, the next request is a *first* attempt
2110 /// by construction — this client's retry never happened, because the request
2111 /// never reached the wire. The position rule then declined to call it a
2112 /// lockout, `classify` fell through to [`GithubError::Forbidden`] — whose
2113 /// documented reading is "the App installation does not grant it" — and the
2114 /// client **stopped backing off entirely**, hammering a credential GitHub
2115 /// had asked it to leave alone. That is the exact inverse of the
2116 /// Definition of Done's "backs off without retrying", and it failed for
2117 /// every lockout outliving one back-off.
2118 ///
2119 /// No counter is needed for that case either, because the response says so
2120 /// itself. GitHub's lockout carries `retry-after` and no parseable message;
2121 /// a permissions refusal carries a message naming what is not accessible and
2122 /// no `retry-after`. Requiring **both** halves of that signature is what
2123 /// keeps this from degenerating into "every `403` is a lockout": a
2124 /// permissions answer has a message, so it never matches, and a secondary
2125 /// rate limit has both a message and `retry-after`, so `is_rate_limited`
2126 /// takes it first.
2127 ///
2128 /// "No parseable message" is deliberately wider than "an empty body", which
2129 /// is how this used to be stated. See [`is_lockout_continuation`] for what
2130 /// else falls into it — a proxy's HTML error page most notably — and for why
2131 /// the resulting false positives are accepted rather than tightened away.
2132 ///
2133 /// This also settles a standing worry about [`MAX_LOCKOUT_BACKOFF`]. With
2134 /// the continuation recognised, the ceiling no longer decides whether the
2135 /// product ever gives up — it only decides how often it re-asks. A lockout
2136 /// longer than the ceiling now re-latches instead of being reported as a
2137 /// permissions failure, so the value is a polling interval rather than a
2138 /// deadline.
2139 fn is_lockout_403(&self, response: &ApiResponse, attempt: Attempt) -> bool {
2140 if is_rate_limited(response) {
2141 return false;
2142 }
2143 match attempt {
2144 Attempt::Retry => true,
2145 Attempt::First => is_lockout_continuation(response),
2146 }
2147 }
2148
2149 fn latch_lockout(&self, headers: &HeaderMap) -> Duration {
2150 // Clamp before latching. An unclamped `Retry-After` is a remote party
2151 // deciding how long this product stays down.
2152 let requested = retry_after(headers).unwrap_or(DEFAULT_LOCKOUT_BACKOFF);
2153 let clamped = requested.min(MAX_LOCKOUT_BACKOFF);
2154
2155 // A span too large for `chrono` must fall back to the default, never to
2156 // `None`: the old code's `.ok()` turned an absurd `Retry-After` into "no
2157 // lockout at all", which fails *open* — the exact inverse of what a
2158 // back-off is for, and reachable by a header alone. The clamp above
2159 // already makes this branch unreachable; it stays because the invariant
2160 // it protects ("latching always latches") is worth more than the line.
2161 let delta = chrono::TimeDelta::from_std(clamped).unwrap_or_else(|_| {
2162 chrono::TimeDelta::from_std(DEFAULT_LOCKOUT_BACKOFF)
2163 .expect("sixty seconds is a representable span")
2164 });
2165 // No third clamp. `clamped` is already `<= MAX_LOCKOUT_BACKOFF`, and
2166 // `TimeDelta` round-trips it exactly, so re-clamping here was dead twice
2167 // over — it could only ever re-apply a bound already applied, and the
2168 // fallback it guarded is `DEFAULT_LOCKOUT_BACKOFF`, which is smaller
2169 // than the ceiling by construction.
2170 let backoff = delta.to_std().unwrap_or(DEFAULT_LOCKOUT_BACKOFF);
2171
2172 let mut state = self.lockout.lock().expect("lockout lock poisoned");
2173 state.backoff = backoff;
2174 state.until = Some(self.clock.now() + delta);
2175 backoff
2176 }
2177
2178 /// One HTTP round trip with the standard headers applied and no
2179 /// interpretation of the result.
2180 async fn send_raw(&self, request: &ApiRequest) -> Result<ApiResponse, GithubError> {
2181 let url = self.resolve(&request.path)?;
2182 let mut builder = self
2183 .http
2184 .request(request.method.clone(), url)
2185 .header(reqwest::header::ACCEPT, GITHUB_ACCEPT)
2186 .header(reqwest::header::USER_AGENT, USER_AGENT)
2187 .header("X-GitHub-Api-Version", GITHUB_API_VERSION)
2188 // The only place the token is ever written onto the wire. It is
2189 // never logged, and `reqwest` does not render headers in its errors.
2190 .header(
2191 reqwest::header::AUTHORIZATION,
2192 format!("Bearer {}", self.bearer()),
2193 );
2194 if !request.query.is_empty() {
2195 builder = builder.query(&request.query);
2196 }
2197 if let Some(body) = &request.body {
2198 builder = builder.json(body);
2199 }
2200 if let Some(validators) = &request.validators {
2201 if let Some(etag) = &validators.etag {
2202 builder = builder.header(reqwest::header::IF_NONE_MATCH, etag);
2203 }
2204 if let Some(modified) = &validators.last_modified {
2205 builder = builder.header(reqwest::header::IF_MODIFIED_SINCE, modified);
2206 }
2207 }
2208
2209 let response = builder.send().await.map_err(transport)?;
2210 let status = response.status();
2211 let headers = response.headers().clone();
2212 // Counted on the headers, before the body: GitHub charged the request
2213 // whether or not the body then arrives whole.
2214 self.traffic.record(status, &headers, self.clock.now());
2215 let body = response.bytes().await.map_err(transport)?.to_vec();
2216
2217 tracing::debug!(
2218 method = request.method.as_str(),
2219 path = %request.path,
2220 status = status.as_u16(),
2221 body_bytes = body.len(),
2222 "github api request"
2223 );
2224
2225 Ok(ApiResponse {
2226 status,
2227 headers,
2228 body,
2229 })
2230 }
2231
2232 fn resolve(&self, path: &str) -> Result<Url, GithubError> {
2233 if path.starts_with("http://") || path.starts_with("https://") {
2234 return Url::parse(path).map_err(|_| GithubError::Malformed {
2235 what: "an absolute request URL",
2236 value: path.to_string(),
2237 });
2238 }
2239 self.endpoints
2240 .api_base
2241 .join(path.trim_start_matches('/'))
2242 .map_err(|_| GithubError::Malformed {
2243 what: "a request path",
2244 value: path.to_string(),
2245 })
2246 }
2247}
2248
2249enum Classified {
2250 Ok,
2251 Unauthorized,
2252 Error(GithubError),
2253}
2254
2255/// Which of a request's at-most-two attempts produced a response.
2256///
2257/// The authentication lockout is defined by *position*, not just by status: it
2258/// is what GitHub answers the retry that follows a `401`. Passing this in makes
2259/// that explicit at both call sites instead of inferring it from a counter that
2260/// outlives the request.
2261#[derive(Debug, Clone, Copy, PartialEq, Eq)]
2262enum Attempt {
2263 First,
2264 Retry,
2265}
2266
2267/// Whether GitHub attributed a failing response to its own rate limit.
2268///
2269/// Reading the evidence, and nothing else — see [`GithubError`]'s note on why
2270/// the headers travel with the error. `c3` decides what to do about it.
2271fn is_rate_limited(response: &ApiResponse) -> bool {
2272 if response.status == StatusCode::TOO_MANY_REQUESTS {
2273 return true;
2274 }
2275 // The primary rate limit's documented signature.
2276 if response
2277 .header("x-ratelimit-remaining")
2278 .is_some_and(|v| v.trim() == "0")
2279 {
2280 return true;
2281 }
2282 // A secondary rate limit sends `retry-after` — but so does the
2283 // authentication lockout, so that header alone cannot tell them apart.
2284 // GitHub's own message ("You have exceeded a secondary rate limit") can.
2285 error_message(&response.body).is_some_and(|m| m.to_ascii_lowercase().contains("rate limit"))
2286}
2287
2288/// Whether a `403` on a *first* attempt is GitHub continuing an authentication
2289/// lockout that outlived this client's back-off.
2290///
2291/// The two halves are both required, and both are GitHub's own evidence rather
2292/// than this client's memory:
2293///
2294/// * **`retry-after` is present.** GitHub sends it when it wants to be left
2295/// alone. A permissions refusal never does — there is nothing to wait for.
2296///
2297/// The header's *presence* is what is tested, not whether it parses, and that
2298/// is a fix rather than laziness. [`retry_after`] reads **integer seconds
2299/// only**, while RFC 9110 §10.2.3 also permits an HTTP-date. Gating detection
2300/// on `retry_after(..).is_some()` meant a date-form header was not recognised
2301/// as a continuation at all, and the bug this function exists to fix came
2302/// straight back for that shape — silently, since the response still looks
2303/// like an ordinary [`GithubError::Forbidden`] on the way out.
2304///
2305/// How long to wait stays a separate question, still answered by the integer
2306/// parse: [`AuthenticatedClient::latch_lockout`] already falls back to
2307/// [`DEFAULT_LOCKOUT_BACKOFF`] for a header it cannot read, so a date-form
2308/// header now latches sixty seconds instead of latching nothing. GitHub sends
2309/// integer seconds in practice; the point is not to depend on that.
2310/// * **The body carries no parseable GitHub message.** A permissions refusal
2311/// always names what is not accessible ("Resource not accessible by
2312/// integration"); the lockout's does not. This is the half that stops the rule
2313/// from swallowing [`GithubError::Forbidden`] entirely.
2314///
2315/// "No parseable GitHub message" is wider than "the body is empty", which is
2316/// how this used to be written, and the difference is worth stating because it
2317/// is what the code actually tests. [`error_message`] returns `None` for *any*
2318/// body that is not JSON carrying a non-empty `message`: an HTML error page
2319/// from a proxy or a CDN, a JSON body carrying only `documentation_url`, plain
2320/// text, a truncated response. Proxies routinely send `Retry-After` too, so a
2321/// `403` that never came from GitHub at all can read as an authentication
2322/// lockout here.
2323///
2324/// That is accepted rather than missed. The cost is bounded — the client
2325/// waits, clamped by [`MAX_LOCKOUT_BACKOFF`], then re-asks — and the direction
2326/// is the safe one: treating a strange `403` as "wait" costs latency, while
2327/// treating a real lockout as a permissions answer costs the back-off
2328/// entirely and tells the operator to fix a grant that is not missing.
2329/// Tightening it would mean asserting the body *is* GitHub's, which is
2330/// precisely the assertion an intercepting proxy makes false.
2331///
2332/// Callers reach this through [`AuthenticatedClient::is_lockout_403`], which
2333/// rules out a rate limit first — a secondary rate limit carries `retry-after`
2334/// *and* a message, so it fails this test on the second half anyway, but the
2335/// ordering makes the precedence explicit rather than incidental.
2336fn is_lockout_continuation(response: &ApiResponse) -> bool {
2337 response.headers.contains_key("retry-after") && error_message(&response.body).is_none()
2338}
2339
2340// ---------------------------------------------------------------------------
2341// Installation discovery
2342// ---------------------------------------------------------------------------
2343
2344/// Whether an installation can reach every repository on its account, or only
2345/// the ones the user picked.
2346#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
2347#[serde(rename_all = "snake_case")]
2348pub enum RepositorySelection {
2349 /// Every repository on the account, including ones created later.
2350 All,
2351 /// Only the repositories the user chose at install time.
2352 Selected,
2353}
2354
2355impl RepositorySelection {
2356 /// `07-security.md`: "`auth status` shows which repositories the token can
2357 /// reach, so an over-broad installation is visible rather than assumed."
2358 /// This is the flag that makes it visible.
2359 #[must_use]
2360 pub fn is_over_broad(self) -> bool {
2361 matches!(self, Self::All)
2362 }
2363}
2364
2365/// Whose account an installation sits on.
2366#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
2367#[serde(rename_all = "snake_case")]
2368pub enum InstallationAccount {
2369 User(String),
2370 Organization(Org),
2371 /// An enterprise account.
2372 ///
2373 /// It is its own variant rather than a [`InstallationAccount::User`]
2374 /// because it is not one, and `auth status` says out loud whose account
2375 /// each installation sits on. Everything GitHub reports without
2376 /// `type: "Organization"` used to fall into `User`, so an enterprise was
2377 /// labelled a user — a wrong statement about the operator's own account, on
2378 /// the one screen that exists to tell them what their credential reaches.
2379 ///
2380 /// It contributes nothing to [`ReachableTargets::organizations`], and that
2381 /// is correct rather than a second bug: an enterprise is not an
2382 /// organization, and `GET /orgs/{org}/actions/runners` does not accept one.
2383 /// The distinction is only visible now because the label is.
2384 Enterprise(String),
2385}
2386
2387impl InstallationAccount {
2388 #[must_use]
2389 pub fn login(&self) -> &str {
2390 match self {
2391 Self::User(login) | Self::Enterprise(login) => login,
2392 Self::Organization(org) => org.as_str(),
2393 }
2394 }
2395
2396 /// The organization, when the account is one. An organization account is a
2397 /// reachable *target* in its own right (D18): a policy may scale for the
2398 /// whole organization.
2399 #[must_use]
2400 pub fn organization(&self) -> Option<&Org> {
2401 match self {
2402 Self::Organization(org) => Some(org),
2403 Self::User(_) | Self::Enterprise(_) => None,
2404 }
2405 }
2406
2407 /// What to call this account in `auth status`. `f1` renders it; nothing in
2408 /// this crate branches on it.
2409 #[must_use]
2410 pub fn kind(&self) -> &'static str {
2411 match self {
2412 Self::User(_) => "user",
2413 Self::Organization(_) => "organization",
2414 Self::Enterprise(_) => "enterprise",
2415 }
2416 }
2417}
2418
2419impl fmt::Display for InstallationAccount {
2420 fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
2421 f.write_str(self.login())
2422 }
2423}
2424
2425/// One installation of the published App, and what it can actually reach.
2426///
2427/// Serializable, as is everything [`InstallationDiscovery`] holds, so that a
2428/// service can answer a discovery for a command that cannot read the
2429/// credential itself (an SSH session on a Mac, whose login keychain it cannot
2430/// unlock). Nothing in it is a secret.
2431#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
2432pub struct Installation {
2433 pub id: u64,
2434 pub account: InstallationAccount,
2435 pub repository_selection: RepositorySelection,
2436 pub repositories: Vec<OwnerRepo>,
2437 /// The permissions GitHub reports for this installation, as
2438 /// `name -> level`. Surfaced verbatim so `auth status` can show a grant the
2439 /// user did not expect rather than assert the published set was applied.
2440 pub permissions: Vec<(String, String)>,
2441}
2442
2443impl Installation {
2444 #[must_use]
2445 pub fn is_over_broad(&self) -> bool {
2446 self.repository_selection.is_over_broad()
2447 }
2448}
2449
2450/// Everything the stored credential can reach.
2451#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
2452pub struct ReachableTargets {
2453 installations: Vec<Installation>,
2454 skipped: usize,
2455}
2456
2457impl ReachableTargets {
2458 #[must_use]
2459 pub fn installations(&self) -> &[Installation] {
2460 &self.installations
2461 }
2462
2463 /// How many installations GitHub reported that this client could not
2464 /// describe, and therefore left out of everything above.
2465 ///
2466 /// Non-zero means this report is **incomplete**, not merely small: whatever
2467 /// those installations reach is absent from
2468 /// [`ReachableTargets::repositories`] and
2469 /// [`ReachableTargets::organizations`]. `auth status` should say so, because
2470 /// the alternative is an operator reading a short list as a complete one.
2471 #[must_use]
2472 pub fn skipped(&self) -> usize {
2473 self.skipped
2474 }
2475
2476 /// Every repository the credential can reach, sorted and de-duplicated.
2477 #[must_use]
2478 pub fn repositories(&self) -> Vec<OwnerRepo> {
2479 let mut all: Vec<OwnerRepo> = self
2480 .installations
2481 .iter()
2482 .flat_map(|i| i.repositories.iter().cloned())
2483 .collect();
2484 all.sort();
2485 all.dedup();
2486 all
2487 }
2488
2489 /// Every organization the App is installed on, sorted and de-duplicated.
2490 #[must_use]
2491 pub fn organizations(&self) -> Vec<Org> {
2492 let mut all: Vec<Org> = self
2493 .installations
2494 .iter()
2495 .filter_map(|i| i.account.organization().cloned())
2496 .collect();
2497 all.sort();
2498 all.dedup();
2499 all
2500 }
2501
2502 /// The installations that hold `repository_selection: all`.
2503 #[must_use]
2504 pub fn over_broad(&self) -> Vec<&Installation> {
2505 self.installations
2506 .iter()
2507 .filter(|i| i.is_over_broad())
2508 .collect()
2509 }
2510
2511 #[must_use]
2512 pub fn is_empty(&self) -> bool {
2513 self.repositories().is_empty() && self.organizations().is_empty()
2514 }
2515}
2516
2517/// What `auth status` and `auth login` show after a successful sign-in.
2518#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
2519#[serde(rename_all = "snake_case")]
2520pub enum InstallationDiscovery {
2521 /// The credential is valid, GitHub reported nothing this client could not
2522 /// describe, and still nothing is reachable: the App is installed nowhere,
2523 /// or on nothing. `03-control-flows.md` flow 1.1 requires the installation
2524 /// URL here, and the URL is the remediation.
2525 NotInstalled {
2526 #[serde(with = "url_text")]
2527 install_url: Url,
2528 },
2529 /// Nothing is reachable, but at least one installation was **skipped**, so
2530 /// this client cannot tell "not installed" from "installed on something it
2531 /// could not describe".
2532 ///
2533 /// # Why this variant exists at all
2534 ///
2535 /// Skipping an unnameable installation is the right trade — one odd
2536 /// installation must not take down `auth status` for every other one — but
2537 /// it was made silently, and the silence flipped a verdict. An account this
2538 /// client cannot name, on the *only* installation the credential has, used
2539 /// to collapse to [`InstallationDiscovery::NotInstalled`], and `auth status`
2540 /// then handed an already-installed operator the "install the App" URL. That
2541 /// is a wrong remediation on the only authentication path there is,
2542 /// contradicted by nothing louder than a `warn!` in a log the operator is
2543 /// not reading.
2544 ///
2545 /// So the skip stays and the verdict does not flip. There is deliberately no
2546 /// `install_url` here: the whole point is that this client does not know
2547 /// whether installing is the remedy, and offering the URL anyway would put
2548 /// the wrong answer back one field over. `f1` says "1 installation could not
2549 /// be described" and stops there, which is true.
2550 Indeterminate { skipped: usize },
2551 /// The credential reaches at least one repository or organization. It may
2552 /// still be an incomplete picture — see [`ReachableTargets::skipped`].
2553 Installed(ReachableTargets),
2554}
2555
2556impl InstallationDiscovery {
2557 #[must_use]
2558 pub fn targets(&self) -> Option<&ReachableTargets> {
2559 match self {
2560 Self::Installed(t) => Some(t),
2561 Self::NotInstalled { .. } | Self::Indeterminate { .. } => None,
2562 }
2563 }
2564
2565 /// The installation URL, and *only* when installing is actually the
2566 /// remediation. See [`InstallationDiscovery::Indeterminate`].
2567 #[must_use]
2568 pub fn install_url(&self) -> Option<&Url> {
2569 match self {
2570 Self::NotInstalled { install_url } => Some(install_url),
2571 Self::Installed(_) | Self::Indeterminate { .. } => None,
2572 }
2573 }
2574
2575 /// How many installations GitHub reported that this client could not
2576 /// describe, whichever verdict was reached. One call for `f1`, so that
2577 /// "this report is incomplete" does not depend on which variant it landed
2578 /// in.
2579 #[must_use]
2580 pub fn skipped(&self) -> usize {
2581 match self {
2582 Self::NotInstalled { .. } => 0,
2583 Self::Indeterminate { skipped } => *skipped,
2584 Self::Installed(targets) => targets.skipped(),
2585 }
2586 }
2587}
2588
2589/// A [`Url`] as its text: this workspace builds `url` without its `serde`
2590/// feature.
2591mod url_text {
2592 use serde::{Deserialize as _, Deserializer, Serializer};
2593 use url::Url;
2594
2595 pub(super) fn serialize<S: Serializer>(url: &Url, serializer: S) -> Result<S::Ok, S::Error> {
2596 serializer.serialize_str(url.as_str())
2597 }
2598
2599 pub(super) fn deserialize<'de, D: Deserializer<'de>>(deserializer: D) -> Result<Url, D::Error> {
2600 let text = String::deserialize(deserializer)?;
2601 Url::parse(&text).map_err(serde::de::Error::custom)
2602 }
2603}
2604
2605#[derive(Debug, Deserialize)]
2606struct InstallationsPage {
2607 /// GitHub reports the size of the whole collection on every page. Decoding
2608 /// it costs nothing and turns silent under-collection into a visible
2609 /// warning — see [`under_collected`].
2610 #[serde(default)]
2611 total_count: Option<u64>,
2612 #[serde(default)]
2613 installations: Vec<RawInstallation>,
2614}
2615
2616#[derive(Debug, Deserialize)]
2617struct RawInstallation {
2618 id: u64,
2619 /// **Nullable.** GitHub's published `installation` schema types `account` as
2620 /// nullable, so a required field here would fail the *whole* decode — and
2621 /// with it all of `discover_installations`, which is all of `auth status` —
2622 /// over one installation whose account this client did not need to name.
2623 #[serde(default)]
2624 account: Option<RawAccount>,
2625 #[serde(default)]
2626 repository_selection: Option<String>,
2627 #[serde(default)]
2628 permissions: std::collections::BTreeMap<String, String>,
2629}
2630
2631/// An installation's account, which is *not* always a simple user.
2632///
2633/// GitHub's schema makes `account` either a simple-user or an enterprise, and an
2634/// enterprise carries `slug` and `name` where a user carries `login`. Requiring
2635/// `login` therefore made an enterprise installation a hard decode failure of
2636/// the entire response. All three are optional here and
2637/// [`RawAccount::display_login`] takes the first usable one.
2638#[derive(Debug, Deserialize)]
2639struct RawAccount {
2640 #[serde(default)]
2641 login: Option<String>,
2642 /// An enterprise account's stable identifier.
2643 #[serde(default)]
2644 slug: Option<String>,
2645 /// An enterprise account's display name, the last resort.
2646 #[serde(default)]
2647 name: Option<String>,
2648 #[serde(rename = "type", default)]
2649 account_type: Option<String>,
2650}
2651
2652impl RawAccount {
2653 fn display_login(&self) -> Option<&str> {
2654 [
2655 self.login.as_deref(),
2656 self.slug.as_deref(),
2657 self.name.as_deref(),
2658 ]
2659 .into_iter()
2660 .flatten()
2661 .find(|value| !value.is_empty())
2662 }
2663
2664 /// An account with no `login` that still names itself is an enterprise:
2665 /// `slug`/`name` is the enterprise shape, and every simple-user and
2666 /// organization account carries `login`.
2667 fn is_enterprise_shaped(&self) -> bool {
2668 self.login.as_deref().is_none_or(str::is_empty)
2669 && (self.slug.as_deref().is_some_and(|s| !s.is_empty())
2670 || self.name.as_deref().is_some_and(|s| !s.is_empty()))
2671 }
2672}
2673
2674#[derive(Debug, Deserialize)]
2675struct RepositoriesPage {
2676 #[serde(default)]
2677 total_count: Option<u64>,
2678 #[serde(default)]
2679 repositories: Vec<RawRepository>,
2680}
2681
2682#[derive(Debug, Deserialize)]
2683struct RawRepository {
2684 full_name: String,
2685}
2686
2687/// How many items a paginated collection said it had, when that is more than
2688/// arrived.
2689///
2690/// This is the cheapest possible check and it is worth more than it looks. The
2691/// `Link`-header parser used to lose the relation whenever a page URL contained
2692/// a comma, which stopped pagination at page 1 — and *nothing* noticed, because
2693/// a short answer and a complete answer are the same shape. Cross-checking the
2694/// count GitHub itself reported turns that class of bug from a wrong answer into
2695/// a logged warning. A collection larger than `total_count` is not reported:
2696/// GitHub can legitimately grow a collection between pages.
2697fn under_collected(collected: usize, total_count: Option<u64>) -> Option<u64> {
2698 let total = total_count?;
2699 (total > collected as u64).then_some(total)
2700}
2701
2702impl AuthenticatedClient {
2703 /// Which repositories and organizations the stored credential can actually
2704 /// reach.
2705 ///
2706 /// Two calls, both paginated: `GET /user/installations`, then
2707 /// `GET /user/installations/{id}/repositories` per installation. The shapes
2708 /// are the ones the D18 spike observed live
2709 /// (`docs/spikes/d18-org-jit-verification.md`, "The permission that
2710 /// authorized it").
2711 ///
2712 /// An installation is reported even when it is broader than the user
2713 /// expected — [`Installation::is_over_broad`] — because `07-security.md`
2714 /// requires that an over-broad installation be *visible* rather than
2715 /// assumed. Nothing here narrows or hides one.
2716 ///
2717 /// # Errors
2718 /// Every variant of [`GithubError`]. A `401` here goes through the same
2719 /// single-flight re-validation as any other request.
2720 pub async fn discover_installations(
2721 &self,
2722 app: &AppRegistration,
2723 ) -> Result<InstallationDiscovery, GithubError> {
2724 let mut installations = Vec::new();
2725 let mut skipped = 0_usize;
2726 for raw in self.all_installations().await? {
2727 // A null or nameless account is skipped rather than fatal. GitHub
2728 // types this field as nullable, and one unnameable installation must
2729 // not take down `auth status` for every other one — but it is also
2730 // not something to swallow quietly, because the repositories behind
2731 // it are then absent from the reported reach. The count is what
2732 // carries that out of here; a `warn!` alone let the skip change the
2733 // verdict with nothing to say so.
2734 let Some(login) = raw.account.as_ref().and_then(RawAccount::display_login) else {
2735 skipped += 1;
2736 tracing::warn!(
2737 installation_id = raw.id,
2738 "skipping an installation GitHub reported with no nameable account; \
2739 anything it reaches is missing from this report"
2740 );
2741 continue;
2742 };
2743 let account_type = raw.account.as_ref().and_then(|a| a.account_type.as_deref());
2744 let account = match account_type {
2745 Some("Organization") => {
2746 InstallationAccount::Organization(Org::new(login).map_err(|_| {
2747 GithubError::Malformed {
2748 what: "an installation account login",
2749 value: login.to_string(),
2750 }
2751 })?)
2752 }
2753 Some("Enterprise") => InstallationAccount::Enterprise(login.to_string()),
2754 // An enterprise is also reported with no `type` at all, carrying
2755 // `slug`/`name` where a user carries `login` — which is the
2756 // shape D18 observed and the shape `display_login` exists for.
2757 // Recognising it by that shape is what stops it being labelled a
2758 // user by default.
2759 _ if raw
2760 .account
2761 .as_ref()
2762 .is_some_and(RawAccount::is_enterprise_shaped) =>
2763 {
2764 InstallationAccount::Enterprise(login.to_string())
2765 }
2766 _ => InstallationAccount::User(login.to_string()),
2767 };
2768 let repository_selection = match raw.repository_selection.as_deref() {
2769 Some("all") => RepositorySelection::All,
2770 _ => RepositorySelection::Selected,
2771 };
2772 installations.push(Installation {
2773 id: raw.id,
2774 account,
2775 repository_selection,
2776 repositories: self.installation_repositories(raw.id).await?,
2777 permissions: raw.permissions.into_iter().collect(),
2778 });
2779 }
2780
2781 let targets = ReachableTargets {
2782 installations,
2783 skipped,
2784 };
2785 if targets.is_empty() {
2786 // "Nothing reachable" and "nothing this client could describe" are
2787 // different answers, and only the first one is fixed by installing
2788 // the App. Reporting them as the same answer is how an
2789 // already-installed operator was handed an install URL.
2790 if skipped > 0 {
2791 tracing::warn!(
2792 skipped,
2793 "every installation GitHub reported was skipped; whether the App is \
2794 installed cannot be determined from this credential"
2795 );
2796 return Ok(InstallationDiscovery::Indeterminate { skipped });
2797 }
2798 let install_url = app.install_url(&self.endpoints);
2799 tracing::info!(
2800 install_url = %install_url,
2801 "the published App is not installed on anything this credential can reach"
2802 );
2803 return Ok(InstallationDiscovery::NotInstalled { install_url });
2804 }
2805 tracing::info!(
2806 repositories = targets.repositories().len(),
2807 organizations = targets.organizations().len(),
2808 over_broad = targets.over_broad().len(),
2809 skipped,
2810 "discovered the targets this credential can reach"
2811 );
2812 Ok(InstallationDiscovery::Installed(targets))
2813 }
2814
2815 async fn all_installations(&self) -> Result<Vec<RawInstallation>, GithubError> {
2816 let mut out = Vec::new();
2817 let mut total_count = None;
2818 let mut next = Some(ApiRequest::get("/user/installations").query("per_page", 100));
2819 let mut pages = 0_usize;
2820 while let Some(request) = next.take() {
2821 let response = self.send(&request).await?;
2822 let page: InstallationsPage = response.json()?;
2823 total_count = page.total_count.or(total_count);
2824 out.extend(page.installations);
2825
2826 pages += 1;
2827 if pages >= MAX_PAGES {
2828 tracing::warn!(
2829 pages,
2830 collected = out.len(),
2831 "stopped following installation pages at the ceiling; a `Link: rel=next` \
2832 that never ends would otherwise loop forever"
2833 );
2834 break;
2835 }
2836 next = response
2837 .next_page()
2838 .map(|url| ApiRequest::get(url.as_str()));
2839 }
2840 if let Some(expected) = under_collected(out.len(), total_count) {
2841 tracing::warn!(
2842 expected,
2843 collected = out.len(),
2844 "GitHub reported more installations than pagination collected; the reported \
2845 reach is incomplete"
2846 );
2847 }
2848 Ok(out)
2849 }
2850
2851 async fn installation_repositories(&self, id: u64) -> Result<Vec<OwnerRepo>, GithubError> {
2852 let mut out = Vec::new();
2853 let mut total_count = None;
2854 let mut next = Some(
2855 ApiRequest::get(format!("/user/installations/{id}/repositories"))
2856 .query("per_page", 100),
2857 );
2858 let mut pages = 0_usize;
2859 while let Some(request) = next.take() {
2860 let response = self.send(&request).await?;
2861 let page: RepositoriesPage = response.json()?;
2862 total_count = page.total_count.or(total_count);
2863 for repo in page.repositories {
2864 out.push(OwnerRepo::parse(&repo.full_name).map_err(|_| {
2865 GithubError::Malformed {
2866 what: "a repository full_name",
2867 value: repo.full_name.clone(),
2868 }
2869 })?);
2870 }
2871
2872 pages += 1;
2873 if pages >= MAX_PAGES {
2874 tracing::warn!(
2875 installation_id = id,
2876 pages,
2877 collected = out.len(),
2878 "stopped following repository pages at the ceiling; a `Link: rel=next` \
2879 that never ends would otherwise loop forever"
2880 );
2881 break;
2882 }
2883 next = response
2884 .next_page()
2885 .map(|url| ApiRequest::get(url.as_str()));
2886 }
2887 if let Some(expected) = under_collected(out.len(), total_count) {
2888 tracing::warn!(
2889 installation_id = id,
2890 expected,
2891 collected = out.len(),
2892 "GitHub reported more repositories than pagination collected; this \
2893 installation's reach is under-reported"
2894 );
2895 }
2896 Ok(out)
2897 }
2898}
2899
2900/// Test support shared by this file and [`device_flow`].
2901///
2902/// It lives inline rather than in `src/testing.rs` on purpose. `a1` laid out
2903/// this crate's five source files — `lib.rs`, `device_flow.rs`, `rest.rs`,
2904/// `demand.rs`, `jit.rs` — and owns every manifest; `c3` and `c4` are working in
2905/// the same directory in parallel, and a new file there is a merge conflict
2906/// waiting to happen for no benefit. An inline `#[cfg(test)]` module is
2907/// reachable as `crate::testing` from every module in the crate and adds nothing
2908/// to a release build.
2909///
2910/// It does not live in `runner-manager-testkit` either, and that one is
2911/// mechanical: `testkit` depends on `runner-manager-github`, so a unit test
2912/// inside this crate that used a `testkit` helper would link a *second* instance
2913/// of this library and the two instances' types would not unify — the same
2914/// hazard `testkit`'s own crate documentation records for `domain`.
2915#[cfg(test)]
2916pub(crate) mod testing {
2917 use super::*;
2918 use serde_json::{Value, json};
2919 use std::sync::{Mutex, atomic::AtomicUsize};
2920 use wiremock::{Request, Respond, ResponseTemplate};
2921
2922 /// Shaped like a real `ghu_` token, and unmistakably not one.
2923 pub const FIXTURE_TOKEN: &str = "ghu_fixtureTOKENnotARealCredential00";
2924 /// Shaped like a real device code, and unmistakably not one.
2925 pub const FIXTURE_DEVICE_CODE: &str = "fixture-device-code-0e37a9c1b4d84f2a";
2926 /// The example user code from RFC 8628.
2927 pub const FIXTURE_USER_CODE: &str = "WDJB-MJHT";
2928
2929 /// A clock the test moves.
2930 ///
2931 /// Deliberately not `runner_manager_testkit::clock::FakeClock`; see this
2932 /// module's documentation for why a `testkit` import is not available here.
2933 #[derive(Debug)]
2934 pub struct TestClock {
2935 now: Mutex<Timestamp>,
2936 }
2937
2938 impl TestClock {
2939 /// # Panics
2940 /// If a previous holder panicked while the lock was held.
2941 pub fn advance_secs(&self, secs: i64) {
2942 let mut now = self.now.lock().expect("TestClock lock poisoned");
2943 *now += chrono::TimeDelta::seconds(secs);
2944 }
2945 }
2946
2947 impl Default for TestClock {
2948 fn default() -> Self {
2949 // 2026-08-21T00:00:00Z, the date this taskflow's decisions were
2950 // locked — the same epoch `testkit`'s clock starts at.
2951 Self {
2952 now: Mutex::new(
2953 chrono::DateTime::from_timestamp(1_787_270_400, 0).expect("a valid instant"),
2954 ),
2955 }
2956 }
2957 }
2958
2959 impl Clock for TestClock {
2960 fn now(&self) -> Timestamp {
2961 *self.now.lock().expect("TestClock lock poisoned")
2962 }
2963 }
2964
2965 /// A sleeper that records what it was asked to wait and returns at once.
2966 ///
2967 /// This is what turns "`slow_down` demonstrably increases the poll interval"
2968 /// into an equality assertion on a `Vec<Duration>`.
2969 #[derive(Debug, Default)]
2970 pub struct RecordingSleeper {
2971 recorded: Mutex<Vec<Duration>>,
2972 }
2973
2974 impl RecordingSleeper {
2975 /// # Panics
2976 /// If a previous holder panicked while the lock was held.
2977 pub fn recorded(&self) -> Vec<Duration> {
2978 self.recorded.lock().expect("sleeper lock poisoned").clone()
2979 }
2980 }
2981
2982 #[async_trait::async_trait]
2983 impl Sleeper for RecordingSleeper {
2984 async fn sleep(&self, duration: Duration) {
2985 self.recorded
2986 .lock()
2987 .expect("sleeper lock poisoned")
2988 .push(duration);
2989 }
2990 }
2991
2992 /// Answers from a fixed script, one entry per call, repeating the last.
2993 pub struct Script {
2994 responses: Vec<ResponseTemplate>,
2995 calls: AtomicUsize,
2996 }
2997
2998 impl Script {
2999 #[must_use]
3000 pub fn new(responses: Vec<ResponseTemplate>) -> Self {
3001 assert!(
3002 !responses.is_empty(),
3003 "a script needs at least one response"
3004 );
3005 Self {
3006 responses,
3007 calls: AtomicUsize::new(0),
3008 }
3009 }
3010 }
3011
3012 impl Respond for Script {
3013 fn respond(&self, _: &Request) -> ResponseTemplate {
3014 let i = self.calls.fetch_add(1, Ordering::SeqCst);
3015 self.responses[i.min(self.responses.len() - 1)].clone()
3016 }
3017 }
3018
3019 /// `POST https://github.com/login/device/code` → `200`, in the shape both
3020 /// spikes observed (`docs/spikes/d17-spike.ps1`).
3021 #[must_use]
3022 pub fn device_code_body(server_uri: &str, interval: u64, expires_in: u64) -> Value {
3023 json!({
3024 "device_code": FIXTURE_DEVICE_CODE,
3025 "user_code": FIXTURE_USER_CODE,
3026 "verification_uri": format!("{server_uri}/login/device"),
3027 "expires_in": expires_in,
3028 "interval": interval
3029 })
3030 }
3031
3032 /// `POST .../login/oauth/access_token` → `200` with an `error` field, which
3033 /// is how GitHub answers every state in the matrix.
3034 #[must_use]
3035 pub fn error_body(code: &str, interval: Option<u64>) -> Value {
3036 let mut body = json!({
3037 "error": code,
3038 "error_description": "see the OAuth 2.0 Device Authorization Grant",
3039 "error_uri": "https://docs.github.com/developers/apps/authorizing-oauth-apps"
3040 });
3041 if let Some(interval) = interval {
3042 body["interval"] = json!(interval);
3043 }
3044 body
3045 }
3046
3047 /// `POST .../login/oauth/access_token` → `200` with an approved token.
3048 #[must_use]
3049 pub fn token_body() -> Value {
3050 json!({ "access_token": FIXTURE_TOKEN, "token_type": "bearer", "scope": "" })
3051 }
3052
3053 /// `GET /user/installations` → `200`. The permission set is the one D18 read
3054 /// back from the live installation.
3055 #[must_use]
3056 pub fn installations_body(entries: &[(u64, &str, &str, &str)]) -> Value {
3057 let installations: Vec<Value> = entries
3058 .iter()
3059 .map(|(id, login, account_type, selection)| {
3060 json!({
3061 "id": id,
3062 "account": { "login": login, "type": account_type },
3063 "repository_selection": selection,
3064 "permissions": {
3065 "actions": "read",
3066 "administration": "write",
3067 "metadata": "read",
3068 "organization_self_hosted_runners": "write"
3069 }
3070 })
3071 })
3072 .collect();
3073 json!({ "total_count": installations.len(), "installations": installations })
3074 }
3075
3076 /// `GET /user/installations/{id}/repositories` → `200`.
3077 #[must_use]
3078 pub fn repositories_body(full_names: &[&str]) -> Value {
3079 let repositories: Vec<Value> = full_names
3080 .iter()
3081 .map(|full_name| json!({ "full_name": full_name }))
3082 .collect();
3083 json!({ "total_count": repositories.len(), "repositories": repositories })
3084 }
3085
3086 // The `tracing` capture subscriber that used to live here now lives in
3087 // `tests/no_secret_reaches_the_logs.rs`, and the move is the point rather
3088 // than tidying. `tracing` caches a callsite's `Interest` process-wide while
3089 // `with_default` installs a subscriber only on the calling *thread*, so a
3090 // scan running alongside the crate's other unit tests captured nothing but
3091 // its own handful of events and passed with a real device-code leak in the
3092 // flow. A scan that is the only test in its process has no concurrent
3093 // thread to be poisoned by, and no `#[cfg(test)]` module here can offer
3094 // that guarantee.
3095 //
3096 // The blinding is a *concurrency* effect and not a permanent
3097 // first-registration one — see that file's header for the measurement that
3098 // separates the two. The distinction matters here because only the
3099 // concurrency reading implies what this comment concludes: that one test
3100 // per process is the fix.
3101}
3102
3103#[cfg(test)]
3104mod tests {
3105
3106 /// Upgrading must not log anybody out, and the App's expiration setting must
3107 /// be safe to turn on -- or back off -- with hosts mid-way through either.
3108 #[test]
3109 fn both_stored_shapes_load_and_a_pair_survives_a_round_trip() {
3110 // What every host stored before renewal existed: a bare token.
3111 let legacy = UserAccessToken::from_stored_document(&SecretString::from("ghu_legacy123"));
3112 assert_eq!(legacy.secret().expose_secret(), "ghu_legacy123");
3113 assert!(
3114 legacy.renewal().is_none(),
3115 "a bare token has no renewal half, and inventing one would make the client try to refresh a credential the App never issued a refresh token for"
3116 );
3117
3118 // A pair, written and read back.
3119 let pair = UserAccessToken::new(SecretString::from("ghu_new")).with_renewal(
3120 Some(SecretString::from("ghr_new")),
3121 Some(28_800),
3122 Some(15_897_600),
3123 );
3124 let stored = pair.to_stored_document();
3125 let read = UserAccessToken::from_stored_document(&stored);
3126 assert_eq!(read.secret().expose_secret(), "ghu_new");
3127 let renewal = read.renewal().expect("the pair survives the round trip");
3128 assert_eq!(renewal.refresh_token().expose_secret(), "ghr_new");
3129 assert!(renewal.access_expires_at.is_some());
3130 assert!(renewal.refresh_expires_at.is_some());
3131
3132 // A credential with no renewal still writes the document shape, and
3133 // still reads back as having none.
3134 let bare_round_trip = UserAccessToken::from_stored_document(&legacy.to_stored_document());
3135 assert_eq!(bare_round_trip.secret().expose_secret(), "ghu_legacy123");
3136 assert!(bare_round_trip.renewal().is_none());
3137 }
3138
3139 /// The refresh token is the more dangerous half -- it mints access tokens
3140 /// for six months -- so it must not reach a log through `Debug`.
3141 #[test]
3142 fn a_refresh_token_never_appears_in_debug_output() {
3143 let pair = UserAccessToken::new(SecretString::from("ghu_x")).with_renewal(
3144 Some(SecretString::from("ghr_SUPERSECRET")),
3145 Some(1),
3146 Some(2),
3147 );
3148 let rendered = format!("{:?}", pair.renewal().expect("a renewal"));
3149 assert!(!rendered.contains("ghr_SUPERSECRET"), "{rendered}");
3150 assert!(rendered.contains("redacted"), "{rendered}");
3151 }
3152 use super::*;
3153 use crate::testing::{FIXTURE_TOKEN, Script, TestClock, installations_body, repositories_body};
3154 use serde_json::json;
3155 use wiremock::{
3156 Mock, MockServer, ResponseTemplate,
3157 matchers::{header, method, path},
3158 };
3159
3160 fn client(server: &MockServer, clock: Arc<TestClock>) -> AuthenticatedClient {
3161 AuthenticatedClient::new(
3162 Endpoints::for_test_server(&server.uri()).unwrap(),
3163 UserAccessToken::new(SecretString::from(FIXTURE_TOKEN)),
3164 clock,
3165 )
3166 .unwrap()
3167 }
3168
3169 fn app() -> AppRegistration {
3170 AppRegistration::new("Iv23liTESTCLIENTID", "runner-manager").unwrap()
3171 }
3172
3173 // -- picking up a credential somebody else stored -------------------------
3174
3175 /// A [`CredentialSource`] over a fixed answer, which is what a store looks
3176 /// like from here.
3177 #[derive(Debug)]
3178 struct StoreHolding(Option<&'static str>);
3179
3180 impl CredentialSource for StoreHolding {
3181 fn reload(&self) -> Option<UserAccessToken> {
3182 self.0
3183 .map(|token| UserAccessToken::new(SecretString::from(token)))
3184 }
3185 }
3186
3187 /// The 28-hour failure, as a test: a daemon holding a dead bare token, an
3188 /// operator who signs in, and nothing that tells the daemon.
3189 ///
3190 /// Bare on purpose. A pair would renew and never reach the store at all,
3191 /// which is why renewal alone did not cover this.
3192 #[tokio::test]
3193 async fn a_daemon_picks_up_a_sign_in_that_happened_after_it_started() {
3194 let server = MockServer::start().await;
3195 Mock::given(method("GET"))
3196 .and(path("/repos/acme/app"))
3197 .and(header("authorization", "Bearer ghu_dead"))
3198 .respond_with(ResponseTemplate::new(401))
3199 .expect(1)
3200 .mount(&server)
3201 .await;
3202 Mock::given(method("GET"))
3203 .and(path("/repos/acme/app"))
3204 .and(header("authorization", "Bearer ghu_freshly_signed_in"))
3205 .respond_with(ResponseTemplate::new(200).set_body_json(json!({"id": 1})))
3206 .expect(1)
3207 .mount(&server)
3208 .await;
3209
3210 let client = AuthenticatedClient::new(
3211 Endpoints::for_test_server(&server.uri()).unwrap(),
3212 UserAccessToken::new(SecretString::from("ghu_dead")),
3213 Arc::new(TestClock::default()),
3214 )
3215 .unwrap()
3216 .with_credential_source(Arc::new(StoreHolding(Some("ghu_freshly_signed_in"))));
3217
3218 client
3219 .send(&ApiRequest::get("/repos/acme/app"))
3220 .await
3221 .expect(
3222 "the 401 is retried with what the store holds now, without anybody \n restarting the daemon",
3223 );
3224 }
3225
3226 /// The other half, and the reason for the comparison in `reload_once`: a
3227 /// store that still holds the token that just failed is not news.
3228 ///
3229 /// Without the check, every `401` would answer "something changed, retry"
3230 /// and a genuinely revoked credential would spend two requests per poll
3231 /// forever instead of being reported.
3232 #[tokio::test]
3233 async fn a_store_holding_the_same_dead_token_is_not_worth_a_retry() {
3234 let server = MockServer::start().await;
3235 Mock::given(method("GET"))
3236 .and(path("/repos/acme/app"))
3237 .respond_with(ResponseTemplate::new(401))
3238 .expect(1)
3239 .mount(&server)
3240 .await;
3241 // The re-validation probe that runs once reload declines.
3242 Mock::given(method("GET"))
3243 .and(path("/user/installations"))
3244 .respond_with(ResponseTemplate::new(401))
3245 .expect(1)
3246 .mount(&server)
3247 .await;
3248
3249 let client = AuthenticatedClient::new(
3250 Endpoints::for_test_server(&server.uri()).unwrap(),
3251 UserAccessToken::new(SecretString::from("ghu_revoked")),
3252 Arc::new(TestClock::default()),
3253 )
3254 .unwrap()
3255 .with_credential_source(Arc::new(StoreHolding(Some("ghu_revoked"))));
3256
3257 let failure = client
3258 .send(&ApiRequest::get("/repos/acme/app"))
3259 .await
3260 .expect_err("a revoked credential is still revoked when the store agrees");
3261 assert!(
3262 matches!(failure, GithubError::AuthenticationFailed),
3263 "{failure:?}"
3264 );
3265 }
3266
3267 /// An unreadable store leaves the `401` exactly where it was, rather than
3268 /// turning a rejection into a different kind of error.
3269 #[tokio::test]
3270 async fn an_unreadable_store_changes_nothing_about_the_rejection() {
3271 let server = MockServer::start().await;
3272 Mock::given(method("GET"))
3273 .and(path("/repos/acme/app"))
3274 .respond_with(ResponseTemplate::new(401))
3275 .expect(1)
3276 .mount(&server)
3277 .await;
3278 Mock::given(method("GET"))
3279 .and(path("/user/installations"))
3280 .respond_with(ResponseTemplate::new(401))
3281 .expect(1)
3282 .mount(&server)
3283 .await;
3284
3285 let client = AuthenticatedClient::new(
3286 Endpoints::for_test_server(&server.uri()).unwrap(),
3287 UserAccessToken::new(SecretString::from("ghu_revoked")),
3288 Arc::new(TestClock::default()),
3289 )
3290 .unwrap()
3291 .with_credential_source(Arc::new(StoreHolding(None)));
3292
3293 let failure = client
3294 .send(&ApiRequest::get("/repos/acme/app"))
3295 .await
3296 .expect_err("nothing to pick up means the rejection stands");
3297 assert!(
3298 matches!(failure, GithubError::AuthenticationFailed),
3299 "{failure:?}"
3300 );
3301 }
3302
3303 // -- headers ------------------------------------------------------------
3304
3305 #[tokio::test]
3306 async fn every_request_states_its_api_version_and_accept_header() {
3307 let server = MockServer::start().await;
3308 Mock::given(method("GET"))
3309 .and(path("/user/installations"))
3310 .and(header("x-github-api-version", GITHUB_API_VERSION))
3311 .and(header("accept", GITHUB_ACCEPT))
3312 .and(header("authorization", format!("Bearer {FIXTURE_TOKEN}")))
3313 .and(header("user-agent", USER_AGENT))
3314 .respond_with(ResponseTemplate::new(200).set_body_json(installations_body(&[])))
3315 .expect(1)
3316 .mount(&server)
3317 .await;
3318
3319 let client = client(&server, Arc::new(TestClock::default()));
3320 client
3321 .send(&ApiRequest::get("/user/installations"))
3322 .await
3323 .expect("the mock only matches when all four headers are present");
3324 }
3325
3326 // -- the 401 path -------------------------------------------------------
3327
3328 #[tokio::test]
3329 async fn a_401_revalidates_once_and_retries_once_then_succeeds() {
3330 let server = MockServer::start().await;
3331 Mock::given(method("GET"))
3332 .and(path("/orgs/acme/actions/runners"))
3333 .respond_with(Script::new(vec![
3334 ResponseTemplate::new(401).set_body_json(json!({"message": "Bad credentials"})),
3335 ResponseTemplate::new(200).set_body_json(json!({"total_count": 0})),
3336 ]))
3337 .expect(2)
3338 .mount(&server)
3339 .await;
3340 Mock::given(method("GET"))
3341 .and(path("/user/installations"))
3342 .respond_with(ResponseTemplate::new(200).set_body_json(installations_body(&[])))
3343 .expect(1)
3344 .mount(&server)
3345 .await;
3346
3347 let client = client(&server, Arc::new(TestClock::default()));
3348 let response = client
3349 .send(&ApiRequest::get("/orgs/acme/actions/runners"))
3350 .await
3351 .expect("the retry succeeds");
3352
3353 assert_eq!(response.status(), StatusCode::OK);
3354 assert_eq!(
3355 client.revalidations_performed(),
3356 1,
3357 "one 401 must produce exactly one re-validation"
3358 );
3359 }
3360
3361 #[tokio::test]
3362 async fn a_second_401_after_the_retry_is_terminal_authentication_failure() {
3363 let server = MockServer::start().await;
3364 Mock::given(method("GET"))
3365 .and(path("/orgs/acme/actions/runners"))
3366 .respond_with(ResponseTemplate::new(401))
3367 .expect(2)
3368 .mount(&server)
3369 .await;
3370 Mock::given(method("GET"))
3371 .and(path("/user/installations"))
3372 .respond_with(ResponseTemplate::new(200).set_body_json(installations_body(&[])))
3373 .mount(&server)
3374 .await;
3375
3376 let client = client(&server, Arc::new(TestClock::default()));
3377 let err = client
3378 .send(&ApiRequest::get("/orgs/acme/actions/runners"))
3379 .await
3380 .expect_err("two 401s is terminal");
3381
3382 assert!(matches!(err, GithubError::AuthenticationFailed), "{err:?}");
3383 assert!(err.is_authentication());
3384 assert!(!err.is_lockout());
3385 }
3386
3387 #[tokio::test]
3388 async fn a_rejected_revalidation_fails_without_spending_the_retry() {
3389 let server = MockServer::start().await;
3390 Mock::given(method("GET"))
3391 .and(path("/orgs/acme/actions/runners"))
3392 .respond_with(ResponseTemplate::new(401))
3393 // Exactly one: a credential GitHub has confirmed dead must not be
3394 // used for a retry.
3395 .expect(1)
3396 .mount(&server)
3397 .await;
3398 Mock::given(method("GET"))
3399 .and(path("/user/installations"))
3400 .respond_with(ResponseTemplate::new(401))
3401 .expect(1)
3402 .mount(&server)
3403 .await;
3404
3405 let client = client(&server, Arc::new(TestClock::default()));
3406 let err = client
3407 .send(&ApiRequest::get("/orgs/acme/actions/runners"))
3408 .await
3409 .expect_err("the credential is dead");
3410 assert!(matches!(err, GithubError::AuthenticationFailed), "{err:?}");
3411 }
3412
3413 /// The Definition of Done's concurrency claim, tested with real concurrent
3414 /// callers on a multi-threaded runtime rather than by reasoning about the
3415 /// mutex.
3416 ///
3417 /// Two things make the assertion deterministic rather than lucky. A barrier
3418 /// releases all eight callers into `send` together, so all eight take their
3419 /// `401` before any of them reaches the gate; and the re-validation endpoint
3420 /// is delayed, so the first caller still holds the gate while the other
3421 /// seven sample the generation counter. Without the delay a caller could
3422 /// legitimately arrive after the first re-validation completed, which is a
3423 /// *new* `401` storm and correctly earns its own attempt.
3424 #[tokio::test(flavor = "multi_thread", worker_threads = 8)]
3425 async fn eight_concurrent_401s_produce_one_revalidation_not_eight() {
3426 const CALLERS: usize = 8;
3427
3428 let server = MockServer::start().await;
3429 Mock::given(method("GET"))
3430 .and(path("/orgs/acme/actions/runners"))
3431 .respond_with(ResponseTemplate::new(401))
3432 .mount(&server)
3433 .await;
3434 Mock::given(method("GET"))
3435 .and(path("/user/installations"))
3436 .respond_with(
3437 ResponseTemplate::new(200)
3438 .set_body_json(installations_body(&[]))
3439 .set_delay(Duration::from_millis(250)),
3440 )
3441 .expect(1)
3442 .mount(&server)
3443 .await;
3444
3445 let client = Arc::new(client(&server, Arc::new(TestClock::default())));
3446 let barrier = Arc::new(tokio::sync::Barrier::new(CALLERS));
3447 let mut tasks = Vec::new();
3448 for _ in 0..CALLERS {
3449 let client = Arc::clone(&client);
3450 let barrier = Arc::clone(&barrier);
3451 tasks.push(tokio::spawn(async move {
3452 barrier.wait().await;
3453 client
3454 .send(&ApiRequest::get("/orgs/acme/actions/runners"))
3455 .await
3456 .expect_err("every caller sees a dead endpoint")
3457 }));
3458 }
3459
3460 let mut outcomes = Vec::new();
3461 for task in tasks {
3462 outcomes.push(task.await.expect("no caller panicked"));
3463 }
3464
3465 assert_eq!(outcomes.len(), CALLERS);
3466 for err in &outcomes {
3467 assert!(matches!(err, GithubError::AuthenticationFailed), "{err:?}");
3468 }
3469 assert_eq!(
3470 client.revalidations_performed(),
3471 1,
3472 "{CALLERS} concurrent 401s must produce ONE attempt, not {CALLERS}"
3473 );
3474
3475 // The same claim, measured from the server rather than from our own
3476 // counter: the mock's `.expect(1)` is verified when the server drops.
3477 let seen = server.received_requests().await.expect("recording is on");
3478 let probes = seen
3479 .iter()
3480 .filter(|r| r.url.path() == "/user/installations")
3481 .count();
3482 assert_eq!(probes, 1, "GitHub itself saw exactly one re-validation");
3483 let attempts = seen
3484 .iter()
3485 .filter(|r| r.url.path() == "/orgs/acme/actions/runners")
3486 .count();
3487 assert_eq!(
3488 attempts,
3489 CALLERS * 2,
3490 "each caller still gets its own single retry"
3491 );
3492 }
3493
3494 // -- the 403 path -------------------------------------------------------
3495
3496 #[tokio::test]
3497 async fn a_403_after_401s_is_a_lockout_and_not_an_authentication_failure() {
3498 let server = MockServer::start().await;
3499 Mock::given(method("GET"))
3500 .and(path("/orgs/acme/actions/runners"))
3501 .respond_with(Script::new(vec![
3502 ResponseTemplate::new(401),
3503 ResponseTemplate::new(403).insert_header("retry-after", "42"),
3504 ]))
3505 .mount(&server)
3506 .await;
3507 Mock::given(method("GET"))
3508 .and(path("/user/installations"))
3509 .respond_with(ResponseTemplate::new(200).set_body_json(installations_body(&[])))
3510 .mount(&server)
3511 .await;
3512
3513 let client = client(&server, Arc::new(TestClock::default()));
3514 let err = client
3515 .send(&ApiRequest::get("/orgs/acme/actions/runners"))
3516 .await
3517 .expect_err("403 after a 401");
3518
3519 match err {
3520 GithubError::AuthenticationLockout { retry_after } => {
3521 assert_eq!(retry_after, Duration::from_secs(42), "honours retry-after");
3522 }
3523 other => panic!("expected a lockout, got {other:?}"),
3524 }
3525 assert!(client.is_locked_out());
3526 }
3527
3528 #[tokio::test]
3529 async fn a_403_with_no_preceding_401_is_a_permissions_answer_not_a_lockout() {
3530 let server = MockServer::start().await;
3531 Mock::given(method("GET"))
3532 .and(path("/orgs/acme/actions/runners"))
3533 .respond_with(
3534 ResponseTemplate::new(403)
3535 .set_body_json(json!({"message": "Resource not accessible by integration"})),
3536 )
3537 .mount(&server)
3538 .await;
3539
3540 let client = client(&server, Arc::new(TestClock::default()));
3541 let err = client
3542 .send(&ApiRequest::get("/orgs/acme/actions/runners"))
3543 .await
3544 .expect_err("403");
3545
3546 assert!(matches!(err, GithubError::Forbidden { .. }), "{err:?}");
3547 assert!(!err.is_lockout());
3548 assert!(!err.is_authentication());
3549 assert!(!client.is_locked_out(), "a permissions 403 must not latch");
3550 }
3551
3552 #[tokio::test]
3553 async fn a_locked_out_client_issues_no_further_http_until_the_backoff_elapses() {
3554 let server = MockServer::start().await;
3555 Mock::given(method("GET"))
3556 .and(path("/orgs/acme/actions/runners"))
3557 .respond_with(Script::new(vec![
3558 ResponseTemplate::new(401),
3559 ResponseTemplate::new(403).insert_header("retry-after", "60"),
3560 ResponseTemplate::new(200).set_body_json(json!({"total_count": 0})),
3561 ]))
3562 .mount(&server)
3563 .await;
3564 Mock::given(method("GET"))
3565 .and(path("/user/installations"))
3566 .respond_with(ResponseTemplate::new(200).set_body_json(installations_body(&[])))
3567 .mount(&server)
3568 .await;
3569
3570 let clock = Arc::new(TestClock::default());
3571 let client = client(&server, Arc::clone(&clock));
3572 let request = ApiRequest::get("/orgs/acme/actions/runners");
3573
3574 let err = client.send(&request).await.expect_err("locks out");
3575 assert!(err.is_lockout(), "{err:?}");
3576
3577 let after_lockout = server.received_requests().await.unwrap().len();
3578
3579 for _ in 0..3 {
3580 let err = client.send(&request).await.expect_err("still locked out");
3581 assert!(err.is_lockout(), "{err:?}");
3582 }
3583 assert_eq!(
3584 server.received_requests().await.unwrap().len(),
3585 after_lockout,
3586 "a backed-off client must open no sockets at all"
3587 );
3588
3589 clock.advance_secs(61);
3590 assert!(!client.is_locked_out(), "the back-off expires on the clock");
3591 let response = client.send(&request).await.expect("traffic resumes");
3592 assert_eq!(response.status(), StatusCode::OK);
3593 assert_eq!(
3594 server.received_requests().await.unwrap().len(),
3595 after_lockout + 1
3596 );
3597 }
3598
3599 /// `consecutive_unauthorized` is reset only by a successful *caller*
3600 /// response, so a request ending in `404`, `422` or `5xx` leaves it set —
3601 /// and in the agent's long-lived reconciliation loop it stays set for as
3602 /// long as nothing succeeds. Before the fix, the next genuine permissions
3603 /// `403` was therefore reported as `AuthenticationLockout`: sixty seconds of
3604 /// client silence, plus an operator message asserting "the credential itself
3605 /// is not the problem" about a credential that was missing
3606 /// `Administration: write` — the failure `04-subsystem-contracts.md` names
3607 /// as the *expected* one for `generate-jitconfig`.
3608 ///
3609 /// The lockout's real signature is narrower: a `403` on the one retry this
3610 /// client issues after this request's own `401`, or a `403` whose own
3611 /// headers and body say GitHub is continuing a lockout. Neither is "the
3612 /// count is non-zero", which is what a stale `401` leaves behind — so the
3613 /// permissions `403` below is a permissions answer whatever happened minutes
3614 /// ago, and it is the *response*, not the history, that decides.
3615 #[tokio::test]
3616 async fn a_stale_401_does_not_turn_a_later_permissions_403_into_a_lockout() {
3617 let server = MockServer::start().await;
3618 // The first request ends in a 404, which leaves the 401 count set
3619 // because only a 2xx clears it.
3620 Mock::given(method("GET"))
3621 .and(path("/orgs/acme/actions/runners"))
3622 .respond_with(Script::new(vec![
3623 ResponseTemplate::new(401),
3624 ResponseTemplate::new(404).set_body_json(json!({"message": "Not Found"})),
3625 ]))
3626 .mount(&server)
3627 .await;
3628 Mock::given(method("GET"))
3629 .and(path("/user/installations"))
3630 .respond_with(ResponseTemplate::new(200).set_body_json(installations_body(&[])))
3631 .mount(&server)
3632 .await;
3633 // Minutes later, a different call is denied for a missing permission.
3634 Mock::given(method("POST"))
3635 .and(path("/orgs/acme/actions/runners/generate-jitconfig"))
3636 .respond_with(
3637 ResponseTemplate::new(403)
3638 .set_body_json(json!({"message": "Resource not accessible by integration"})),
3639 )
3640 .mount(&server)
3641 .await;
3642
3643 let client = client(&server, Arc::new(TestClock::default()));
3644 let err = client
3645 .send(&ApiRequest::get("/orgs/acme/actions/runners"))
3646 .await
3647 .expect_err("404");
3648 assert!(
3649 matches!(err, GithubError::Status { status: 404, .. }),
3650 "{err:?}"
3651 );
3652
3653 let err = client
3654 .send(&ApiRequest::new(
3655 Method::POST,
3656 "/orgs/acme/actions/runners/generate-jitconfig",
3657 ))
3658 .await
3659 .expect_err("403");
3660
3661 assert!(
3662 matches!(err, GithubError::Forbidden { .. }),
3663 "a fresh first-attempt 403 is a permissions answer, not a lockout: {err:?}"
3664 );
3665 assert!(!err.is_lockout());
3666 assert!(
3667 !client.is_locked_out(),
3668 "a stale 401 must not be able to silence the client for a minute"
3669 );
3670 }
3671
3672 /// GitHub's own rate limit is not an answer about the credential, and must
3673 /// not be reported as one. `classify` reached the `403` branch before
3674 /// anything looked at the rate-limit headers, so a primary rate limit
3675 /// arriving during a `401` storm was announced as an authentication lockout
3676 /// with the message "the credential itself is not the problem" — about a
3677 /// response that never mentioned the credential.
3678 #[tokio::test]
3679 async fn a_rate_limited_403_is_not_reported_as_an_authentication_lockout() {
3680 let server = MockServer::start().await;
3681 Mock::given(method("GET"))
3682 .and(path("/orgs/acme/actions/runners"))
3683 .respond_with(Script::new(vec![
3684 ResponseTemplate::new(401),
3685 ResponseTemplate::new(403)
3686 .insert_header("x-ratelimit-remaining", "0")
3687 .insert_header("x-ratelimit-reset", "1787270460")
3688 .insert_header("retry-after", "30")
3689 .set_body_json(json!({"message": "API rate limit exceeded"})),
3690 ]))
3691 .mount(&server)
3692 .await;
3693 Mock::given(method("GET"))
3694 .and(path("/user/installations"))
3695 .respond_with(ResponseTemplate::new(200).set_body_json(installations_body(&[])))
3696 .mount(&server)
3697 .await;
3698
3699 let client = client(&server, Arc::new(TestClock::default()));
3700 let err = client
3701 .send(&ApiRequest::get("/orgs/acme/actions/runners"))
3702 .await
3703 .expect_err("rate limited");
3704
3705 assert!(
3706 matches!(err, GithubError::Forbidden { .. }),
3707 "a rate limit is not an authentication outcome: {err:?}"
3708 );
3709 assert!(!err.is_lockout());
3710 assert!(!err.is_authentication());
3711 assert!(
3712 !client.is_locked_out(),
3713 "a rate limit must not latch this crate's authentication back-off"
3714 );
3715
3716 // And `c3` gets the evidence it needs to apply the policy that is its
3717 // own, without editing this file.
3718 let evidence = err
3719 .rate_limit()
3720 .expect("the headers survived classification");
3721 assert_eq!(evidence.remaining, Some(0));
3722 assert_eq!(evidence.reset_unix_secs, Some(1_787_270_460));
3723 assert_eq!(evidence.retry_after, Some(Duration::from_secs(30)));
3724 }
3725
3726 /// The same claim for the variant `429` lands in.
3727 #[tokio::test]
3728 async fn a_429_carries_its_retry_after_across_the_c2_c3_seam() {
3729 let server = MockServer::start().await;
3730 Mock::given(method("GET"))
3731 .and(path("/orgs/acme/actions/runners"))
3732 .respond_with(
3733 ResponseTemplate::new(429)
3734 .insert_header("retry-after", "17")
3735 .insert_header("x-ratelimit-remaining", "0")
3736 .set_body_json(json!({"message": "You have exceeded a secondary rate limit"})),
3737 )
3738 .mount(&server)
3739 .await;
3740
3741 let client = client(&server, Arc::new(TestClock::default()));
3742 let err = client
3743 .send(&ApiRequest::get("/orgs/acme/actions/runners"))
3744 .await
3745 .expect_err("429");
3746
3747 assert!(
3748 matches!(err, GithubError::Status { status: 429, .. }),
3749 "{err:?}"
3750 );
3751 assert_eq!(
3752 err.retry_after(),
3753 Some(Duration::from_secs(17)),
3754 "destroying this header is what made `c3`'s Definition of Done unmeetable"
3755 );
3756 assert_eq!(
3757 err.headers().and_then(|h| h.get("x-ratelimit-remaining")),
3758 Some(&reqwest::header::HeaderValue::from_static("0"))
3759 );
3760 }
3761
3762 /// A back-off is a safety mechanism, and this one had both failure modes at
3763 /// once: no ceiling, so `Retry-After: 86400` latched a silent twenty-four
3764 /// hour outage; and `TimeDelta::from_std(...).ok()` on a value too large to
3765 /// convert, which yielded `until = None` — *not locked out at all*, the
3766 /// exact inverse of the requirement, reachable by a header alone.
3767 #[tokio::test]
3768 async fn an_extreme_retry_after_is_clamped_and_never_fails_open() {
3769 async fn lockout_for(header: &str) -> (GithubError, bool, Option<Duration>) {
3770 let server = MockServer::start().await;
3771 Mock::given(method("GET"))
3772 .and(path("/orgs/acme/actions/runners"))
3773 .respond_with(Script::new(vec![
3774 ResponseTemplate::new(401),
3775 ResponseTemplate::new(403).insert_header("retry-after", header),
3776 ]))
3777 .mount(&server)
3778 .await;
3779 Mock::given(method("GET"))
3780 .and(path("/user/installations"))
3781 .respond_with(ResponseTemplate::new(200).set_body_json(installations_body(&[])))
3782 .mount(&server)
3783 .await;
3784
3785 let client = AuthenticatedClient::new(
3786 Endpoints::for_test_server(&server.uri()).unwrap(),
3787 UserAccessToken::new(SecretString::from(FIXTURE_TOKEN)),
3788 Arc::new(TestClock::default()),
3789 )
3790 .unwrap();
3791 let err = client
3792 .send(&ApiRequest::get("/orgs/acme/actions/runners"))
3793 .await
3794 .expect_err("403 after a 401");
3795 let locked = client.is_locked_out();
3796 let remaining = client.lockout_remaining();
3797 (err, locked, remaining)
3798 }
3799
3800 // A day-long back-off is clamped to the ceiling.
3801 let (err, locked, remaining) = lockout_for("86400").await;
3802 let GithubError::AuthenticationLockout { retry_after } = &err else {
3803 panic!("expected a lockout, got {err:?}");
3804 };
3805 assert_eq!(
3806 *retry_after, MAX_LOCKOUT_BACKOFF,
3807 "an unclamped Retry-After lets a remote party decide how long this product \
3808 stays down"
3809 );
3810 assert!(locked);
3811 assert!(remaining.is_some_and(|r| r <= MAX_LOCKOUT_BACKOFF));
3812
3813 // A value too large for `chrono` must still lock out. Before the fix
3814 // this produced `until = None`: the more extreme the header, the less
3815 // protection it bought.
3816 let (err, locked, remaining) = lockout_for(&u64::MAX.to_string()).await;
3817 assert!(err.is_lockout(), "{err:?}");
3818 assert!(
3819 locked,
3820 "an absurd Retry-After must not mean `not locked out at all` — that fails open"
3821 );
3822 assert!(remaining.is_some_and(|r| r <= MAX_LOCKOUT_BACKOFF));
3823 }
3824
3825 /// The lockout's own contract is "this client issues no HTTP at all", and
3826 /// `revalidate` is HTTP. It documented an `AuthenticationLockout` it could
3827 /// never return, which made the one direct entry point into the probe the
3828 /// single exception to the rule.
3829 #[tokio::test]
3830 async fn a_direct_revalidation_is_refused_while_the_lockout_is_backing_off() {
3831 let server = MockServer::start().await;
3832 Mock::given(method("GET"))
3833 .and(path("/orgs/acme/actions/runners"))
3834 .respond_with(Script::new(vec![
3835 ResponseTemplate::new(401),
3836 ResponseTemplate::new(403).insert_header("retry-after", "60"),
3837 ]))
3838 .mount(&server)
3839 .await;
3840 Mock::given(method("GET"))
3841 .and(path("/user/installations"))
3842 .respond_with(ResponseTemplate::new(200).set_body_json(installations_body(&[])))
3843 .mount(&server)
3844 .await;
3845
3846 let client = client(&server, Arc::new(TestClock::default()));
3847 client
3848 .send(&ApiRequest::get("/orgs/acme/actions/runners"))
3849 .await
3850 .expect_err("locks out");
3851 assert!(client.is_locked_out());
3852
3853 let before = server.received_requests().await.unwrap().len();
3854 let err = client
3855 .revalidate()
3856 .await
3857 .expect_err("the documented lockout error is now reachable");
3858 assert!(err.is_lockout(), "{err:?}");
3859 assert_eq!(
3860 server.received_requests().await.unwrap().len(),
3861 before,
3862 "a locked-out client opens no socket, and the probe is not an exception"
3863 );
3864 }
3865
3866 /// The position rule fixed `classify` and left the same defect one function
3867 /// over, behind a comment asserting it could not happen: "the probe only
3868 /// ever runs after a `401`, so it is always in the retry position". Making
3869 /// [`AuthenticatedClient::revalidate`] public — the previous round's own
3870 /// change — is exactly what made that untrue.
3871 ///
3872 /// The sequence is the agent's, not a contrivance. A request 401s, the probe
3873 /// says the credential is fine, the retry answers `404` — which does *not*
3874 /// reset the counter, by design. Minutes later `f1` renders `auth status`,
3875 /// which probes directly, and the probe meets an ordinary permissions `403`.
3876 /// A stale `401` then latched a sixty-second client-wide lockout and told
3877 /// the operator to wait, when the real answer was a missing grant.
3878 #[tokio::test]
3879 async fn a_directly_requested_probe_does_not_latch_a_lockout_from_a_stale_401() {
3880 let server = MockServer::start().await;
3881 Mock::given(method("GET"))
3882 .and(path("/orgs/acme/actions/runners"))
3883 .respond_with(Script::new(vec![
3884 ResponseTemplate::new(401),
3885 // The retry misses. A `404` leaves `consecutive_unauthorized`
3886 // set, which is the whole premise of the position rule.
3887 ResponseTemplate::new(404).set_body_json(json!({"message": "Not Found"})),
3888 ]))
3889 .mount(&server)
3890 .await;
3891 Mock::given(method("GET"))
3892 .and(path("/user/installations"))
3893 .respond_with(Script::new(vec![
3894 // The probe that accompanies the 401 above.
3895 ResponseTemplate::new(200).set_body_json(installations_body(&[])),
3896 // The direct probe, minutes later: a plain permissions answer,
3897 // with no `retry-after` and a message that names a grant.
3898 ResponseTemplate::new(403)
3899 .set_body_json(json!({"message": "Resource not accessible by integration"})),
3900 ]))
3901 .mount(&server)
3902 .await;
3903
3904 let clock = Arc::new(TestClock::default());
3905 let client = client(&server, clock.clone());
3906 client
3907 .send(&ApiRequest::get("/orgs/acme/actions/runners"))
3908 .await
3909 .expect_err("the retry 404s");
3910 assert!(
3911 !client.is_locked_out(),
3912 "a 404 on the retry is not a lockout"
3913 );
3914
3915 clock.advance_secs(300);
3916 let outcome = client
3917 .revalidate()
3918 .await
3919 .expect("a direct probe is not a lockout error");
3920
3921 assert_eq!(
3922 outcome,
3923 Revalidation::Unavailable,
3924 "a 403 on the probe teaches this client nothing about the credential"
3925 );
3926 assert!(
3927 !client.is_locked_out(),
3928 "a caller-initiated probe is a *first* attempt, not the retry that follows a 401: \
3929 latching here converts a stale 401 into a 60-second client-wide outage and \
3930 reports a missing permission as `the credential is fine, please wait`"
3931 );
3932 assert_eq!(client.lockout_remaining(), None);
3933 }
3934
3935 /// The square the other three leave empty, and the one where a defect in the
3936 /// composition would hide.
3937 ///
3938 /// Covered elsewhere: a first-attempt continuation through `send`, a
3939 /// first-attempt permissions `403` through `send`, and a direct probe
3940 /// meeting a permissions `403` (immediately above). A direct probe meeting a
3941 /// *continuation-shaped* `403` is the fourth square, and it is the
3942 /// composition point of the two rules that pull in opposite directions —
3943 /// the narrowing to `Attempt::First` that fixed the stale-`401` lockout, and
3944 /// the continuation rule that re-widens `First` on GitHub's own evidence.
3945 ///
3946 /// If the narrowing swallowed the continuation here, every other test in
3947 /// this file would still pass, and `f1`'s `auth status` would poll a
3948 /// credential GitHub had asked it to leave alone — reporting each refusal as
3949 /// a missing grant.
3950 #[tokio::test]
3951 async fn a_directly_requested_probe_latches_a_continuation_shaped_lockout() {
3952 let server = MockServer::start().await;
3953 Mock::given(method("GET"))
3954 .and(path("/user/installations"))
3955 // The continuation's signature: `retry-after`, and a body with no
3956 // message for `error_message` to find.
3957 .respond_with(ResponseTemplate::new(403).insert_header("retry-after", "60"))
3958 .mount(&server)
3959 .await;
3960
3961 let client = client(&server, Arc::new(TestClock::default()));
3962
3963 // No preceding traffic whatsoever: a first attempt in the strongest
3964 // sense, which is exactly the position the narrowed rule refused to
3965 // latch in.
3966 let err = client
3967 .revalidate()
3968 .await
3969 .expect_err("a probe that latches a lockout reports it rather than `Unavailable`");
3970 assert!(
3971 err.is_lockout(),
3972 "`revalidate` latched a client-wide lockout and must say so; answering \
3973 `Ok(Unavailable)` leaves `f1` to discover a 15-minute outage through a separate \
3974 `is_locked_out()` call it has no reason to make: {err:?}"
3975 );
3976 assert!(
3977 client.is_locked_out(),
3978 "a 403 carrying `retry-after` with no message is GitHub continuing a lockout, \
3979 whoever asked for the request that met it"
3980 );
3981 assert_eq!(client.lockout_remaining(), Some(Duration::from_secs(60)));
3982
3983 // And the back-off is real, not just a renamed error.
3984 let before = server.received_requests().await.unwrap().len();
3985 let err = client.revalidate().await.expect_err("still locked out");
3986 assert!(err.is_lockout(), "{err:?}");
3987 assert_eq!(
3988 server.received_requests().await.unwrap().len(),
3989 before,
3990 "latching must actually stop traffic"
3991 );
3992 }
3993
3994 /// [`retry_after`] parses integer seconds only; RFC 9110 §10.2.3 also
3995 /// permits an HTTP-date. Detection used to gate on that parse succeeding, so
3996 /// a date-form `Retry-After` was not recognised as a continuation at all and
3997 /// the hole the continuation rule exists to close reopened for that shape —
3998 /// silently, because the response leaves as an ordinary `Forbidden`.
3999 ///
4000 /// GitHub sends integer seconds in practice. This pins the crate to not
4001 /// depending on that, and records where the two halves part company:
4002 /// presence decides *whether* it is a lockout, the integer parse decides
4003 /// only *how long*, and `latch_lockout` already had a default for the header
4004 /// it could not read.
4005 #[tokio::test]
4006 async fn a_date_form_retry_after_is_still_recognised_as_a_continuation() {
4007 let server = MockServer::start().await;
4008 Mock::given(method("GET"))
4009 .and(path("/user/installations"))
4010 .respond_with(
4011 ResponseTemplate::new(403)
4012 .insert_header("retry-after", "Wed, 21 Oct 2026 07:28:00 GMT"),
4013 )
4014 .mount(&server)
4015 .await;
4016
4017 let client = client(&server, Arc::new(TestClock::default()));
4018 let err = client
4019 .revalidate()
4020 .await
4021 .expect_err("a date-form `retry-after` is still GitHub asking to be left alone");
4022 assert!(
4023 err.is_lockout(),
4024 "gating detection on an integer parse hands the continuation bug back for every \
4025 lockout GitHub chose to date-stamp: {err:?}"
4026 );
4027 assert_eq!(
4028 client.lockout_remaining(),
4029 Some(DEFAULT_LOCKOUT_BACKOFF),
4030 "the date form is recognised for detection; the duration falls back to the \
4031 default, which is what `latch_lockout` already did with a header it could not \
4032 parse as seconds"
4033 );
4034 }
4035
4036 /// The narrowing that fixed the stale-`401` lockout opened a hole at the
4037 /// other end of the same back-off.
4038 ///
4039 /// While GitHub is still locking the credential out after the back-off
4040 /// elapses, the next request is a *first* attempt by construction — this
4041 /// client's own retry never happened, because the request never reached the
4042 /// wire. So the position rule declined to call it a lockout and `classify`
4043 /// fell through to [`GithubError::Forbidden`], whose documented reading is
4044 /// "the App installation does not grant it". The client then stopped backing
4045 /// off entirely and hammered a credential GitHub had asked it to leave
4046 /// alone, which is the exact inverse of "backs off without retrying".
4047 ///
4048 /// A continuation is distinguishable from a permissions answer without any
4049 /// counter: GitHub sends `retry-after` and no message body for the lockout,
4050 /// and a message and no `retry-after` for a permissions refusal.
4051 #[tokio::test]
4052 async fn a_lockout_outliving_its_backoff_re_latches_instead_of_reporting_a_permissions_answer()
4053 {
4054 let server = MockServer::start().await;
4055 Mock::given(method("GET"))
4056 .and(path("/orgs/acme/actions/runners"))
4057 .respond_with(Script::new(vec![
4058 ResponseTemplate::new(401),
4059 // The retry: the lockout latches here, in the retry position.
4060 ResponseTemplate::new(403).insert_header("retry-after", "60"),
4061 // The continuation, once the back-off has elapsed. Same shape,
4062 // first position.
4063 ResponseTemplate::new(403).insert_header("retry-after", "60"),
4064 ]))
4065 .mount(&server)
4066 .await;
4067 Mock::given(method("GET"))
4068 .and(path("/user/installations"))
4069 .respond_with(ResponseTemplate::new(200).set_body_json(installations_body(&[])))
4070 .mount(&server)
4071 .await;
4072
4073 let clock = Arc::new(TestClock::default());
4074 let client = client(&server, clock.clone());
4075 let err = client
4076 .send(&ApiRequest::get("/orgs/acme/actions/runners"))
4077 .await
4078 .expect_err("403 on the retry");
4079 assert!(err.is_lockout(), "{err:?}");
4080 assert!(client.is_locked_out());
4081
4082 // The back-off elapses with GitHub unchanged.
4083 clock.advance_secs(61);
4084 assert!(!client.is_locked_out(), "the back-off has run out");
4085
4086 let err = client
4087 .send(&ApiRequest::get("/orgs/acme/actions/runners"))
4088 .await
4089 .expect_err("GitHub is still locking the credential out");
4090
4091 assert!(
4092 err.is_lockout(),
4093 "a 403 carrying `retry-after` with no message is GitHub continuing the lockout, \
4094 not the App installation refusing a permission; reporting `Forbidden` here \
4095 tells the operator to fix a grant that is not missing: {err:?}"
4096 );
4097 assert!(
4098 client.is_locked_out(),
4099 "`backs off without retrying` fails for any lockout that outlives one back-off \
4100 if the continuation does not re-latch"
4101 );
4102 let GithubError::AuthenticationLockout { retry_after } = err else {
4103 unreachable!("asserted above")
4104 };
4105 assert_eq!(
4106 retry_after,
4107 Duration::from_secs(60),
4108 "the continuation's own `retry-after` sets the new back-off"
4109 );
4110
4111 // And the next request is suppressed before a socket is opened, which is
4112 // the property the whole back-off exists for.
4113 let before = server.received_requests().await.unwrap().len();
4114 let err = client
4115 .send(&ApiRequest::get("/orgs/acme/actions/runners"))
4116 .await
4117 .expect_err("still locked out");
4118 assert!(err.is_lockout(), "{err:?}");
4119 assert_eq!(
4120 server.received_requests().await.unwrap().len(),
4121 before,
4122 "re-latching must actually stop traffic, not merely rename the error"
4123 );
4124 }
4125
4126 /// A permissions `403` on a first attempt is still a permissions answer, and
4127 /// the continuation rule above must not swallow it. This is the test that
4128 /// keeps that rule from becoming "every 403 is a lockout".
4129 #[tokio::test]
4130 async fn a_first_attempt_permissions_403_is_still_reported_as_forbidden() {
4131 let server = MockServer::start().await;
4132 Mock::given(method("GET"))
4133 .and(path("/orgs/acme/actions/runners"))
4134 .respond_with(
4135 ResponseTemplate::new(403)
4136 .set_body_json(json!({"message": "Resource not accessible by integration"})),
4137 )
4138 .mount(&server)
4139 .await;
4140
4141 let client = client(&server, Arc::new(TestClock::default()));
4142 let err = client
4143 .send(&ApiRequest::get("/orgs/acme/actions/runners"))
4144 .await
4145 .expect_err("403");
4146
4147 assert!(
4148 matches!(err, GithubError::Forbidden { .. }),
4149 "a message and no `retry-after` is GitHub naming a missing grant: {err:?}"
4150 );
4151 assert!(!client.is_locked_out());
4152 }
4153
4154 /// The `consecutive_unauthorized > 0` conjunct that used to sit alongside
4155 /// the position rule added no signal — `Attempt::Retry` already implies this
4156 /// request's own `401` incremented the counter — and added a fail-open race:
4157 /// any concurrent success `store(0)`s the counter between the `401` and the
4158 /// retry, and a real lockout is then reported as a permissions answer.
4159 ///
4160 /// The race is driven directly rather than by scheduling two requests and
4161 /// hoping: `store(0)` is the *only* thing the concurrent success contributes,
4162 /// so performing it between the `401` and the classification reproduces the
4163 /// race deterministically and on every run.
4164 #[tokio::test]
4165 async fn a_concurrent_success_cannot_downgrade_a_lockout_to_a_permissions_answer() {
4166 let server = MockServer::start().await;
4167 let client = client(&server, Arc::new(TestClock::default()));
4168
4169 // This request's own 401 has landed: the retry position is established.
4170 client
4171 .consecutive_unauthorized
4172 .fetch_add(1, Ordering::SeqCst);
4173 // ... and a request on another task succeeds in the same instant.
4174 client.consecutive_unauthorized.store(0, Ordering::SeqCst);
4175
4176 let mut headers = HeaderMap::new();
4177 headers.insert("retry-after", "60".parse().unwrap());
4178 let lockout = ApiResponse {
4179 status: StatusCode::FORBIDDEN,
4180 headers,
4181 body: Vec::new(),
4182 };
4183
4184 assert!(
4185 client.is_lockout_403(&lockout, Attempt::Retry),
4186 "`Attempt::Retry` already means this request's own 401 incremented the counter, so \
4187 reading the counter again adds no signal and only lets an unrelated success \
4188 downgrade a real lockout to `Forbidden`"
4189 );
4190
4191 // The counter must stay irrelevant in the other direction too: a
4192 // permissions `403` on a first attempt is not a lockout however many
4193 // `401`s are on the count.
4194 client.consecutive_unauthorized.store(7, Ordering::SeqCst);
4195 let permissions = ApiResponse {
4196 status: StatusCode::FORBIDDEN,
4197 headers: HeaderMap::new(),
4198 body: br#"{"message":"Resource not accessible by integration"}"#.to_vec(),
4199 };
4200 assert!(!client.is_lockout_403(&permissions, Attempt::First));
4201 }
4202
4203 // -- installation discovery ---------------------------------------------
4204
4205 #[tokio::test]
4206 async fn discovery_returns_the_reachable_repository_and_organization_set() {
4207 let server = MockServer::start().await;
4208 Mock::given(method("GET"))
4209 .and(path("/user/installations"))
4210 .respond_with(
4211 ResponseTemplate::new(200).set_body_json(installations_body(&[
4212 (11, "IvanMurzak", "User", "selected"),
4213 (22, "Tap-Top-Fun", "Organization", "all"),
4214 ])),
4215 )
4216 .mount(&server)
4217 .await;
4218 Mock::given(method("GET"))
4219 .and(path("/user/installations/11/repositories"))
4220 .respond_with(
4221 ResponseTemplate::new(200)
4222 .set_body_json(repositories_body(&["IvanMurzak/GitHub-Runner-Scaler-UI"])),
4223 )
4224 .mount(&server)
4225 .await;
4226 Mock::given(method("GET"))
4227 .and(path("/user/installations/22/repositories"))
4228 .respond_with(
4229 ResponseTemplate::new(200)
4230 .set_body_json(repositories_body(&["Tap-Top-Fun/game", "Tap-Top-Fun/site"])),
4231 )
4232 .mount(&server)
4233 .await;
4234
4235 let client = client(&server, Arc::new(TestClock::default()));
4236 let discovery = client.discover_installations(&app()).await.unwrap();
4237
4238 let targets = discovery.targets().expect("installed");
4239 assert_eq!(
4240 targets
4241 .repositories()
4242 .iter()
4243 .map(ToString::to_string)
4244 .collect::<Vec<_>>(),
4245 [
4246 "IvanMurzak/GitHub-Runner-Scaler-UI",
4247 "Tap-Top-Fun/game",
4248 "Tap-Top-Fun/site"
4249 ]
4250 );
4251 assert_eq!(
4252 targets
4253 .organizations()
4254 .iter()
4255 .map(ToString::to_string)
4256 .collect::<Vec<_>>(),
4257 ["Tap-Top-Fun"],
4258 "a User account is not an organization target"
4259 );
4260 assert!(discovery.install_url().is_none());
4261 }
4262
4263 #[tokio::test]
4264 async fn an_over_broad_installation_is_visible_rather_than_assumed() {
4265 let server = MockServer::start().await;
4266 Mock::given(method("GET"))
4267 .and(path("/user/installations"))
4268 .respond_with(
4269 ResponseTemplate::new(200).set_body_json(installations_body(&[
4270 (11, "IvanMurzak", "User", "selected"),
4271 (22, "Tap-Top-Fun", "Organization", "all"),
4272 ])),
4273 )
4274 .mount(&server)
4275 .await;
4276 Mock::given(method("GET"))
4277 .and(path("/user/installations/11/repositories"))
4278 .respond_with(ResponseTemplate::new(200).set_body_json(repositories_body(&["a/b"])))
4279 .mount(&server)
4280 .await;
4281 Mock::given(method("GET"))
4282 .and(path("/user/installations/22/repositories"))
4283 .respond_with(ResponseTemplate::new(200).set_body_json(repositories_body(&["c/d"])))
4284 .mount(&server)
4285 .await;
4286
4287 let client = client(&server, Arc::new(TestClock::default()));
4288 let targets = client
4289 .discover_installations(&app())
4290 .await
4291 .unwrap()
4292 .targets()
4293 .cloned()
4294 .expect("installed");
4295
4296 let over_broad = targets.over_broad();
4297 assert_eq!(over_broad.len(), 1);
4298 assert_eq!(over_broad[0].account.login(), "Tap-Top-Fun");
4299 assert!(over_broad[0].is_over_broad());
4300 assert_eq!(
4301 over_broad[0].repository_selection,
4302 RepositorySelection::All,
4303 "`repository_selection: all` reaches repositories created later too"
4304 );
4305 assert!(
4306 targets.installations().iter().any(|i| i
4307 .permissions
4308 .iter()
4309 .any(|(k, v)| k == "administration" && v == "write")),
4310 "the grant GitHub reports is surfaced verbatim, not assumed from the design"
4311 );
4312 }
4313
4314 #[tokio::test]
4315 async fn discovery_returns_the_installation_url_when_the_set_is_empty() {
4316 let server = MockServer::start().await;
4317 Mock::given(method("GET"))
4318 .and(path("/user/installations"))
4319 .respond_with(ResponseTemplate::new(200).set_body_json(installations_body(&[])))
4320 .mount(&server)
4321 .await;
4322
4323 let client = client(&server, Arc::new(TestClock::default()));
4324 let discovery = client.discover_installations(&app()).await.unwrap();
4325
4326 let url = discovery
4327 .install_url()
4328 .expect("an empty set must yield the installation URL");
4329 assert_eq!(url.path(), "/apps/runner-manager/installations/new");
4330 assert!(discovery.targets().is_none());
4331 }
4332
4333 #[tokio::test]
4334 async fn an_installation_that_reaches_no_repository_is_still_not_installed() {
4335 let server = MockServer::start().await;
4336 Mock::given(method("GET"))
4337 .and(path("/user/installations"))
4338 .respond_with(
4339 ResponseTemplate::new(200).set_body_json(installations_body(&[(
4340 11,
4341 "IvanMurzak",
4342 "User",
4343 "selected",
4344 )])),
4345 )
4346 .mount(&server)
4347 .await;
4348 Mock::given(method("GET"))
4349 .and(path("/user/installations/11/repositories"))
4350 .respond_with(ResponseTemplate::new(200).set_body_json(repositories_body(&[])))
4351 .mount(&server)
4352 .await;
4353
4354 let client = client(&server, Arc::new(TestClock::default()));
4355 let discovery = client.discover_installations(&app()).await.unwrap();
4356 assert!(
4357 discovery.install_url().is_some(),
4358 "a user installation that selected no repository reaches nothing"
4359 );
4360 }
4361
4362 #[tokio::test]
4363 async fn discovery_follows_every_page_rather_than_trusting_the_first() {
4364 let server = MockServer::start().await;
4365 let next = format!("<{}/user/installations?page=2>; rel=\"next\"", server.uri());
4366 Mock::given(method("GET"))
4367 .and(path("/user/installations"))
4368 .respond_with(Script::new(vec![
4369 ResponseTemplate::new(200)
4370 .set_body_json(installations_body(&[(
4371 11,
4372 "one",
4373 "Organization",
4374 "selected",
4375 )]))
4376 .insert_header("link", next.as_str()),
4377 ResponseTemplate::new(200).set_body_json(installations_body(&[(
4378 22,
4379 "two",
4380 "Organization",
4381 "selected",
4382 )])),
4383 ]))
4384 .mount(&server)
4385 .await;
4386 Mock::given(method("GET"))
4387 .and(path("/user/installations/11/repositories"))
4388 .respond_with(ResponseTemplate::new(200).set_body_json(repositories_body(&["one/a"])))
4389 .mount(&server)
4390 .await;
4391 Mock::given(method("GET"))
4392 .and(path("/user/installations/22/repositories"))
4393 .respond_with(ResponseTemplate::new(200).set_body_json(repositories_body(&["two/b"])))
4394 .mount(&server)
4395 .await;
4396
4397 let client = client(&server, Arc::new(TestClock::default()));
4398 let targets = client
4399 .discover_installations(&app())
4400 .await
4401 .unwrap()
4402 .targets()
4403 .cloned()
4404 .expect("installed");
4405 assert_eq!(
4406 targets
4407 .organizations()
4408 .iter()
4409 .map(ToString::to_string)
4410 .collect::<Vec<_>>(),
4411 ["one", "two"],
4412 "the second page must not be dropped"
4413 );
4414 }
4415
4416 /// GitHub's published `installation` schema types `account` as **nullable**,
4417 /// and as either a simple-user *or* an enterprise — which carries
4418 /// `slug`/`name` where a user carries `login`. A required `RawAccount` with
4419 /// a required `login` made either shape a hard `response.json()` failure,
4420 /// which takes down all of `discover_installations`, which is all of
4421 /// `auth status`. One unusual installation must not blind the command that
4422 /// exists to show the user what their credential can reach.
4423 #[tokio::test]
4424 async fn an_installation_with_a_null_or_enterprise_account_does_not_fail_the_whole_decode() {
4425 let server = MockServer::start().await;
4426 Mock::given(method("GET"))
4427 .and(path("/user/installations"))
4428 .respond_with(ResponseTemplate::new(200).set_body_json(json!({
4429 "total_count": 3,
4430 "installations": [
4431 // Nullable, per the published schema.
4432 { "id": 10, "account": null, "repository_selection": "selected" },
4433 // An enterprise: no `login` at all.
4434 {
4435 "id": 20,
4436 "account": { "slug": "acme-enterprise", "name": "Acme Inc" },
4437 "repository_selection": "selected"
4438 },
4439 // And an ordinary user alongside them.
4440 {
4441 "id": 30,
4442 "account": { "login": "IvanMurzak", "type": "User" },
4443 "repository_selection": "selected"
4444 }
4445 ]
4446 })))
4447 .mount(&server)
4448 .await;
4449 for (id, repo) in [(20_u64, "acme-enterprise/tools"), (30, "IvanMurzak/app")] {
4450 Mock::given(method("GET"))
4451 .and(path(format!("/user/installations/{id}/repositories")))
4452 .respond_with(ResponseTemplate::new(200).set_body_json(repositories_body(&[repo])))
4453 .mount(&server)
4454 .await;
4455 }
4456
4457 let client = client(&server, Arc::new(TestClock::default()));
4458 let targets = client
4459 .discover_installations(&app())
4460 .await
4461 .expect("one odd account must not fail the whole discovery")
4462 .targets()
4463 .cloned()
4464 .expect("installed");
4465
4466 // Membership rather than order: what matters here is that neither
4467 // installation was lost, not how `OwnerRepo` collates.
4468 let reached = targets
4469 .repositories()
4470 .iter()
4471 .map(ToString::to_string)
4472 .collect::<Vec<_>>();
4473 assert!(
4474 reached.contains(&"acme-enterprise/tools".to_string()),
4475 "the enterprise installation is named from `slug` rather than dropped: {reached:?}"
4476 );
4477 assert!(
4478 reached.contains(&"IvanMurzak/app".to_string()),
4479 "the ordinary installation alongside it survives too: {reached:?}"
4480 );
4481 assert_eq!(reached.len(), 2);
4482 assert_eq!(
4483 targets.installations().len(),
4484 2,
4485 "the null account is skipped, and only it"
4486 );
4487 assert_eq!(
4488 targets.skipped(),
4489 1,
4490 "the skip is the right trade, but it must travel with the answer: everything the \
4491 skipped installation reaches is missing from the lists above, and a short list \
4492 reads exactly like a complete one"
4493 );
4494
4495 // The enterprise is labelled an enterprise. It used to fall through to
4496 // `User`, so `auth status` told the operator their enterprise was a
4497 // personal account.
4498 let enterprise = targets
4499 .installations()
4500 .iter()
4501 .find(|i| i.id == 20)
4502 .expect("the enterprise installation survived");
4503 assert_eq!(
4504 enterprise.account,
4505 InstallationAccount::Enterprise("acme-enterprise".to_string()),
4506 "an account with no `login` that names itself through `slug` is an enterprise, \
4507 and calling it a user is a wrong statement about the operator's own account"
4508 );
4509 assert_eq!(enterprise.account.kind(), "enterprise");
4510 assert!(
4511 enterprise.account.organization().is_none(),
4512 "an enterprise is not an organization target: `GET /orgs/{{org}}/actions/runners` \
4513 does not accept one, so contributing nothing to `organizations()` is correct"
4514 );
4515 assert!(
4516 !targets
4517 .organizations()
4518 .iter()
4519 .any(|o| o.as_str() == "acme-enterprise"),
4520 "and it must not be smuggled in as one either"
4521 );
4522 }
4523
4524 /// The skip is right; the verdict flip was not.
4525 ///
4526 /// A null-account installation that is the *only* installation used to
4527 /// collapse to `NotInstalled`, so `auth status` handed an operator who **is**
4528 /// installed the "install the App" URL — a wrong remediation on the only
4529 /// authentication path there is, contradicted by nothing but a `warn!`.
4530 #[tokio::test]
4531 async fn a_credential_whose_only_installation_was_skipped_is_not_reported_as_not_installed() {
4532 let server = MockServer::start().await;
4533 Mock::given(method("GET"))
4534 .and(path("/user/installations"))
4535 .respond_with(ResponseTemplate::new(200).set_body_json(json!({
4536 "total_count": 1,
4537 "installations": [
4538 { "id": 10, "account": null, "repository_selection": "selected" }
4539 ]
4540 })))
4541 .mount(&server)
4542 .await;
4543
4544 let client = client(&server, Arc::new(TestClock::default()));
4545 let discovery = client.discover_installations(&app()).await.unwrap();
4546
4547 assert_eq!(
4548 discovery,
4549 InstallationDiscovery::Indeterminate { skipped: 1 },
4550 "GitHub reported an installation; this client could not describe it. That is not \
4551 the same answer as `not installed`, and only one of the two is fixed by \
4552 installing the App"
4553 );
4554 assert_eq!(
4555 discovery.install_url(),
4556 None,
4557 "offering the install URL here is the wrong remediation, and putting it one field \
4558 over from the right verdict would just relocate the defect"
4559 );
4560 assert_eq!(discovery.skipped(), 1);
4561 assert!(discovery.targets().is_none());
4562 }
4563
4564 /// The other side of the same rule: with nothing skipped, an empty reach is
4565 /// still an empty reach, and the install URL is still the remediation.
4566 #[tokio::test]
4567 async fn an_empty_reach_with_nothing_skipped_is_still_not_installed() {
4568 let server = MockServer::start().await;
4569 Mock::given(method("GET"))
4570 .and(path("/user/installations"))
4571 .respond_with(ResponseTemplate::new(200).set_body_json(installations_body(&[])))
4572 .mount(&server)
4573 .await;
4574
4575 let client = client(&server, Arc::new(TestClock::default()));
4576 let discovery = client.discover_installations(&app()).await.unwrap();
4577
4578 assert!(discovery.install_url().is_some(), "{discovery:?}");
4579 assert_eq!(discovery.skipped(), 0);
4580 }
4581
4582 /// A `Link: rel="next"` that points back at the page it arrived on is an
4583 /// infinite loop inside the agent's reconciliation loop — the one place in
4584 /// this product that must not be able to wedge. The ceiling is what makes
4585 /// this test terminate at all.
4586 #[tokio::test]
4587 async fn a_self_referential_link_header_stops_at_the_page_ceiling() {
4588 let server = MockServer::start().await;
4589 let self_link = format!("<{}/user/installations?page=2>; rel=\"next\"", server.uri());
4590 Mock::given(method("GET"))
4591 .and(path("/user/installations"))
4592 .respond_with(
4593 ResponseTemplate::new(200)
4594 .set_body_json(installations_body(&[]))
4595 .insert_header("link", self_link.as_str()),
4596 )
4597 .mount(&server)
4598 .await;
4599
4600 let client = client(&server, Arc::new(TestClock::default()));
4601 let discovery = client
4602 .discover_installations(&app())
4603 .await
4604 .expect("the ceiling is what makes this return at all");
4605
4606 assert!(
4607 discovery.install_url().is_some(),
4608 "no installation was found"
4609 );
4610 assert_eq!(
4611 server.received_requests().await.unwrap().len(),
4612 MAX_PAGES,
4613 "pagination must stop at the ceiling rather than follow the loop forever"
4614 );
4615 }
4616
4617 #[test]
4618 fn a_link_header_yields_only_the_next_relation() {
4619 let header = "<https://api.github.com/user/installations?page=3>; rel=\"next\", \
4620 <https://api.github.com/user/installations?page=9>; rel=\"last\"";
4621 assert_eq!(
4622 parse_link_next(header).map(|u| u.to_string()),
4623 Some("https://api.github.com/user/installations?page=3".to_string())
4624 );
4625 assert!(parse_link_next("<https://x/>; rel=\"last\"").is_none());
4626 assert!(parse_link_next("nonsense").is_none());
4627 }
4628
4629 /// A comma is legal inside a URL and GitHub sends such URLs routinely — a
4630 /// runner query carries `labels=self-hosted,windows`. Splitting the header
4631 /// on `,` before recognising `<...>` tore that URL in half, found no
4632 /// relation, and stopped paginating at page 1 while reporting success. That
4633 /// is precisely what `04-subsystem-contracts.md` forbids ("the dashboard
4634 /// must not treat a first page as a complete inventory"), in the one shared
4635 /// reader `c3`'s inventory also goes through.
4636 #[test]
4637 fn a_next_url_containing_a_comma_still_paginates() {
4638 let header = "<https://api.github.com/repos/o/r/actions/runners\
4639 ?labels=self-hosted,windows&page=2>; rel=\"next\"";
4640 assert_eq!(
4641 parse_link_next(header).map(|u| u.to_string()),
4642 Some(
4643 "https://api.github.com/repos/o/r/actions/runners\
4644 ?labels=self-hosted,windows&page=2"
4645 .to_string()
4646 ),
4647 "a comma inside the URL must not end the link-value"
4648 );
4649
4650 // The same URL as the second link-value, so the scan has to walk past a
4651 // comma-bearing target to reach the relation it wants.
4652 let header = "<https://api.github.com/x?a=1,2&page=1>; rel=\"prev\", \
4653 <https://api.github.com/x?a=1,2&page=3>; rel=\"next\"";
4654 assert_eq!(
4655 parse_link_next(header).map(|u| u.to_string()),
4656 Some("https://api.github.com/x?a=1,2&page=3".to_string())
4657 );
4658 }
4659
4660 /// `rel="next"` is not always the first link-value, and both quoted and
4661 /// unquoted forms are legal.
4662 #[test]
4663 fn the_next_relation_is_found_wherever_it_sits_in_the_header() {
4664 let not_first = "<https://api.github.com/u?page=1>; rel=\"first\", \
4665 <https://api.github.com/u?page=9>; rel=\"last\", \
4666 <https://api.github.com/u?page=4>; rel=\"next\"";
4667 assert_eq!(
4668 parse_link_next(not_first).map(|u| u.to_string()),
4669 Some("https://api.github.com/u?page=4".to_string())
4670 );
4671
4672 let unquoted = "<https://api.github.com/u?page=1>; rel=prev, \
4673 <https://api.github.com/u?page=3>; rel=next";
4674 assert_eq!(
4675 parse_link_next(unquoted).map(|u| u.to_string()),
4676 Some("https://api.github.com/u?page=3".to_string())
4677 );
4678
4679 assert!(
4680 parse_link_next("<https://api.github.com/u?page=2; rel=\"next\"").is_none(),
4681 "an unterminated target is not a link-value"
4682 );
4683 }
4684
4685 /// The free cross-check that would have caught the comma bug on its own.
4686 #[test]
4687 fn a_short_collection_is_measured_against_the_count_github_reported() {
4688 assert_eq!(under_collected(1, Some(2)), Some(2), "page 2 was dropped");
4689 assert_eq!(under_collected(2, Some(2)), None, "complete");
4690 assert_eq!(
4691 under_collected(3, Some(2)),
4692 None,
4693 "a collection that grew between pages is not an under-collection"
4694 );
4695 assert_eq!(under_collected(0, None), None, "no count, no claim");
4696 }
4697
4698 // -- redaction ----------------------------------------------------------
4699
4700 #[test]
4701 fn no_type_in_this_crate_renders_a_secret_through_debug() {
4702 let token = UserAccessToken::new(SecretString::from(FIXTURE_TOKEN));
4703 let rendered = format!("{token:?}");
4704 assert!(!rendered.contains(FIXTURE_TOKEN), "{rendered}");
4705 assert!(rendered.contains("[REDACTED]"));
4706 assert!(
4707 rendered.contains("ghu_"),
4708 "the family prefix is diagnostic and is not the secret"
4709 );
4710
4711 let request = ApiRequest::post_json("/x", &json!({"encoded_jit_config": "SECRETBLOB"}))
4712 .expect("serializes");
4713 let rendered = format!("{request:?}");
4714 assert!(!rendered.contains("SECRETBLOB"), "{rendered}");
4715
4716 let response = ApiResponse {
4717 status: StatusCode::OK,
4718 headers: HeaderMap::new(),
4719 body: b"{\"encoded_jit_config\":\"SECRETBLOB\"}".to_vec(),
4720 };
4721 let rendered = format!("{response:?}");
4722 assert!(!rendered.contains("SECRETBLOB"), "{rendered}");
4723 }
4724
4725 // The Definition of Done's log scan is `tests/no_secret_reaches_the_logs.rs`
4726 // and not a unit test here. See the note at the end of `mod testing` for the
4727 // `tracing` callsite-cache reason it cannot be one.
4728
4729 // -- the crate-shape scans ----------------------------------------------
4730 //
4731 // The three gates below share these helpers on purpose. The previous round
4732 // defined `normalise` twice — once in the scan and once in the meta-test
4733 // that checks it — which left the meta-test structurally unable to notice a
4734 // change to the real one. One definition, used by both, is the only shape
4735 // in which a meta-test proves anything.
4736
4737 /// Spelled in halves so that this file's own source does not trip the scan
4738 /// it runs: normalising `concat!("refresh", "token")` leaves the
4739 /// quote-comma-quote between the halves, so no needle ever appears whole.
4740 // The renewal guard that used to live here is gone, and its absence is the
4741 // point. It forbade this crate from naming a refresh token at all, on the
4742 // reasoning that the published App opts out of user-token expiration "so
4743 // GitHub issues nothing to renew". That reasoning rested on a second claim
4744 // -- that renewing needs a confidential client credential -- which GitHub's
4745 // own documentation contradicts for the device flow, and which was then
4746 // disproved against live GitHub: a refresh exchange with `client_id` alone
4747 // answers `200`.
4748 //
4749 // What the guard below still forbids is the part that was always true and
4750 // is the reason renewal is safe here: no confidential credential in this
4751 // crate. Renewal was added *without* one, so the remaining half of this
4752 // scan is now evidence for the design rather than against it.
4753 const CONFIDENTIAL: &[&str] = &[concat!("client", "secret"), concat!("app", "secret")];
4754
4755 const MANIFEST: (&str, &str) = ("Cargo.toml", include_str!("../Cargo.toml"));
4756
4757 /// Every `.rs` file at or below `src/`, named by its `/`-joined path
4758 /// relative to `src/` — so a nested module is `("rest/runners.rs",
4759 /// include_str!("rest/runners.rs"))`, not just its file name.
4760 ///
4761 /// This list used to *be* the claim "every source file in the crate", and a
4762 /// hard-coded list is not that claim — it is a snapshot of it. `c3` and `c4`
4763 /// are the tasks that will add files to this directory, so the list was
4764 /// guaranteed to go stale on exactly the work that most needed scanning: a
4765 /// new `pagination.rs` holding a confidential credential passed silently.
4766 /// [`the_confidential_credential_scan_covers_every_source_file`] pins this
4767 /// by walking the directory tree, so adding a file — at the top level or in
4768 /// a subdirectory — and not adding it here fails.
4769 const CRATE_SOURCES: &[(&str, &str)] = &[
4770 ("conditional.rs", include_str!("conditional.rs")),
4771 ("demand.rs", include_str!("demand.rs")),
4772 ("device_flow.rs", include_str!("device_flow.rs")),
4773 ("jit.rs", include_str!("jit.rs")),
4774 ("lib.rs", include_str!("lib.rs")),
4775 ("rest.rs", include_str!("rest.rs")),
4776 ("traffic.rs", include_str!("traffic.rs")),
4777 ];
4778
4779 /// The two source files `c2` owns, plus the manifest. The renewal half of
4780 /// the scan stays inside this boundary; see the scan's own documentation.
4781 const SOURCES_OWNED_BY_C2: &[(&str, &str)] = &[
4782 ("device_flow.rs", include_str!("device_flow.rs")),
4783 ("lib.rs", include_str!("lib.rs")),
4784 MANIFEST,
4785 ];
4786
4787 /// Lower-cased with `_` removed, so that one needle catches the snake,
4788 /// camel, Pascal and screaming-snake spellings of an identifier at once.
4789 /// (Those four spellings cannot be written out here: they are exactly what
4790 /// the gate forbids, which is the constraint on documentation this scan
4791 /// imposes and defends below.)
4792 ///
4793 /// # Why `-` is *not* stripped from Rust source
4794 ///
4795 /// It used to be, and that rejected ordinary English. `c3`'s own file opens
4796 /// with a line stating that the gateway holds no such credential, written
4797 /// with the compound adjective English requires — and stripping `-` turned
4798 /// that sentence into the needle, so the gate accused `c3` of naming a
4799 /// confidential credential in the very line that says it holds none. A
4800 /// compound adjective is not an evasion; it is how the language works, and
4801 /// this brief, this crate's documentation and that line all use one.
4802 ///
4803 /// Nothing is lost, because **a Rust identifier cannot contain `-`**.
4804 /// Stripping it never bought identifier coverage: every casing an identifier
4805 /// can actually take is `_`-separated or unseparated, and all of those still
4806 /// collapse onto the needle. What it bought was coverage of a *kebab-case
4807 /// string literal*, and the residual gap is stated plainly rather than
4808 /// papered over: a `.rs` file that wrote this credential's name as a
4809 /// hyphenated string would not be caught here. That gap is narrow on
4810 /// purpose — OAuth 2.0 and GitHub both spell the field `_`-separated, which
4811 /// this catches — and it is the price of a gate that ordinary prose can
4812 /// coexist with. A gate that fires on correct English is not a stricter
4813 /// gate; it is a gate that gets deleted.
4814 ///
4815 /// The alternatives were weighed. Requiring identifier context needs a Rust
4816 /// lexer to tell `a client-secret-free design` from a TOML key, and gets the
4817 /// wrong answer for both string literals and comments. Excluding comment
4818 /// text needs the same lexer to avoid mangling `//` inside a string, and
4819 /// would stop the gate catching a `TODO` comment proposing to read the
4820 /// credential from the environment — which is precisely the drift worth
4821 /// catching early, while it is still a comment. Stripping one character
4822 /// fewer needs neither, which is why it wins.
4823 ///
4824 /// A space is not stripped either, and for the same reason: it is what lets
4825 /// this crate's prose discuss a "client secret" as two words.
4826 fn normalise_source(source: &str) -> String {
4827 source.to_ascii_lowercase().replace('_', "")
4828 }
4829
4830 /// The manifest keeps `-` stripped: TOML keys and crate names are kebab-case
4831 /// by convention, so `-` there is a word separator rather than a hyphen, and
4832 /// a manifest carries no hyphenated English for it to break.
4833 fn normalise_manifest(manifest: &str) -> String {
4834 manifest.to_ascii_lowercase().replace(['_', '-'], "")
4835 }
4836
4837 /// Which normaliser a scanned file gets. The manifest is the only file whose
4838 /// `-` is a separator rather than punctuation.
4839 fn normalise(name: &str, contents: &str) -> String {
4840 if name == MANIFEST.0 {
4841 normalise_manifest(contents)
4842 } else {
4843 normalise_source(contents)
4844 }
4845 }
4846
4847 /// The part of a source file that is not test code.
4848 ///
4849 /// The boundary is the first line that is **exactly** `#[cfg(test)]`, and
4850 /// the word "exactly" is the fix. Splitting on that literal wherever it
4851 /// appeared also split on it in *prose*, and `lib.rs` has carried such a
4852 /// mention since the `testing` module was documented — so the scan below
4853 /// already stopped nine lines early, today, with nothing to say so. A file
4854 /// whose module documentation happened to mention an inline test module
4855 /// would have had its scanned region truncated to a few dozen lines, after
4856 /// which a real `std::fs::write` in non-test code passed silently. That is
4857 /// the same class of defect as the log scan that captured only its own
4858 /// events and the credential scan that claimed a scope it did not have: a
4859 /// gate whose description outran what it did.
4860 fn non_test_prefix(source: &str) -> &str {
4861 let mut offset = 0;
4862 for line in source.split_inclusive('\n') {
4863 if line.trim() == "#[cfg(test)]" {
4864 return &source[..offset];
4865 }
4866 offset += line.len();
4867 }
4868 source
4869 }
4870
4871 /// The Definition of Done's second item, made checkable rather than
4872 /// reviewed: "no renewal token code path exists, and no client secret
4873 /// appears anywhere in the crate **or its configuration**".
4874 ///
4875 /// # Normalised, because a literal scan is evaded by naming
4876 ///
4877 /// This used to be a case-sensitive `contains` over two snake-case
4878 /// spellings, which is a gate that any ordinary Rust or JSON identifier
4879 /// walks straight through: the camel-cased, Pascal-cased and
4880 /// screaming-snake spellings of the very same two identifiers were all
4881 /// invisible to it. None of those is exotic — several are what the
4882 /// surrounding ecosystem actually calls these fields — so evading this gate
4883 /// never had to be deliberate. See [`normalise_source`] for what is
4884 /// collapsed, what is deliberately not, and why.
4885 ///
4886 /// The consequence is that this crate's *prose* may not write those
4887 /// identifiers either, in any casing: it says "renewal token" and "client
4888 /// secret" as separate words, which normalisation preserves and the scan
4889 /// therefore ignores. That is a real constraint on the documentation, and it
4890 /// is the right way round — a gate loosened until the comments compile is
4891 /// not a gate. It is a constraint on *identifier spellings*, though, and
4892 /// never on English: hyphenating a compound adjective is not writing an
4893 /// identifier, and a gate that could not tell those apart is what this round
4894 /// fixed.
4895 ///
4896 /// # Two different scopes, for two different reasons
4897 ///
4898 /// The **renewal** half stays scoped to the two files `c2` owns plus the
4899 /// manifest. A renewal path in `c3`'s or `c4`'s file would be their finding;
4900 /// failing here on their work would be this task reaching across an
4901 /// ownership boundary.
4902 ///
4903 /// The **client secret** half covers every source file in the crate. That is
4904 /// not a boundary crossing but the opposite: a public client cannot hold a
4905 /// client secret at all (D3, `07-security.md`), so one appearing *anywhere*
4906 /// in this crate is a product defect rather than a matter of whose file it
4907 /// is, and `c2` is the designated owner of that clause. "Every source file"
4908 /// is a claim about the directory, so it is checked against the directory —
4909 /// see [`the_confidential_credential_scan_covers_every_source_file`].
4910 #[test]
4911 fn no_confidential_credential_in_this_crate() {
4912 for &(name, source) in CRATE_SOURCES.iter().chain(std::iter::once(&MANIFEST)) {
4913 let haystack = normalise(name, source);
4914 for forbidden in CONFIDENTIAL {
4915 assert!(
4916 !haystack.contains(forbidden),
4917 "{name} names {forbidden:?} in some spelling: a public client cannot \
4918 secure a confidential credential, and this design never tries to (D3)"
4919 );
4920 }
4921 }
4922 }
4923
4924 /// "Every source file in the crate" is a claim about a directory, and the
4925 /// scan above states it as a hard-coded list. A list is a snapshot: the
4926 /// moment `c3` or `c4` adds a file to `src/`, the claim is false and nothing
4927 /// says so. A `src/pagination.rs` holding a confidential credential passed
4928 /// the gate that exists to catch exactly that.
4929 ///
4930 /// Reading the directory here is what turns the claim back into a claim. It
4931 /// cannot be done in the scan itself — `include_str!` needs a literal path
4932 /// at compile time — so the list stays, and this pins it.
4933 ///
4934 /// # Why it walks the tree instead of listing one directory
4935 ///
4936 /// It used to call `read_dir("src")` once and keep the entries ending in
4937 /// `.rs`. That reads like a directory scan and is not one: a subdirectory
4938 /// module — `src/rest/mod.rs`, `src/rest/runners.rs` — arrives as the single
4939 /// entry `rest`, which does not end in `.rs`, so the filter dropped it and
4940 /// took the files underneath with it. The pin went on passing while those
4941 /// files were scanned by nothing at all.
4942 ///
4943 /// That defeated the pin in exactly the case it was written for. `c3` is the
4944 /// REST inventory gateway, a module directory is the ordinary Rust shape for
4945 /// it, and the failure is silent on both sides: the credential scan does not
4946 /// read the file, and the test whose whole job is to notice that reports
4947 /// success.
4948 ///
4949 /// Recursing is the fix, rather than asserting that `src/` holds no
4950 /// subdirectories. The claim being pinned is about *files*, not about
4951 /// layout; banning the directory would fail `c3` for choosing a normal
4952 /// module shape, and a gate that fails correct work is a gate the next round
4953 /// loosens to get its own work compiling — which is how the normalisation
4954 /// half of this same scan was weakened once already.
4955 ///
4956 /// Names are `/`-joined paths relative to `src/`, which is what
4957 /// `include_str!` takes on every platform, so a nested file is listed as
4958 /// `("rest/runners.rs", include_str!("rest/runners.rs"))` and the two sides
4959 /// compare directly.
4960 #[test]
4961 fn the_confidential_credential_scan_covers_every_source_file() {
4962 // Every `.rs` file at or below `dir`, named by its `/`-joined path
4963 // relative to `src/`.
4964 //
4965 // `file_type()` is deliberately not followed through symlinks: a link
4966 // cannot walk this into a cycle, and a symlinked `.rs` file still lands
4967 // in the list through the extension test. A directory is recursed into
4968 // before the extension is considered, so a directory named `foo.rs`
4969 // is walked rather than mistaken for a file.
4970 fn collect(dir: &std::path::Path, prefix: &str, found: &mut Vec<String>) {
4971 for entry in std::fs::read_dir(dir).expect("the source directory is readable") {
4972 let entry = entry.expect("a readable directory entry");
4973 let name = entry.file_name().to_string_lossy().into_owned();
4974 let relative = if prefix.is_empty() {
4975 name.clone()
4976 } else {
4977 format!("{prefix}/{name}")
4978 };
4979 if entry.file_type().expect("a readable entry type").is_dir() {
4980 collect(&entry.path(), &relative, found);
4981 } else if name.ends_with(".rs") {
4982 found.push(relative);
4983 }
4984 }
4985 }
4986
4987 let mut on_disk = Vec::new();
4988 collect(
4989 std::path::Path::new(concat!(env!("CARGO_MANIFEST_DIR"), "/src")),
4990 "",
4991 &mut on_disk,
4992 );
4993 on_disk.sort();
4994
4995 // Sorted, rather than taken in declaration order. Comparing a sorted
4996 // `on_disk` against an unsorted `scanned` made this assertion depend on
4997 // `CRATE_SOURCES` happening to be declared alphabetically. It is — but
4998 // nothing said so, and the failure that would follow from reordering the
4999 // list is a diff of two lists holding the same names, which reads as a
5000 // coverage gap rather than as the ordering nit it would actually be.
5001 let mut scanned: Vec<String> = CRATE_SOURCES
5002 .iter()
5003 .map(|(name, _)| (*name).to_string())
5004 .collect();
5005 scanned.sort();
5006
5007 assert_eq!(
5008 scanned, on_disk,
5009 "`src/` and the scanned list have diverged. Add the new file to `CRATE_SOURCES` \
5010 with an `include_str!`, naming it by its `/`-joined path relative to `src/`; \
5011 leaving it out means the confidential-credential scan silently stops covering \
5012 `every source file in the crate`, which is the claim it makes."
5013 );
5014 }
5015
5016 /// The scan above, shown to actually catch the spellings it claims to — and
5017 /// to leave alone the ones it claims to leave alone.
5018 ///
5019 /// Without this, "the gate is case-insensitive now" is a comment rather than
5020 /// a fact, and the finding that produced it was precisely a gate whose
5021 /// description outran what it did. It calls [`normalise`], the same function
5022 /// the scan calls, because a meta-test with its own private copy of the
5023 /// thing under test cannot detect a change to it.
5024 #[test]
5025 fn the_confidential_credential_scan_is_not_evaded_by_naming() {
5026 // Assembled at run time rather than written out, for the same reason the
5027 // needles are spelled in halves: a test that contained these spellings
5028 // literally would fail the scan it is checking.
5029 let source_evasions = [
5030 format!("let {}Token = fetch()", "refresh"),
5031 format!("struct {}Token;", "Refresh"),
5032 format!("{}_TOKEN", "REFRESH"),
5033 format!("{}Secret", "client"),
5034 format!("{}_SECRET", "CLIENT"),
5035 format!("{}_secret", "app"),
5036 ];
5037 for evasion in &source_evasions {
5038 let normalised = normalise("lib.rs", evasion);
5039 assert!(
5040 normalised.contains(concat!("refresh", "token"))
5041 || normalised.contains(concat!("client", "secret"))
5042 || normalised.contains(concat!("app", "secret")),
5043 "{evasion:?} would walk straight through the scan"
5044 );
5045 }
5046
5047 // The manifest is where kebab-case is a word separator rather than a
5048 // hyphen, so that is where it is still collapsed.
5049 let manifest_evasion = format!("{}-secret = \"...\"", "client");
5050 assert!(
5051 normalise(MANIFEST.0, &manifest_evasion).contains(concat!("client", "secret")),
5052 "a kebab-case TOML key is an identifier, and the manifest normaliser must \
5053 still collapse it"
5054 );
5055
5056 // And the prose the crate legitimately writes must still pass, or the
5057 // gate would be unusable and would be weakened again to make it usable.
5058 for allowed in [
5059 "a public client cannot hold a client secret",
5060 "the published App issues no renewal token",
5061 // The line that fails the old normalisation, quoted from `c3`'s own
5062 // file. It says the *opposite* of what the gate accused it of.
5063 "//! This gateway is deliberately client-secret-free, as D3 requires.",
5064 // The same shape, for the renewal half.
5065 "a refresh-free credential model",
5066 ] {
5067 let normalised = normalise("lib.rs", allowed);
5068 assert!(
5069 !normalised.contains(concat!("client", "secret"))
5070 && !normalised.contains(concat!("refresh", "token")),
5071 "{allowed:?} is English, not an identifier, and must not trip the scan"
5072 );
5073 }
5074 }
5075
5076 /// The storage boundary, made checkable the same way. `c2` returns the token
5077 /// and never persists it; the machine-scoped store is `d2` and the wiring is
5078 /// `f1`. A dependency on `runner-manager-platform`, or a filesystem write,
5079 /// would silently move that boundary.
5080 #[test]
5081 fn this_crate_persists_nothing_and_does_not_depend_on_the_platform_crate() {
5082 assert!(
5083 !MANIFEST.1.contains("runner-manager-platform"),
5084 "the gateway must be testable with no platform dependency at all"
5085 );
5086
5087 for &(name, source) in SOURCES_OWNED_BY_C2 {
5088 if name == MANIFEST.0 {
5089 continue;
5090 }
5091 // Everything below `#[cfg(test)]` is test code; the boundary is about
5092 // non-test code, and the tests above legitimately read this file.
5093 let non_test = non_test_prefix(source);
5094 // `OpenOptions`, `File::options` and `std::io::Write` are on this
5095 // list because the original four named only the *obvious* ways to
5096 // write a file. A store built with `OpenOptions::new().create(true)`
5097 // would have moved the persistence boundary silently, which is the
5098 // one thing this scan exists to prevent.
5099 for forbidden in [
5100 "std::fs",
5101 "fs::write",
5102 "File::create",
5103 "File::options",
5104 "OpenOptions",
5105 "std::io::Write",
5106 "tokio::fs",
5107 ] {
5108 assert!(
5109 !non_test.contains(forbidden),
5110 "{name} performs a filesystem operation ({forbidden:?}) outside its tests"
5111 );
5112 }
5113 }
5114 }
5115
5116 /// The scan above, shown to be looking at what it says it is looking at.
5117 ///
5118 /// `split("#[cfg(test)]")` matched that literal **anywhere**, prose
5119 /// included. One ordinary sentence in a module's documentation truncated the
5120 /// scanned region to whatever preceded it, and every filesystem call after
5121 /// that point became invisible — with the scan still reporting `ok`. This is
5122 /// the third gate in this crate found describing more than it did, so it
5123 /// gets the same treatment as the other two: a synthetic file where the
5124 /// difference is decisive, and an assertion about the real ones.
5125 #[test]
5126 fn the_non_test_boundary_is_a_line_and_not_a_mention() {
5127 // A file shaped like this crate's own: prose that names the attribute,
5128 // then real non-test code, then the actual module.
5129 let file = "//! Test helpers live in an inline #[cfg(test)] module near the bottom.\n\
5130 \n\
5131 fn persist() { std::fs::write(\"x\", b\"y\").unwrap(); }\n\
5132 \n\
5133 #[cfg(test)]\n\
5134 mod tests {\n\
5135 fn helper() { std::fs::write(\"ok-in-tests\", b\"\").unwrap(); }\n\
5136 }\n";
5137
5138 let non_test = non_test_prefix(file);
5139 assert!(
5140 non_test.contains("fn persist"),
5141 "a prose mention of the attribute truncated the scanned region, and every \
5142 filesystem call below it stopped being scanned — silently:\n{non_test}"
5143 );
5144 assert!(
5145 !non_test.contains("ok-in-tests"),
5146 "the boundary must still exclude the real test module:\n{non_test}"
5147 );
5148
5149 // And on the real files, whose module documentation contains such a
5150 // mention today. `lib.rs` has carried one since `mod testing` was
5151 // written, so this crate was shipping the truncated scan.
5152 for &(name, source) in SOURCES_OWNED_BY_C2 {
5153 if name == MANIFEST.0 {
5154 continue;
5155 }
5156 let expected = source
5157 .lines()
5158 .position(|line| line.trim() == "#[cfg(test)]")
5159 .expect("each source file has an inline test module");
5160 let scanned = non_test_prefix(source).lines().count();
5161 assert_eq!(
5162 scanned, expected,
5163 "{name}: the scanned region ends at line {scanned} but the test module starts \
5164 at line {expected}. The gap is code that claims to be scanned and is not."
5165 );
5166 }
5167 }
5168
5169 #[derive(Debug)]
5170 struct SpyRenewal {
5171 invocations: std::sync::atomic::AtomicUsize,
5172 }
5173 #[async_trait::async_trait]
5174 impl CredentialRenewal for SpyRenewal {
5175 async fn renew(
5176 &self,
5177 _refresh_token: &SecretString,
5178 ) -> Result<UserAccessToken, RenewalError> {
5179 self.invocations
5180 .fetch_add(1, std::sync::atomic::Ordering::SeqCst);
5181 Ok(UserAccessToken::new(SecretString::from("ghu_renewed")))
5182 }
5183 }
5184
5185 #[tokio::test]
5186 async fn reload_happens_before_renew_to_prevent_cross_process_races() {
5187 let server = MockServer::start().await;
5188 Mock::given(method("GET"))
5189 .and(path("/repos/acme/app"))
5190 .and(header("authorization", "Bearer ghu_initial"))
5191 .respond_with(ResponseTemplate::new(401))
5192 .expect(1)
5193 .mount(&server)
5194 .await;
5195 Mock::given(method("GET"))
5196 .and(path("/repos/acme/app"))
5197 .and(header("authorization", "Bearer ghu_reloaded"))
5198 .respond_with(ResponseTemplate::new(200).set_body_json(serde_json::json!({"id": 1})))
5199 .expect(1)
5200 .mount(&server)
5201 .await;
5202
5203 let renewal = Arc::new(SpyRenewal {
5204 invocations: std::sync::atomic::AtomicUsize::new(0),
5205 });
5206
5207 let document = serde_json::json!({
5208 "access_token": "ghu_initial",
5209 "refresh_token": "ghr_initial",
5210 });
5211 let initial_token =
5212 UserAccessToken::from_stored_document(&SecretString::from(document.to_string()));
5213
5214 let client = AuthenticatedClient::new(
5215 Endpoints::for_test_server(&server.uri()).unwrap(),
5216 initial_token,
5217 Arc::new(TestClock::default()),
5218 )
5219 .unwrap()
5220 .with_credential_source(Arc::new(StoreHolding(Some("ghu_reloaded"))))
5221 .with_renewal(renewal.clone());
5222
5223 let response = client
5224 .get_json::<serde_json::Value>("/repos/acme/app")
5225 .await
5226 .expect("the reloaded token should succeed");
5227 assert_eq!(response["id"], 1);
5228 assert_eq!(
5229 renewal
5230 .invocations
5231 .load(std::sync::atomic::Ordering::SeqCst),
5232 0,
5233 "renew should not be called because reload succeeded"
5234 );
5235 }
5236
5237 #[tokio::test]
5238 async fn proactive_renewal_waits_until_the_access_token_is_near_expiry() {
5239 let renewal = Arc::new(SpyRenewal {
5240 invocations: std::sync::atomic::AtomicUsize::new(0),
5241 });
5242 let document = serde_json::json!({
5243 "access_token": "ghu_initial",
5244 "refresh_token": "ghr_initial",
5245 "access_expires_at": "2026-08-21T02:00:00Z",
5246 "refresh_expires_at": "2027-02-21T00:00:00Z"
5247 });
5248 let clock = Arc::new(TestClock::default());
5249 let client = AuthenticatedClient::new(
5250 Endpoints::production(),
5251 UserAccessToken::from_stored_document(&SecretString::from(document.to_string())),
5252 clock.clone(),
5253 )
5254 .unwrap()
5255 .with_renewal(renewal.clone());
5256
5257 assert!(
5258 !client
5259 .renew_if_expiring_within(Duration::from_secs(30 * 60))
5260 .await
5261 );
5262 assert_eq!(
5263 renewal
5264 .invocations
5265 .load(std::sync::atomic::Ordering::SeqCst),
5266 0
5267 );
5268
5269 clock.advance_secs(91 * 60);
5270 assert!(
5271 client
5272 .renew_if_expiring_within(Duration::from_secs(30 * 60))
5273 .await
5274 );
5275 assert_eq!(
5276 renewal
5277 .invocations
5278 .load(std::sync::atomic::Ordering::SeqCst),
5279 1
5280 );
5281 }
5282
5283 #[tokio::test]
5284 async fn proactive_renewal_repairs_a_pair_with_no_expiry_metadata() {
5285 let renewal = Arc::new(SpyRenewal {
5286 invocations: std::sync::atomic::AtomicUsize::new(0),
5287 });
5288 let document = serde_json::json!({
5289 "access_token": "ghu_initial",
5290 "refresh_token": "ghr_initial"
5291 });
5292 let client = AuthenticatedClient::new(
5293 Endpoints::production(),
5294 UserAccessToken::from_stored_document(&SecretString::from(document.to_string())),
5295 Arc::new(TestClock::default()),
5296 )
5297 .unwrap()
5298 .with_renewal(renewal.clone());
5299
5300 assert!(
5301 client
5302 .renew_if_expiring_within(Duration::from_secs(30 * 60))
5303 .await
5304 );
5305 assert_eq!(
5306 renewal
5307 .invocations
5308 .load(std::sync::atomic::Ordering::SeqCst),
5309 1
5310 );
5311 }
5312}