Skip to main content

runner_manager_platform/
logging.rs

1// owner: d1-platform-core
2
3//! The structured log sink, and the redaction that is not optional.
4//!
5//! `07-security.md`'s threat table, on *"Logs disclose repository secrets"*:
6//! **"Structured allowlist logging with unconditional redaction of tokens,
7//! headers, JIT blobs, and paths."** Its security gate is a *secret-injection
8//! log scan*, and its release gate is that *"the user access token and the
9//! encoded JIT configuration are absent from logs, databases, snapshots, crash
10//! reports, and CLI output"*.
11//!
12//! # Allowlist, not denylist — and why that is the whole design
13//!
14//! A denylist redacts the fields somebody remembered to name. It is correct on
15//! the day it is written and wrong on the day a later task adds a field, which
16//! is the day it matters. So this sink inverts the default: **a field whose
17//! name is not in [`ALLOWED_FIELDS`] has its value replaced outright.** A task
18//! that adds `runner_token` to a log call gets `[redacted]` with no review, no
19//! ceremony, and no leak. Adding a field to the allowlist is a deliberate edit
20//! to this file that shows up in a diff.
21//!
22//! The field name itself is kept. Knowing that an event carried a
23//! `runner_token` field is useful, and the name is not the secret.
24//!
25//! # Two layers, because one is not enough
26//!
27//! The allowlist protects *fields*. It cannot protect the message body, which
28//! has to be allowed or the logs say nothing — and a message body is exactly
29//! where a secret ends up when somebody writes
30//! `info!("failed with {authorization}")`. So every string that survives the
31//! allowlist is then scrubbed by value shape ([`redact`]): GitHub token
32//! prefixes, credential header names, long opaque runs such as an encoded JIT
33//! configuration, and filesystem paths.
34//!
35//! Scrubbing by shape over-redacts, deliberately. Some of that is worth knowing
36//! about before it surprises somebody:
37//!
38//! - A slash-rooted word is treated as a filesystem path, and a URL *path* on
39//!   its own looks exactly like one. Log a full URL — `https://api.github.com/…`
40//!   — and it survives intact; log a bare `/repos/owner/repo` and it becomes
41//!   `[path]`.
42//! - Any unbroken run of 40 or more base64, base64url, or hex characters is
43//!   treated as an opaque secret. There is exactly one carve-out: a value that
44//!   is precisely 64 lowercase hex characters is a SHA-256 digest, and renders
45//!   as a labelled 12-character prefix rather than disappearing. A digest is
46//!   not a secret, and `07-security.md` makes checksum verification a security
47//!   gate whose most useful diagnostic is expected-versus-actual.
48//! - A JSON Web Token is redacted whole. Its `.` separators split it into runs
49//!   the opaque-run rule is too short-sighted to catch, so it is recognised by
50//!   shape instead: two or three base64url segments whose header begins `eyJ`.
51//! - A URL keeps its scheme, host and path and loses everything that
52//!   authenticates it: the `user:password@` userinfo, the query string, and the
53//!   fragment. The userinfo matters more than it looks — a
54//!   token-authenticated git remote is
55//!   `https://x-access-token:ghu_…@github.com/owner/repo.git`, so it is the
56//!   shape a clone or fetch failure arrives in. The path stays diagnosable but
57//!   is not exempt: each segment goes through the same shape rules on its own,
58//!   because a token in `…/raw/ghu_…/f` is a token, and the alternative was the
59//!   one place the belt never ran. A 40-character git object name is opaque
60//!   enough to go with it.
61//! - A word ending in `:` or `=` whose stem is a credential header name causes
62//!   the next two words to be redacted, so `Authorization: Bearer ghu_…` loses
63//!   both the scheme and the token.
64//!
65//! A word is cut on structural punctuation — `,`, `;`, `{`, `}`, `[`, `]`,
66//! `<`, `>` and `&` — before any of that runs, and each fragment is then judged
67//! the way a whole word is: unwrapped, judged on its core, and re-emitted with
68//! its punctuation put back. Without that cut only the *first* key/value pair
69//! in a compact structure is ever examined, and redaction becomes a function of
70//! field order: `{"encoded_jit_config":"…"}` was caught and
71//! `{"runner_id":42,"encoded_jit_config":"…"}` was not, while
72//! `serde_json::to_string` is what decides which of the two an error body is.
73//! Nesting, a form-encoded body, a `;`-separated connection string and a plist
74//! element are all that same defect in different punctuation. So is an array
75//! element, one step further down — a fragment judged *with* its quote still
76//! attached matches no shape rule at all, and an array element is the one
77//! fragment that has no key of its own to give it away.
78//!
79//! Because the cut is flat, a credential's value does not have to be in the
80//! same fragment as the key that names it — `{"password":["hunter2"]}`,
81//! `{"password":{"v":"hunter2"}}` and a plist's
82//! `<key>password</key><string>hunter2</string>` all put it one or more
83//! fragments away — so a *carry* is threaded along the fragments to say that a
84//! key is still waiting for its value, or that a quoted value was cut before
85//! its closing quote. It steps over element names, because markup is not the
86//! value a key named, and it stops at the punctuation that visibly closes the
87//! value, because a redaction reported where no secret was is a false signal in
88//! the one log a reader consults to find out whether anything leaked.
89//!
90//! A URL is cut out of the text around it rather than being allowed to own the
91//! rest of the word. Its scheme is the run of scheme characters immediately
92//! before the `://`, and it ends at the first character that cannot appear in a
93//! URL — plus, *ahead of its query string only*, at a `;` or an `&`, which are
94//! structural characters everywhere else and are query syntax after the `?`.
95//! Everything on either side goes back through the rules. So
96//! `{"documentation_url":"https://…","token":"ghu_…"}` — which is what a GitHub
97//! REST error body looks like — keeps the URL and redacts the token, rather
98//! than the URL swallowing the token, and
99//! `Server=https://vault.local/api;Password=…` keeps the URL and redacts the
100//! password rather than the URL swallowing the `;` that separates them.
101//!
102//! **The text after a URL is iterated over, not recursed into.** That is a
103//! memory-safety property, not a style: recursing there made stack depth linear
104//! in the length of the message, and ~86 KB of URL-carrying JSON exited
105//! `STATUS_STACK_OVERFLOW`. A stack overflow is not catchable and takes the
106//! process with it, so an attacker-influenceable error body could kill the
107//! agent from inside its own log sink — and a sink that is not running redacts
108//! nothing at all. For the same reason every search in `split_url` is bounded
109//! by the URL's own end: a pass that never returns is as effective a denial as
110//! one that overflows.
111//!
112//! A key is also trimmed of backslashes, which a value never is: `Debug` on a
113//! `String` escapes the quotes inside it, so a body reached through
114//! `error!(reason = ?err)` spells its keys `\"password\"`. The `trim_key`
115//! function documents why the same trim must not be applied to a value; it is
116//! named rather than linked because it is private and this module's
117//! documentation is not.
118//!
119//! Every one of those is a case where being wrong costs a slightly less
120//! readable log line, against a case where being wrong the other way costs a
121//! disclosed credential.
122//!
123//! # What this does not do
124//!
125//! It does not stop a caller printing to standard output, and it does not
126//! redact a `Debug` derive somewhere else in the program. Those are held by
127//! different controls: `secrecy::SecretString` for the two values that matter
128//! (`07-security.md`'s credential inventory), and
129//! [`crate::process::SpawnSpec::spawn_with_handoff`] for the command line.
130
131use std::fmt;
132use std::io::Write;
133use std::path::{Path, PathBuf};
134
135use serde_json::{Map, Value};
136use tracing::field::{Field, Visit};
137use tracing::{Event, Subscriber};
138use tracing_subscriber::layer::{Context, Layer};
139use tracing_subscriber::registry::LookupSpan;
140
141/// What replaces a value this sink will not emit.
142pub const REDACTION: &str = "[redacted]";
143
144/// What replaces something that was recognisably a filesystem path. Distinct
145/// from [`REDACTION`] so a reader can tell "a path was here" from "a secret was
146/// here" without either being disclosed.
147pub const PATH_REDACTION: &str = "[path]";
148
149/// The field names this sink emits verbatim. Everything else is replaced with
150/// [`REDACTION`].
151///
152/// ---
153///
154/// **Read this before adding a name.** A field on this list still passes
155/// through [`redact`], so adding one does not switch redaction off — but it
156/// does mean the field's value reaches the scrubber instead of being discarded,
157/// and the scrubber only recognises shapes it was taught. Add a name only when
158/// the value is structurally incapable of carrying a credential: an
159/// identifier, an enumerated state, a count, a duration. Never add a name whose
160/// value is free text supplied by GitHub or by a workflow.
161///
162/// Kept sorted, and a test enforces that, so an addition is one line in a diff
163/// rather than a name buried in the middle of a list.
164pub const ALLOWED_FIELDS: &[&str] = &[
165    "arch",
166    "attempt",
167    "attempt_id",
168    "attempt_state",
169    "capacity",
170    "count",
171    // Seconds a rate-limited client waits before its next request.
172    "delay_secs",
173    "demand",
174    "desired",
175    "duration_ms",
176    "elapsed_ms",
177    "error_kind",
178    "event",
179    "exit_code",
180    "headroom",
181    "host_id",
182    "http_status",
183    "installation_id",
184    "job_id",
185    // A closed enum's `Display`, such as a rate limit's primary/secondary kind.
186    "kind",
187    "label",
188    "lock",
189    "lock_wait_ms",
190    "materialize_ms",
191    "message",
192    "mode",
193    "os",
194    "outcome",
195    "pid",
196    "policy_id",
197    "policy_state",
198    "reason",
199    "register_ms",
200    // GitHub's `x-ratelimit-remaining`: a request count.
201    "remaining",
202    "retry_in_ms",
203    "runner_id",
204    "scope",
205    "spawn_ms",
206    "start_mode",
207    "state",
208    "target",
209    "version",
210];
211
212/// Header and parameter names whose value is a credential.
213///
214/// Compared case-insensitively with `-` and `_` treated as the same character,
215/// because the same header arrives as `Authorization`, `authorization`, and
216/// `x_api_key` depending on who wrote the line.
217const CREDENTIAL_KEYS: &[&str] = &[
218    "access.token",
219    "api.key",
220    "apikey",
221    "auth",
222    "authorization",
223    "client.secret",
224    "cookie",
225    "credential",
226    "encoded.jit.config",
227    "jit",
228    "jit.config",
229    "jitconfig",
230    "password",
231    "private.token",
232    "proxy.authorization",
233    "refresh.token",
234    "secret",
235    "set.cookie",
236    "token",
237    "www.authenticate",
238    "x.api.key",
239    "x.auth.token",
240    "x.github.token",
241    "x.hub.signature",
242    "x.hub.signature.256",
243];
244
245/// Authentication scheme words. Whatever follows one of these is the
246/// credential.
247const SCHEME_WORDS: &[&str] = &["basic", "bearer", "digest", "negotiate", "token"];
248
249/// Prefixes GitHub gives its credentials. Present as a belt on top of the
250/// opaque-run rule below, because a short-lived or truncated token can be under
251/// the length threshold while still being a live credential.
252const TOKEN_PREFIXES: &[&str] = &["ghp_", "gho_", "ghu_", "ghs_", "ghr_", "github_pat_", "gh_"];
253
254/// How long an unbroken run of opaque characters has to be before it is assumed
255/// to be a secret.
256///
257/// 40 rather than something shorter so that ordinary identifiers, hyphenated
258/// words, and UUIDs (36 characters, and not secret) survive; short enough that
259/// every GitHub credential format and every encoded JIT configuration is well
260/// past it.
261const OPAQUE_RUN_THRESHOLD: usize = 40;
262
263/// Characters that make up base64, base64url, and hex.
264fn is_opaque_char(c: char) -> bool {
265    c.is_ascii_alphanumeric() || matches!(c, '+' | '/' | '=' | '_' | '-')
266}
267
268/// Punctuation that may wrap a value without being part of it.
269const WRAPPERS: &[char] = &[
270    '"', '\'', '`', '(', ')', '[', ']', '{', '}', '<', '>', ',', ';', '.', '!', '?',
271];
272
273/// Punctuation that separates one key/value pair from the next *inside* a
274/// single whitespace-delimited word.
275///
276/// [`redact_core`] judges a fragment by splitting it once, on the first `=` or
277/// `:` it finds. That is enough for `key=value` and for a one-field object,
278/// and it is why every shape this module was tested against put the credential
279/// in the first pair. It is not enough for anything `serde_json::to_string`
280/// actually emits: a struct with two fields becomes
281/// `{"runner_id":42,"encoded_jit_config":"…"}`, where the only pair ever
282/// examined is `runner_id`. Nesting is the same defect one level down, and a
283/// form-encoded body is the same defect spelled with `&`.
284///
285/// So a word is cut on these before any of that runs, and each fragment is
286/// judged on its own, by [`redact_fragment`]. The separators go back verbatim,
287/// because the structure around a redaction is what keeps the line diagnosable.
288///
289/// `;` is here for the reason `&` is: it separates the pairs of a Windows
290/// connection string (`Server=host;Database=x;Password=…`), of a credential
291/// string, and of a cookie header written without a space after the separator.
292/// It was already in [`WRAPPERS`] — recognised as punctuation, and so never
293/// used to cut — which left `Set-Cookie: theme=dark; session=…` safe only
294/// because of the space. `d2` logs keychain, DPAPI and libsecret failures, and
295/// that is the shape they arrive in.
296///
297/// `<` and `>` are here because [`split_wrappers`] strips only the *outermost*
298/// pair, so `<string>ghu_…</string>` reached the rules as
299/// `string>ghu_…</string`, which is on no list and matches no shape. `d3`'s
300/// installers handle launchd plists, which is where that shape comes from.
301const STRUCTURAL: &[char] = &[',', ';', '{', '}', '[', ']', '<', '>', '&'];
302
303/// Whether this sink will emit a field's value rather than replacing it.
304#[must_use]
305pub fn is_field_allowed(name: &str) -> bool {
306    ALLOWED_FIELDS.binary_search(&name).is_ok()
307}
308
309/// Trims the punctuation a *key* can arrive wrapped in: [`WRAPPERS`], plus the
310/// backslash.
311///
312/// The backslash is here rather than in [`WRAPPERS`] on purpose, and the
313/// distinction is load-bearing in both directions.
314///
315/// It has to be trimmed somewhere. `tracing::error!(reason = ?err)` reaches
316/// this module through `record_debug` and `format!("{:?}")`, and `Debug` on a
317/// `String` escapes the quotes inside it — so an error whose `Debug` embeds an
318/// HTTP body arrives with its keys spelled `\"password\"`. Trimming only
319/// [`WRAPPERS`] leaves the backslash welded on, and `\"password\` is on no
320/// list. That is `d2`'s shape: a secret-store failure carrying the body it was
321/// handed.
322///
323/// It must not be trimmed unconditionally. [`split_wrappers`] runs before
324/// [`looks_like_path`], so a `\` in [`WRAPPERS`] would strip the leading
325/// `\\` that UNC detection keys on, and `\\server\share` would stop being
326/// recognised as a path. A key is the one place a backslash can never be part
327/// of the value, so it is the one place the trim is unconditional;
328/// [`trim_start_wrappers`] takes the same backslash off a *value* only when it
329/// is escaping punctuation.
330fn trim_key(key: &str) -> &str {
331    key.trim()
332        .trim_matches(|c: char| c == '\\' || WRAPPERS.contains(&c))
333}
334
335fn normalise_key(key: &str) -> String {
336    trim_key(key).to_ascii_lowercase().replace(['-', '_'], ".")
337}
338
339fn is_credential_key(key: &str) -> bool {
340    let key = normalise_key(key);
341    CREDENTIAL_KEYS.contains(&key.as_str())
342}
343
344fn is_scheme_word(word: &str) -> bool {
345    SCHEME_WORDS.contains(&word.to_ascii_lowercase().as_str())
346}
347
348/// Scrubs a string of anything that looks like a credential, an encoded JIT
349/// configuration, or a filesystem path.
350///
351/// Applied to every string this sink emits, including the ones whose field name
352/// is on [`ALLOWED_FIELDS`]. Also public so that the TUI and the CLI can put a
353/// value through the same rules before showing it, rather than inventing a
354/// second, differently wrong set.
355#[must_use]
356pub fn redact(text: &str) -> String {
357    let mut out = String::with_capacity(text.len());
358    // How many upcoming words to replace outright, because a credential key or
359    // an authentication scheme word said the value comes next.
360    let mut pending: u32 = 0;
361
362    for chunk in text.split_inclusive(char::is_whitespace) {
363        let (word, whitespace) = split_trailing_whitespace(chunk);
364
365        if word.is_empty() {
366            out.push_str(whitespace);
367            continue;
368        }
369
370        if pending > 0 {
371            pending -= 1;
372            out.push_str(REDACTION);
373            out.push_str(whitespace);
374            // A header value ends at a comma or semicolon; anything after that
375            // is the next header's name and is not a secret.
376            if word.ends_with([',', ';']) {
377                pending = 0;
378            }
379            continue;
380        }
381
382        let (rendered, follow_on) = redact_word(word);
383        out.push_str(&rendered);
384        out.push_str(whitespace);
385        pending = follow_on;
386    }
387
388    out
389}
390
391/// Splits a chunk produced by `split_inclusive` into its word and the single
392/// whitespace character that terminated it, if any.
393fn split_trailing_whitespace(chunk: &str) -> (&str, &str) {
394    match chunk.char_indices().next_back() {
395        Some((index, last)) if last.is_whitespace() => chunk.split_at(index),
396        _ => (chunk, ""),
397    }
398}
399
400/// Splits a fragment into its leading wrapping punctuation, its core, and its
401/// trailing wrapping punctuation.
402///
403/// Shared by [`redact_word`] and [`redact_core`], because both have to judge a
404/// fragment on its core while emitting it with its punctuation intact.
405///
406/// [`WRAPPERS`], and an *escaped* quote as well. Escaped text is exactly what
407/// a `Debug` rendering of a string is, and in it every quote arrives with a
408/// backslash welded on — so a value spelled `\"ghu_…\"` is wrapped in the
409/// same way `"ghu_…"` is, and none of the shape rules can see the token until
410/// the wrapper comes off. Trimming the key alone is not enough: `runner_token`
411/// is deliberately *not* on [`CREDENTIAL_KEYS`], because a name is not what
412/// makes a value a secret, so that pair is caught by the token-prefix rule
413/// reading its value or it is not caught at all.
414fn split_wrappers(fragment: &str) -> (&str, &str, &str) {
415    let leading = fragment.len() - trim_start_wrappers(fragment).len();
416    let (prefix, rest) = fragment.split_at(leading);
417    let core_len = trim_end_wrappers(rest).len();
418    let (core, suffix) = rest.split_at(core_len);
419    (prefix, core, suffix)
420}
421
422/// Trims wrapping punctuation from the front of a fragment.
423///
424/// A backslash counts as punctuation **only when it escapes punctuation**, and
425/// that restriction is the whole of what keeps this safe. `\"value` is a
426/// quoted value spelled the way an escaped rendering spells it, and the
427/// backslash is not part of the value. `\\server\share` is a UNC path whose
428/// leading backslashes escape nothing and *are* the value — trimming them is
429/// exactly what would stop [`looks_like_path`] recognising it, and this runs
430/// before [`looks_like_path`] does. `\\` is not a backslash-escaping-a-
431/// wrapper, so the two cases separate cleanly.
432fn trim_start_wrappers(fragment: &str) -> &str {
433    let mut rest = fragment;
434    loop {
435        let trimmed = rest.trim_start_matches(WRAPPERS);
436        let trimmed = match trimmed.strip_prefix('\\') {
437            Some(after) if after.starts_with(WRAPPERS) => after,
438            _ => trimmed,
439        };
440        if trimmed.len() == rest.len() {
441            return rest;
442        }
443        rest = trimmed;
444    }
445}
446
447/// Trims wrapping punctuation from the end of a fragment.
448///
449/// A trailing backslash goes with it, and needs no adjacency test: trimming
450/// runs right to left, so a backslash that has reached the end is one whose
451/// quote has already been taken off. A path that ends in a separator is still
452/// a path without it.
453fn trim_end_wrappers(fragment: &str) -> &str {
454    let mut rest = fragment;
455    loop {
456        let trimmed = rest.trim_end_matches(WRAPPERS).trim_end_matches('\\');
457        if trimmed.len() == rest.len() {
458            return rest;
459        }
460        rest = trimmed;
461    }
462}
463
464/// What a credential key left outstanding at the end of the text that named it.
465///
466/// The structural cut is flat and a credential's value does not have to be in
467/// the same fragment as the key that names it, so something has to carry the
468/// key across the cut. Nothing did, which is [`redact_core`]'s empty-value
469/// case: the comment there claimed the value would be reached "in the next
470/// fragment, where the structural cut reaches it", and no such thing happened.
471///
472/// The two variants are not the same claim, and collapsing them over-redacts:
473///
474/// - [`Carry::Expecting`] is *"a key named a value that has not appeared yet"*.
475///   It has to step over markup, because `<key>password</key><string>…</string>`
476///   puts `/key` and `string` between the key and its value.
477/// - [`Carry::Unclosed`] is *"a quoted value was redacted and its closing quote
478///   is not in this fragment"*, so the value continues past the cut. Inside an
479///   open quote `<` and `>` are literal text rather than markup, so this one
480///   must *not* step over anything.
481#[derive(Clone, Copy, PartialEq, Eq, Debug)]
482enum Carry {
483    /// Nothing outstanding.
484    None,
485    /// A credential key supplied no value of its own.
486    Expecting,
487    /// A redacted credential value was cut before its closing quote.
488    Unclosed,
489}
490
491/// Whether wrapping punctuation closes the value it followed.
492///
493/// A quote closes a string and a bracket closes a structure, so either one
494/// means the credential value ended here and a carry must stop. `,` and `;` are
495/// on the list for the reason [`redact`] already stops its word-level follow-on
496/// at them: they end a header value, and what comes after is the next header's
497/// name.
498///
499/// `>` is deliberately absent. It closes a *tag*, not a value —
500/// `<key>password</key>` ends in one with the credential's value still to come
501/// — and treating it as a terminator is what would leave the multi-word plist
502/// spelling leaking.
503fn closes_a_value(wrappers: &str) -> bool {
504    wrappers.contains(['"', '\'', '`', ',', ';', ']', '}', ')'])
505}
506
507/// Whether a value opened a quote that its own fragment did not close.
508fn opens_an_unclosed_quote(lead: &str, trail: &str) -> bool {
509    const QUOTES: [char; 3] = ['"', '\'', '`'];
510    lead.contains(QUOTES) && !trail.contains(QUOTES)
511}
512
513/// Whether a fragment sits where an element *name* goes rather than where its
514/// content does.
515///
516/// `<` and `>` are in [`STRUCTURAL`] because `<string>ghu_…</string>` reached
517/// the rules as `string>ghu_…</string` and matched nothing. Having cut on them,
518/// the loop knows which side of a tag it is on, and a [`Carry::Expecting`] must
519/// not be spent on `/key` or `string` when the value it is waiting for is the
520/// element's content. A tag name is still judged by every other rule — this
521/// says only that it is not the value a credential key named.
522fn is_tag_name(preceding: Option<char>, following: Option<char>) -> bool {
523    preceding == Some('<') && following == Some('>')
524}
525
526/// Redacts one whitespace-delimited word, and says how many following words the
527/// word implicates.
528fn redact_word(word: &str) -> (String, u32) {
529    // Wrapping punctuation is kept so that JSON-ish and prose context survives
530    // — `("ghu_…")` should become `("[redacted]")`, not `[redacted]`.
531    let (prefix, core, suffix) = split_wrappers(word);
532
533    if core.is_empty() {
534        return (word.to_string(), 0);
535    }
536
537    // A bare credential key or scheme word: the value is the next word.
538    //
539    // The stem is unwrapped before it is judged: spaced JSON writes the key as
540    // `"password":`, which leaves `password"` welded to a quote, and that is on
541    // no list. A long value survived this anyway by way of the opaque-run rule,
542    // so the gap only ever showed on a short one.
543    let stem = core.trim_end_matches([':', '=']);
544    if stem.len() < core.len() && is_credential_key(stem) {
545        // Two, so that `Authorization: Bearer <token>` loses the scheme and the
546        // token rather than only the scheme.
547        return (word.to_string(), 2);
548    }
549    if is_scheme_word(core) {
550        return (word.to_string(), 1);
551    }
552
553    let (rendered, carry) = redact_core(core, suffix);
554
555    // A value the word did not finish is the *next* word's problem, which is
556    // the follow-on rule the word level already has, reached from one level
557    // down. A pretty-printed plist is why it is needed: `<key>password</key>`
558    // and `<string>hunter2</string>` are on separate lines, so the key and its
559    // value are not even in the same word.
560    //
561    // The two carries ask for different amounts, for the same reason the stem
562    // rule above asks for two words and not one:
563    //
564    // - [`Carry::Expecting`] means the value has not been seen *at all*, so it
565    //   may be introduced by a word of its own — `<string>correct` and then
566    //   `horse</string>`. Two, exactly as `Authorization: Bearer <token>` needs
567    //   two.
568    // - [`Carry::Unclosed`] means the value has already started and was cut, so
569    //   only its remainder is outstanding. One.
570    //
571    // Trailing punctuation that closes the value withdraws the claim, for the
572    // same reason pass one declines an empty value: `{"password":""}` names a
573    // credential and supplies nothing, and redacting the next word there would
574    // report a secret where none was.
575    let follow_on = if closes_a_value(suffix) {
576        0
577    } else {
578        match carry {
579            Carry::None => 0,
580            Carry::Unclosed => 1,
581            Carry::Expecting => 2,
582        }
583    };
584    (format!("{prefix}{rendered}{suffix}"), follow_on)
585}
586
587/// Redacts one fragment of a cut-up word, judging it on its core.
588///
589/// [`redact_core`] used to recurse on the *raw* fragment, and its terminal
590/// fallback then handed that fragment to [`redact_value`] with its wrapping
591/// punctuation still attached. A fragment carrying no `:` or `=` of its own —
592/// which is exactly what an array element is — therefore reached the shape
593/// rules as `"ghu_…`, where `starts_with("ghu_")` fails, [`is_opaque_char`]
594/// fails on the quote, the `eyJ` test fails, and [`looks_like_path`] never gets
595/// a clean look at `"C:\Users\…`.
596///
597/// That is the defect the structural cut was written to close, one step further
598/// down: *"only the first key/value pair is ever examined"* became *"a value
599/// with no key of its own is never examined"*. It survived a round because the
600/// secret-injection scan had no array among its shapes, which is the same
601/// lesson in a different place — a check that cannot fail proves nothing.
602///
603/// So a fragment is treated the way [`redact_word`] treats a word: split,
604/// judge the core, put the punctuation back.
605///
606/// The *word-level* follow-on rule is still not repeated here, and the reason
607/// stands: *"the value is the next word"* is a statement about whitespace, and
608/// there is no next word inside a fragment. What that argument does not cover —
609/// and what an earlier spelling of it wrongly took itself to have settled — is
610/// the *fragment-level* claim: **the value is the next fragment**. That one is
611/// a statement about the structural cut, it is true, and nothing implemented
612/// it, which is why `{"password":["hunter2"]}` and
613/// `<key>password</key><string>hunter2</string>` went out whole. [`Carry`] is
614/// that claim, threaded through the loop in [`redact_core`].
615///
616/// `trailing` says this is the last fragment of its core, and it is what keeps
617/// the carry from escaping a word it has already been spent inside: consuming
618/// a claim in the middle of a core settles it, while consuming it at the end
619/// leaves open the possibility that the value runs on into the next word.
620fn redact_fragment(
621    fragment: &str,
622    carry: Carry,
623    tag_name: bool,
624    trailing: bool,
625) -> (String, Carry) {
626    let (prefix, core, suffix) = split_wrappers(fragment);
627    if core.is_empty() {
628        return (fragment.to_string(), carry);
629    }
630
631    let claimed = match carry {
632        Carry::None => false,
633        // Markup is not the value a credential key named.
634        Carry::Expecting => !tag_name,
635        // Inside an open quote there is no markup, only text.
636        Carry::Unclosed => true,
637    };
638
639    if claimed {
640        // Two different reasons to think the value has more to come, and one
641        // answer: it is `Unclosed` from here, never the `Expecting` it may have
642        // arrived as, because the value has now started.
643        //
644        // - `carry == Unclosed` — still inside an open quote, so the value runs
645        //   into the next fragment as well.
646        // - `trailing` — a claim spent on the *last* fragment of a core is one
647        //   whose value reached the end of the word with nothing closing it, so
648        //   it may continue into the next word.
649        //   `<key>password</key><string>` + `correct horse</string>` is that
650        //   shape.
651        //
652        // Anything else was spent in the middle of a core, where the value was
653        // and is done — and punctuation that closes a string or a structure
654        // settles it either way.
655        let next = if !closes_a_value(suffix) && (carry == Carry::Unclosed || trailing) {
656            Carry::Unclosed
657        } else {
658            Carry::None
659        };
660        return (format!("{prefix}{REDACTION}{suffix}"), next);
661    }
662
663    let (rendered, next) = redact_core(core, suffix);
664    let next = if closes_a_value(suffix) {
665        Carry::None
666    } else if next == Carry::None {
667        // **An unclaimed fragment preserves the claim rather than clearing
668        // it.** This is the whole of what lets a plist work: `/key` and
669        // `string` sit between `<key>password</key>` and the `<string>` content
670        // that is the value, they are tag names so they do not spend the claim,
671        // and a fragment that neither spends nor arms one must leave it exactly
672        // as it found it. Overwriting with this fragment's own (empty) result
673        // is how `<key>password</key><string>hunter2</string>` kept leaking
674        // after the carry existed.
675        carry
676    } else {
677        next
678    };
679    (format!("{prefix}{rendered}{suffix}"), next)
680}
681
682/// Redacts one word with its wrapping punctuation already removed, and reports
683/// what it left outstanding for whatever follows it.
684///
685/// `closing` is the wrapping punctuation the caller stripped off the end of
686/// this core. It is not emitted here — the caller puts it back — but the loop
687/// below has to see it, because the character that follows the *last* fragment
688/// is what says whether that fragment is an element name or an element's
689/// content.
690fn redact_core(core: &str, closing: &str) -> (String, Carry) {
691    // A URL survives, minus everything on it that carries a credential. The
692    // scheme, host and path are what makes a log line diagnosable.
693    //
694    // The URL is *cut out* of the text around it rather than being allowed to
695    // own the rest of the fragment, and both sides of the cut come back through
696    // here. [`split_url`] documents what the old unbounded `split_once("://")`
697    // cost. This still runs ahead of the structural cut, because a query string
698    // is `redact_url`'s to own: `?` and `#` are not structural characters, and
699    // an OAuth implicit-flow response puts the token after one of them.
700    //
701    // **The tail is iterated, not recursed into, and that is a memory-safety
702    // property rather than a matter of taste.** An earlier spelling called
703    // `redact_core` on the remainder, and argued only that the recursion
704    // *terminates*: every call is on a strictly shorter slice, which is true
705    // and is not enough. Termination says nothing about **depth**, and depth
706    // was linear in the length of the input — one frame per URL. A message of
707    // ~2000 `{"url":"https://…"},` items, about 86 KB, exited
708    // `0xc00000fd STATUS_STACK_OVERFLOW`.
709    //
710    // A stack overflow is not a `panic`: it is not catchable, it does not
711    // unwind, and it takes the process with it. A large HTTP error body is
712    // content an attacker can influence, so that was a way to kill the agent
713    // from inside its own log sink — and a sink that is not running redacts
714    // nothing at all, which is worse than any single leak.
715    //
716    // With the loop, every remaining re-entry is depth-bounded by construction
717    // rather than by argument:
718    //
719    // - `prefix` holds everything before the *first* `://`, so it contains no
720    //   `://` and cannot reach this branch again.
721    // - `rest` is what is left when the loop finds no further `://`, for the
722    //   same reason.
723    // - the structural branch below calls [`redact_fragment`], whose fragments
724    //   contain no structural character by construction and so cannot reach it
725    //   again either.
726    //
727    // Three levels, whatever the input is. Anything added here that recurses on
728    // a slice whose length is not bounded by a constant re-opens this.
729    if let Some((prefix, scheme, url, remainder)) = split_url(core) {
730        let mut out = String::with_capacity(core.len());
731        if !prefix.is_empty() {
732            out.push_str(&redact_core(prefix, "").0);
733        }
734        out.push_str(&redact_url(scheme, url));
735
736        let mut rest = remainder;
737        while let Some((prefix, scheme, url, remainder)) = split_url(rest) {
738            if !prefix.is_empty() {
739                out.push_str(&redact_core(prefix, "").0);
740            }
741            out.push_str(&redact_url(scheme, url));
742            rest = remainder;
743        }
744        if !rest.is_empty() {
745            out.push_str(&redact_core(rest, "").0);
746        }
747
748        // A URL is a complete value: it carries its own credentials in its own
749        // places, and `redact_url` has already dealt with them. Nothing is
750        // outstanding, which is also what keeps `{"token":"https://…"}` from
751        // arming a claim against whatever follows the URL.
752        return (out, Carry::None);
753    }
754
755    // A Windows drive path is `key:value`-shaped by accident, and
756    // `looks_like_path` only recognises the drive letter at position zero.
757    // Splitting such a path on its colon would hand the tail to a rule that
758    // matches nothing, so it has to be judged whole, before the separators
759    // get at it.
760    if looks_like_path(core) {
761        return (PATH_REDACTION.to_string(), Carry::None);
762    }
763
764    // A compact structure holds more than one key/value pair, and the rules
765    // below examine exactly one of them: `split_once` stops at the first
766    // separator it finds. So `{"encoded_jit_config":"…"}` was caught and
767    // `{"runner_id":42,"encoded_jit_config":"…"}` was not — the two differ by
768    // field order and by nothing else, and `serde_json::to_string` is what
769    // chooses the order. Nesting was the same defect: the value recursion
770    // below ends at `redact_value`, which is a leaf and never comes back
771    // here, so an object inside an object went out whole. A form-encoded body
772    // was the same defect again, with `&` as a separator nothing knew.
773    //
774    // Cutting on [`STRUCTURAL`] first turns all three into the shape the rules
775    // already handle, and does it at any depth: the cut is flat, so
776    // `{"a":{"b":{"token":"…"}}}` yields the same fragments a one-level object
777    // would.
778    //
779    // Each fragment goes through [`redact_fragment`] rather than straight back
780    // in here, because a fragment carries its own wrapping punctuation and a
781    // value judged with its quote attached matches nothing at all.
782    //
783    // The recursion terminates *and is depth-bounded*: a fragment contains no
784    // structural character by construction, so `redact_fragment` cannot reach
785    // this branch again, and the URL branch above iterates over its tail rather
786    // than recursing into it. The loop here is a loop for the same reason — a
787    // fragment per separator, at whatever length the input has.
788    //
789    // A [`Carry`] is threaded along it because the cut is flat and a
790    // credential's value need not be in the same fragment as its key. Without
791    // it the empty-value skip in pass one was a promise nothing kept: the value
792    // was said to be reachable "in the next fragment", and no next fragment
793    // ever heard about the key.
794    if core.contains(STRUCTURAL) {
795        let mut out = String::with_capacity(core.len());
796        let mut carry = Carry::None;
797        let mut rest = core;
798        // The separator on each side of a fragment is what says whether it is
799        // an element name or an element's content.
800        let mut preceding: Option<char> = None;
801        while let Some(index) = rest.find(STRUCTURAL) {
802            let (fragment, tail) = rest.split_at(index);
803            // Every character in `STRUCTURAL` is ASCII, so the separator is
804            // one byte, this cannot split a code point, and the byte is the
805            // whole character.
806            let separator = char::from(tail.as_bytes()[0]);
807            if !fragment.is_empty() {
808                let (rendered, next) = redact_fragment(
809                    fragment,
810                    carry,
811                    is_tag_name(preceding, Some(separator)),
812                    false,
813                );
814                out.push_str(&rendered);
815                carry = next;
816            }
817            out.push_str(&tail[..1]);
818            preceding = Some(separator);
819            rest = &tail[1..];
820        }
821        if !rest.is_empty() {
822            // The last fragment's *following* character is the caller's closing
823            // punctuation: in `<key>password</key>` the final `/key` is a tag
824            // name only because the `>` that ends it was stripped as a wrapper
825            // before this ran.
826            let following = closing.chars().next();
827            let (rendered, next) =
828                redact_fragment(rest, carry, is_tag_name(preceding, following), true);
829            out.push_str(&rendered);
830            carry = next;
831        }
832        return (out, carry);
833    }
834
835    // `key=value` and `key:value` in a single fragment.
836    //
837    // Pass one asks only whether either separator names a credential, and it
838    // runs to completion before any value is inspected, because the *first*
839    // separator in a fragment is not necessarily the one that names the key.
840    //
841    // The witness is a webhook signature header in its compact-JSON spelling,
842    // `{"x-hub-signature-256":"sha256=<hmac>"}`, and
843    // `a_credential_header_loses_its_scheme_and_its_value` holds it. The `=`
844    // comes first and names nothing, but what follows it is a whole HMAC —
845    // *non-empty*, so a merged pass inspects that value, renders it as a
846    // digest prefix, and returns without ever reaching the `:` that names
847    // `x-hub-signature-256`. `{"password":"a=b"}` is the same witness with
848    // nothing else going on: merged, it comes out whole.
849    //
850    // Two earlier spellings of this comment named
851    // `{"encoded_jit_config":"eyJ…In0="}` and then
852    // `{"encoded_jit_config":"eyJ…In0=","runner_id":42}`, and neither
853    // demonstrates the invariant: base64 padding is trailing-only, so the `=`
854    // split yields a value that `split_wrappers` trims to empty, and the
855    // `value.is_empty()` guard in pass two falls through to the `:` anyway. A
856    // comment naming a case that does not demonstrate its own invariant is how
857    // the invariant gets deleted by the next person, so the witness above is
858    // one that reds a named test when the two passes are merged.
859    //
860    // Both halves are judged through `trim_key`: compact JSON welds a quote to
861    // each, so the key arrives as `encoded_jit_config"`, and a `Debug`
862    // rendering welds a backslash as well.
863    // Whether pass one saw a credential key and was given nothing to redact.
864    let mut names_a_credential = false;
865
866    for separator in ['=', ':'] {
867        if let Some((key, raw_value)) = core.split_once(separator)
868            && is_credential_key(key)
869        {
870            // The value's own wrapping punctuation goes back, exactly as pass
871            // two puts it back. Dropping it emitted `{"password":[redacted]"}`
872            // — an unbalanced quote in a line this module argues, correctly,
873            // has to stay diagnosable, and a reader who cannot parse the line
874            // cannot tell a redaction from a truncation.
875            let (lead, value, trail) = split_wrappers(raw_value);
876
877            // An empty value is nothing to redact, and pass two declines one
878            // for the same reason. Claiming `[redacted]` here says a secret was
879            // somewhere none was, which is a false signal in the one log a
880            // reader consults to find out whether anything leaked.
881            //
882            // It is also load-bearing for the URL branch above, which sends
883            // the text before a URL back through here: `{"token":"https://…"}`
884            // leaves this pass a key of `token` and a value of nothing, and
885            // redacting that emitted `token":"[redacted]https://…` — the URL
886            // still standing behind the redaction meant to have replaced it.
887            //
888            // **What it is not is a reason to forget the key.** An earlier
889            // spelling of this comment said the value would be found "in the
890            // next fragment or the next word, where the structural cut and the
891            // follow-on rule reach it", and half of that was false: the
892            // follow-on rule does reach the next *word*, and nothing whatever
893            // reached the next *fragment*, because `redact_fragment`
894            // deliberately carries no follow-on. So `{"password":["hunter2"]}`,
895            // `{"password":{"v":"hunter2"}}` and `Password=;hunter2` — a
896            // credential key with its value one fragment away, which is what an
897            // array, a nested object and an empty pair all are — went out
898            // whole, with the key sitting in plain sight next to them.
899            //
900            // That is the failure this module warns about two comments down: a
901            // comment naming a case that does not demonstrate its own
902            // invariant. The claim is now true because [`Carry::Expecting`]
903            // implements it, and `the_fragment_carry_stops_where_the_value_does`
904            // holds both halves — that the claim is made, and that it is
905            // withdrawn where the value visibly ended.
906            if value.is_empty() {
907                names_a_credential = true;
908                continue;
909            }
910
911            // A quoted value whose closing quote is not in this fragment is a
912            // value the cut ran through the middle of: `{"password":"p&ss"}`
913            // reaches here as `password":"p`, and `ss` is the rest of the
914            // secret. Adding `;`, `<` and `>` to `STRUCTURAL` this round made
915            // three more characters able to do that, and punctuated passwords
916            // are ordinary.
917            let carry = if opens_an_unclosed_quote(lead, trail) {
918                Carry::Unclosed
919            } else {
920                Carry::None
921            };
922            return (format!("{key}{separator}{lead}{REDACTION}{trail}"), carry);
923        }
924    }
925
926    // Pass two: not a credential key, but the value may still be a path or a
927    // token: `runtime=/var/lib/runner-manager/…`. Applied to `:` as well as
928    // to `=`, because `:` is what compact JSON and a bare `key:value` use,
929    // and recursing for `=` alone is what let an encoded JIT configuration
930    // and a `runner_token:ghu_…` pair through verbatim.
931    for separator in ['=', ':'] {
932        if let Some((key, raw_value)) = core.split_once(separator) {
933            let (lead, value, trail) = split_wrappers(raw_value);
934            if value.is_empty() {
935                continue;
936            }
937            let redacted = redact_value(value);
938            // `as_sha256_digest` re-attaches its own `sha256:` label, so an
939            // already-labelled digest would come back doubled as
940            // `sha256:sha256:9f86d081884c…`. Dropping the redundant label
941            // keeps the caller's own key and separator, and leaves the digest
942            // truncated -- which the unrecursed `:` path never did, and which
943            // is worth having, because an HMAC-SHA256 signature has exactly a
944            // digest's shape and was previously printed in full.
945            if trim_key(key).eq_ignore_ascii_case("sha256")
946                && let Some(bare) = redacted.strip_prefix("sha256:")
947            {
948                return (format!("{key}{separator}{lead}{bare}{trail}"), Carry::None);
949            }
950            return (
951                format!("{key}{separator}{lead}{redacted}{trail}"),
952                Carry::None,
953            );
954        }
955    }
956
957    // A credential key with no value at all — either `password:` with nothing
958    // after it, or a bare `password` between a plist's tags. Both name a value
959    // that is somewhere else, and the caller is the only one who can see where.
960    let carry = if names_a_credential || is_credential_key(core) {
961        Carry::Expecting
962    } else {
963        Carry::None
964    };
965    (redact_value(core), carry)
966}
967
968/// Characters a URL scheme is made of: RFC 3986 allows a letter followed by
969/// letters, digits, `+`, `-` and `.`.
970fn is_scheme_byte(byte: u8) -> bool {
971    byte.is_ascii_alphanumeric() || matches!(byte, b'+' | b'-' | b'.')
972}
973
974/// Characters that cannot appear inside a URL, and therefore end one.
975///
976/// `\` is deliberately absent, even though it cannot appear in a URL either.
977/// The shape it would have been there for is a `Debug`-escaped body, which
978/// spells its quotes `\"` — and the quote already ends the URL. Making the
979/// backslash a terminator as well costs a real redaction: a Windows path used
980/// as a URL password, `https://x-access-token:C:\Users\…@github.com/o/r.git`,
981/// would then end its URL at the first `\`, which puts the `@` that identifies
982/// the userinfo *outside* the span, and [`redact_url`] echoes what it is given.
983/// Measured: with `\` a terminator, that line emits the path verbatim; without
984/// it, the userinfo is replaced as it should be.
985fn is_url_terminator(c: char) -> bool {
986    c.is_whitespace() || matches!(c, '"' | ',' | '{' | '}' | '[' | ']' | '<' | '>')
987}
988
989/// The same, plus the two [`STRUCTURAL`] characters that end a URL only *ahead
990/// of its query string*.
991///
992/// [`STRUCTURAL`] has nine characters and [`is_url_terminator`] recognises seven
993/// of them; `;` and `&` were missing. Because the URL branch runs ahead of the
994/// structural cut, a URL earlier in a word made [`split_url`] swallow
995/// everything to the next terminator — including the separator that would have
996/// cut the word — and [`redact_url`]'s terminal arm echoes what it is given. So
997/// `Server=https://vault.local/api;Password=hunter2` and
998/// `cb=https://a.com/x&access_token=ghu_…` went out whole: the fix that bounds
999/// a URL and the fix that cuts on `;` and `&` each worked alone and did not
1000/// compose.
1001///
1002/// **Adding them to [`is_url_terminator`] outright is the obvious fix and the
1003/// wrong one**, which is why there are two predicates rather than one extended
1004/// list. `&` is what separates the parameters *inside* a query string, and
1005/// [`redact_url`] replaces a query wholesale precisely because a token can be
1006/// in any parameter and this module does not guess which. Measured:
1007/// `https://a.com/cb?code=1&state=hunter2` goes from `?[redacted]` to
1008/// `?[redacted]&state=hunter2`. `;` is the same story — it is a legal query
1009/// separator too.
1010///
1011/// Ahead of the `?` there is no such conflict: a `;` or an `&` in an authority
1012/// or a path is not URL syntax this module needs to keep, and something else in
1013/// the word is much the likelier reading.
1014fn is_url_head_terminator(c: char) -> bool {
1015    is_url_terminator(c) || matches!(c, ';' | '&')
1016}
1017
1018/// Finds the first URL in a fragment and cuts it out of the text around it.
1019///
1020/// Returns the text before the URL, its scheme, the URL itself with the `://`
1021/// removed, and the text after it — or `None` when the fragment holds no URL.
1022///
1023/// **Both bounds are the point.** `split_once("://")` treated *everything*
1024/// before the separator as the scheme and everything after it as the URL, and
1025/// the terminal arm of [`redact_url`] echoes both verbatim — so a single URL
1026/// anywhere in a word put the whole of the rest of that word beyond every rule
1027/// below it. `documentation_url` is in essentially every GitHub REST error
1028/// body, so that was *any* such body logged alongside a credential:
1029///
1030/// ```text
1031/// {"documentation_url":"https://docs.github.com/rest","token":"ghu_…"}
1032/// ```
1033///
1034/// came out intact. The reverse order leaked for the mirror reason —
1035/// everything before the `://` became the "scheme" — and a nested object behind
1036/// a URL was missed even when the URL's own userinfo was caught, because the
1037/// miss and the catch happened in the same call. The only thing that saved the
1038/// shape at all was a `?` or `#` *inside* the URL, which made [`redact_url`]
1039/// replace the tail.
1040///
1041/// An empty scheme run means the `://` is not introducing a URL, and the
1042/// fragment is left to the rules below rather than handed over as a URL with no
1043/// scheme.
1044fn split_url(fragment: &str) -> Option<(&str, &str, &str, &str)> {
1045    let separator = fragment.find("://")?;
1046    let before = &fragment[..separator];
1047
1048    // Scheme characters are ASCII, so a byte count is a character boundary.
1049    let scheme_len = before
1050        .bytes()
1051        .rev()
1052        .take_while(|byte| is_scheme_byte(*byte))
1053        .count();
1054    if scheme_len == 0 {
1055        return None;
1056    }
1057    let (prefix, scheme) = before.split_at(before.len() - scheme_len);
1058
1059    let after = &fragment[separator + "://".len()..];
1060
1061    // Where the URL ends no matter what, and **the bound every other search
1062    // here is taken inside**. Searching the rest of the fragment first is a
1063    // correct answer computed quadratically: a body of *n* URLs would scan the
1064    // whole remaining text once per URL looking for a `?` that is not there,
1065    // which on the 860 KB regression case in
1066    // `a_large_message_does_not_overflow_the_stack` is 17 billion character
1067    // comparisons. That test is a guard against this module taking the process
1068    // down, and a redaction pass that never returns takes it down just as
1069    // effectively as an overflow does.
1070    let end = after.find(is_url_terminator).unwrap_or(after.len());
1071    let url = &after[..end];
1072
1073    // The head is the authority and path — everything ahead of the first `?` or
1074    // `#`. Only there do `;` and `&` end the URL; inside a query string they
1075    // are query syntax, and the query is `redact_url`'s to replace wholesale.
1076    let query = url.find(['?', '#']).unwrap_or(url.len());
1077    let end = url[..query].find(is_url_head_terminator).unwrap_or(end);
1078    let (url, remainder) = after.split_at(end);
1079
1080    Some((prefix, scheme, url, remainder))
1081}
1082
1083/// Redacts the three places a URL can carry a credential, keeping the rest.
1084///
1085/// `scheme` is everything before `://` and `rest` everything after it.
1086///
1087/// The **userinfo** is the one this module used to miss, and it is not an
1088/// exotic shape: `https://x-access-token:ghu_…@github.com/owner/repo.git` is
1089/// the canonical token-authenticated git remote, so it is what `e2` and `e3`
1090/// will have in hand when a clone or a download fails and they log the error
1091/// they were given. The old `://` branch returned before the token-prefix and
1092/// opaque-run rules could run, so that URL came out intact.
1093///
1094/// A **fragment** is stripped for the same reason a query string is: an OAuth
1095/// implicit-flow response puts the token after the `#`, and the fragment is
1096/// never load-bearing for diagnosing an HTTP call.
1097fn redact_url(scheme: &str, rest: &str) -> String {
1098    // The authority runs to the first `/`, `?` or `#`.
1099    let authority_end = rest.find(['/', '?', '#']).unwrap_or(rest.len());
1100    let (authority, tail) = rest.split_at(authority_end);
1101
1102    // `rsplit_once`, not `split_once`: a password may itself contain an `@`,
1103    // and the host is what follows the *last* one.
1104    let host = match authority.rsplit_once('@') {
1105        // Replaced rather than deleted. That the request carried credentials at
1106        // all is diagnostic — it is often the answer to "why did this 401?" —
1107        // and it is the credential, not its existence, that must not be here.
1108        Some((_userinfo, host)) => format!("{REDACTION}@{host}"),
1109        None => authority.to_string(),
1110    };
1111
1112    // Whichever of `?` and `#` comes first ends the diagnosable part; anything
1113    // after it is replaced wholesale rather than parsed, because a token can be
1114    // in any parameter and this module does not guess which.
1115    match tail.find(['?', '#']) {
1116        Some(cut) => {
1117            let (path, query) = tail.split_at(cut);
1118            let separator = &query[..1];
1119            format!(
1120                "{scheme}://{host}{}{separator}{REDACTION}",
1121                redact_path(path)
1122            )
1123        }
1124        None => format!("{scheme}://{host}{}", redact_path(tail)),
1125    }
1126}
1127
1128/// Applies the shape rules to a URL path, one segment at a time.
1129///
1130/// The path is the diagnosable part of a URL and stays that way: a segment is
1131/// judged on its own, and an ordinary one — `repos`, `owner`, `actions-runner-
1132/// linux-x64-2.330.0.tar.gz` — is not a secret and is not touched. What this
1133/// closes is that [`redact_url`] previously applied *no* rule to a path at all,
1134/// so `https://github.com/o/r/raw/ghu_…/f` was echoed whole while the identical
1135/// token one character to the left of the `/` would have been replaced. The
1136/// token-prefix rule is documented as a belt that catches a credential
1137/// *anywhere*, and a path was the one place it never ran.
1138///
1139/// Segment at a time rather than whole, because [`redact_value`] would
1140/// otherwise see the `/` and hand the whole path to [`looks_like_path`], which
1141/// is exactly the over-redaction the module documentation promises a full URL
1142/// escapes.
1143///
1144/// Worth knowing: [`OPAQUE_RUN_THRESHOLD`] is 40, and a git object name written
1145/// as hex is exactly 40 characters, so a `…/raw/<sha1>/f` URL loses its commit
1146/// to `[redacted]`. That is the module's standing trade — a less readable line
1147/// against a disclosed credential — and it is called out here because a commit
1148/// SHA is the one path segment somebody may miss.
1149fn redact_path(path: &str) -> String {
1150    let mut out = String::with_capacity(path.len());
1151    for (index, segment) in path.split('/').enumerate() {
1152        if index > 0 {
1153            out.push('/');
1154        }
1155        if !segment.is_empty() {
1156            out.push_str(&redact_value(segment));
1157        }
1158    }
1159    out
1160}
1161
1162/// The shape rules, applied to a bare value.
1163fn redact_value(value: &str) -> String {
1164    let lower = value.to_ascii_lowercase();
1165    if TOKEN_PREFIXES
1166        .iter()
1167        .any(|prefix| lower.starts_with(prefix))
1168    {
1169        return REDACTION.to_string();
1170    }
1171
1172    if looks_like_path(value) {
1173        return PATH_REDACTION.to_string();
1174    }
1175
1176    // A digest is not a secret, and `07-security.md` makes checksum
1177    // verification a security gate. The most useful thing `e2` can write when
1178    // that gate fails is expected-versus-actual, and until this carve-out
1179    // existed both sides came out as `[redacted]` — a gate that reports it
1180    // failed and refuses to say how.
1181    if let Some(digest) = as_sha256_digest(value) {
1182        return digest;
1183    }
1184
1185    if looks_like_jwt(value) {
1186        return REDACTION.to_string();
1187    }
1188
1189    if value.len() >= OPAQUE_RUN_THRESHOLD && value.chars().all(is_opaque_char) {
1190        return REDACTION.to_string();
1191    }
1192
1193    value.to_string()
1194}
1195
1196/// Fields whose values are this product's own closed vocabulary: `snake_case`
1197/// constants such as `late_ephemeral_workspace_could_not_be_removed`.
1198const CLOSED_VOCABULARY_FIELDS: &[&str] = &["kind", "reason"];
1199
1200/// Whether a closed-vocabulary field's value is emitted as it is.
1201///
1202/// The long `reason` constants are over [`OPAQUE_RUN_THRESHOLD`], so the
1203/// opaque-run rule redacted the very field that says what went wrong. Exempt
1204/// only in these fields, never in a message or any other field: there a long
1205/// run is still treated as a secret. Within them, a lowercase `snake_case`
1206/// identifier that does not start with a token prefix is not something a
1207/// credential or a key generator produces.
1208fn is_closed_vocabulary(name: &str, value: &str) -> bool {
1209    CLOSED_VOCABULARY_FIELDS.contains(&name)
1210        && !TOKEN_PREFIXES
1211            .iter()
1212            .any(|prefix| value.starts_with(prefix))
1213        && is_snake_case_identifier(value)
1214}
1215
1216/// Whether a value is lowercase words joined by single underscores.
1217fn is_snake_case_identifier(value: &str) -> bool {
1218    value.contains('_')
1219        && value
1220            .split('_')
1221            .all(|word| !word.is_empty() && word.chars().all(|c| c.is_ascii_lowercase()))
1222}
1223
1224/// The length of a SHA-256 digest written as lowercase hex.
1225const SHA256_HEX_LEN: usize = 64;
1226
1227/// How much of a digest is shown.
1228///
1229/// 12 is the short-digest convention git and the OCI tooling use: 48 bits, far
1230/// more than enough to tell an expected digest from the one that was actually
1231/// computed, which is the only comparison a checksum failure needs. Truncating
1232/// is also what makes this carve-out safe to have at all — a 64-character
1233/// lowercase hex run is *usually* a digest, but an HMAC-SHA256 signature has
1234/// the same shape, and 12 of its 64 characters are of no use to anybody.
1235const DIGEST_PREFIX_LEN: usize = 12;
1236
1237/// Renders a value that is exactly a lowercase SHA-256 digest as a labelled
1238/// prefix, or `None` when it is not one.
1239///
1240/// Deliberately strict: exactly 64 characters, and lowercase only. An uppercase
1241/// or mixed-case run falls through to the opaque-run rule and is redacted,
1242/// because the narrower this exception is, the less there is to reason about.
1243fn as_sha256_digest(value: &str) -> Option<String> {
1244    let is_digest = value.len() == SHA256_HEX_LEN
1245        && value
1246            .chars()
1247            .all(|c| c.is_ascii_digit() || c.is_ascii_lowercase() && c.is_ascii_hexdigit());
1248
1249    is_digest.then(|| format!("sha256:{}…", &value[..DIGEST_PREFIX_LEN]))
1250}
1251
1252/// The most `.`-separated segments a JSON Web Token has: header, payload,
1253/// signature.
1254const JWT_SEGMENTS: usize = 3;
1255
1256/// Whether a value is a JSON Web Token.
1257///
1258/// The opaque-run rule cannot see one. [`is_opaque_char`] excludes `.`, and a
1259/// JWT is base64url runs joined by two of them, so a 100-character credential
1260/// arrives as three runs that are each under the threshold and prints
1261/// verbatim. `Authorization: Bearer <jwt>` is caught by the scheme-word rule
1262/// and a credential-keyed one by the key rule, so this only ever bit a token
1263/// logged bare or under a name nobody listed — which is what a GitHub App
1264/// installation assertion is when it reaches a log at all.
1265///
1266/// Narrow deliberately, because the obvious fix is the wrong one: adding `.`
1267/// to [`is_opaque_char`] would swallow every long dotted word there is —
1268/// package names, dated filenames, dotted identifiers. A JWT header is
1269/// base64url of `{"alg":…`, which always begins `eyJ`, and that is a
1270/// discriminator ordinary text does not have.
1271fn looks_like_jwt(value: &str) -> bool {
1272    if value.len() < OPAQUE_RUN_THRESHOLD || !value.starts_with("eyJ") {
1273        return false;
1274    }
1275
1276    let mut segments = 0usize;
1277    for segment in value.split('.') {
1278        segments += 1;
1279        if !segment.chars().all(is_opaque_char) {
1280            return false;
1281        }
1282    }
1283
1284    // Two segments as well as three: an unsigned token has an empty signature,
1285    // and the trailing `.` that would have made the third is trimmed as
1286    // wrapping punctuation before this is reached.
1287    (2..=JWT_SEGMENTS).contains(&segments)
1288}
1289
1290fn looks_like_path(value: &str) -> bool {
1291    let bytes = value.as_bytes();
1292
1293    // `C:\Users\…` and `C:/Users/…`.
1294    if bytes.len() >= 3
1295        && bytes[0].is_ascii_alphabetic()
1296        && bytes[1] == b':'
1297        && (bytes[2] == b'\\' || bytes[2] == b'/')
1298    {
1299        return true;
1300    }
1301
1302    // `\\server\share\…`.
1303    if value.starts_with("\\\\") {
1304        return true;
1305    }
1306
1307    // `~/…`.
1308    if value.starts_with("~/") || value.starts_with("~\\") {
1309        return true;
1310    }
1311
1312    // `/var/lib/…`. Two segments required, so a lone `/` and a bare `/tmp`
1313    // stay readable; see the module documentation for why a URL path is caught
1314    // by this too.
1315    if let Some(rest) = value.strip_prefix('/') {
1316        return rest.contains('/') && !rest.is_empty();
1317    }
1318
1319    false
1320}
1321
1322// ---------------------------------------------------------------------------
1323// The tracing layer
1324// ---------------------------------------------------------------------------
1325
1326/// The redacted fields recorded on a span, stashed in the span's extensions so
1327/// that an event inside the span can carry them.
1328#[derive(Debug, Clone)]
1329struct SpanFields(Map<String, Value>);
1330
1331/// Collects a `tracing` record into JSON, applying the allowlist and the
1332/// scrubber as it goes.
1333#[derive(Debug, Default)]
1334struct RedactingVisitor {
1335    fields: Map<String, Value>,
1336}
1337
1338impl RedactingVisitor {
1339    fn put_str(&mut self, name: &str, value: &str) {
1340        let rendered = if !is_field_allowed(name) {
1341            REDACTION.to_string()
1342        } else if is_closed_vocabulary(name, value) {
1343            value.to_string()
1344        } else {
1345            redact(value)
1346        };
1347        self.fields
1348            .insert(name.to_string(), Value::String(rendered));
1349    }
1350
1351    fn put_value(&mut self, name: &str, value: Value) {
1352        if is_field_allowed(name) {
1353            self.fields.insert(name.to_string(), value);
1354        } else {
1355            // Numbers and booleans go the same way as strings. A credential is
1356            // never an `i64`, but the rule that makes this sink trustworthy is
1357            // that it has *no* exception for a name nobody listed.
1358            self.fields
1359                .insert(name.to_string(), Value::String(REDACTION.to_string()));
1360        }
1361    }
1362}
1363
1364impl Visit for RedactingVisitor {
1365    fn record_str(&mut self, field: &Field, value: &str) {
1366        self.put_str(field.name(), value);
1367    }
1368
1369    fn record_debug(&mut self, field: &Field, value: &dyn fmt::Debug) {
1370        // The event message arrives here, as `format_args!` output.
1371        self.put_str(field.name(), &format!("{value:?}"));
1372    }
1373
1374    fn record_error(&mut self, field: &Field, value: &(dyn std::error::Error + 'static)) {
1375        self.put_str(field.name(), &value.to_string());
1376    }
1377
1378    fn record_bool(&mut self, field: &Field, value: bool) {
1379        self.put_value(field.name(), Value::Bool(value));
1380    }
1381
1382    fn record_i64(&mut self, field: &Field, value: i64) {
1383        self.put_value(field.name(), Value::from(value));
1384    }
1385
1386    fn record_u64(&mut self, field: &Field, value: u64) {
1387        self.put_value(field.name(), Value::from(value));
1388    }
1389
1390    fn record_f64(&mut self, field: &Field, value: f64) {
1391        self.put_value(field.name(), Value::from(value));
1392    }
1393
1394    fn record_i128(&mut self, field: &Field, value: i128) {
1395        self.put_str(field.name(), &value.to_string());
1396    }
1397
1398    fn record_u128(&mut self, field: &Field, value: u128) {
1399        self.put_str(field.name(), &value.to_string());
1400    }
1401}
1402
1403/// A `tracing` layer that writes one redacted JSON object per event.
1404///
1405/// Written here rather than assembled from `tracing_subscriber::fmt` because
1406/// redaction has to happen *before* formatting, and a `tracing::Event`'s fields
1407/// cannot be rewritten for a downstream formatter to pick up. Owning the
1408/// formatting is what makes "unconditional" true: there is no path from an
1409/// event to the output that does not go through
1410/// [`RedactingVisitor`].
1411#[derive(Debug, Clone)]
1412pub struct RedactingLayer<W> {
1413    writer: W,
1414}
1415
1416impl<W> RedactingLayer<W> {
1417    /// Wraps a writer factory — a file appender, a capture buffer, or
1418    /// `std::io::stderr`.
1419    pub const fn new(writer: W) -> Self {
1420        Self { writer }
1421    }
1422}
1423
1424impl<S, W> Layer<S> for RedactingLayer<W>
1425where
1426    S: Subscriber + for<'a> LookupSpan<'a>,
1427    W: for<'a> tracing_subscriber::fmt::MakeWriter<'a> + 'static,
1428{
1429    fn on_new_span(
1430        &self,
1431        attrs: &tracing::span::Attributes<'_>,
1432        id: &tracing::span::Id,
1433        ctx: Context<'_, S>,
1434    ) {
1435        let mut visitor = RedactingVisitor::default();
1436        attrs.record(&mut visitor);
1437        if let Some(span) = ctx.span(id) {
1438            span.extensions_mut().insert(SpanFields(visitor.fields));
1439        }
1440    }
1441
1442    fn on_record(
1443        &self,
1444        id: &tracing::span::Id,
1445        values: &tracing::span::Record<'_>,
1446        ctx: Context<'_, S>,
1447    ) {
1448        let mut visitor = RedactingVisitor::default();
1449        values.record(&mut visitor);
1450        if let Some(span) = ctx.span(id) {
1451            let mut extensions = span.extensions_mut();
1452            if let Some(existing) = extensions.get_mut::<SpanFields>() {
1453                existing.0.extend(visitor.fields);
1454            } else {
1455                extensions.insert(SpanFields(visitor.fields));
1456            }
1457        }
1458    }
1459
1460    fn on_event(&self, event: &Event<'_>, ctx: Context<'_, S>) {
1461        let mut visitor = RedactingVisitor::default();
1462        event.record(&mut visitor);
1463
1464        let mut record = Map::new();
1465        record.insert(
1466            "timestamp".to_string(),
1467            Value::String(chrono::Utc::now().to_rfc3339()),
1468        );
1469        record.insert(
1470            "level".to_string(),
1471            Value::String(event.metadata().level().to_string()),
1472        );
1473        // Named `logger` and not `target`: `target` is also a domain field —
1474        // the repository or organization a policy points at — and two different
1475        // things under one key is how a diagnostic becomes a puzzle.
1476        record.insert(
1477            "logger".to_string(),
1478            Value::String(event.metadata().target().to_string()),
1479        );
1480        record.insert("fields".to_string(), Value::Object(visitor.fields));
1481
1482        let spans: Vec<Value> = ctx
1483            .event_scope(event)
1484            .into_iter()
1485            .flat_map(tracing_subscriber::registry::Scope::from_root)
1486            .map(|span| {
1487                let mut entry = Map::new();
1488                entry.insert("name".to_string(), Value::String(span.name().to_string()));
1489                if let Some(fields) = span.extensions().get::<SpanFields>() {
1490                    entry.insert("fields".to_string(), Value::Object(fields.0.clone()));
1491                }
1492                Value::Object(entry)
1493            })
1494            .collect();
1495        if !spans.is_empty() {
1496            record.insert("spans".to_string(), Value::Array(spans));
1497        }
1498
1499        let mut line = serde_json::to_string(&Value::Object(record)).unwrap_or_else(|_| {
1500            String::from(
1501                r#"{"level":"ERROR","fields":{"message":"a log record could not be encoded"}}"#,
1502            )
1503        });
1504        line.push('\n');
1505
1506        // A failed write to the diagnostics file must not take the agent down,
1507        // and must not be reported through `tracing` either — that would
1508        // recurse straight back into this method.
1509        let mut writer = self.writer.make_writer();
1510        let _ = writer.write_all(line.as_bytes());
1511        let _ = writer.flush();
1512    }
1513}
1514
1515// ---------------------------------------------------------------------------
1516// Installing the sink
1517// ---------------------------------------------------------------------------
1518
1519/// Whose diagnostics these are, and therefore which file they go in.
1520///
1521/// # Why the daemon does not share the operator's file
1522///
1523/// On the two Unixes a boot-mode registration runs the daemon as `root` while
1524/// the four application-data directories stay in the operator's profile —
1525/// `05-infrastructure.md` puts them there and `service install` records those
1526/// paths into the plist. So two accounts write into one `logs/` directory, and
1527/// the appender creates its file with the umask default, `0644`. Whichever
1528/// account opened today's file first owns it, and if that was `root` the
1529/// operator's own `runner-manager status` can no longer append to it.
1530///
1531/// That was not a degraded log. `tracing_appender::rolling::daily` **panics**
1532/// when it cannot open the file, so every CLI command on such a host died with
1533/// a backtrace before it did anything — reported on 0.1.17, on a host whose
1534/// daemon had rolled the file over at midnight as `root`:
1535///
1536/// ```text
1537/// thread 'main' panicked at rolling.rs:156:14:
1538/// initializing rolling file appender failed: InitError { context: "failed to
1539/// create log file", source: Os { code: 13, kind: PermissionDenied } }
1540/// ```
1541///
1542/// [`install`] no longer panics — see there — but not panicking would only have
1543/// turned a crash into an operator who never gets diagnostics again. The two
1544/// writers are separated instead, so neither can take the other's file: the
1545/// account that runs the daemon owns [`SERVICE_LOG_STEM`] and the operator owns
1546/// [`OPERATOR_LOG_STEM`], for as long as the registration lives.
1547#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
1548pub enum LogRole {
1549    /// A command the operator ran, or a daemon they started in the foreground.
1550    Operator,
1551    /// The daemon a service manager started, which on a boot-mode host is a
1552    /// different account from the operator's.
1553    Service,
1554}
1555
1556/// The file stem [`LogRole::Operator`] writes.
1557pub const OPERATOR_LOG_STEM: &str = "runner-manager.log";
1558/// The file stem [`LogRole::Service`] writes, and what `service status`
1559/// reports as the daemon's log.
1560pub const SERVICE_LOG_STEM: &str = "runner-manager.service.log";
1561
1562impl LogRole {
1563    /// The file stem this role writes, before the appender's date suffix.
1564    #[must_use]
1565    pub const fn file_stem(self) -> &'static str {
1566        match self {
1567            Self::Operator => OPERATOR_LOG_STEM,
1568            Self::Service => SERVICE_LOG_STEM,
1569        }
1570    }
1571}
1572
1573impl fmt::Display for LogRole {
1574    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
1575        f.write_str(match self {
1576            Self::Operator => "operator",
1577            Self::Service => "service",
1578        })
1579    }
1580}
1581
1582/// The sink could not be installed.
1583#[derive(Debug, thiserror::Error)]
1584pub enum LoggingError {
1585    /// The application-data directories could not be created.
1586    ///
1587    /// Carries a [`crate::paths::PathsError`] rather than a bare
1588    /// [`std::io::Error`] because [`install`] creates `logs/` through
1589    /// [`crate::paths::AppPaths::create_all`], which is what applies the `0700`
1590    /// restriction; the source therefore names whichever of the four
1591    /// directories actually failed, and that need not be `logs/`.
1592    #[error("cannot create the log directory {}: {source}", directory.display())]
1593    Directory {
1594        /// The diagnostics directory [`install`] was asked to write into.
1595        directory: PathBuf,
1596        /// The underlying error.
1597        #[source]
1598        source: crate::paths::PathsError,
1599    },
1600
1601    /// No diagnostics file in `logs/` could be opened for appending.
1602    ///
1603    /// Both stems are named because [`install`] tries two — the role's own, and
1604    /// then one qualified by the account — and an operator who is told only
1605    /// about the second would go looking for a file this process never reached
1606    /// for first.
1607    ///
1608    /// The two attempts are composed into one string rather than carried as
1609    /// four fields. As four this variant reached 128 bytes on Windows — where
1610    /// `PathBuf` is wider than on the Unixes — and tripped
1611    /// `clippy::result_large_err`, which is a lint about every *success* path
1612    /// paying for the size of a failure. The message is the same either way,
1613    /// and only this error is ever built.
1614    #[error("cannot open a diagnostics file in {}: {attempts}", directory.display())]
1615    Appender {
1616        /// The diagnostics directory.
1617        directory: PathBuf,
1618        /// Both stems that were tried, and why each was refused.
1619        attempts: String,
1620    },
1621
1622    /// A global subscriber was already installed.
1623    #[error("a tracing subscriber is already installed for this process: {message}")]
1624    AlreadyInstalled {
1625        /// What `tracing_subscriber` said.
1626        message: String,
1627    },
1628}
1629
1630/// Keeps the background log-writing thread alive.
1631///
1632/// Dropping it flushes and stops that thread, so the value must be held for as
1633/// long as the program expects its diagnostics to be written. Losing it is a
1634/// silent, total loss of logs, which is why it is `#[must_use]`.
1635#[must_use = "dropping the guard stops the log writer and silently discards later diagnostics"]
1636#[derive(Debug)]
1637pub struct LoggingGuard {
1638    _worker: tracing_appender::non_blocking::WorkerGuard,
1639}
1640
1641/// Installs the redacting sink as this process's global subscriber, writing
1642/// daily-rotating files into `logs/`.
1643///
1644/// `role` decides the file, for the reason [`LogRole`] gives. `default_filter`
1645/// applies when `RUST_LOG` is unset or unparseable.
1646///
1647/// # Errors
1648///
1649/// [`LoggingError::Directory`], [`LoggingError::Appender`] and
1650/// [`LoggingError::AlreadyInstalled`]. **None of them is fatal to the caller**,
1651/// and the CLI treats all three as a warning: a `host show` that refused to
1652/// print a capacity because a log file could not be opened would be a worse
1653/// failure than the one it was reporting.
1654pub fn install(
1655    paths: &crate::paths::AppPaths,
1656    role: LogRole,
1657    default_filter: &str,
1658) -> Result<LoggingGuard, LoggingError> {
1659    use tracing_subscriber::layer::SubscriberExt as _;
1660    use tracing_subscriber::util::SubscriberInitExt as _;
1661
1662    // `AppPaths::create_all`, not `create_dir_all`. The claim `create_all`
1663    // documents — that a diagnostics file is not readable by other local
1664    // accounts — is only true if `logs/` is created at `0700`, and this is the
1665    // path a running daemon actually takes. Creating the directory here with a
1666    // bare `create_dir_all` left it at the umask default whenever `install`
1667    // won the race to create it, and `tracing_appender` then wrote 0644 files
1668    // into it: the invariant held in the test that asserts it and nowhere else.
1669    let directory = paths.logs_dir().to_path_buf();
1670    paths
1671        .create_all()
1672        .map_err(|source| LoggingError::Directory {
1673            directory: directory.clone(),
1674            source,
1675        })?;
1676
1677    let appender = open_appender(&directory, role)?;
1678    let (writer, worker) = tracing_appender::non_blocking(appender);
1679
1680    let filter = tracing_subscriber::EnvFilter::try_from_default_env()
1681        .unwrap_or_else(|_| tracing_subscriber::EnvFilter::new(default_filter));
1682
1683    tracing_subscriber::registry()
1684        .with(filter)
1685        .with(RedactingLayer::new(writer))
1686        .try_init()
1687        .map_err(|error| LoggingError::AlreadyInstalled {
1688            message: error.to_string(),
1689        })?;
1690
1691    Ok(LoggingGuard { _worker: worker })
1692}
1693
1694/// Opens today's diagnostics file, falling back to one only this account can
1695/// have created.
1696///
1697/// # Why there is a second attempt at all
1698///
1699/// [`LogRole`] keeps the daemon and the operator apart, which is enough on a
1700/// host where nothing else ever writes here. Two things still put a file the
1701/// caller cannot append to under the stem it wants:
1702///
1703/// 1. **A privileged command.** Signing in to the machine-scoped store is
1704///    `sudo runner-manager auth login`, and that run writes the *operator's*
1705///    stem as `root`. Every unprivileged command for the rest of that day then
1706///    finds a `root`-owned file under it.
1707/// 2. **A host upgraded into this change.** Files written before the roles were
1708///    split are all under [`OPERATOR_LOG_STEM`], and on a boot-mode host the
1709///    ones from the last few days belong to `root`.
1710///
1711/// Neither is worth losing diagnostics over, and neither can be repaired from
1712/// inside an unprivileged process — it may not chown the file, may not change
1713/// its mode, and must not delete an existing log. So it writes beside it, under
1714/// a stem carrying this account's identity, which no other account will choose.
1715///
1716/// The fallback is deliberately *not* the first choice. A file named after
1717/// whoever happened to open it first is a file an operator has to go looking
1718/// for; the plain stem stays the plain stem, and the qualified one appears only
1719/// on a host that has the collision.
1720fn open_appender(
1721    directory: &Path,
1722    role: LogRole,
1723) -> Result<tracing_appender::rolling::RollingFileAppender, LoggingError> {
1724    let stem = role.file_stem();
1725    let first = match build_appender(directory, stem) {
1726        Ok(appender) => return Ok(appender),
1727        Err(error) => error,
1728    };
1729
1730    let qualified = account_qualified_stem(stem);
1731    match build_appender(directory, &qualified) {
1732        Ok(appender) => Ok(appender),
1733        // Both stems are named. An operator told only about the second would go
1734        // looking for a file this process never reached for first.
1735        Err(second) => Err(LoggingError::Appender {
1736            directory: directory.to_path_buf(),
1737            attempts: format!(
1738                "neither {stem}.<date> ({first}) nor {qualified}.<date> ({second}) could be \
1739                 appended to"
1740            ),
1741        }),
1742    }
1743}
1744
1745/// One attempt, through the constructor that **returns** its failure.
1746///
1747/// `tracing_appender::rolling::daily` is the same thing with an `.expect` on
1748/// the end, and that `.expect` is a panic in the middle of an operator's `status`
1749/// command. The whole reason this function exists is that the builder hands the
1750/// error back instead.
1751fn build_appender(
1752    directory: &Path,
1753    stem: &str,
1754) -> Result<tracing_appender::rolling::RollingFileAppender, String> {
1755    tracing_appender::rolling::RollingFileAppender::builder()
1756        .rotation(tracing_appender::rolling::Rotation::DAILY)
1757        .filename_prefix(stem)
1758        .max_log_files(30)
1759        .build(directory)
1760        .map_err(|error| error.to_string())
1761}
1762
1763/// `runner-manager.log` becomes `runner-manager.<account>.log`.
1764///
1765/// The stem carries its own `.log`, so the account goes before it rather than
1766/// after: the appender appends `.<date>`, and `runner-manager.log.uid-501` would
1767/// read as a rotated file rather than as another account's.
1768fn account_qualified_stem(stem: &str) -> String {
1769    let account = account_tag();
1770    match stem.rsplit_once('.') {
1771        Some((head, tail)) => format!("{head}.{account}.{tail}"),
1772        None => format!("{stem}.{account}"),
1773    }
1774}
1775
1776/// Something stable, filename-safe, and different for every local account.
1777///
1778/// The numeric effective user id rather than a name: it needs no lookup, cannot
1779/// contain a path separator, and is the identity the filesystem actually
1780/// compared when it refused the open.
1781#[cfg(unix)]
1782fn account_tag() -> String {
1783    // SAFETY: `geteuid` takes no argument, touches no memory this process owns,
1784    // and is documented never to fail.
1785    let uid = unsafe { libc::geteuid() };
1786    format!("uid-{uid}")
1787}
1788
1789/// As the Unix half, from the one identity Windows exposes without a lookup.
1790///
1791/// Sanitized rather than trusted: this becomes a file name, and `USERNAME` is
1792/// an ordinary environment variable that a caller may set to anything at all,
1793/// including something holding a path separator.
1794#[cfg(windows)]
1795fn account_tag() -> String {
1796    let raw = std::env::var("USERNAME").unwrap_or_default();
1797    let safe: String = raw
1798        .chars()
1799        .filter(|c| c.is_ascii_alphanumeric() || *c == '-' || *c == '_')
1800        .take(32)
1801        .collect();
1802    if safe.is_empty() {
1803        "other-account".to_string()
1804    } else {
1805        format!("user-{safe}")
1806    }
1807}
1808
1809#[cfg(test)]
1810mod tests {
1811    use super::*;
1812
1813    use std::sync::{Arc, Mutex};
1814
1815    use tracing_subscriber::layer::SubscriberExt as _;
1816
1817    #[test]
1818    fn opening_the_sink_bounds_rotated_logs_without_pruning_attempt_summaries() {
1819        let root = tempfile::tempdir().unwrap();
1820        let today = chrono::Utc::now();
1821        for days in 1..=35 {
1822            let date = (today - chrono::Duration::days(days)).format("%Y-%m-%d");
1823            std::fs::write(
1824                root.path().join(format!("{SERVICE_LOG_STEM}.{date}")),
1825                b"old",
1826            )
1827            .unwrap();
1828        }
1829        let summary = root
1830            .path()
1831            .join(format!("{}.log", uuid::Uuid::from_u128(1)));
1832        std::fs::write(&summary, b"attempt summary").unwrap();
1833        let mut appender = build_appender(root.path(), SERVICE_LOG_STEM).unwrap();
1834        appender.write_all(b"current diagnostics\n").unwrap();
1835        appender.flush().unwrap();
1836        let rotated = std::fs::read_dir(root.path())
1837            .unwrap()
1838            .map(Result::unwrap)
1839            .filter(|entry| {
1840                entry
1841                    .file_name()
1842                    .to_string_lossy()
1843                    .starts_with(SERVICE_LOG_STEM)
1844            })
1845            .count();
1846        assert_eq!(rotated, 30);
1847        assert_eq!(std::fs::read(summary).unwrap(), b"attempt summary");
1848    }
1849
1850    // -----------------------------------------------------------------------
1851    // Capturing what the sink actually wrote
1852    // -----------------------------------------------------------------------
1853
1854    #[derive(Clone, Default)]
1855    struct Capture(Arc<Mutex<Vec<u8>>>);
1856
1857    impl Capture {
1858        fn text(&self) -> String {
1859            String::from_utf8_lossy(&self.0.lock().expect("not poisoned")).into_owned()
1860        }
1861    }
1862
1863    struct CaptureWriter(Arc<Mutex<Vec<u8>>>);
1864
1865    impl Write for CaptureWriter {
1866        fn write(&mut self, buf: &[u8]) -> std::io::Result<usize> {
1867            self.0.lock().expect("not poisoned").extend_from_slice(buf);
1868            Ok(buf.len())
1869        }
1870
1871        fn flush(&mut self) -> std::io::Result<()> {
1872            Ok(())
1873        }
1874    }
1875
1876    impl<'a> tracing_subscriber::fmt::MakeWriter<'a> for Capture {
1877        type Writer = CaptureWriter;
1878
1879        fn make_writer(&'a self) -> Self::Writer {
1880            CaptureWriter(Arc::clone(&self.0))
1881        }
1882    }
1883
1884    /// A sink with the same plumbing and no redaction at all.
1885    ///
1886    /// Exists so that `the_scan_catches_a_sink_that_does_not_redact` can prove
1887    /// the secret-injection scan below is capable of failing. Without it, a scan
1888    /// that found nothing would be indistinguishable from a scan that looked
1889    /// nowhere.
1890    #[derive(Debug, Clone)]
1891    struct PassthroughLayer<W>(W);
1892
1893    #[derive(Default)]
1894    struct PassthroughVisitor(Map<String, Value>);
1895
1896    impl Visit for PassthroughVisitor {
1897        fn record_str(&mut self, field: &Field, value: &str) {
1898            self.0
1899                .insert(field.name().to_string(), Value::String(value.to_string()));
1900        }
1901
1902        fn record_debug(&mut self, field: &Field, value: &dyn fmt::Debug) {
1903            self.0.insert(
1904                field.name().to_string(),
1905                Value::String(format!("{value:?}")),
1906            );
1907        }
1908    }
1909
1910    impl<S, W> Layer<S> for PassthroughLayer<W>
1911    where
1912        S: Subscriber + for<'a> LookupSpan<'a>,
1913        W: for<'a> tracing_subscriber::fmt::MakeWriter<'a> + 'static,
1914    {
1915        fn on_event(&self, event: &Event<'_>, _ctx: Context<'_, S>) {
1916            let mut visitor = PassthroughVisitor::default();
1917            event.record(&mut visitor);
1918            let mut line = serde_json::to_string(&Value::Object(visitor.0)).unwrap_or_default();
1919            line.push('\n');
1920            let _ = self.0.make_writer().write_all(line.as_bytes());
1921        }
1922    }
1923
1924    // -----------------------------------------------------------------------
1925    // The secret-injection log scan (`07-security.md`, security gate)
1926    // -----------------------------------------------------------------------
1927
1928    /// A user access token, in GitHub's user-to-server format.
1929    const USER_TOKEN: &str = "ghu_16C7e42F292c6912E7710c838347Ae178B4a";
1930    /// An installation-style token, to prove the prefix rule is not one-off.
1931    const SERVER_TOKEN: &str = "ghs_1CGGYnBAtn5ov3M0aTHhP7l3ZKuMhIB3pnPd";
1932    /// A fine-grained personal access token.
1933    const FINE_GRAINED: &str =
1934        "github_pat_11ABCDEFG0abcdefghijkl_ZYXWVUTSRQPONMLKJIHGFEDCBA9876543210zyxwv";
1935    /// Stands in for an encoded JIT configuration: base64, and long.
1936    const JIT_BLOB: &str = "eyJhZ2VudE5hbWUiOiJydW5uZXItbWFuYWdlciIsImVuY29kZWQiOiJhYmNkZWZnaGlqa2xtbm9wcXJzdHV2d3h5ejAxMjM0NTY3ODkrLz09In0=";
1937    /// A workspace path, which `07-security.md` also requires redacted.
1938    const WORKSPACE: &str = "/var/lib/runner-manager/runtime/9f2c/attempt-1";
1939    /// The same, on the platform the persona is least likely to be using but
1940    /// the CI matrix definitely is.
1941    const WINDOWS_WORKSPACE: &str = r"C:\Users\operator\AppData\Local\runner-manager\runtime\9f2c";
1942    /// A GitHub App installation assertion: a JSON Web Token.
1943    ///
1944    /// Its two `.` separators are the point. `is_opaque_char` excludes `.`, so
1945    /// the opaque-run rule sees three short runs rather than one long one, and
1946    /// every other secret here is caught by that rule when it stands alone.
1947    const JWT: &str = "eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.\
1948                       eyJpc3MiOiIxMjM0NTYiLCJpYXQiOjE3MDAwMDAwMDAsImV4cCI6MTcwMDAwMDYwMH0.\
1949                       c2lnbmF0dXJlLXRoYXQtaXMtb3BhcXVlLWFuZC1sb25nLWVub3VnaC10by1tYXR0ZXI";
1950
1951    /// An error whose `Debug` carries the HTTP body it was given.
1952    ///
1953    /// This is the shape `d2`'s secret store and `d3`'s installer will hand to
1954    /// `tracing::error!(reason = ?err)`, and it is not the same shape as any of
1955    /// the JSON above: `Debug` on a `String` escapes the quotes inside it, so
1956    /// the keys arrive spelled `\"password\"` rather than `"password"`.
1957    #[derive(Debug)]
1958    struct StoreError {
1959        body: String,
1960    }
1961
1962    fn secrets() -> Vec<&'static str> {
1963        vec![
1964            USER_TOKEN,
1965            SERVER_TOKEN,
1966            FINE_GRAINED,
1967            JIT_BLOB,
1968            JWT,
1969            WORKSPACE,
1970            WINDOWS_WORKSPACE,
1971        ]
1972    }
1973
1974    /// Routes every secret through `sink` in every shape a caller might use,
1975    /// and reports the first one that came out the other side.
1976    ///
1977    /// A helper returning `Result` rather than a test body, because the same
1978    /// injection has to be run against a deliberately non-redacting sink to
1979    /// show that it is capable of finding anything at all.
1980    fn scan_for_leaks<L>(layer: L, capture: &Capture) -> Result<String, String>
1981    where
1982        L: Layer<tracing_subscriber::Registry> + Send + Sync + 'static,
1983    {
1984        let subscriber = tracing_subscriber::registry().with(layer);
1985
1986        tracing::subscriber::with_default(subscriber, || {
1987            for secret in secrets() {
1988                // 1. In the message body, which is the one field that must be
1989                //    allowed and therefore cannot be protected by the allowlist.
1990                tracing::info!("starting runner with {secret}");
1991
1992                // 2. As a credential header in the message body, scheme and all.
1993                tracing::info!("request failed; Authorization: Bearer {secret}");
1994                tracing::warn!("retrying with x-api-key={secret}");
1995
1996                // 3. In a field nobody listed — the case that must be safe with
1997                //    no edit to this file.
1998                tracing::info!(runner_token = %secret, "registered");
1999
2000                // 4. In a field that *is* listed, which is where the allowlist
2001                //    alone would not save anything.
2002                tracing::info!(outcome = %secret, event = "started", "registered");
2003                tracing::info!(message = %secret, "ignored");
2004
2005                // 5. Through `Debug` rather than `Display`.
2006                tracing::info!(?secret, "debug shaped");
2007
2008                // 6. Embedded in a structured value rather than standing
2009                //    alone. Every shape above presents the secret as its
2010                //    own whitespace-delimited word, which is the one shape
2011                //    the opaque-run rule catches for free -- so the scan
2012                //    could pass while the sink leaked anything embedded.
2013                //    Compact JSON is what `serde_json::to_string` emits and
2014                //    what an HTTP error body arrives as, and it was emitted
2015                //    verbatim: `redact_core` recursed into the value for
2016                //    `=` only, and the JSON quote left the key spelled
2017                //    `encoded_jit_config"`, which is not on CREDENTIAL_KEYS.
2018                for key in ["encoded_jit_config", "runner_token", "pat"] {
2019                    tracing::error!("registration failed: {{\"{key}\":\"{secret}\"}}");
2020                    tracing::error!("registration failed: {{ \"{key}\": \"{secret}\" }}");
2021                    tracing::error!("registration failed: {key}:{secret}");
2022                }
2023
2024                // 7. An `Authorization` header value, in the same three
2025                //    embedded shapes.
2026                tracing::error!("response: {{\"authorization\":\"Bearer {secret}\"}}");
2027                tracing::error!("response: {{ \"authorization\": \"Bearer {secret}\" }}");
2028                tracing::error!("response: authorization:Bearer {secret}");
2029
2030                // 9. A compact object with **more than one field**, the
2031                //    credential key second. Shape 6 above differs from this by
2032                //    field order and by nothing else, and `serde_json::to_string`
2033                //    is what chooses the order -- so shape 6 could pass while an
2034                //    error body from a struct with two fields leaked. The rules
2035                //    split a word once, on the first separator they find, so
2036                //    only `runner_id` was ever examined.
2037                for key in ["encoded_jit_config", "runner_token", "access_token"] {
2038                    tracing::error!(
2039                        "registration failed: {{\"runner_id\":42,\"{key}\":\"{secret}\"}}"
2040                    );
2041                    tracing::error!(
2042                        "registration failed: {{\"status\":422,\"message\":\"bad\",\"{key}\":\"{secret}\"}}"
2043                    );
2044                    // The spaced spelling of the same thing, which was already
2045                    // safe -- by accident, because the value is its own word.
2046                    tracing::error!(
2047                        "registration failed: {{ \"runner_id\": 42, \"{key}\": \"{secret}\" }}"
2048                    );
2049                }
2050
2051                // 10. Nested, one level and two. The value recursion goes to
2052                //     `redact_value`, which is a leaf and never re-enters
2053                //     `redact_core`, so an object inside an object was emitted
2054                //     whole.
2055                tracing::error!("response: {{\"body\":{{\"runner_token\":\"{secret}\"}}}}");
2056                tracing::error!(
2057                    "response: {{\"error\":{{\"status\":422,\"body\":{{\"encoded_jit_config\":\"{secret}\"}}}}}}"
2058                );
2059
2060                // 11. A form-encoded body with the credential parameter second.
2061                //     `&` was not a separator this module knew, so the whole
2062                //     body was one word and only `scope` was ever judged.
2063                tracing::error!("token exchange failed: scope=repo&access_token={secret}");
2064                tracing::error!(
2065                    "token exchange failed: grant_type=refresh&refresh_token={secret}&scope=repo"
2066                );
2067
2068                // 12. Backslash-escaped JSON, through `Debug` rather than
2069                //     `Display`. Shape 5 is also `Debug`, but it puts the
2070                //     secret on an *unlisted* field, which is dropped wholesale
2071                //     -- so the `Debug` lens and the compact-JSON lens were both
2072                //     here and had never been crossed. `reason` is allowed, so
2073                //     this one reaches the scrubber, with its keys spelled
2074                //     `\"runner_token\"`.
2075                let failure = StoreError {
2076                    body: format!("{{\"runner_token\":\"{secret}\"}}"),
2077                };
2078                tracing::error!(reason = ?failure, "the secret store rejected the request");
2079
2080                // 13. A URL earlier in the same word. `redact_core` split on the
2081                //     first `://` and handed everything before it to
2082                //     `redact_url` as a "scheme" and everything after it as
2083                //     the URL -- and the terminal arm of `redact_url` echoes
2084                //     both verbatim. So one URL anywhere in a word put the
2085                //     whole of the rest of that word beyond every rule below.
2086                //     `documentation_url` is in essentially every GitHub REST
2087                //     error body, which makes that "any error body logged
2088                //     alongside a credential".
2089                tracing::error!(
2090                    "api failed: {{\"message\":\"Bad credentials\",\"documentation_url\":\"https://docs.github.com/rest\",\"token\":\"{secret}\"}}"
2091                );
2092                // The reverse order leaked for the mirror reason: everything
2093                // before the `://` became the scheme, and was echoed.
2094                tracing::error!(
2095                    "api failed: {{\"token\":\"{secret}\",\"documentation_url\":\"https://docs.github.com/rest\"}}"
2096                );
2097                // A nested object behind a URL, which is the shape a clone
2098                // failure arrives in.
2099                tracing::error!(
2100                    "clone failed: {{\"remote\":\"https://github.com/o/r.git\",\"body\":{{\"password\":\"{secret}\"}}}}"
2101                );
2102
2103                // 14. A value with no key of its own: an array element. The
2104                //     structural cut recursed on the *raw* fragment, and the
2105                //     terminal fallback then handed it to `redact_value` with
2106                //     its quote still attached -- so `"ghu_…` failed
2107                //     `starts_with("ghu_")`, failed `is_opaque_char` on the
2108                //     quote, failed the `eyJ` test, and failed
2109                //     `looks_like_path`. This is shape 9's defect one step
2110                //     down: "only the first pair is examined" became "a value
2111                //     with no key of its own is never examined", and it
2112                //     survived because there was no array among the twelve
2113                //     shapes above.
2114                tracing::error!("registration failed: {{\"tokens\":[\"{secret}\",\"x\"]}}");
2115                tracing::error!("registration failed: {{\"tokens\":[\"x\",\"{secret}\"]}}");
2116                tracing::error!("registration failed: [\"x\",\"{secret}\"]");
2117                tracing::error!("registration failed: [{{\"id\":1}},[\"{secret}\"]]");
2118                let listed = StoreError {
2119                    body: format!("{{\"tokens\":[\"x\",\"{secret}\"]}}"),
2120                };
2121                tracing::error!(reason = ?listed, "the secret store rejected the list");
2122
2123                // 15. `;` as the separator. It is in `WRAPPERS`, so it was
2124                //     recognised as punctuation, and it was not in
2125                //     `STRUCTURAL`, so it never cut a word. It is what
2126                //     separates the pairs of a Windows connection string, of a
2127                //     credential string, and of a cookie header written
2128                //     without a space -- `d2`'s territory exactly.
2129                tracing::error!("store rejected: Server=host;Database=x;Password={secret};");
2130                tracing::error!("store rejected: user=operator;password={secret}");
2131                tracing::error!("store rejected: theme=dark;session={secret}");
2132
2133                // 16. Wrapped in an element rather than in a quote.
2134                //     `split_wrappers` strips only the *outermost* `<` and
2135                //     `>`, so `<string>…</string>` arrived as
2136                //     `string>…</string`, which matches no rule at all. `d3`'s
2137                //     installers handle launchd plists, which is where this
2138                //     shape comes from.
2139                tracing::error!("plist rejected: <string>{secret}</string>");
2140                tracing::error!("plist rejected: <key>token</key><string>{secret}</string>");
2141
2142                // 17. A `;` or an `&` *after a URL in the same word*. This is
2143                //     shape 15 and shape 11 with one thing changed: a URL
2144                //     earlier in the word. `STRUCTURAL` has nine characters and
2145                //     `is_url_terminator` recognised seven of them -- `;` and
2146                //     `&` were missing -- and the URL branch runs *ahead* of
2147                //     the structural cut. So `split_url` swallowed everything
2148                //     to the next terminator, including the separator that
2149                //     would have cut the word, and `redact_url`'s terminal arm
2150                //     echoed it verbatim.
2151                //
2152                //     The L3 fix (bounding the URL) and the L1 fix (cutting on
2153                //     `;` and `&`) each work alone and did not compose: this
2154                //     commit added `;` to `STRUCTURAL` *for connection strings*
2155                //     while leaving the path a URL opens straight through it.
2156                //     A connection string whose `Server=` is a URL is exactly
2157                //     `d2`'s shape, and a systemd `Environment=` line is
2158                //     `d3`'s.
2159                tracing::error!(
2160                    "store rejected: Server=https://vault.local/api;Password={secret};"
2161                );
2162                tracing::error!("keychain error: url=https://kc.local;secret={secret}");
2163                tracing::error!("unit rejected: Environment=API=https://a.com/v1;TOKEN={secret}");
2164                tracing::error!("token exchange failed: cb=https://a.com/x&access_token={secret}");
2165                tracing::error!(
2166                    "dsn rejected: dsn=https://sentry.local/1;password={secret};user=x"
2167                );
2168                tracing::error!(
2169                    "callback failed: redirect=https://a.com/cb&state=1&access_token={secret}"
2170                );
2171                let routed = StoreError {
2172                    body: format!("Server=https://vault.local/api;Password={secret};"),
2173                };
2174                tracing::error!(reason = ?routed, "the secret store rejected the connection string");
2175
2176                // 18. A credential key whose value is not in its own fragment.
2177                //     The empty-value skip is right, but its justification --
2178                //     "the value is in the next fragment, where the structural
2179                //     cut reaches it" -- was false: `redact_fragment` carried
2180                //     nothing across the cut, so nothing ever reached it. An
2181                //     array, a nested object, an empty pair and a plist
2182                //     key/value pair are four spellings of the same gap.
2183                tracing::error!("store rejected: {{\"password\":[\"{secret}\"]}}");
2184                tracing::error!("store rejected: {{\"password\":{{\"v\":\"{secret}\"}}}}");
2185                tracing::error!("store rejected: Password=;{secret}");
2186                tracing::error!("plist rejected: <key>password</key><string>{secret}</string>");
2187
2188                // 19. A structural character *inside* a credential value. `,`
2189                //     and `&` could already split a secret; this commit added
2190                //     `;`, `<` and `>`, so each of those became a new character
2191                //     that can cut a punctuated password in half and leave the
2192                //     tail standing.
2193                tracing::error!("store rejected: {{\"password\":\"a<{secret}>b\"}}");
2194                tracing::error!("store rejected: {{\"password\":\"a&{secret},b\"}}");
2195
2196                // 20. A secret in a URL *path*. `redact_url` applies no shape
2197                //     rule to the path at all, so a token that reaches one is
2198                //     echoed whole -- and the token-prefix rule is documented
2199                //     as a belt that catches a credential anywhere.
2200                //
2201                //     The two workspace paths are excluded, and the exclusion
2202                //     is a real limitation rather than a convenience. A path is
2203                //     judged one segment at a time, because judging the whole
2204                //     of a URL path with `looks_like_path` would redact every
2205                //     URL with two segments in it -- which is the
2206                //     over-redaction the module documentation explicitly
2207                //     promises a full URL escapes. A filesystem path pasted
2208                //     into a URL path is therefore indistinguishable from an
2209                //     ordinary deep URL path: `/var/lib/…` and `/repos/o/r/…`
2210                //     are the same shape, segment by segment. A *credential* in
2211                //     a path is caught, because a credential has a shape of its
2212                //     own; a path in a path is not.
2213                if !looks_like_path(secret) {
2214                    tracing::error!("download failed: https://github.com/o/r/raw/{secret}/f");
2215                    tracing::error!("download failed: https://github.com/o/r/raw/{secret}");
2216                }
2217
2218                // 8. Carried on a span rather than on the event.
2219                let span = tracing::info_span!("attempt", jit_config = %secret);
2220                let _entered = span.enter();
2221                tracing::info!(event = "inside_span", "in a span");
2222            }
2223        });
2224
2225        let output = capture.text();
2226        for secret in secrets() {
2227            for needle in needles(secret) {
2228                if output.contains(&needle) {
2229                    return Err(format!(
2230                        "the sink emitted a secret verbatim: {secret}\n\
2231                         (found as: {needle})\n--- output ---\n{output}"
2232                    ));
2233                }
2234            }
2235        }
2236        Ok(output)
2237    }
2238
2239    /// Every spelling a secret can have in the sink's output.
2240    ///
2241    /// The sink writes JSON, so a secret containing a character `serde_json`
2242    /// escapes never appears in the output as it was written. `WINDOWS_WORKSPACE`
2243    /// is a raw string with single backslashes and is emitted with doubled ones,
2244    /// so scanning for the literal could not have matched it — not even against
2245    /// `PassthroughLayer`, which redacts nothing at all. The path really is
2246    /// redacted, so this was a hole in the coverage rather than a leak, but a
2247    /// needle that cannot match is a check that cannot fail.
2248    fn needles(secret: &str) -> Vec<String> {
2249        // Strip the quotes `to_string` adds; what is left is the text exactly
2250        // as it appears inside a JSON string literal.
2251        let escape = |text: &str| {
2252            let json = serde_json::to_string(text).expect("a string is serialisable");
2253            json[1..json.len() - 1].to_string()
2254        };
2255
2256        let once = escape(secret);
2257        // Twice, because a secret can be escaped twice on the way out, and
2258        // shape 12 is where that happens: `Debug` on a `String` escapes the
2259        // quotes *and* the backslashes inside it, and the sink then
2260        // JSON-encodes what `Debug` produced. A Windows path arrives in that
2261        // line with four backslashes where the secret has one, so the
2262        // once-escaped needle cannot match it there -- not even against
2263        // `PassthroughLayer`, which redacts nothing at all. That cell of the
2264        // matrix was a check incapable of failing, which is the same defect
2265        // this helper was written to fix one level down.
2266        let twice = escape(&once);
2267
2268        let mut spellings = vec![secret.to_string(), once, twice];
2269        // Consecutive-only is enough: each level of escaping is a superset of
2270        // the last, so equal spellings are always adjacent.
2271        spellings.dedup();
2272        spellings
2273    }
2274
2275    #[test]
2276    fn the_scan_looks_for_a_needle_that_can_actually_occur() {
2277        // Guards the helper above: if `needles` ever stops producing the
2278        // JSON-escaped spelling, the Windows workspace silently stops being
2279        // scanned for and every assertion about it becomes vacuous.
2280        let windows = needles(WINDOWS_WORKSPACE);
2281        assert_eq!(
2282            windows.len(),
2283            3,
2284            "a backslash path has three spellings, one per level of escaping it \
2285             can pass through on the way out: {windows:?}"
2286        );
2287        assert!(
2288            windows[1].contains(r"\\Users\\operator"),
2289            "the once-escaped spelling is what an ordinary JSON line contains: {windows:?}"
2290        );
2291        assert!(
2292            windows[2].contains(r"\\\\Users\\\\operator"),
2293            "the twice-escaped spelling is what a Debug rendering inside a JSON \
2294             line contains: {windows:?}"
2295        );
2296
2297        // The third spelling is not hypothetical: it is exactly what the sink
2298        // writes for shape 12, and without it that cell of the scan was a
2299        // check that could not fail.
2300        let shape_twelve = serde_json::to_string(&Value::String(format!(
2301            "{:?}",
2302            StoreError {
2303                body: format!("{{\"runner_token\":\"{WINDOWS_WORKSPACE}\"}}"),
2304            }
2305        )))
2306        .expect("serialisable");
2307        assert!(
2308            shape_twelve.contains(&windows[2]),
2309            "the twice-escaped needle must be findable in what shape 12 emits: {shape_twelve}"
2310        );
2311        assert!(
2312            !shape_twelve.contains(&windows[1]),
2313            "and the once-escaped one must not be, or the gap was never there: {shape_twelve}"
2314        );
2315
2316        // Confirms the premise: the raw spelling genuinely cannot occur in the
2317        // sink's output, which is why scanning only for it proved nothing.
2318        let rendered = serde_json::to_string(&Value::String(WINDOWS_WORKSPACE.to_string()))
2319            .expect("serialisable");
2320        assert!(
2321            !rendered.contains(WINDOWS_WORKSPACE),
2322            "if this ever contains the raw path, the original scan was fine after all: {rendered}"
2323        );
2324        assert!(rendered.contains(&windows[1]), "{rendered}");
2325
2326        // A secret with nothing to escape has exactly one spelling.
2327        assert_eq!(needles(USER_TOKEN), vec![USER_TOKEN.to_string()]);
2328    }
2329
2330    #[test]
2331    fn the_secret_injection_scan_finds_nothing() {
2332        let capture = Capture::default();
2333        let output = scan_for_leaks(RedactingLayer::new(capture.clone()), &capture)
2334            .unwrap_or_else(|complaint| panic!("{complaint}"));
2335
2336        // Something must have been written, or "no secrets found" is trivially
2337        // true and means nothing.
2338        assert!(!output.trim().is_empty(), "the sink wrote nothing at all");
2339        assert!(
2340            output.contains(REDACTION),
2341            "nothing was redacted, so nothing was routed through the sink:\n{output}"
2342        );
2343    }
2344
2345    #[test]
2346    fn the_scan_catches_a_sink_that_does_not_redact() {
2347        let capture = Capture::default();
2348        let complaint = scan_for_leaks(PassthroughLayer(capture.clone()), &capture)
2349            .expect_err("a sink with no redaction must be caught");
2350        assert!(
2351            complaint.contains("emitted a secret verbatim"),
2352            "the complaint must name the failure mode: {complaint}"
2353        );
2354    }
2355
2356    // -----------------------------------------------------------------------
2357    // The allowlist
2358    // -----------------------------------------------------------------------
2359
2360    fn emit(capture: &Capture, body: impl FnOnce()) -> String {
2361        let subscriber = tracing_subscriber::registry().with(RedactingLayer::new(capture.clone()));
2362        tracing::subscriber::with_default(subscriber, body);
2363        capture.text()
2364    }
2365
2366    #[test]
2367    fn a_field_nobody_listed_is_redacted_by_default() {
2368        let capture = Capture::default();
2369        let output = emit(&capture, || {
2370            tracing::info!(
2371                a_field_added_by_a_later_task = "supersecret-value",
2372                "an event"
2373            );
2374        });
2375
2376        assert!(
2377            !output.contains("supersecret-value"),
2378            "an unlisted field leaked its value:\n{output}"
2379        );
2380        assert!(output.contains(REDACTION), "{output}");
2381        // The name survives: knowing the field was there is useful, and a field
2382        // name is not a credential.
2383        assert!(
2384            output.contains("a_field_added_by_a_later_task"),
2385            "the field name should be kept:\n{output}"
2386        );
2387    }
2388
2389    #[test]
2390    fn an_unlisted_numeric_field_is_redacted_too() {
2391        // The rule has no exception for a type, because "a credential is never
2392        // an integer" is exactly the kind of reasoning that stops being true
2393        // once somebody logs an installation-scoped identifier they should not.
2394        let capture = Capture::default();
2395        let output = emit(&capture, || {
2396            tracing::info!(unlisted_number = 8_675_309_u64, "an event");
2397        });
2398        assert!(!output.contains("8675309"), "{output}");
2399        assert!(output.contains(REDACTION), "{output}");
2400    }
2401
2402    #[test]
2403    fn listed_fields_survive() {
2404        let capture = Capture::default();
2405        let output = emit(&capture, || {
2406            tracing::info!(
2407                event = "reconciled",
2408                policy_id = "9f2c1a44-0000-4000-8000-000000000001",
2409                count = 3,
2410                outcome = "started",
2411                "reconciliation finished"
2412            );
2413        });
2414
2415        for expected in [
2416            "reconciled",
2417            "9f2c1a44-0000-4000-8000-000000000001",
2418            "started",
2419            "reconciliation finished",
2420        ] {
2421            assert!(
2422                output.contains(expected),
2423                "{expected} missing from:\n{output}"
2424            );
2425        }
2426        assert!(output.contains("\"count\":3"), "{output}");
2427    }
2428
2429    /// The rate-limit warning's fields and a long closed-vocabulary reason
2430    /// reach the log, rather than `[redacted]` in place of the diagnosis.
2431    #[test]
2432    fn non_secret_diagnostic_fields_survive() {
2433        let capture = Capture::default();
2434        let output = emit(&capture, || {
2435            tracing::warn!(
2436                kind = "secondary",
2437                delay_secs = 61_u64,
2438                remaining = 0_u64,
2439                reason = "late_ephemeral_workspace_could_not_be_removed",
2440                "rate limited"
2441            );
2442        });
2443        for expected in [
2444            "\"kind\":\"secondary\"",
2445            "\"delay_secs\":61",
2446            "\"remaining\":0",
2447            "\"reason\":\"late_ephemeral_workspace_could_not_be_removed\"",
2448        ] {
2449            assert!(
2450                output.contains(expected),
2451                "{expected} missing from:
2452{output}"
2453            );
2454        }
2455    }
2456
2457    /// The carve-out is for identifiers in closed-vocabulary fields, not for
2458    /// anything long with an underscore in it, and not for message text.
2459    #[test]
2460    fn only_a_snake_case_reason_field_escapes_the_opaque_rule() {
2461        let reason = "late_ephemeral_workspace_could_not_be_removed";
2462        assert!(is_closed_vocabulary("reason", reason));
2463        assert!(!is_closed_vocabulary("message", reason));
2464        assert_eq!(redact(reason), REDACTION, "free text kept a long run");
2465        for opaque in [
2466            // A capital, a digit, a doubled or trailing underscore, and no
2467            // underscore at all: each is a shape a secret can take.
2468            "late_ephemeral_workspace_could_not_be_Removed",
2469            "late_ephemeral_workspace_could_not_be_removed2",
2470            "late_ephemeral__workspace_could_not_be_removed",
2471            "late_ephemeral_workspace_could_not_be_removed_",
2472            "lateephemeralworkspacecouldnotberemovedquite",
2473            "ghs_ephemeral_workspace_could_not_be_removed",
2474        ] {
2475            assert!(!is_closed_vocabulary("reason", opaque), "{opaque}");
2476        }
2477    }
2478
2479    #[test]
2480    fn the_allowlist_is_sorted_and_has_no_duplicates() {
2481        // `is_field_allowed` uses a binary search, so an unsorted list would not
2482        // merely be untidy — it would silently start redacting fields that are
2483        // on it.
2484        let mut sorted = ALLOWED_FIELDS.to_vec();
2485        sorted.sort_unstable();
2486        assert_eq!(
2487            ALLOWED_FIELDS,
2488            &sorted[..],
2489            "ALLOWED_FIELDS must stay sorted"
2490        );
2491
2492        let mut deduped = sorted.clone();
2493        deduped.dedup();
2494        assert_eq!(sorted.len(), deduped.len(), "ALLOWED_FIELDS has duplicates");
2495
2496        for name in ALLOWED_FIELDS {
2497            assert!(is_field_allowed(name), "{name} is listed but not allowed");
2498        }
2499        assert!(!is_field_allowed("authorization"));
2500        assert!(!is_field_allowed("runner_token"));
2501    }
2502
2503    #[test]
2504    fn every_record_is_one_parseable_json_object_per_line() {
2505        let capture = Capture::default();
2506        let output = emit(&capture, || {
2507            tracing::info!(event = "one", "first");
2508            tracing::warn!(event = "two", "second");
2509        });
2510
2511        let lines: Vec<&str> = output
2512            .lines()
2513            .filter(|line| !line.trim().is_empty())
2514            .collect();
2515        assert_eq!(lines.len(), 2, "{output}");
2516        for line in lines {
2517            let value: Value = serde_json::from_str(line)
2518                .unwrap_or_else(|error| panic!("not JSON: {error}\n{line}"));
2519            assert!(value.get("timestamp").is_some(), "{line}");
2520            assert!(value.get("level").is_some(), "{line}");
2521            assert!(value.get("logger").is_some(), "{line}");
2522            assert!(value.get("fields").is_some(), "{line}");
2523        }
2524    }
2525
2526    #[test]
2527    fn span_fields_are_redacted_and_carried() {
2528        let capture = Capture::default();
2529        let output = emit(&capture, || {
2530            let span = tracing::info_span!("attempt", attempt_id = "abc-123", jit = %JIT_BLOB);
2531            let _entered = span.enter();
2532            tracing::info!(event = "inside", "in the span");
2533        });
2534
2535        assert!(output.contains("\"name\":\"attempt\""), "{output}");
2536        assert!(
2537            output.contains("abc-123"),
2538            "the listed span field survives:\n{output}"
2539        );
2540        assert!(!output.contains(JIT_BLOB), "a span field leaked:\n{output}");
2541    }
2542
2543    /// Asserts that [`install`] created its directories the restrictive way.
2544    ///
2545    /// `install` used to call `std::fs::create_dir_all(logs_dir)` directly.
2546    /// That is invisible on Windows and *nearly* invisible on Unix — the
2547    /// directory exists either way, and `AppPaths::create_all`'s own test still
2548    /// passed, because it tests `create_all` rather than the path a running
2549    /// daemon actually takes. So this checks the two things that distinguish
2550    /// them:
2551    ///
2552    /// 1. **All four directories exist.** `create_dir_all(logs_dir)` makes
2553    ///    exactly one. This half runs on every platform, which matters because
2554    ///    Windows is the leg most likely to be run locally.
2555    /// 2. **On Unix the mode is `0700`.** This is the invariant `create_all`
2556    ///    documents — that a diagnostics file is not readable by other local
2557    ///    accounts — and `tracing_appender` writes 0644 files into whatever
2558    ///    directory it is given, so a `0755` `logs/` defeats it entirely.
2559    fn assert_install_created_restricted_directories(paths: &crate::paths::AppPaths) {
2560        for (purpose, path) in paths.all() {
2561            assert!(
2562                path.is_dir(),
2563                "install must create the {purpose} directory too: going through \
2564                 AppPaths::create_all is what applies the restriction, and creating only \
2565                 logs/ is the bug this asserts against ({})",
2566                path.display()
2567            );
2568
2569            #[cfg(unix)]
2570            {
2571                use std::os::unix::fs::PermissionsExt as _;
2572
2573                let mode = std::fs::metadata(path)
2574                    .expect("the directory exists")
2575                    .permissions()
2576                    .mode()
2577                    & 0o777;
2578                assert_eq!(
2579                    mode, 0o700,
2580                    "the {purpose} directory is mode {mode:04o}; a diagnostics file under a \
2581                     group- or world-readable directory is readable by other local accounts"
2582                );
2583            }
2584        }
2585    }
2586
2587    /// The one test that exercises [`install`] rather than assembling a
2588    /// subscriber by hand: the rolling file appender, the non-blocking writer,
2589    /// and the guard that has to be held for any of it to reach disk.
2590    ///
2591    /// It installs a *global* subscriber, which a process can only do once, so
2592    /// it is the only test here that may. Everything else uses
2593    /// `with_default`, which is thread-local and does not conflict — including
2594    /// on threads that run after this one.
2595    #[test]
2596    #[serial_test::serial(global_subscriber)]
2597    fn install_writes_redacted_json_into_the_logs_directory() {
2598        // `install` honours `RUST_LOG`, and a developer who has set it to
2599        // something restrictive would otherwise see this fail for a reason that
2600        // has nothing to do with the code. CI sets no such variable.
2601        if std::env::var_os("RUST_LOG").is_some() {
2602            return;
2603        }
2604
2605        let root = tempfile::tempdir().expect("a temporary directory");
2606        let paths = crate::paths::AppPaths::rooted_at(root.path());
2607
2608        let outcome = install(&paths, LogRole::Operator, "trace");
2609
2610        // Asserted before the early return below, because `install` creates the
2611        // directories before it touches the global subscriber: this holds
2612        // whether or not this test won the race to install one.
2613        assert_install_created_restricted_directories(&paths);
2614
2615        let Ok(guard) = outcome else {
2616            // Another test binary in the same process already installed one.
2617            // Not this test's failure, and not worth making the suite
2618            // order-dependent over.
2619            return;
2620        };
2621
2622        tracing::info!(event = "installed", runner_token = %USER_TOKEN, "hello from the sink");
2623
2624        // Dropping the guard flushes and stops the background writer, which is
2625        // the documented way to be sure the line reached disk.
2626        drop(guard);
2627
2628        let written: Vec<PathBuf> = std::fs::read_dir(paths.logs_dir())
2629            .expect("the log directory was created")
2630            .filter_map(Result::ok)
2631            .map(|entry| entry.path())
2632            .collect();
2633        assert_eq!(
2634            written.len(),
2635            1,
2636            "expected one rotating log file: {written:?}"
2637        );
2638
2639        let contents = std::fs::read_to_string(&written[0]).expect("readable");
2640        assert!(contents.contains("hello from the sink"), "{contents}");
2641        assert!(
2642            !contents.contains(USER_TOKEN),
2643            "the file sink must redact exactly like the in-memory one:\n{contents}"
2644        );
2645        assert!(contents.contains(REDACTION), "{contents}");
2646        serde_json::from_str::<Value>(contents.lines().next().expect("a line"))
2647            .expect("each line is a JSON object");
2648    }
2649
2650    // -----------------------------------------------------------------------
2651    // Two accounts, one logs/ directory
2652    // -----------------------------------------------------------------------
2653
2654    /// The daemon and the operator do not reach for the same file.
2655    ///
2656    /// Cheap, and it is the whole of the separation: everything else about
2657    /// [`LogRole`] follows from these two names being different.
2658    #[test]
2659    fn the_daemon_and_the_operator_write_different_files() {
2660        assert_ne!(
2661            LogRole::Service.file_stem(),
2662            LogRole::Operator.file_stem(),
2663            "a boot-mode daemon runs as another account and creates its file 0644; sharing a \
2664             stem hands whichever of them opened it first the day's file and locks the other out"
2665        );
2666        assert_eq!(LogRole::Operator.file_stem(), OPERATOR_LOG_STEM);
2667        assert_eq!(LogRole::Service.file_stem(), SERVICE_LOG_STEM);
2668    }
2669
2670    /// A file this account may not append to is written *beside*, not panicked
2671    /// on.
2672    ///
2673    /// This is the reported 0.1.17 crash, reproduced at the mode that caused
2674    /// it. On the host it was ownership — `root` had rolled the file over at
2675    /// midnight — and an unprivileged test cannot make a `root`-owned file, so
2676    /// it makes an unwritable one instead: the appender's `OpenOptions` refuse
2677    /// both with the same `EACCES`, and refusing was what
2678    /// `tracing_appender::rolling::daily` turned into a panic.
2679    #[cfg(unix)]
2680    #[test]
2681    fn a_log_file_this_account_cannot_append_to_is_written_beside() {
2682        use std::os::unix::fs::PermissionsExt as _;
2683
2684        // `root` is not subject to the mode bits, so it would sail through the
2685        // very open this test needs to fail.
2686        if account_tag() == "uid-0" {
2687            return;
2688        }
2689
2690        let root = tempfile::tempdir().expect("a temporary directory");
2691        let paths = crate::paths::AppPaths::rooted_at(root.path());
2692        paths
2693            .create_all()
2694            .expect("the four directories are created");
2695
2696        // Every date the appender might choose, so the test cannot flake on a
2697        // run that straddles midnight UTC.
2698        let logs = paths.logs_dir().to_path_buf();
2699        let today = chrono::Utc::now();
2700        for date in [
2701            (today - chrono::Duration::days(1))
2702                .format("%Y-%m-%d")
2703                .to_string(),
2704            today.format("%Y-%m-%d").to_string(),
2705            (today + chrono::Duration::days(1))
2706                .format("%Y-%m-%d")
2707                .to_string(),
2708        ] {
2709            let taken = logs.join(format!("{OPERATOR_LOG_STEM}.{date}"));
2710            std::fs::write(&taken, b"another account's diagnostics").expect("writable");
2711            std::fs::set_permissions(&taken, std::fs::Permissions::from_mode(0o444))
2712                .expect("the mode is applied");
2713        }
2714
2715        let mut appender = open_appender(&logs, LogRole::Operator)
2716            .expect("a diagnostics file is opened beside the one this account may not append to");
2717        appender
2718            .write_all(b"a line\n")
2719            .expect("the line is written");
2720        appender.flush().expect("the line is flushed");
2721
2722        let qualified = account_qualified_stem(OPERATOR_LOG_STEM);
2723        let written: Vec<String> = std::fs::read_dir(&logs)
2724            .expect("the directory is readable")
2725            .flatten()
2726            .map(|entry| entry.file_name().to_string_lossy().into_owned())
2727            .collect();
2728        assert!(
2729            written.iter().any(|name| name.starts_with(&qualified)),
2730            "expected a file under {qualified:?} beside the unwritable ones, and found {written:?}"
2731        );
2732        for name in &written {
2733            if name.starts_with(&qualified) {
2734                continue;
2735            }
2736            assert_eq!(
2737                std::fs::read(logs.join(name)).expect("readable"),
2738                b"another account's diagnostics",
2739                "{name} belongs to another account and must not have been touched"
2740            );
2741        }
2742    }
2743
2744    // -----------------------------------------------------------------------
2745    // The scrubber, in isolation
2746    // -----------------------------------------------------------------------
2747
2748    #[test]
2749    fn tokens_are_redacted_wherever_they_appear() {
2750        for token in [USER_TOKEN, SERVER_TOKEN, FINE_GRAINED] {
2751            for shape in [
2752                token.to_string(),
2753                format!("using {token} now"),
2754                format!("(\"{token}\")"),
2755                format!("token={token}"),
2756                format!("Authorization: Bearer {token}"),
2757                format!("authorization={token}"),
2758                // A URL's userinfo. This is the canonical token-authenticated
2759                // git remote, so it is what a clone or fetch error carries, and
2760                // it is the shape the `://` branch used to hand back verbatim.
2761                format!("https://x-access-token:{token}@github.com/owner/repo.git"),
2762                format!("fatal: could not read from https://{token}@github.com/o/r"),
2763                // Userinfo with no password half.
2764                format!("https://{token}@github.com/o/r.git"),
2765                // A fragment, which used to be stripped only when it was a
2766                // query string.
2767                format!("https://github.com/login/oauth#access_token={token}"),
2768                format!("https://github.com/x?a=1#token={token}"),
2769            ] {
2770                let redacted = redact(&shape);
2771                assert!(
2772                    !redacted.contains(token),
2773                    "{shape:?} survived redaction as {redacted:?}"
2774                );
2775            }
2776        }
2777    }
2778
2779    #[test]
2780    fn a_url_keeps_what_diagnoses_it_and_loses_what_authenticates_it() {
2781        // The host and the path are the diagnosable part and must survive, or
2782        // nobody keeps this redaction.
2783        assert_eq!(
2784            redact(
2785                "https://x-access-token:ghu_16C7e42F292c6912E7710c838347Ae178B4a@github.com/owner/repo.git"
2786            ),
2787            format!("https://{REDACTION}@github.com/owner/repo.git")
2788        );
2789        assert_eq!(
2790            redact("https://user:hunter2@api.github.com/repos/o/r?page=2"),
2791            format!("https://{REDACTION}@api.github.com/repos/o/r?{REDACTION}")
2792        );
2793        // A password containing an `@`: the host is what follows the last one.
2794        assert_eq!(
2795            redact("https://user:p@ss@github.com/o/r"),
2796            format!("https://{REDACTION}@github.com/o/r")
2797        );
2798        // Userinfo on a bare authority, with no path at all.
2799        assert_eq!(
2800            redact("https://token@github.com"),
2801            format!("https://{REDACTION}@github.com")
2802        );
2803        // An `@` after the first `/` is part of the path, not userinfo.
2804        assert_eq!(
2805            redact("https://github.com/@owner/repo"),
2806            "https://github.com/@owner/repo"
2807        );
2808    }
2809
2810    #[test]
2811    fn an_encoded_jit_configuration_is_redacted() {
2812        assert_eq!(redact(JIT_BLOB), REDACTION);
2813        let sentence = format!("handing off {JIT_BLOB} to the runner");
2814        let redacted = redact(&sentence);
2815        assert!(!redacted.contains(JIT_BLOB), "{redacted}");
2816        assert!(redacted.starts_with("handing off "), "{redacted}");
2817        assert!(redacted.ends_with(" to the runner"), "{redacted}");
2818    }
2819
2820    #[test]
2821    fn a_secret_embedded_in_a_structured_value_is_redacted() {
2822        // Compact JSON is what `serde_json::to_string` emits and what an
2823        // HTTP error body arrives as; spaced JSON is the only shape that
2824        // used to be safe, and it was safe by accident -- the value is its
2825        // own word there, so the opaque-run rule caught it without the key
2826        // ever being recognised.
2827        for shape in [
2828            format!("{{\"encoded_jit_config\":\"{JIT_BLOB}\"}}"),
2829            format!("{{ \"encoded_jit_config\": \"{JIT_BLOB}\" }}"),
2830            format!("encoded_jit_config:{JIT_BLOB}"),
2831            format!("{{\"runner_token\":\"{USER_TOKEN}\"}}"),
2832            format!("runner_token:{USER_TOKEN}"),
2833            format!("pat:{USER_TOKEN}"),
2834            format!("{{\"authorization\":\"Bearer {USER_TOKEN}\"}}"),
2835        ] {
2836            let redacted = redact(&shape);
2837            assert!(
2838                !redacted.contains(JIT_BLOB) && !redacted.contains(USER_TOKEN),
2839                "leaked from {shape}:\n{redacted}"
2840            );
2841        }
2842
2843        // A credential short enough to clear the opaque-run threshold and
2844        // without a GitHub prefix has nothing but the key to give it away,
2845        // so it leaked from the spaced shape too: `\"password\":` left the
2846        // key spelled `password\"`, which is on no list.
2847        for shape in [
2848            "{\"password\":\"hunter2\"}",
2849            "{ \"password\": \"hunter2\" }",
2850        ] {
2851            let redacted = redact(shape);
2852            assert!(
2853                !redacted.contains("hunter2"),
2854                "leaked from {shape}: {redacted}"
2855            );
2856        }
2857
2858        // The neighbouring context still survives: this is a scrubber, not
2859        // a deleter.
2860        let body = format!("registration failed: {{\"encoded_jit_config\":\"{JIT_BLOB}\"}}");
2861        let redacted = redact(&body);
2862        assert!(redacted.starts_with("registration failed: "), "{redacted}");
2863        assert!(redacted.contains("encoded_jit_config"), "{redacted}");
2864    }
2865
2866    #[test]
2867    fn a_credential_key_is_found_wherever_it_sits_in_a_compact_structure() {
2868        // Every shape the test above covers puts the credential in the
2869        // *first* key/value pair, and the rules split a word once, on the
2870        // first separator they find. So redaction was a function of field
2871        // order -- and `serde_json::to_string` is what decides field order for
2872        // any struct with more than one field, which is what an error body is.
2873        //
2874        // Nesting and a form-encoded body are the same defect wearing
2875        // different punctuation: the value recursion ends at `redact_value`,
2876        // which is a leaf, and `&` was not a separator at all.
2877        for shape in [
2878            // The credential key second, in a compact object.
2879            format!("{{\"runner_id\":42,\"encoded_jit_config\":\"{JIT_BLOB}\"}}"),
2880            format!("{{\"status\":422,\"message\":\"bad\",\"runner_token\":\"{USER_TOKEN}\"}}"),
2881            // Nested one level, and two.
2882            format!("{{\"body\":{{\"runner_token\":\"{USER_TOKEN}\"}}}}"),
2883            format!(
2884                "{{\"error\":{{\"status\":422,\"body\":{{\"encoded_jit_config\":\"{JIT_BLOB}\"}}}}}}"
2885            ),
2886            // An array of objects, which is what a list endpoint returns.
2887            format!("[{{\"id\":1}},{{\"access_token\":\"{USER_TOKEN}\"}}]"),
2888            // Form-encoded, credential second and in the middle.
2889            format!("scope=repo&access_token={USER_TOKEN}"),
2890            format!("grant_type=refresh&refresh_token={USER_TOKEN}&scope=repo"),
2891        ] {
2892            let redacted = redact(&shape);
2893            assert!(
2894                !redacted.contains(JIT_BLOB) && !redacted.contains(USER_TOKEN),
2895                "leaked from {shape}:\n{redacted}"
2896            );
2897            assert!(redacted.contains(REDACTION), "{shape} -> {redacted}");
2898        }
2899
2900        // The pairs around it stay legible. A body whose every field came back
2901        // `[redacted]` diagnoses nothing, and is how a redaction gets turned
2902        // off rather than fixed.
2903        let redacted = redact(&format!(
2904            "failed: {{\"runner_id\":42,\"encoded_jit_config\":\"{JIT_BLOB}\"}}"
2905        ));
2906        assert!(redacted.starts_with("failed: "), "{redacted}");
2907        assert!(redacted.contains("\"runner_id\":42"), "{redacted}");
2908        assert!(redacted.contains("encoded_jit_config"), "{redacted}");
2909
2910        // A credential short enough to clear the opaque-run threshold and
2911        // carrying no GitHub prefix has nothing but its key to give it away,
2912        // so it is the case the shape rules cannot rescue.
2913        for shape in [
2914            "{\"user\":\"operator\",\"password\":\"hunter2\"}",
2915            "{\"body\":{\"password\":\"hunter2\"}}",
2916            "user=operator&password=hunter2",
2917        ] {
2918            assert!(
2919                !redact(shape).contains("hunter2"),
2920                "leaked from {shape}: {}",
2921                redact(shape)
2922            );
2923        }
2924
2925        // Ordinary punctuation still survives the cut, or the structure that
2926        // makes a log line readable is gone with the secret.
2927        assert_eq!(
2928            redact("desired 3, active 1, headroom 2"),
2929            "desired 3, active 1, headroom 2"
2930        );
2931        assert_eq!(
2932            redact("labels=[linux,x64,self-hosted]"),
2933            "labels=[linux,x64,self-hosted]"
2934        );
2935    }
2936
2937    #[test]
2938    fn backslash_escaped_json_reaches_the_key_rules_and_the_value_rules() {
2939        // `tracing::error!(reason = ?err)` reaches this module through
2940        // `record_debug` and `format!("{:?}")`, and `Debug` on a `String`
2941        // escapes the quotes inside it. So an error whose `Debug` embeds an
2942        // HTTP body arrives with its keys spelled `\"password\"` -- and `\`
2943        // is not in `WRAPPERS`, so `trim_matches(WRAPPERS)` left it welded on
2944        // and no list contained the result. `reason` is on ALLOWED_FIELDS, so
2945        // this reaches the scrubber rather than being dropped.
2946        let failure = StoreError {
2947            body: "{\"password\":\"hunter2\"}".to_string(),
2948        };
2949        // The fixture carries the secret before anything has looked at it. A
2950        // premise, asserted rather than assumed: a fixture that quietly
2951        // stopped carrying one would make every assertion below vacuously
2952        // true.
2953        assert!(failure.body.contains("hunter2"), "{}", failure.body);
2954        let rendered = format!("{failure:?}");
2955        // The premise, asserted rather than assumed: `Debug` really does
2956        // produce the escaped spelling. If it ever stops, this test is
2957        // measuring something else.
2958        assert!(
2959            rendered.contains("\\\"password\\\""),
2960            "the premise is that Debug escapes the quotes: {rendered}"
2961        );
2962        assert!(
2963            !redact(&rendered).contains("hunter2"),
2964            "leaked: {}",
2965            redact(&rendered)
2966        );
2967
2968        for shape in [
2969            "{\\\"password\\\":\\\"hunter2\\\"}",
2970            "Error { body: \"{\\\"access_token\\\":\\\"hunter2\\\"}\" }",
2971        ] {
2972            assert!(
2973                !redact(shape).contains("hunter2"),
2974                "leaked from {shape}: {}",
2975                redact(shape)
2976            );
2977        }
2978
2979        // The escaped spelling of a long secret leaked too, and for the same
2980        // reason: the key was unrecognised, so nothing below it ever ran.
2981        let escaped = format!("{{\\\"encoded_jit_config\\\":\\\"{JIT_BLOB}\\\"}}");
2982        assert!(!redact(&escaped).contains(JIT_BLOB), "{}", redact(&escaped));
2983
2984        // Trimming the key is only half of it, and it is the half that is
2985        // easy to mistake for the whole. `runner_token` is deliberately *not*
2986        // on CREDENTIAL_KEYS -- a field name is not what makes a value a
2987        // secret, and this module's own documentation says so -- so that pair
2988        // is caught by the token-prefix rule reading its *value*, or it is not
2989        // caught at all. The escaped quote hid the value from that rule
2990        // exactly as it hid the key from the key rules, and a key-only fix
2991        // leaves this one leaking through the sink.
2992        assert!(
2993            !is_credential_key("runner_token"),
2994            "the premise of this case is an unlisted key; once it is listed, \
2995             this stops exercising the value side at all"
2996        );
2997        let escaped_value = format!("{{\\\"runner_token\\\":\\\"{USER_TOKEN}\\\"}}");
2998        assert!(
2999            !redact(&escaped_value).contains(USER_TOKEN),
3000            "the value side leaked: {}",
3001            redact(&escaped_value)
3002        );
3003
3004        // The trim is unconditional on a key and conditional on a value: a
3005        // backslash comes off a value only where it is escaping punctuation.
3006        // `split_wrappers` runs before `looks_like_path`, so putting `\` in
3007        // `WRAPPERS` would trim away the leading `\\` that UNC detection keys
3008        // on and turn a share path back into ordinary text.
3009        assert_eq!(redact(r"\\fileserver\share\jit"), PATH_REDACTION);
3010        assert_eq!(redact(r"\\?\C:\Users\operator\runtime"), PATH_REDACTION);
3011        // And the escaped spelling of a share path is still a share path:
3012        // `\\` escapes nothing, so it survives the trim that `\"` does not.
3013        assert_eq!(redact(r"\\\\fileserver\\share\\jit"), PATH_REDACTION);
3014        // A path ending in its own separator is still a path without it. The
3015        // separator is punctuation, so it is trimmed, judged, and put back.
3016        let trailing = redact("C:\\Users\\operator\\runtime\\");
3017        assert!(trailing.starts_with(PATH_REDACTION), "{trailing}");
3018        assert!(!trailing.contains("operator"), "{trailing}");
3019    }
3020
3021    #[test]
3022    fn a_url_does_not_make_the_rest_of_its_word_unredactable() {
3023        // `redact_core` split on the first `://` and handed *everything*
3024        // before it to `redact_url` as a scheme and everything after it as a
3025        // URL. The terminal arm of `redact_url` echoes both verbatim, so a
3026        // single URL anywhere in a word turned the whole of the rest of that
3027        // word into text no rule below could reach. The only thing that saved
3028        // the shape was a `?` or `#` inside the URL, which made `redact_url`
3029        // replace the tail.
3030        //
3031        // `documentation_url` is in essentially every GitHub REST error body,
3032        // so this was any such body logged alongside a credential.
3033        let body = format!(
3034            "{{\"message\":\"Bad credentials\",\"documentation_url\":\"https://docs.github.com/rest\",\"token\":\"{USER_TOKEN}\"}}"
3035        );
3036        let redacted = redact(&body);
3037        assert!(!redacted.contains(USER_TOKEN), "leaked: {redacted}");
3038        // The URL is the diagnosable part and must survive, or nobody keeps
3039        // this redaction.
3040        assert!(
3041            redacted.contains("https://docs.github.com/rest"),
3042            "the URL should survive: {redacted}"
3043        );
3044
3045        // The mirror order: everything before the `://` became the "scheme".
3046        let reversed = format!(
3047            "{{\"token\":\"{USER_TOKEN}\",\"documentation_url\":\"https://docs.github.com/rest\"}}"
3048        );
3049        let redacted = redact(&reversed);
3050        assert!(!redacted.contains(USER_TOKEN), "leaked: {redacted}");
3051
3052        // A nested object behind a URL was missed even when the URL's own
3053        // userinfo was caught, because the miss and the catch happened in the
3054        // same call.
3055        let clone = format!(
3056            "{{\"remote\":\"https://x-access-token:{USER_TOKEN}@github.com/o/r.git\",\"body\":{{\"password\":\"hunter2\"}}}}"
3057        );
3058        let redacted = redact(&clone);
3059        assert!(!redacted.contains(USER_TOKEN), "leaked: {redacted}");
3060        assert!(!redacted.contains("hunter2"), "leaked: {redacted}");
3061        assert!(
3062            redacted.contains("@github.com/o/r.git"),
3063            "the remote should stay diagnosable: {redacted}"
3064        );
3065
3066        // A URL still owns its own query string, which is why the URL check
3067        // sits ahead of the structural cut: `?` and `#` are not structural
3068        // characters, and an OAuth response puts the token after one of them.
3069        assert_eq!(
3070            redact(&format!(
3071                "{{\"url\":\"https://api.github.com/x?token={USER_TOKEN}\"}}"
3072            )),
3073            format!("{{\"url\":\"https://api.github.com/x?{REDACTION}\"}}")
3074        );
3075        // And a URL standing on its own is untouched.
3076        assert_eq!(
3077            redact("GET https://api.github.com/repos/o/r/actions/runners"),
3078            "GET https://api.github.com/repos/o/r/actions/runners"
3079        );
3080
3081        // A URL that is a credential key's *own* value keeps its scheme, host
3082        // and path like every other URL -- a URL is not a token, and this
3083        // module keeps exactly that much of one everywhere else. What must not
3084        // happen is the key's empty value being redacted on the way past,
3085        // which put the URL out behind a `[redacted]` that had replaced
3086        // nothing: `token=[redacted]https://evil.example/x`. Bounding the URL
3087        // is what created that shape, and pass one declining an empty value is
3088        // what closes it.
3089        assert_eq!(
3090            redact("token=https://evil.example/x"),
3091            "token=https://evil.example/x"
3092        );
3093        assert_eq!(
3094            redact("{\"token\":\"https://evil.example/x\"}"),
3095            "{\"token\":\"https://evil.example/x\"}"
3096        );
3097        // An empty credential value is nothing to redact wherever it sits, and
3098        // saying otherwise reports a secret in a place none was.
3099        assert_eq!(redact("{\"password\":\"\"}"), "{\"password\":\"\"}");
3100    }
3101
3102    #[test]
3103    fn an_array_element_is_judged_without_the_quote_that_wraps_it() {
3104        // The structural cut recursed on the *raw* fragment, and the terminal
3105        // fallback then called `redact_value` with the wrappers still on. A
3106        // fragment carrying no `:` or `=` of its own -- which is exactly what
3107        // an array element is -- therefore reached the shape rules as
3108        // `"ghu_…`: `starts_with("ghu_")` fails, `is_opaque_char` fails on the
3109        // quote, `starts_with("eyJ")` fails, and `looks_like_path` never gets
3110        // a clean look at `"C:\Users\…`.
3111        //
3112        // This is the defect the structural cut was written to close, one step
3113        // further down: "only the first key/value pair is ever examined"
3114        // became "a value with no key of its own is never examined". It
3115        // survived a round because `scan_for_leaks` had no array among its
3116        // twelve shapes.
3117        for shape in [
3118            format!("{{\"tokens\":[\"{USER_TOKEN}\",\"x\"]}}"),
3119            format!("{{\"tokens\":[\"x\",\"{USER_TOKEN}\"]}}"),
3120            format!("[\"x\",\"{USER_TOKEN}\"]"),
3121            format!("[{{\"id\":1}},[\"{USER_TOKEN}\"]]"),
3122            // The `Debug`-escaped spelling of the same thing.
3123            format!("{{\\\"tokens\\\":[\\\"x\\\",\\\"{USER_TOKEN}\\\"]}}"),
3124        ] {
3125            let redacted = redact(&shape);
3126            assert!(
3127                !redacted.contains(USER_TOKEN),
3128                "leaked from {shape}: {redacted}"
3129            );
3130            assert!(redacted.contains(REDACTION), "{shape} -> {redacted}");
3131        }
3132
3133        // Every other shape rule was reachable the same way.
3134        let jit = format!("{{\"items\":[\"{JIT_BLOB}\"]}}");
3135        assert!(!redact(&jit).contains(JIT_BLOB), "{}", redact(&jit));
3136        let assertions = format!("{{\"assertions\":[\"{JWT}\"]}}");
3137        assert!(
3138            !redact(&assertions).contains(JWT),
3139            "{}",
3140            redact(&assertions)
3141        );
3142        let opaque = "ZYXWVUTSRQPONMLKJIHGFEDCBA9876543210zyxwvut";
3143        assert!(opaque.len() > OPAQUE_RUN_THRESHOLD, "{}", opaque.len());
3144        let listed = format!("[\"x\",\"{opaque}\"]");
3145        assert!(!redact(&listed).contains(opaque), "{}", redact(&listed));
3146
3147        // A path in an array. `07-security.md` requires paths redacted, and
3148        // only `looks_like_path` can see one -- with a clean look at the value
3149        // and not before.
3150        let roots = format!("{{\"roots\":[\"x\",\"{WINDOWS_WORKSPACE}\"]}}");
3151        let redacted = redact(&roots);
3152        assert!(!redacted.contains(WINDOWS_WORKSPACE), "leaked: {redacted}");
3153        assert!(redacted.contains(PATH_REDACTION), "{redacted}");
3154
3155        // The array survives being scrubbed: this is a scrubber, not a
3156        // deleter.
3157        assert_eq!(
3158            redact("labels=[\"linux\",\"x64\"]"),
3159            "labels=[\"linux\",\"x64\"]"
3160        );
3161    }
3162
3163    #[test]
3164    fn a_semicolon_cuts_a_word_the_way_a_comma_does() {
3165        // `;` was in `WRAPPERS` -- recognised as punctuation -- and not in
3166        // `STRUCTURAL`, so it never cut a word. It is the separator for a
3167        // Windows connection string, for a credential string, and for cookie
3168        // pairs written without a space, which is `d2`'s territory exactly.
3169        // `Set-Cookie: theme=dark; session=…` was safe only because of the
3170        // space after the `;`.
3171        for shape in [
3172            "store rejected: Server=host;Database=x;Password=hunter2;",
3173            "store rejected: user=operator;password=hunter2",
3174            "Server=host;Password=hunter2;Database=x",
3175        ] {
3176            let redacted = redact(shape);
3177            assert!(
3178                !redacted.contains("hunter2"),
3179                "leaked from {shape}: {redacted}"
3180            );
3181        }
3182
3183        let cookies = format!("theme=dark;session={USER_TOKEN}");
3184        let redacted = redact(&cookies);
3185        assert!(!redacted.contains(USER_TOKEN), "leaked: {redacted}");
3186
3187        // Separators go back verbatim, so ordinary prose is unaffected.
3188        assert_eq!(
3189            redact("started; then reconciled; then idled"),
3190            "started; then reconciled; then idled"
3191        );
3192        assert_eq!(
3193            redact("desired 3;active 1;headroom 2"),
3194            "desired 3;active 1;headroom 2"
3195        );
3196    }
3197
3198    #[test]
3199    fn an_element_wrapped_value_is_redacted() {
3200        // `split_wrappers` strips only the *outermost* `<` and `>`, so
3201        // `<string>ghu_…</string>` arrived as `string>ghu_…</string`, which
3202        // matches no rule at all. `d3`'s installers handle launchd plists,
3203        // which is where this shape comes from; a systemd unit line is the
3204        // `key=value` spelling and was already covered.
3205        for shape in [
3206            format!("<string>{USER_TOKEN}</string>"),
3207            format!("<key>token</key><string>{USER_TOKEN}</string>"),
3208            format!("<dict><key>Token</key><string>{USER_TOKEN}</string></dict>"),
3209        ] {
3210            let redacted = redact(&shape);
3211            assert!(
3212                !redacted.contains(USER_TOKEN),
3213                "leaked from {shape}: {redacted}"
3214            );
3215            assert!(redacted.contains(REDACTION), "{shape} -> {redacted}");
3216        }
3217
3218        // The element names survive: they are what says the line was about a
3219        // plist at all.
3220        let plist = format!("<key>token</key><string>{JIT_BLOB}</string>");
3221        let redacted = redact(&plist);
3222        assert!(!redacted.contains(JIT_BLOB), "leaked: {redacted}");
3223        assert!(redacted.contains("<key>token</key>"), "{redacted}");
3224
3225        // An angle bracket in ordinary text is re-emitted verbatim.
3226        assert_eq!(redact("Custom<Io>"), "Custom<Io>");
3227    }
3228
3229    #[test]
3230    fn a_redacted_value_keeps_the_punctuation_that_wrapped_it() {
3231        // Pass one returned `format!("{key}{separator}{REDACTION}")` and
3232        // dropped the value's trailing wrapper, so a redacted object came out
3233        // with an unbalanced quote: `{"password":[redacted]"}`. Pass two
3234        // already put `lead` and `trail` back.
3235        //
3236        // Worth fixing because this module argues, correctly, that the
3237        // structure around a redaction is what keeps a line diagnosable -- and
3238        // a reader who cannot parse the line cannot tell a redaction from a
3239        // truncation.
3240        assert_eq!(
3241            redact("{\"password\":\"hunter2\"}"),
3242            format!("{{\"password\":\"{REDACTION}\"}}")
3243        );
3244        assert_eq!(
3245            redact(&format!("{{\"access_token\":\"{USER_TOKEN}\"}}")),
3246            format!("{{\"access_token\":\"{REDACTION}\"}}")
3247        );
3248        assert_eq!(
3249            redact(&format!(
3250                "[{{\"id\":1}},{{\"access_token\":\"{USER_TOKEN}\"}}]"
3251            )),
3252            format!("[{{\"id\":1}},{{\"access_token\":\"{REDACTION}\"}}]")
3253        );
3254        // The `Debug`-escaped spelling keeps its escaped quotes.
3255        assert_eq!(
3256            redact("{\\\"password\\\":\\\"hunter2\\\"}"),
3257            format!("{{\\\"password\\\":\\\"{REDACTION}\\\"}}")
3258        );
3259
3260        // The point of all of it: what the sink writes is still the JSON it
3261        // was handed, minus the secret.
3262        let line = redact(&format!(
3263            "{{\"runner_id\":42,\"access_token\":\"{USER_TOKEN}\"}}"
3264        ));
3265        serde_json::from_str::<Value>(&line)
3266            .unwrap_or_else(|error| panic!("a redacted body must still parse: {line} ({error})"));
3267    }
3268
3269    #[test]
3270    fn a_bare_jwt_is_redacted_despite_its_dots() {
3271        // `is_opaque_char` excludes `.`, so a JWT is three opaque runs rather
3272        // than one, each under the threshold, and a 100-character credential
3273        // printed verbatim. `Authorization: Bearer <jwt>` is caught by the
3274        // scheme rule and a credential-keyed one by the key rule, so this bit
3275        // only where a token stood alone or under a name nobody listed.
3276        assert!(
3277            JWT.len() > 100,
3278            "the premise is a long token: {}",
3279            JWT.len()
3280        );
3281        assert_eq!(redact(JWT), REDACTION);
3282        assert_eq!(
3283            redact(&format!("minted {JWT} for the installation")),
3284            format!("minted {REDACTION} for the installation")
3285        );
3286        // Sentence-final punctuation is trimmed as a wrapper first, so the
3287        // token is still recognised.
3288        assert!(!redact(&format!("minted {JWT}.")).contains(JWT));
3289        // Under a key nobody listed.
3290        assert!(!redact(&format!("assertion={JWT}")).contains(JWT));
3291        assert!(!redact(&format!("{{\"assertion\":\"{JWT}\"}}")).contains(JWT));
3292
3293        // Narrow on purpose. Adding `.` to `is_opaque_char` is the obvious fix
3294        // and the wrong one: it swallows every long dotted word there is.
3295        for ordinary in [
3296            "com.example.runner.manager.platform.process.identity.token",
3297            "runner-manager.2026.08.21.log",
3298            "api.github.com",
3299            "9f2c1a44-0000-4000-8000-000000000001.attempt.json",
3300        ] {
3301            assert_eq!(redact(ordinary), ordinary, "over-redacted {ordinary}");
3302        }
3303    }
3304
3305    #[test]
3306    fn the_sink_redacts_a_short_credential_that_only_its_key_gives_away() {
3307        // `scan_for_leaks` cannot carry this case. A credential short enough
3308        // to clear the opaque-run threshold and carrying no GitHub prefix is
3309        // indistinguishable from an ordinary word when it is logged bare, so
3310        // injecting it through shape 1 would demand a redaction no shape rule
3311        // can deliver. Here the key is always present, which is the situation
3312        // `d2`'s secret store is actually in -- and it is through the sink,
3313        // not through `redact` alone, because that distinction is why these
3314        // shapes survived a round.
3315        let capture = Capture::default();
3316        let output = emit(&capture, || {
3317            tracing::error!("store rejected: {{\"user\":\"operator\",\"password\":\"hunter2\"}}");
3318            tracing::error!("store rejected: {{\"body\":{{\"password\":\"hunter2\"}}}}");
3319            tracing::error!("store rejected: user=operator&password=hunter2");
3320            let failure = StoreError {
3321                body: "{\"password\":\"hunter2\"}".to_string(),
3322            };
3323            tracing::error!(reason = ?failure, "store rejected the request");
3324
3325            // A `;` or an `&` after a URL in the same word. The URL branch runs
3326            // ahead of the structural cut and `split_url` did not stop at
3327            // either character, so the separator that would have cut the word
3328            // was swallowed into the URL and echoed.
3329            tracing::error!("store rejected: Server=https://vault.local/api;Password=hunter2;");
3330            tracing::error!("keychain error: url=https://kc.local;password=hunter2");
3331            tracing::error!("unit rejected: Environment=API=https://a.com/v1;PASSWORD=hunter2");
3332            tracing::error!("token exchange failed: cb=https://a.com/x&password=hunter2");
3333
3334            // A credential key whose value is in a *later* fragment. The
3335            // empty-value skip is right; the claim that the structural cut
3336            // reaches such a value was not, because nothing carried the key
3337            // across the cut.
3338            tracing::error!("store rejected: {{\"password\":[\"hunter2\"]}}");
3339            tracing::error!("store rejected: {{\"password\":{{\"v\":\"hunter2\"}}}}");
3340            tracing::error!("store rejected: Password=;hunter2");
3341            tracing::error!("plist rejected: <key>password</key><string>hunter2</string>");
3342            tracing::error!(
3343                "plist rejected: <dict><key>password</key><string>hunter2</string></dict>"
3344            );
3345
3346            // A structural character inside the value. `,` and `&` could
3347            // already split a secret; this commit added `;`, `<` and `>`.
3348            //
3349            // The pieces are spelled so that none of them is a substring of
3350            // anything the line legitimately keeps -- `ss` would have "found" a
3351            // leak in the surviving key `password`, which is a check that fails
3352            // for the wrong reason and is no better than one that cannot fail.
3353            tracing::error!("store rejected: {{\"password\":\"qq<vv>xx\"}}");
3354            tracing::error!("store rejected: {{\"password\":\"j1&k2,m3\"}}");
3355            tracing::error!("store rejected: {{\"password\":\"n4;p5<q6>r7,t8\"}}");
3356        });
3357
3358        assert!(
3359            !output.contains("hunter2"),
3360            "a short credential leaked through the sink:\n{output}"
3361        );
3362        // The punctuated passwords: every piece a structural character can cut
3363        // one into has to go, not just the piece that kept the key company.
3364        for piece in [
3365            "qq", "vv", "xx", "j1", "k2", "m3", "n4", "p5", "q6", "r7", "t8",
3366        ] {
3367            assert!(
3368                !output.contains(piece),
3369                "a structural character split a secret and the tail survived \
3370                 ({piece}):\n{output}"
3371            );
3372        }
3373        assert!(output.contains(REDACTION), "{output}");
3374        // Sixteen lines, or the sink was never reached and "no secret found" is
3375        // true of an empty string.
3376        assert_eq!(
3377            output
3378                .lines()
3379                .filter(|line| !line.trim().is_empty())
3380                .count(),
3381            16,
3382            "{output}"
3383        );
3384    }
3385
3386    /// The multi-word plist spelling, which is worse than the one-word one:
3387    /// the credential key and the value are in the same word, but the value
3388    /// itself is two words, so closing it needs the fragment carry to reach the
3389    /// word-level follow-on rule.
3390    #[test]
3391    fn a_credential_value_that_runs_past_its_own_word_is_still_redacted() {
3392        let capture = Capture::default();
3393        let output = emit(&capture, || {
3394            tracing::error!("plist rejected: <key>password</key><string>correct horse</string>");
3395            tracing::error!("plist rejected: <key>password</key> <string>correct horse</string>");
3396        });
3397        for piece in ["correct", "horse"] {
3398            assert!(
3399                !output.contains(piece),
3400                "a multi-word credential value leaked ({piece}):\n{output}"
3401            );
3402        }
3403    }
3404
3405    /// The other half of the fragment carry: it must not swallow the ordinary,
3406    /// *closed* pairs sitting next to a credential.
3407    ///
3408    /// Every shape here is one the carry has an opportunity to over-redact —
3409    /// a closed empty value, a neighbouring key, the pairs of a connection
3410    /// string, and the element names of a plist. Redaction that eats the
3411    /// diagnostics is redaction nobody keeps, and a carry with no terminator
3412    /// is exactly how that happens.
3413    #[test]
3414    fn the_fragment_carry_stops_where_the_value_does() {
3415        for (input, expected) in [
3416            (
3417                "{\"password\":\"abc\",\"user\":\"bob\"}",
3418                format!("{{\"password\":\"{REDACTION}\",\"user\":\"bob\"}}"),
3419            ),
3420            (
3421                "Server=host;Password=hunter2;User=bob",
3422                format!("Server=host;Password={REDACTION};User=bob"),
3423            ),
3424            (
3425                "{\"password\":\"\",\"user\":\"bob\"}",
3426                "{\"password\":\"\",\"user\":\"bob\"}".to_string(),
3427            ),
3428            (
3429                "<dict><key>password</key><string>hunter2</string></dict>",
3430                format!("<dict><key>password</key><string>{REDACTION}</string></dict>"),
3431            ),
3432            (
3433                "{\"password\":\"qq<vv>xx\"}",
3434                format!("{{\"password\":\"{REDACTION}<{REDACTION}>{REDACTION}\"}}"),
3435            ),
3436        ] {
3437            assert_eq!(redact(input), expected, "from {input}");
3438        }
3439
3440        // A closed value ends the claim, so the word after it is ordinary text.
3441        assert_eq!(
3442            redact("{\"password\":\"x\"} ok"),
3443            format!("{{\"password\":\"{REDACTION}\"}} ok")
3444        );
3445        assert_eq!(
3446            redact("Server=host;Password=hunter2; ok"),
3447            format!("Server=host;Password={REDACTION}; ok")
3448        );
3449
3450        // `{"password":""} ok` is *not* on that list, and the reason is worth
3451        // recording rather than asserting away. `split_wrappers` strips the
3452        // empty value with the rest of the trailing punctuation, so the word
3453        // reaches `redact_word` as the core `password":` — a bare credential
3454        // key, which the stem rule has always read as "the value is the next
3455        // word". That rule predates the carry, is untouched by it, and fires
3456        // first; the empty-value skip below it never gets a look. So the `ok`
3457        // is redacted, exactly as it was before any of this.
3458        assert_eq!(
3459            redact("{\"password\":\"\"} ok"),
3460            format!("{{\"password\":\"\"}} {REDACTION}")
3461        );
3462        // What the empty-value skip does still guarantee is the thing it was
3463        // written for: no `[redacted]` is invented *inside* the pair itself.
3464        assert_eq!(redact("{\"password\":\"\"}"), "{\"password\":\"\"}");
3465
3466        // Ordinary prose is untouched: the carry is only ever armed by a
3467        // credential key.
3468        assert_eq!(
3469            redact("started; then reconciled; then idled"),
3470            "started; then reconciled; then idled"
3471        );
3472        assert_eq!(redact("Custom<Io>"), "Custom<Io>");
3473    }
3474
3475    #[test]
3476    fn paths_are_redacted_on_both_families() {
3477        assert_eq!(redact(WORKSPACE), PATH_REDACTION);
3478        assert_eq!(redact(WINDOWS_WORKSPACE), PATH_REDACTION);
3479        assert_eq!(redact("\\\\fileserver\\share\\jit"), PATH_REDACTION);
3480        assert_eq!(
3481            redact("~/Library/Application Support/x"),
3482            format!("{PATH_REDACTION} Support/x")
3483        );
3484        assert_eq!(
3485            redact("runtime=/var/lib/runner-manager/runtime"),
3486            format!("runtime={PATH_REDACTION}")
3487        );
3488    }
3489
3490    #[test]
3491    fn a_url_survives_but_its_query_string_and_fragment_do_not() {
3492        assert_eq!(
3493            redact("GET https://api.github.com/repos/owner/repo/actions/runners"),
3494            "GET https://api.github.com/repos/owner/repo/actions/runners"
3495        );
3496        assert_eq!(
3497            redact("https://api.github.com/x?access_token=ghu_secret"),
3498            format!("https://api.github.com/x?{REDACTION}")
3499        );
3500        // A fragment carries a token in an OAuth implicit-flow response, and
3501        // is never needed to diagnose an HTTP call.
3502        assert_eq!(
3503            redact("https://api.github.com/x#access_token=ghu_secret"),
3504            format!("https://api.github.com/x#{REDACTION}")
3505        );
3506        // Whichever comes first ends the diagnosable part.
3507        assert_eq!(
3508            redact("https://api.github.com/x?page=2#access_token=ghu_secret"),
3509            format!("https://api.github.com/x?{REDACTION}")
3510        );
3511    }
3512
3513    /// `;` and `&` end a URL, but only ahead of its query string.
3514    ///
3515    /// `STRUCTURAL` has nine characters and `is_url_terminator` recognised
3516    /// seven of them. Because the URL branch runs *ahead* of the structural
3517    /// cut, a URL earlier in a word made `split_url` swallow everything to the
3518    /// next terminator — including the `;` or `&` that would have cut the word
3519    /// — and `redact_url`'s terminal arm echoes what it is given. So the L3 fix
3520    /// (bounding a URL) and the L1 fix (cutting on `;` and `&`) did not
3521    /// compose, and this commit added `;` to `STRUCTURAL` for connection
3522    /// strings while leaving the path a URL opens straight through it.
3523    #[test]
3524    fn a_separator_after_a_url_still_cuts_the_word() {
3525        for shape in [
3526            format!("store rejected: Server=https://vault.local/api;Password={USER_TOKEN};"),
3527            format!("keychain error: url=https://kc.local;secret={USER_TOKEN}"),
3528            format!("unit rejected: Environment=API=https://a.com/v1;TOKEN={USER_TOKEN}"),
3529            format!("token exchange failed: cb=https://a.com/x&access_token={USER_TOKEN}"),
3530            format!("dsn rejected: dsn=https://sentry.local/1;password={USER_TOKEN};user=x"),
3531        ] {
3532            let redacted = redact(&shape);
3533            assert!(
3534                !redacted.contains(USER_TOKEN),
3535                "leaked from {shape}:\n{redacted}"
3536            );
3537        }
3538
3539        // The URL itself still survives, or nobody keeps this redaction.
3540        assert_eq!(
3541            redact("store rejected: Server=https://vault.local/api;Password=hunter2;"),
3542            format!("store rejected: Server=https://vault.local/api;Password={REDACTION};")
3543        );
3544
3545        // The obvious fix is the wrong one, and this is what it costs. Making
3546        // `;` and `&` terminators everywhere ends a URL *inside* its own query
3547        // string, and `redact_url` replaces a query wholesale precisely because
3548        // a token can be in any parameter and this module does not guess which:
3549        // `?[redacted]` would become `?[redacted]&state=hunter2`. So the cut is
3550        // restricted to the part of the URL ahead of the first `?` or `#`.
3551        assert_eq!(
3552            redact("https://a.com/cb?code=1&state=hunter2"),
3553            format!("https://a.com/cb?{REDACTION}")
3554        );
3555        assert_eq!(
3556            redact("https://a.com/cb#code=1&state=hunter2"),
3557            format!("https://a.com/cb#{REDACTION}")
3558        );
3559        assert_eq!(
3560            redact(&format!(
3561                "https://a.com/cb?a=1;b=2&access_token={USER_TOKEN}"
3562            )),
3563            format!("https://a.com/cb?{REDACTION}")
3564        );
3565
3566        // And leaving `\` out of the terminators is still right: a Windows
3567        // path used as a URL password puts the `@` that identifies the
3568        // userinfo behind the first backslash, and `redact_url` echoes what it
3569        // is given.
3570        assert_eq!(
3571            redact(r"https://x-access-token:C:\Users\op\p@github.com/o/r.git"),
3572            format!("https://{REDACTION}@github.com/o/r.git")
3573        );
3574    }
3575
3576    /// A secret in a URL *path* is echoed verbatim.
3577    ///
3578    /// `redact_url` applied no shape rule to the path at all. The design
3579    /// intends a path to be diagnosable — and it still is — but the
3580    /// token-prefix rule is documented as a belt that catches a credential
3581    /// *anywhere*, and a path was the one place it never ran.
3582    #[test]
3583    fn a_secret_in_a_url_path_is_redacted_and_the_rest_of_the_path_is_not() {
3584        for shape in [
3585            format!("https://github.com/o/r/raw/{USER_TOKEN}/f"),
3586            format!("https://api.github.com/{JIT_BLOB}"),
3587            format!("https://a.com/x/{JWT}?page=2"),
3588        ] {
3589            let redacted = redact(&shape);
3590            assert!(
3591                !redacted.contains(USER_TOKEN)
3592                    && !redacted.contains(JIT_BLOB)
3593                    && !redacted.contains(JWT),
3594                "leaked from {shape}:\n{redacted}"
3595            );
3596        }
3597
3598        assert_eq!(
3599            redact(&format!("https://github.com/o/r/raw/{USER_TOKEN}/f")),
3600            format!("https://github.com/o/r/raw/{REDACTION}/f")
3601        );
3602
3603        // The rest of the path is exactly as diagnosable as it was: a segment
3604        // is judged on its own, and an ordinary one is not a secret.
3605        for url in [
3606            "https://api.github.com/repos/owner/repo/actions/runners",
3607            "https://github.com/actions/runner/releases/download/v2.330.0/actions-runner-linux-x64-2.330.0.tar.gz",
3608            "https://github.com/o/r.git",
3609            "https://github.com/@owner/repo",
3610            "https://api.github.com/repos/o/r/actions/runners/42",
3611        ] {
3612            assert_eq!(redact(url), url, "over-redacted a diagnosable path: {url}");
3613        }
3614    }
3615
3616    /// A large message must not take the process down.
3617    ///
3618    /// The URL branch recursed into `redact_core` on both sides of the cut. The
3619    /// termination argument was sound — every call is on a strictly shorter
3620    /// slice — and it was silent on *depth*, which was linear in the length of
3621    /// the input. A message of ~2000 URL-carrying items, about 86 KB, exited
3622    /// `0xc00000fd STATUS_STACK_OVERFLOW`.
3623    ///
3624    /// **A stack overflow is not catchable — it aborts the process.** A large
3625    /// HTTP error body is attacker-influenceable content, so this was a way to
3626    /// kill the agent from inside its own log sink, which is worse than any
3627    /// leak: a sink that is not running redacts nothing.
3628    ///
3629    /// Note what this test can and cannot do. An overflow aborts the test
3630    /// binary rather than failing an assertion, so the value of this test is
3631    /// that **the suite completes at all**; the assertions below only confirm
3632    /// it did the work rather than skipping it.
3633    #[test]
3634    fn a_large_message_does_not_overflow_the_stack() {
3635        // 20000 items is ~860 KB and an order of magnitude past the ~2000 that
3636        // was measured to abort. It is also what the commit *before* the URL
3637        // branch survived, because that branch returned without recursing --
3638        // so this is a regression guard with the margin the regression had.
3639        const ITEMS: usize = 20_000;
3640        let body = r#"{"url":"https://api.github.com/repos/o/r"},"#.repeat(ITEMS);
3641        assert!(body.len() > 800_000, "the premise is a large body");
3642
3643        let redacted = redact(&body);
3644        assert!(
3645            redacted.contains("https://api.github.com/repos/o/r"),
3646            "the URLs are the diagnosable part and must survive"
3647        );
3648
3649        // The same depth, reached through the structural cut and through a
3650        // credential value rather than through a bare URL.
3651        let nested = format!("{{\"error\":{{\"body\":{{\"list\":[{}]}}}}}}", body);
3652        assert!(!redact(&nested).is_empty());
3653
3654        // A long run with no URL in it at all, so the structural loop is
3655        // exercised on its own.
3656        let flat = "{\"a\":1},".repeat(ITEMS);
3657        assert!(!redact(&flat).is_empty());
3658
3659        // And one long unbroken URL, where the *path* loop is what iterates.
3660        let long_path = format!("https://a.com/{}", "seg/".repeat(ITEMS));
3661        assert!(!redact(&long_path).is_empty());
3662    }
3663
3664    #[test]
3665    fn a_credential_header_loses_its_scheme_and_its_value() {
3666        assert_eq!(
3667            redact("Authorization: Bearer abc123"),
3668            format!("Authorization: {REDACTION} {REDACTION}")
3669        );
3670        assert_eq!(
3671            redact("Authorization: Bearer abc123, Accept: application/json"),
3672            // The comma ends the header value, so the next header's name is not
3673            // swallowed.
3674            format!("Authorization: {REDACTION} {REDACTION} Accept: application/json")
3675        );
3676        assert_eq!(
3677            redact("x-api-key=hunter2"),
3678            format!("x-api-key={REDACTION}")
3679        );
3680        assert_eq!(
3681            redact("Cookie: session=abc"),
3682            format!("Cookie: {REDACTION}")
3683        );
3684
3685        // A webhook signature is not a credential the way a token is -- it is
3686        // derived, and it verifies rather than authenticates -- but it has a
3687        // SHA-256's exact shape, so the digest carve-out rendered it as a
3688        // labelled 12-character prefix rather than redacting it. Twelve hex
3689        // characters of an HMAC are of no use to anybody, which is why the
3690        // carve-out is safe in general; the header is on the list anyway,
3691        // because there is no diagnostic worth having in a signature and one
3692        // entry is cheaper than the argument.
3693        let hmac = "9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08";
3694        assert_eq!(
3695            redact(&format!("X-Hub-Signature-256: sha256={hmac}")),
3696            format!("X-Hub-Signature-256: {REDACTION}")
3697        );
3698        // Pass one puts the value's wrapping quote back, so what comes out is
3699        // still the object that went in. It used to emit
3700        // `{"x-hub-signature-256":[redacted]"}` -- an unbalanced quote in a
3701        // line this module argues has to stay diagnosable.
3702        //
3703        // This case is also `redact_core`'s two-pass witness, and the comment
3704        // there names it: the `=` comes first and names nothing, but what
3705        // follows it is a whole HMAC -- non-empty -- so a merged pass inspects
3706        // that value and returns before ever reaching the `:` that names
3707        // `x-hub-signature-256`.
3708        assert_eq!(
3709            redact(&format!("{{\"x-hub-signature-256\":\"sha256={hmac}\"}}")),
3710            format!("{{\"x-hub-signature-256\":\"{REDACTION}\"}}")
3711        );
3712        // The SHA-1 spelling GitHub still sends alongside it.
3713        assert_eq!(
3714            redact(&format!("X-Hub-Signature: sha1={hmac}")),
3715            format!("X-Hub-Signature: {REDACTION}")
3716        );
3717        // The carve-out itself is untouched: a bare digest still renders as a
3718        // prefix, because a checksum gate that cannot say which digest it got
3719        // is a gate nobody can act on.
3720        assert_eq!(redact(hmac), "sha256:9f86d081884c…");
3721    }
3722
3723    #[test]
3724    fn ordinary_text_survives_intact() {
3725        // Redaction that eats the diagnostics is redaction nobody keeps. These
3726        // are the shapes this product's own log lines actually take.
3727        for text in [
3728            "reconciliation finished",
3729            "runner exited idle without work",
3730            "desired 3, active 1, headroom 2",
3731            "attempt 9f2c1a44-0000-4000-8000-000000000001 is busy",
3732            "http 403 after 2 retries",
3733            "windows/x64 is a documented pair",
3734        ] {
3735            assert_eq!(redact(text), text, "redaction damaged an ordinary message");
3736        }
3737    }
3738
3739    /// A SHA-256 is 64 opaque characters and would otherwise be swallowed by
3740    /// the opaque-run rule. `07-security.md` makes checksum verification a
3741    /// security gate, and a gate that cannot say *which* digest it got is a
3742    /// gate nobody can act on, so a digest renders as a labelled prefix.
3743    #[test]
3744    fn a_digest_renders_as_a_prefix_rather_than_disappearing() {
3745        let digest = "9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08";
3746
3747        assert_eq!(redact(digest), "sha256:9f86d081884c…");
3748        // The point of the change: expected-versus-actual stays legible.
3749        let other = "60303ae22b998861bce3b28f33eec1be758a213c86c93c076dbe9f558c11c752";
3750        assert_ne!(
3751            redact(digest),
3752            redact(other),
3753            "two digests must stay distinguishable"
3754        );
3755
3756        // In the sentence shape `e2` will actually write.
3757        assert_eq!(
3758            redact(&format!("checksum mismatch: expected {digest} got {other}")),
3759            "checksum mismatch: expected sha256:9f86d081884c… got sha256:60303ae22b99…"
3760        );
3761
3762        // A prefix a caller logged itself is below the threshold and untouched.
3763        assert_eq!(redact(&digest[..12]), digest[..12].to_string());
3764
3765        // An already-labelled digest is labelled once, not twice, and is
3766        // truncated like a bare one.
3767        //
3768        // Before `redact_core` recursed into a `:` value this arrived at
3769        // `redact_value` whole, matched nothing there -- a `:` is not an
3770        // opaque character -- and was printed in full. Recursing hands
3771        // `as_sha256_digest` the digest on its own, which re-attaches its own
3772        // label, so the redundant one is dropped rather than doubled.
3773        //
3774        // The truncation is the part worth having: a 64-character lowercase
3775        // hex run is only *usually* a digest, an HMAC-SHA256 signature has
3776        // the same shape, and `sha256:<hmac>` used to be emitted whole.
3777        assert_eq!(redact(&format!("sha256:{digest}")), "sha256:9f86d081884c…");
3778        // The `=` spelling had the same doubling and is fixed with it; the
3779        // caller's own separator survives either way.
3780        assert_eq!(redact(&format!("sha256={digest}")), "sha256=9f86d081884c…");
3781    }
3782
3783    #[test]
3784    fn the_digest_exception_is_exactly_sixty_four_lowercase_hex_and_nothing_else() {
3785        // The narrower this carve-out is, the less there is to reason about.
3786        // Anything that is not precisely a lowercase SHA-256 stays redacted by
3787        // the opaque-run rule.
3788        let digest = "9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08";
3789
3790        assert_eq!(
3791            redact(&digest.to_ascii_uppercase()),
3792            REDACTION,
3793            "uppercase hex is not the shape this exception recognises"
3794        );
3795        assert_eq!(
3796            redact(&format!("{digest}0")),
3797            REDACTION,
3798            "65 characters is not a SHA-256"
3799        );
3800        // 64 characters of base64, which is what an encoded secret of that
3801        // length looks like: not hex, so not exempt.
3802        let base64ish = "ZYXWVUTSRQPONMLKJIHGFEDCBA9876543210zyxwvutsrqponmlkjihgfedcba98";
3803        assert_eq!(base64ish.len(), 64);
3804        assert_eq!(redact(base64ish), REDACTION);
3805
3806        // And the JIT blob, which is the thing the opaque-run rule exists for,
3807        // must not have been weakened by any of this.
3808        assert_eq!(redact(JIT_BLOB), REDACTION);
3809    }
3810
3811    #[test]
3812    fn a_uuid_survives() {
3813        // 36 characters, below the opaque-run threshold on purpose: every
3814        // identifier in this product's domain model is a UUID, and redacting
3815        // them would make the logs useless.
3816        let id = "9f2c1a44-0000-4000-8000-000000000001";
3817        assert_eq!(redact(id), id);
3818        assert!(id.len() < OPAQUE_RUN_THRESHOLD);
3819    }
3820
3821    #[test]
3822    fn redaction_preserves_whitespace_and_line_structure() {
3823        let input = "line one\nAuthorization: Bearer x\nline three";
3824        let redacted = redact(input);
3825        assert_eq!(redacted.lines().count(), 3, "{redacted}");
3826        assert!(redacted.starts_with("line one\n"), "{redacted}");
3827        assert!(redacted.ends_with("\nline three"), "{redacted}");
3828    }
3829}