Skip to main content

runner_manager_platform/
logging.rs

1// owner: d1-platform-core
2
3//! The structured log sink, and the redaction that is not optional.
4//!
5//! `07-security.md`'s threat table, on *"Logs disclose repository secrets"*:
6//! **"Structured allowlist logging with unconditional redaction of tokens,
7//! headers, JIT blobs, and paths."** Its security gate is a *secret-injection
8//! log scan*, and its release gate is that *"the user access token and the
9//! encoded JIT configuration are absent from logs, databases, snapshots, crash
10//! reports, and CLI output"*.
11//!
12//! # Allowlist, not denylist — and why that is the whole design
13//!
14//! A denylist redacts the fields somebody remembered to name. It is correct on
15//! the day it is written and wrong on the day a later task adds a field, which
16//! is the day it matters. So this sink inverts the default: **a field whose
17//! name is not in [`ALLOWED_FIELDS`] has its value replaced outright.** A task
18//! that adds `runner_token` to a log call gets `[redacted]` with no review, no
19//! ceremony, and no leak. Adding a field to the allowlist is a deliberate edit
20//! to this file that shows up in a diff.
21//!
22//! The field name itself is kept. Knowing that an event carried a
23//! `runner_token` field is useful, and the name is not the secret.
24//!
25//! # Two layers, because one is not enough
26//!
27//! The allowlist protects *fields*. It cannot protect the message body, which
28//! has to be allowed or the logs say nothing — and a message body is exactly
29//! where a secret ends up when somebody writes
30//! `info!("failed with {authorization}")`. So every string that survives the
31//! allowlist is then scrubbed by value shape ([`redact`]): GitHub token
32//! prefixes, credential header names, long opaque runs such as an encoded JIT
33//! configuration, and filesystem paths.
34//!
35//! Scrubbing by shape over-redacts, deliberately. Some of that is worth knowing
36//! about before it surprises somebody:
37//!
38//! - A slash-rooted word is treated as a filesystem path, and a URL *path* on
39//!   its own looks exactly like one. Log a full URL — `https://api.github.com/…`
40//!   — and it survives intact; log a bare `/repos/owner/repo` and it becomes
41//!   `[path]`.
42//! - Any unbroken run of 40 or more base64, base64url, or hex characters is
43//!   treated as an opaque secret. There is exactly one carve-out: a value that
44//!   is precisely 64 lowercase hex characters is a SHA-256 digest, and renders
45//!   as a labelled 12-character prefix rather than disappearing. A digest is
46//!   not a secret, and `07-security.md` makes checksum verification a security
47//!   gate whose most useful diagnostic is expected-versus-actual.
48//! - A JSON Web Token is redacted whole. Its `.` separators split it into runs
49//!   the opaque-run rule is too short-sighted to catch, so it is recognised by
50//!   shape instead: two or three base64url segments whose header begins `eyJ`.
51//! - A URL keeps its scheme, host and path and loses everything that
52//!   authenticates it: the `user:password@` userinfo, the query string, and the
53//!   fragment. The userinfo matters more than it looks — a
54//!   token-authenticated git remote is
55//!   `https://x-access-token:ghu_…@github.com/owner/repo.git`, so it is the
56//!   shape a clone or fetch failure arrives in. The path stays diagnosable but
57//!   is not exempt: each segment goes through the same shape rules on its own,
58//!   because a token in `…/raw/ghu_…/f` is a token, and the alternative was the
59//!   one place the belt never ran. A 40-character git object name is opaque
60//!   enough to go with it.
61//! - A word ending in `:` or `=` whose stem is a credential header name causes
62//!   the next two words to be redacted, so `Authorization: Bearer ghu_…` loses
63//!   both the scheme and the token.
64//!
65//! A word is cut on structural punctuation — `,`, `;`, `{`, `}`, `[`, `]`,
66//! `<`, `>` and `&` — before any of that runs, and each fragment is then judged
67//! the way a whole word is: unwrapped, judged on its core, and re-emitted with
68//! its punctuation put back. Without that cut only the *first* key/value pair
69//! in a compact structure is ever examined, and redaction becomes a function of
70//! field order: `{"encoded_jit_config":"…"}` was caught and
71//! `{"runner_id":42,"encoded_jit_config":"…"}` was not, while
72//! `serde_json::to_string` is what decides which of the two an error body is.
73//! Nesting, a form-encoded body, a `;`-separated connection string and a plist
74//! element are all that same defect in different punctuation. So is an array
75//! element, one step further down — a fragment judged *with* its quote still
76//! attached matches no shape rule at all, and an array element is the one
77//! fragment that has no key of its own to give it away.
78//!
79//! Because the cut is flat, a credential's value does not have to be in the
80//! same fragment as the key that names it — `{"password":["hunter2"]}`,
81//! `{"password":{"v":"hunter2"}}` and a plist's
82//! `<key>password</key><string>hunter2</string>` all put it one or more
83//! fragments away — so a *carry* is threaded along the fragments to say that a
84//! key is still waiting for its value, or that a quoted value was cut before
85//! its closing quote. It steps over element names, because markup is not the
86//! value a key named, and it stops at the punctuation that visibly closes the
87//! value, because a redaction reported where no secret was is a false signal in
88//! the one log a reader consults to find out whether anything leaked.
89//!
90//! A URL is cut out of the text around it rather than being allowed to own the
91//! rest of the word. Its scheme is the run of scheme characters immediately
92//! before the `://`, and it ends at the first character that cannot appear in a
93//! URL — plus, *ahead of its query string only*, at a `;` or an `&`, which are
94//! structural characters everywhere else and are query syntax after the `?`.
95//! Everything on either side goes back through the rules. So
96//! `{"documentation_url":"https://…","token":"ghu_…"}` — which is what a GitHub
97//! REST error body looks like — keeps the URL and redacts the token, rather
98//! than the URL swallowing the token, and
99//! `Server=https://vault.local/api;Password=…` keeps the URL and redacts the
100//! password rather than the URL swallowing the `;` that separates them.
101//!
102//! **The text after a URL is iterated over, not recursed into.** That is a
103//! memory-safety property, not a style: recursing there made stack depth linear
104//! in the length of the message, and ~86 KB of URL-carrying JSON exited
105//! `STATUS_STACK_OVERFLOW`. A stack overflow is not catchable and takes the
106//! process with it, so an attacker-influenceable error body could kill the
107//! agent from inside its own log sink — and a sink that is not running redacts
108//! nothing at all. For the same reason every search in `split_url` is bounded
109//! by the URL's own end: a pass that never returns is as effective a denial as
110//! one that overflows.
111//!
112//! A key is also trimmed of backslashes, which a value never is: `Debug` on a
113//! `String` escapes the quotes inside it, so a body reached through
114//! `error!(reason = ?err)` spells its keys `\"password\"`. The `trim_key`
115//! function documents why the same trim must not be applied to a value; it is
116//! named rather than linked because it is private and this module's
117//! documentation is not.
118//!
119//! Every one of those is a case where being wrong costs a slightly less
120//! readable log line, against a case where being wrong the other way costs a
121//! disclosed credential.
122//!
123//! # What this does not do
124//!
125//! It does not stop a caller printing to standard output, and it does not
126//! redact a `Debug` derive somewhere else in the program. Those are held by
127//! different controls: `secrecy::SecretString` for the two values that matter
128//! (`07-security.md`'s credential inventory), and
129//! [`crate::process::SpawnSpec::spawn_with_handoff`] for the command line.
130
131use std::fmt;
132use std::io::Write;
133use std::path::{Path, PathBuf};
134
135use serde_json::{Map, Value};
136use tracing::field::{Field, Visit};
137use tracing::{Event, Subscriber};
138use tracing_subscriber::layer::{Context, Layer};
139use tracing_subscriber::registry::LookupSpan;
140
141/// What replaces a value this sink will not emit.
142pub const REDACTION: &str = "[redacted]";
143
144/// What replaces something that was recognisably a filesystem path. Distinct
145/// from [`REDACTION`] so a reader can tell "a path was here" from "a secret was
146/// here" without either being disclosed.
147pub const PATH_REDACTION: &str = "[path]";
148
149/// The field names this sink emits verbatim. Everything else is replaced with
150/// [`REDACTION`].
151///
152/// ---
153///
154/// **Read this before adding a name.** A field on this list still passes
155/// through [`redact`], so adding one does not switch redaction off — but it
156/// does mean the field's value reaches the scrubber instead of being discarded,
157/// and the scrubber only recognises shapes it was taught. Add a name only when
158/// the value is structurally incapable of carrying a credential: an
159/// identifier, an enumerated state, a count, a duration. Never add a name whose
160/// value is free text supplied by GitHub or by a workflow.
161///
162/// Kept sorted, and a test enforces that, so an addition is one line in a diff
163/// rather than a name buried in the middle of a list.
164pub const ALLOWED_FIELDS: &[&str] = &[
165    "arch",
166    "attempt",
167    "attempt_id",
168    "attempt_state",
169    "capacity",
170    "count",
171    "demand",
172    "desired",
173    "duration_ms",
174    "elapsed_ms",
175    "error_kind",
176    "event",
177    "exit_code",
178    "headroom",
179    "host_id",
180    "http_status",
181    "installation_id",
182    "job_id",
183    "label",
184    "lock",
185    "message",
186    "mode",
187    "os",
188    "outcome",
189    "pid",
190    "policy_id",
191    "policy_state",
192    "reason",
193    "retry_in_ms",
194    "runner_id",
195    "scope",
196    "start_mode",
197    "state",
198    "target",
199    "version",
200];
201
202/// Header and parameter names whose value is a credential.
203///
204/// Compared case-insensitively with `-` and `_` treated as the same character,
205/// because the same header arrives as `Authorization`, `authorization`, and
206/// `x_api_key` depending on who wrote the line.
207const CREDENTIAL_KEYS: &[&str] = &[
208    "access.token",
209    "api.key",
210    "apikey",
211    "auth",
212    "authorization",
213    "client.secret",
214    "cookie",
215    "credential",
216    "encoded.jit.config",
217    "jit",
218    "jit.config",
219    "jitconfig",
220    "password",
221    "private.token",
222    "proxy.authorization",
223    "refresh.token",
224    "secret",
225    "set.cookie",
226    "token",
227    "www.authenticate",
228    "x.api.key",
229    "x.auth.token",
230    "x.github.token",
231    "x.hub.signature",
232    "x.hub.signature.256",
233];
234
235/// Authentication scheme words. Whatever follows one of these is the
236/// credential.
237const SCHEME_WORDS: &[&str] = &["basic", "bearer", "digest", "negotiate", "token"];
238
239/// Prefixes GitHub gives its credentials. Present as a belt on top of the
240/// opaque-run rule below, because a short-lived or truncated token can be under
241/// the length threshold while still being a live credential.
242const TOKEN_PREFIXES: &[&str] = &["ghp_", "gho_", "ghu_", "ghs_", "ghr_", "github_pat_", "gh_"];
243
244/// How long an unbroken run of opaque characters has to be before it is assumed
245/// to be a secret.
246///
247/// 40 rather than something shorter so that ordinary identifiers, hyphenated
248/// words, and UUIDs (36 characters, and not secret) survive; short enough that
249/// every GitHub credential format and every encoded JIT configuration is well
250/// past it.
251const OPAQUE_RUN_THRESHOLD: usize = 40;
252
253/// Characters that make up base64, base64url, and hex.
254fn is_opaque_char(c: char) -> bool {
255    c.is_ascii_alphanumeric() || matches!(c, '+' | '/' | '=' | '_' | '-')
256}
257
258/// Punctuation that may wrap a value without being part of it.
259const WRAPPERS: &[char] = &[
260    '"', '\'', '`', '(', ')', '[', ']', '{', '}', '<', '>', ',', ';', '.', '!', '?',
261];
262
263/// Punctuation that separates one key/value pair from the next *inside* a
264/// single whitespace-delimited word.
265///
266/// [`redact_core`] judges a fragment by splitting it once, on the first `=` or
267/// `:` it finds. That is enough for `key=value` and for a one-field object,
268/// and it is why every shape this module was tested against put the credential
269/// in the first pair. It is not enough for anything `serde_json::to_string`
270/// actually emits: a struct with two fields becomes
271/// `{"runner_id":42,"encoded_jit_config":"…"}`, where the only pair ever
272/// examined is `runner_id`. Nesting is the same defect one level down, and a
273/// form-encoded body is the same defect spelled with `&`.
274///
275/// So a word is cut on these before any of that runs, and each fragment is
276/// judged on its own, by [`redact_fragment`]. The separators go back verbatim,
277/// because the structure around a redaction is what keeps the line diagnosable.
278///
279/// `;` is here for the reason `&` is: it separates the pairs of a Windows
280/// connection string (`Server=host;Database=x;Password=…`), of a credential
281/// string, and of a cookie header written without a space after the separator.
282/// It was already in [`WRAPPERS`] — recognised as punctuation, and so never
283/// used to cut — which left `Set-Cookie: theme=dark; session=…` safe only
284/// because of the space. `d2` logs keychain, DPAPI and libsecret failures, and
285/// that is the shape they arrive in.
286///
287/// `<` and `>` are here because [`split_wrappers`] strips only the *outermost*
288/// pair, so `<string>ghu_…</string>` reached the rules as
289/// `string>ghu_…</string`, which is on no list and matches no shape. `d3`'s
290/// installers handle launchd plists, which is where that shape comes from.
291const STRUCTURAL: &[char] = &[',', ';', '{', '}', '[', ']', '<', '>', '&'];
292
293/// Whether this sink will emit a field's value rather than replacing it.
294#[must_use]
295pub fn is_field_allowed(name: &str) -> bool {
296    ALLOWED_FIELDS.binary_search(&name).is_ok()
297}
298
299/// Trims the punctuation a *key* can arrive wrapped in: [`WRAPPERS`], plus the
300/// backslash.
301///
302/// The backslash is here rather than in [`WRAPPERS`] on purpose, and the
303/// distinction is load-bearing in both directions.
304///
305/// It has to be trimmed somewhere. `tracing::error!(reason = ?err)` reaches
306/// this module through `record_debug` and `format!("{:?}")`, and `Debug` on a
307/// `String` escapes the quotes inside it — so an error whose `Debug` embeds an
308/// HTTP body arrives with its keys spelled `\"password\"`. Trimming only
309/// [`WRAPPERS`] leaves the backslash welded on, and `\"password\` is on no
310/// list. That is `d2`'s shape: a secret-store failure carrying the body it was
311/// handed.
312///
313/// It must not be trimmed unconditionally. [`split_wrappers`] runs before
314/// [`looks_like_path`], so a `\` in [`WRAPPERS`] would strip the leading
315/// `\\` that UNC detection keys on, and `\\server\share` would stop being
316/// recognised as a path. A key is the one place a backslash can never be part
317/// of the value, so it is the one place the trim is unconditional;
318/// [`trim_start_wrappers`] takes the same backslash off a *value* only when it
319/// is escaping punctuation.
320fn trim_key(key: &str) -> &str {
321    key.trim()
322        .trim_matches(|c: char| c == '\\' || WRAPPERS.contains(&c))
323}
324
325fn normalise_key(key: &str) -> String {
326    trim_key(key).to_ascii_lowercase().replace(['-', '_'], ".")
327}
328
329fn is_credential_key(key: &str) -> bool {
330    let key = normalise_key(key);
331    CREDENTIAL_KEYS.contains(&key.as_str())
332}
333
334fn is_scheme_word(word: &str) -> bool {
335    SCHEME_WORDS.contains(&word.to_ascii_lowercase().as_str())
336}
337
338/// Scrubs a string of anything that looks like a credential, an encoded JIT
339/// configuration, or a filesystem path.
340///
341/// Applied to every string this sink emits, including the ones whose field name
342/// is on [`ALLOWED_FIELDS`]. Also public so that the TUI and the CLI can put a
343/// value through the same rules before showing it, rather than inventing a
344/// second, differently wrong set.
345#[must_use]
346pub fn redact(text: &str) -> String {
347    let mut out = String::with_capacity(text.len());
348    // How many upcoming words to replace outright, because a credential key or
349    // an authentication scheme word said the value comes next.
350    let mut pending: u32 = 0;
351
352    for chunk in text.split_inclusive(char::is_whitespace) {
353        let (word, whitespace) = split_trailing_whitespace(chunk);
354
355        if word.is_empty() {
356            out.push_str(whitespace);
357            continue;
358        }
359
360        if pending > 0 {
361            pending -= 1;
362            out.push_str(REDACTION);
363            out.push_str(whitespace);
364            // A header value ends at a comma or semicolon; anything after that
365            // is the next header's name and is not a secret.
366            if word.ends_with([',', ';']) {
367                pending = 0;
368            }
369            continue;
370        }
371
372        let (rendered, follow_on) = redact_word(word);
373        out.push_str(&rendered);
374        out.push_str(whitespace);
375        pending = follow_on;
376    }
377
378    out
379}
380
381/// Splits a chunk produced by `split_inclusive` into its word and the single
382/// whitespace character that terminated it, if any.
383fn split_trailing_whitespace(chunk: &str) -> (&str, &str) {
384    match chunk.char_indices().next_back() {
385        Some((index, last)) if last.is_whitespace() => chunk.split_at(index),
386        _ => (chunk, ""),
387    }
388}
389
390/// Splits a fragment into its leading wrapping punctuation, its core, and its
391/// trailing wrapping punctuation.
392///
393/// Shared by [`redact_word`] and [`redact_core`], because both have to judge a
394/// fragment on its core while emitting it with its punctuation intact.
395///
396/// [`WRAPPERS`], and an *escaped* quote as well. Escaped text is exactly what
397/// a `Debug` rendering of a string is, and in it every quote arrives with a
398/// backslash welded on — so a value spelled `\"ghu_…\"` is wrapped in the
399/// same way `"ghu_…"` is, and none of the shape rules can see the token until
400/// the wrapper comes off. Trimming the key alone is not enough: `runner_token`
401/// is deliberately *not* on [`CREDENTIAL_KEYS`], because a name is not what
402/// makes a value a secret, so that pair is caught by the token-prefix rule
403/// reading its value or it is not caught at all.
404fn split_wrappers(fragment: &str) -> (&str, &str, &str) {
405    let leading = fragment.len() - trim_start_wrappers(fragment).len();
406    let (prefix, rest) = fragment.split_at(leading);
407    let core_len = trim_end_wrappers(rest).len();
408    let (core, suffix) = rest.split_at(core_len);
409    (prefix, core, suffix)
410}
411
412/// Trims wrapping punctuation from the front of a fragment.
413///
414/// A backslash counts as punctuation **only when it escapes punctuation**, and
415/// that restriction is the whole of what keeps this safe. `\"value` is a
416/// quoted value spelled the way an escaped rendering spells it, and the
417/// backslash is not part of the value. `\\server\share` is a UNC path whose
418/// leading backslashes escape nothing and *are* the value — trimming them is
419/// exactly what would stop [`looks_like_path`] recognising it, and this runs
420/// before [`looks_like_path`] does. `\\` is not a backslash-escaping-a-
421/// wrapper, so the two cases separate cleanly.
422fn trim_start_wrappers(fragment: &str) -> &str {
423    let mut rest = fragment;
424    loop {
425        let trimmed = rest.trim_start_matches(WRAPPERS);
426        let trimmed = match trimmed.strip_prefix('\\') {
427            Some(after) if after.starts_with(WRAPPERS) => after,
428            _ => trimmed,
429        };
430        if trimmed.len() == rest.len() {
431            return rest;
432        }
433        rest = trimmed;
434    }
435}
436
437/// Trims wrapping punctuation from the end of a fragment.
438///
439/// A trailing backslash goes with it, and needs no adjacency test: trimming
440/// runs right to left, so a backslash that has reached the end is one whose
441/// quote has already been taken off. A path that ends in a separator is still
442/// a path without it.
443fn trim_end_wrappers(fragment: &str) -> &str {
444    let mut rest = fragment;
445    loop {
446        let trimmed = rest.trim_end_matches(WRAPPERS).trim_end_matches('\\');
447        if trimmed.len() == rest.len() {
448            return rest;
449        }
450        rest = trimmed;
451    }
452}
453
454/// What a credential key left outstanding at the end of the text that named it.
455///
456/// The structural cut is flat and a credential's value does not have to be in
457/// the same fragment as the key that names it, so something has to carry the
458/// key across the cut. Nothing did, which is [`redact_core`]'s empty-value
459/// case: the comment there claimed the value would be reached "in the next
460/// fragment, where the structural cut reaches it", and no such thing happened.
461///
462/// The two variants are not the same claim, and collapsing them over-redacts:
463///
464/// - [`Carry::Expecting`] is *"a key named a value that has not appeared yet"*.
465///   It has to step over markup, because `<key>password</key><string>…</string>`
466///   puts `/key` and `string` between the key and its value.
467/// - [`Carry::Unclosed`] is *"a quoted value was redacted and its closing quote
468///   is not in this fragment"*, so the value continues past the cut. Inside an
469///   open quote `<` and `>` are literal text rather than markup, so this one
470///   must *not* step over anything.
471#[derive(Clone, Copy, PartialEq, Eq, Debug)]
472enum Carry {
473    /// Nothing outstanding.
474    None,
475    /// A credential key supplied no value of its own.
476    Expecting,
477    /// A redacted credential value was cut before its closing quote.
478    Unclosed,
479}
480
481/// Whether wrapping punctuation closes the value it followed.
482///
483/// A quote closes a string and a bracket closes a structure, so either one
484/// means the credential value ended here and a carry must stop. `,` and `;` are
485/// on the list for the reason [`redact`] already stops its word-level follow-on
486/// at them: they end a header value, and what comes after is the next header's
487/// name.
488///
489/// `>` is deliberately absent. It closes a *tag*, not a value —
490/// `<key>password</key>` ends in one with the credential's value still to come
491/// — and treating it as a terminator is what would leave the multi-word plist
492/// spelling leaking.
493fn closes_a_value(wrappers: &str) -> bool {
494    wrappers.contains(['"', '\'', '`', ',', ';', ']', '}', ')'])
495}
496
497/// Whether a value opened a quote that its own fragment did not close.
498fn opens_an_unclosed_quote(lead: &str, trail: &str) -> bool {
499    const QUOTES: [char; 3] = ['"', '\'', '`'];
500    lead.contains(QUOTES) && !trail.contains(QUOTES)
501}
502
503/// Whether a fragment sits where an element *name* goes rather than where its
504/// content does.
505///
506/// `<` and `>` are in [`STRUCTURAL`] because `<string>ghu_…</string>` reached
507/// the rules as `string>ghu_…</string` and matched nothing. Having cut on them,
508/// the loop knows which side of a tag it is on, and a [`Carry::Expecting`] must
509/// not be spent on `/key` or `string` when the value it is waiting for is the
510/// element's content. A tag name is still judged by every other rule — this
511/// says only that it is not the value a credential key named.
512fn is_tag_name(preceding: Option<char>, following: Option<char>) -> bool {
513    preceding == Some('<') && following == Some('>')
514}
515
516/// Redacts one whitespace-delimited word, and says how many following words the
517/// word implicates.
518fn redact_word(word: &str) -> (String, u32) {
519    // Wrapping punctuation is kept so that JSON-ish and prose context survives
520    // — `("ghu_…")` should become `("[redacted]")`, not `[redacted]`.
521    let (prefix, core, suffix) = split_wrappers(word);
522
523    if core.is_empty() {
524        return (word.to_string(), 0);
525    }
526
527    // A bare credential key or scheme word: the value is the next word.
528    //
529    // The stem is unwrapped before it is judged: spaced JSON writes the key as
530    // `"password":`, which leaves `password"` welded to a quote, and that is on
531    // no list. A long value survived this anyway by way of the opaque-run rule,
532    // so the gap only ever showed on a short one.
533    let stem = core.trim_end_matches([':', '=']);
534    if stem.len() < core.len() && is_credential_key(stem) {
535        // Two, so that `Authorization: Bearer <token>` loses the scheme and the
536        // token rather than only the scheme.
537        return (word.to_string(), 2);
538    }
539    if is_scheme_word(core) {
540        return (word.to_string(), 1);
541    }
542
543    let (rendered, carry) = redact_core(core, suffix);
544
545    // A value the word did not finish is the *next* word's problem, which is
546    // the follow-on rule the word level already has, reached from one level
547    // down. A pretty-printed plist is why it is needed: `<key>password</key>`
548    // and `<string>hunter2</string>` are on separate lines, so the key and its
549    // value are not even in the same word.
550    //
551    // The two carries ask for different amounts, for the same reason the stem
552    // rule above asks for two words and not one:
553    //
554    // - [`Carry::Expecting`] means the value has not been seen *at all*, so it
555    //   may be introduced by a word of its own — `<string>correct` and then
556    //   `horse</string>`. Two, exactly as `Authorization: Bearer <token>` needs
557    //   two.
558    // - [`Carry::Unclosed`] means the value has already started and was cut, so
559    //   only its remainder is outstanding. One.
560    //
561    // Trailing punctuation that closes the value withdraws the claim, for the
562    // same reason pass one declines an empty value: `{"password":""}` names a
563    // credential and supplies nothing, and redacting the next word there would
564    // report a secret where none was.
565    let follow_on = if closes_a_value(suffix) {
566        0
567    } else {
568        match carry {
569            Carry::None => 0,
570            Carry::Unclosed => 1,
571            Carry::Expecting => 2,
572        }
573    };
574    (format!("{prefix}{rendered}{suffix}"), follow_on)
575}
576
577/// Redacts one fragment of a cut-up word, judging it on its core.
578///
579/// [`redact_core`] used to recurse on the *raw* fragment, and its terminal
580/// fallback then handed that fragment to [`redact_value`] with its wrapping
581/// punctuation still attached. A fragment carrying no `:` or `=` of its own —
582/// which is exactly what an array element is — therefore reached the shape
583/// rules as `"ghu_…`, where `starts_with("ghu_")` fails, [`is_opaque_char`]
584/// fails on the quote, the `eyJ` test fails, and [`looks_like_path`] never gets
585/// a clean look at `"C:\Users\…`.
586///
587/// That is the defect the structural cut was written to close, one step further
588/// down: *"only the first key/value pair is ever examined"* became *"a value
589/// with no key of its own is never examined"*. It survived a round because the
590/// secret-injection scan had no array among its shapes, which is the same
591/// lesson in a different place — a check that cannot fail proves nothing.
592///
593/// So a fragment is treated the way [`redact_word`] treats a word: split,
594/// judge the core, put the punctuation back.
595///
596/// The *word-level* follow-on rule is still not repeated here, and the reason
597/// stands: *"the value is the next word"* is a statement about whitespace, and
598/// there is no next word inside a fragment. What that argument does not cover —
599/// and what an earlier spelling of it wrongly took itself to have settled — is
600/// the *fragment-level* claim: **the value is the next fragment**. That one is
601/// a statement about the structural cut, it is true, and nothing implemented
602/// it, which is why `{"password":["hunter2"]}` and
603/// `<key>password</key><string>hunter2</string>` went out whole. [`Carry`] is
604/// that claim, threaded through the loop in [`redact_core`].
605///
606/// `trailing` says this is the last fragment of its core, and it is what keeps
607/// the carry from escaping a word it has already been spent inside: consuming
608/// a claim in the middle of a core settles it, while consuming it at the end
609/// leaves open the possibility that the value runs on into the next word.
610fn redact_fragment(
611    fragment: &str,
612    carry: Carry,
613    tag_name: bool,
614    trailing: bool,
615) -> (String, Carry) {
616    let (prefix, core, suffix) = split_wrappers(fragment);
617    if core.is_empty() {
618        return (fragment.to_string(), carry);
619    }
620
621    let claimed = match carry {
622        Carry::None => false,
623        // Markup is not the value a credential key named.
624        Carry::Expecting => !tag_name,
625        // Inside an open quote there is no markup, only text.
626        Carry::Unclosed => true,
627    };
628
629    if claimed {
630        // Two different reasons to think the value has more to come, and one
631        // answer: it is `Unclosed` from here, never the `Expecting` it may have
632        // arrived as, because the value has now started.
633        //
634        // - `carry == Unclosed` — still inside an open quote, so the value runs
635        //   into the next fragment as well.
636        // - `trailing` — a claim spent on the *last* fragment of a core is one
637        //   whose value reached the end of the word with nothing closing it, so
638        //   it may continue into the next word.
639        //   `<key>password</key><string>` + `correct horse</string>` is that
640        //   shape.
641        //
642        // Anything else was spent in the middle of a core, where the value was
643        // and is done — and punctuation that closes a string or a structure
644        // settles it either way.
645        let next = if !closes_a_value(suffix) && (carry == Carry::Unclosed || trailing) {
646            Carry::Unclosed
647        } else {
648            Carry::None
649        };
650        return (format!("{prefix}{REDACTION}{suffix}"), next);
651    }
652
653    let (rendered, next) = redact_core(core, suffix);
654    let next = if closes_a_value(suffix) {
655        Carry::None
656    } else if next == Carry::None {
657        // **An unclaimed fragment preserves the claim rather than clearing
658        // it.** This is the whole of what lets a plist work: `/key` and
659        // `string` sit between `<key>password</key>` and the `<string>` content
660        // that is the value, they are tag names so they do not spend the claim,
661        // and a fragment that neither spends nor arms one must leave it exactly
662        // as it found it. Overwriting with this fragment's own (empty) result
663        // is how `<key>password</key><string>hunter2</string>` kept leaking
664        // after the carry existed.
665        carry
666    } else {
667        next
668    };
669    (format!("{prefix}{rendered}{suffix}"), next)
670}
671
672/// Redacts one word with its wrapping punctuation already removed, and reports
673/// what it left outstanding for whatever follows it.
674///
675/// `closing` is the wrapping punctuation the caller stripped off the end of
676/// this core. It is not emitted here — the caller puts it back — but the loop
677/// below has to see it, because the character that follows the *last* fragment
678/// is what says whether that fragment is an element name or an element's
679/// content.
680fn redact_core(core: &str, closing: &str) -> (String, Carry) {
681    // A URL survives, minus everything on it that carries a credential. The
682    // scheme, host and path are what makes a log line diagnosable.
683    //
684    // The URL is *cut out* of the text around it rather than being allowed to
685    // own the rest of the fragment, and both sides of the cut come back through
686    // here. [`split_url`] documents what the old unbounded `split_once("://")`
687    // cost. This still runs ahead of the structural cut, because a query string
688    // is `redact_url`'s to own: `?` and `#` are not structural characters, and
689    // an OAuth implicit-flow response puts the token after one of them.
690    //
691    // **The tail is iterated, not recursed into, and that is a memory-safety
692    // property rather than a matter of taste.** An earlier spelling called
693    // `redact_core` on the remainder, and argued only that the recursion
694    // *terminates*: every call is on a strictly shorter slice, which is true
695    // and is not enough. Termination says nothing about **depth**, and depth
696    // was linear in the length of the input — one frame per URL. A message of
697    // ~2000 `{"url":"https://…"},` items, about 86 KB, exited
698    // `0xc00000fd STATUS_STACK_OVERFLOW`.
699    //
700    // A stack overflow is not a `panic`: it is not catchable, it does not
701    // unwind, and it takes the process with it. A large HTTP error body is
702    // content an attacker can influence, so that was a way to kill the agent
703    // from inside its own log sink — and a sink that is not running redacts
704    // nothing at all, which is worse than any single leak.
705    //
706    // With the loop, every remaining re-entry is depth-bounded by construction
707    // rather than by argument:
708    //
709    // - `prefix` holds everything before the *first* `://`, so it contains no
710    //   `://` and cannot reach this branch again.
711    // - `rest` is what is left when the loop finds no further `://`, for the
712    //   same reason.
713    // - the structural branch below calls [`redact_fragment`], whose fragments
714    //   contain no structural character by construction and so cannot reach it
715    //   again either.
716    //
717    // Three levels, whatever the input is. Anything added here that recurses on
718    // a slice whose length is not bounded by a constant re-opens this.
719    if let Some((prefix, scheme, url, remainder)) = split_url(core) {
720        let mut out = String::with_capacity(core.len());
721        if !prefix.is_empty() {
722            out.push_str(&redact_core(prefix, "").0);
723        }
724        out.push_str(&redact_url(scheme, url));
725
726        let mut rest = remainder;
727        while let Some((prefix, scheme, url, remainder)) = split_url(rest) {
728            if !prefix.is_empty() {
729                out.push_str(&redact_core(prefix, "").0);
730            }
731            out.push_str(&redact_url(scheme, url));
732            rest = remainder;
733        }
734        if !rest.is_empty() {
735            out.push_str(&redact_core(rest, "").0);
736        }
737
738        // A URL is a complete value: it carries its own credentials in its own
739        // places, and `redact_url` has already dealt with them. Nothing is
740        // outstanding, which is also what keeps `{"token":"https://…"}` from
741        // arming a claim against whatever follows the URL.
742        return (out, Carry::None);
743    }
744
745    // A Windows drive path is `key:value`-shaped by accident, and
746    // `looks_like_path` only recognises the drive letter at position zero.
747    // Splitting such a path on its colon would hand the tail to a rule that
748    // matches nothing, so it has to be judged whole, before the separators
749    // get at it.
750    if looks_like_path(core) {
751        return (PATH_REDACTION.to_string(), Carry::None);
752    }
753
754    // A compact structure holds more than one key/value pair, and the rules
755    // below examine exactly one of them: `split_once` stops at the first
756    // separator it finds. So `{"encoded_jit_config":"…"}` was caught and
757    // `{"runner_id":42,"encoded_jit_config":"…"}` was not — the two differ by
758    // field order and by nothing else, and `serde_json::to_string` is what
759    // chooses the order. Nesting was the same defect: the value recursion
760    // below ends at `redact_value`, which is a leaf and never comes back
761    // here, so an object inside an object went out whole. A form-encoded body
762    // was the same defect again, with `&` as a separator nothing knew.
763    //
764    // Cutting on [`STRUCTURAL`] first turns all three into the shape the rules
765    // already handle, and does it at any depth: the cut is flat, so
766    // `{"a":{"b":{"token":"…"}}}` yields the same fragments a one-level object
767    // would.
768    //
769    // Each fragment goes through [`redact_fragment`] rather than straight back
770    // in here, because a fragment carries its own wrapping punctuation and a
771    // value judged with its quote attached matches nothing at all.
772    //
773    // The recursion terminates *and is depth-bounded*: a fragment contains no
774    // structural character by construction, so `redact_fragment` cannot reach
775    // this branch again, and the URL branch above iterates over its tail rather
776    // than recursing into it. The loop here is a loop for the same reason — a
777    // fragment per separator, at whatever length the input has.
778    //
779    // A [`Carry`] is threaded along it because the cut is flat and a
780    // credential's value need not be in the same fragment as its key. Without
781    // it the empty-value skip in pass one was a promise nothing kept: the value
782    // was said to be reachable "in the next fragment", and no next fragment
783    // ever heard about the key.
784    if core.contains(STRUCTURAL) {
785        let mut out = String::with_capacity(core.len());
786        let mut carry = Carry::None;
787        let mut rest = core;
788        // The separator on each side of a fragment is what says whether it is
789        // an element name or an element's content.
790        let mut preceding: Option<char> = None;
791        while let Some(index) = rest.find(STRUCTURAL) {
792            let (fragment, tail) = rest.split_at(index);
793            // Every character in `STRUCTURAL` is ASCII, so the separator is
794            // one byte, this cannot split a code point, and the byte is the
795            // whole character.
796            let separator = char::from(tail.as_bytes()[0]);
797            if !fragment.is_empty() {
798                let (rendered, next) = redact_fragment(
799                    fragment,
800                    carry,
801                    is_tag_name(preceding, Some(separator)),
802                    false,
803                );
804                out.push_str(&rendered);
805                carry = next;
806            }
807            out.push_str(&tail[..1]);
808            preceding = Some(separator);
809            rest = &tail[1..];
810        }
811        if !rest.is_empty() {
812            // The last fragment's *following* character is the caller's closing
813            // punctuation: in `<key>password</key>` the final `/key` is a tag
814            // name only because the `>` that ends it was stripped as a wrapper
815            // before this ran.
816            let following = closing.chars().next();
817            let (rendered, next) =
818                redact_fragment(rest, carry, is_tag_name(preceding, following), true);
819            out.push_str(&rendered);
820            carry = next;
821        }
822        return (out, carry);
823    }
824
825    // `key=value` and `key:value` in a single fragment.
826    //
827    // Pass one asks only whether either separator names a credential, and it
828    // runs to completion before any value is inspected, because the *first*
829    // separator in a fragment is not necessarily the one that names the key.
830    //
831    // The witness is a webhook signature header in its compact-JSON spelling,
832    // `{"x-hub-signature-256":"sha256=<hmac>"}`, and
833    // `a_credential_header_loses_its_scheme_and_its_value` holds it. The `=`
834    // comes first and names nothing, but what follows it is a whole HMAC —
835    // *non-empty*, so a merged pass inspects that value, renders it as a
836    // digest prefix, and returns without ever reaching the `:` that names
837    // `x-hub-signature-256`. `{"password":"a=b"}` is the same witness with
838    // nothing else going on: merged, it comes out whole.
839    //
840    // Two earlier spellings of this comment named
841    // `{"encoded_jit_config":"eyJ…In0="}` and then
842    // `{"encoded_jit_config":"eyJ…In0=","runner_id":42}`, and neither
843    // demonstrates the invariant: base64 padding is trailing-only, so the `=`
844    // split yields a value that `split_wrappers` trims to empty, and the
845    // `value.is_empty()` guard in pass two falls through to the `:` anyway. A
846    // comment naming a case that does not demonstrate its own invariant is how
847    // the invariant gets deleted by the next person, so the witness above is
848    // one that reds a named test when the two passes are merged.
849    //
850    // Both halves are judged through `trim_key`: compact JSON welds a quote to
851    // each, so the key arrives as `encoded_jit_config"`, and a `Debug`
852    // rendering welds a backslash as well.
853    // Whether pass one saw a credential key and was given nothing to redact.
854    let mut names_a_credential = false;
855
856    for separator in ['=', ':'] {
857        if let Some((key, raw_value)) = core.split_once(separator)
858            && is_credential_key(key)
859        {
860            // The value's own wrapping punctuation goes back, exactly as pass
861            // two puts it back. Dropping it emitted `{"password":[redacted]"}`
862            // — an unbalanced quote in a line this module argues, correctly,
863            // has to stay diagnosable, and a reader who cannot parse the line
864            // cannot tell a redaction from a truncation.
865            let (lead, value, trail) = split_wrappers(raw_value);
866
867            // An empty value is nothing to redact, and pass two declines one
868            // for the same reason. Claiming `[redacted]` here says a secret was
869            // somewhere none was, which is a false signal in the one log a
870            // reader consults to find out whether anything leaked.
871            //
872            // It is also load-bearing for the URL branch above, which sends
873            // the text before a URL back through here: `{"token":"https://…"}`
874            // leaves this pass a key of `token` and a value of nothing, and
875            // redacting that emitted `token":"[redacted]https://…` — the URL
876            // still standing behind the redaction meant to have replaced it.
877            //
878            // **What it is not is a reason to forget the key.** An earlier
879            // spelling of this comment said the value would be found "in the
880            // next fragment or the next word, where the structural cut and the
881            // follow-on rule reach it", and half of that was false: the
882            // follow-on rule does reach the next *word*, and nothing whatever
883            // reached the next *fragment*, because `redact_fragment`
884            // deliberately carries no follow-on. So `{"password":["hunter2"]}`,
885            // `{"password":{"v":"hunter2"}}` and `Password=;hunter2` — a
886            // credential key with its value one fragment away, which is what an
887            // array, a nested object and an empty pair all are — went out
888            // whole, with the key sitting in plain sight next to them.
889            //
890            // That is the failure this module warns about two comments down: a
891            // comment naming a case that does not demonstrate its own
892            // invariant. The claim is now true because [`Carry::Expecting`]
893            // implements it, and `the_fragment_carry_stops_where_the_value_does`
894            // holds both halves — that the claim is made, and that it is
895            // withdrawn where the value visibly ended.
896            if value.is_empty() {
897                names_a_credential = true;
898                continue;
899            }
900
901            // A quoted value whose closing quote is not in this fragment is a
902            // value the cut ran through the middle of: `{"password":"p&ss"}`
903            // reaches here as `password":"p`, and `ss` is the rest of the
904            // secret. Adding `;`, `<` and `>` to `STRUCTURAL` this round made
905            // three more characters able to do that, and punctuated passwords
906            // are ordinary.
907            let carry = if opens_an_unclosed_quote(lead, trail) {
908                Carry::Unclosed
909            } else {
910                Carry::None
911            };
912            return (format!("{key}{separator}{lead}{REDACTION}{trail}"), carry);
913        }
914    }
915
916    // Pass two: not a credential key, but the value may still be a path or a
917    // token: `runtime=/var/lib/runner-manager/…`. Applied to `:` as well as
918    // to `=`, because `:` is what compact JSON and a bare `key:value` use,
919    // and recursing for `=` alone is what let an encoded JIT configuration
920    // and a `runner_token:ghu_…` pair through verbatim.
921    for separator in ['=', ':'] {
922        if let Some((key, raw_value)) = core.split_once(separator) {
923            let (lead, value, trail) = split_wrappers(raw_value);
924            if value.is_empty() {
925                continue;
926            }
927            let redacted = redact_value(value);
928            // `as_sha256_digest` re-attaches its own `sha256:` label, so an
929            // already-labelled digest would come back doubled as
930            // `sha256:sha256:9f86d081884c…`. Dropping the redundant label
931            // keeps the caller's own key and separator, and leaves the digest
932            // truncated -- which the unrecursed `:` path never did, and which
933            // is worth having, because an HMAC-SHA256 signature has exactly a
934            // digest's shape and was previously printed in full.
935            if trim_key(key).eq_ignore_ascii_case("sha256")
936                && let Some(bare) = redacted.strip_prefix("sha256:")
937            {
938                return (format!("{key}{separator}{lead}{bare}{trail}"), Carry::None);
939            }
940            return (
941                format!("{key}{separator}{lead}{redacted}{trail}"),
942                Carry::None,
943            );
944        }
945    }
946
947    // A credential key with no value at all — either `password:` with nothing
948    // after it, or a bare `password` between a plist's tags. Both name a value
949    // that is somewhere else, and the caller is the only one who can see where.
950    let carry = if names_a_credential || is_credential_key(core) {
951        Carry::Expecting
952    } else {
953        Carry::None
954    };
955    (redact_value(core), carry)
956}
957
958/// Characters a URL scheme is made of: RFC 3986 allows a letter followed by
959/// letters, digits, `+`, `-` and `.`.
960fn is_scheme_byte(byte: u8) -> bool {
961    byte.is_ascii_alphanumeric() || matches!(byte, b'+' | b'-' | b'.')
962}
963
964/// Characters that cannot appear inside a URL, and therefore end one.
965///
966/// `\` is deliberately absent, even though it cannot appear in a URL either.
967/// The shape it would have been there for is a `Debug`-escaped body, which
968/// spells its quotes `\"` — and the quote already ends the URL. Making the
969/// backslash a terminator as well costs a real redaction: a Windows path used
970/// as a URL password, `https://x-access-token:C:\Users\…@github.com/o/r.git`,
971/// would then end its URL at the first `\`, which puts the `@` that identifies
972/// the userinfo *outside* the span, and [`redact_url`] echoes what it is given.
973/// Measured: with `\` a terminator, that line emits the path verbatim; without
974/// it, the userinfo is replaced as it should be.
975fn is_url_terminator(c: char) -> bool {
976    c.is_whitespace() || matches!(c, '"' | ',' | '{' | '}' | '[' | ']' | '<' | '>')
977}
978
979/// The same, plus the two [`STRUCTURAL`] characters that end a URL only *ahead
980/// of its query string*.
981///
982/// [`STRUCTURAL`] has nine characters and [`is_url_terminator`] recognises seven
983/// of them; `;` and `&` were missing. Because the URL branch runs ahead of the
984/// structural cut, a URL earlier in a word made [`split_url`] swallow
985/// everything to the next terminator — including the separator that would have
986/// cut the word — and [`redact_url`]'s terminal arm echoes what it is given. So
987/// `Server=https://vault.local/api;Password=hunter2` and
988/// `cb=https://a.com/x&access_token=ghu_…` went out whole: the fix that bounds
989/// a URL and the fix that cuts on `;` and `&` each worked alone and did not
990/// compose.
991///
992/// **Adding them to [`is_url_terminator`] outright is the obvious fix and the
993/// wrong one**, which is why there are two predicates rather than one extended
994/// list. `&` is what separates the parameters *inside* a query string, and
995/// [`redact_url`] replaces a query wholesale precisely because a token can be
996/// in any parameter and this module does not guess which. Measured:
997/// `https://a.com/cb?code=1&state=hunter2` goes from `?[redacted]` to
998/// `?[redacted]&state=hunter2`. `;` is the same story — it is a legal query
999/// separator too.
1000///
1001/// Ahead of the `?` there is no such conflict: a `;` or an `&` in an authority
1002/// or a path is not URL syntax this module needs to keep, and something else in
1003/// the word is much the likelier reading.
1004fn is_url_head_terminator(c: char) -> bool {
1005    is_url_terminator(c) || matches!(c, ';' | '&')
1006}
1007
1008/// Finds the first URL in a fragment and cuts it out of the text around it.
1009///
1010/// Returns the text before the URL, its scheme, the URL itself with the `://`
1011/// removed, and the text after it — or `None` when the fragment holds no URL.
1012///
1013/// **Both bounds are the point.** `split_once("://")` treated *everything*
1014/// before the separator as the scheme and everything after it as the URL, and
1015/// the terminal arm of [`redact_url`] echoes both verbatim — so a single URL
1016/// anywhere in a word put the whole of the rest of that word beyond every rule
1017/// below it. `documentation_url` is in essentially every GitHub REST error
1018/// body, so that was *any* such body logged alongside a credential:
1019///
1020/// ```text
1021/// {"documentation_url":"https://docs.github.com/rest","token":"ghu_…"}
1022/// ```
1023///
1024/// came out intact. The reverse order leaked for the mirror reason —
1025/// everything before the `://` became the "scheme" — and a nested object behind
1026/// a URL was missed even when the URL's own userinfo was caught, because the
1027/// miss and the catch happened in the same call. The only thing that saved the
1028/// shape at all was a `?` or `#` *inside* the URL, which made [`redact_url`]
1029/// replace the tail.
1030///
1031/// An empty scheme run means the `://` is not introducing a URL, and the
1032/// fragment is left to the rules below rather than handed over as a URL with no
1033/// scheme.
1034fn split_url(fragment: &str) -> Option<(&str, &str, &str, &str)> {
1035    let separator = fragment.find("://")?;
1036    let before = &fragment[..separator];
1037
1038    // Scheme characters are ASCII, so a byte count is a character boundary.
1039    let scheme_len = before
1040        .bytes()
1041        .rev()
1042        .take_while(|byte| is_scheme_byte(*byte))
1043        .count();
1044    if scheme_len == 0 {
1045        return None;
1046    }
1047    let (prefix, scheme) = before.split_at(before.len() - scheme_len);
1048
1049    let after = &fragment[separator + "://".len()..];
1050
1051    // Where the URL ends no matter what, and **the bound every other search
1052    // here is taken inside**. Searching the rest of the fragment first is a
1053    // correct answer computed quadratically: a body of *n* URLs would scan the
1054    // whole remaining text once per URL looking for a `?` that is not there,
1055    // which on the 860 KB regression case in
1056    // `a_large_message_does_not_overflow_the_stack` is 17 billion character
1057    // comparisons. That test is a guard against this module taking the process
1058    // down, and a redaction pass that never returns takes it down just as
1059    // effectively as an overflow does.
1060    let end = after.find(is_url_terminator).unwrap_or(after.len());
1061    let url = &after[..end];
1062
1063    // The head is the authority and path — everything ahead of the first `?` or
1064    // `#`. Only there do `;` and `&` end the URL; inside a query string they
1065    // are query syntax, and the query is `redact_url`'s to replace wholesale.
1066    let query = url.find(['?', '#']).unwrap_or(url.len());
1067    let end = url[..query].find(is_url_head_terminator).unwrap_or(end);
1068    let (url, remainder) = after.split_at(end);
1069
1070    Some((prefix, scheme, url, remainder))
1071}
1072
1073/// Redacts the three places a URL can carry a credential, keeping the rest.
1074///
1075/// `scheme` is everything before `://` and `rest` everything after it.
1076///
1077/// The **userinfo** is the one this module used to miss, and it is not an
1078/// exotic shape: `https://x-access-token:ghu_…@github.com/owner/repo.git` is
1079/// the canonical token-authenticated git remote, so it is what `e2` and `e3`
1080/// will have in hand when a clone or a download fails and they log the error
1081/// they were given. The old `://` branch returned before the token-prefix and
1082/// opaque-run rules could run, so that URL came out intact.
1083///
1084/// A **fragment** is stripped for the same reason a query string is: an OAuth
1085/// implicit-flow response puts the token after the `#`, and the fragment is
1086/// never load-bearing for diagnosing an HTTP call.
1087fn redact_url(scheme: &str, rest: &str) -> String {
1088    // The authority runs to the first `/`, `?` or `#`.
1089    let authority_end = rest.find(['/', '?', '#']).unwrap_or(rest.len());
1090    let (authority, tail) = rest.split_at(authority_end);
1091
1092    // `rsplit_once`, not `split_once`: a password may itself contain an `@`,
1093    // and the host is what follows the *last* one.
1094    let host = match authority.rsplit_once('@') {
1095        // Replaced rather than deleted. That the request carried credentials at
1096        // all is diagnostic — it is often the answer to "why did this 401?" —
1097        // and it is the credential, not its existence, that must not be here.
1098        Some((_userinfo, host)) => format!("{REDACTION}@{host}"),
1099        None => authority.to_string(),
1100    };
1101
1102    // Whichever of `?` and `#` comes first ends the diagnosable part; anything
1103    // after it is replaced wholesale rather than parsed, because a token can be
1104    // in any parameter and this module does not guess which.
1105    match tail.find(['?', '#']) {
1106        Some(cut) => {
1107            let (path, query) = tail.split_at(cut);
1108            let separator = &query[..1];
1109            format!(
1110                "{scheme}://{host}{}{separator}{REDACTION}",
1111                redact_path(path)
1112            )
1113        }
1114        None => format!("{scheme}://{host}{}", redact_path(tail)),
1115    }
1116}
1117
1118/// Applies the shape rules to a URL path, one segment at a time.
1119///
1120/// The path is the diagnosable part of a URL and stays that way: a segment is
1121/// judged on its own, and an ordinary one — `repos`, `owner`, `actions-runner-
1122/// linux-x64-2.330.0.tar.gz` — is not a secret and is not touched. What this
1123/// closes is that [`redact_url`] previously applied *no* rule to a path at all,
1124/// so `https://github.com/o/r/raw/ghu_…/f` was echoed whole while the identical
1125/// token one character to the left of the `/` would have been replaced. The
1126/// token-prefix rule is documented as a belt that catches a credential
1127/// *anywhere*, and a path was the one place it never ran.
1128///
1129/// Segment at a time rather than whole, because [`redact_value`] would
1130/// otherwise see the `/` and hand the whole path to [`looks_like_path`], which
1131/// is exactly the over-redaction the module documentation promises a full URL
1132/// escapes.
1133///
1134/// Worth knowing: [`OPAQUE_RUN_THRESHOLD`] is 40, and a git object name written
1135/// as hex is exactly 40 characters, so a `…/raw/<sha1>/f` URL loses its commit
1136/// to `[redacted]`. That is the module's standing trade — a less readable line
1137/// against a disclosed credential — and it is called out here because a commit
1138/// SHA is the one path segment somebody may miss.
1139fn redact_path(path: &str) -> String {
1140    let mut out = String::with_capacity(path.len());
1141    for (index, segment) in path.split('/').enumerate() {
1142        if index > 0 {
1143            out.push('/');
1144        }
1145        if !segment.is_empty() {
1146            out.push_str(&redact_value(segment));
1147        }
1148    }
1149    out
1150}
1151
1152/// The shape rules, applied to a bare value.
1153fn redact_value(value: &str) -> String {
1154    let lower = value.to_ascii_lowercase();
1155    if TOKEN_PREFIXES
1156        .iter()
1157        .any(|prefix| lower.starts_with(prefix))
1158    {
1159        return REDACTION.to_string();
1160    }
1161
1162    if looks_like_path(value) {
1163        return PATH_REDACTION.to_string();
1164    }
1165
1166    // A digest is not a secret, and `07-security.md` makes checksum
1167    // verification a security gate. The most useful thing `e2` can write when
1168    // that gate fails is expected-versus-actual, and until this carve-out
1169    // existed both sides came out as `[redacted]` — a gate that reports it
1170    // failed and refuses to say how.
1171    if let Some(digest) = as_sha256_digest(value) {
1172        return digest;
1173    }
1174
1175    if looks_like_jwt(value) {
1176        return REDACTION.to_string();
1177    }
1178
1179    if value.len() >= OPAQUE_RUN_THRESHOLD && value.chars().all(is_opaque_char) {
1180        return REDACTION.to_string();
1181    }
1182
1183    value.to_string()
1184}
1185
1186/// The length of a SHA-256 digest written as lowercase hex.
1187const SHA256_HEX_LEN: usize = 64;
1188
1189/// How much of a digest is shown.
1190///
1191/// 12 is the short-digest convention git and the OCI tooling use: 48 bits, far
1192/// more than enough to tell an expected digest from the one that was actually
1193/// computed, which is the only comparison a checksum failure needs. Truncating
1194/// is also what makes this carve-out safe to have at all — a 64-character
1195/// lowercase hex run is *usually* a digest, but an HMAC-SHA256 signature has
1196/// the same shape, and 12 of its 64 characters are of no use to anybody.
1197const DIGEST_PREFIX_LEN: usize = 12;
1198
1199/// Renders a value that is exactly a lowercase SHA-256 digest as a labelled
1200/// prefix, or `None` when it is not one.
1201///
1202/// Deliberately strict: exactly 64 characters, and lowercase only. An uppercase
1203/// or mixed-case run falls through to the opaque-run rule and is redacted,
1204/// because the narrower this exception is, the less there is to reason about.
1205fn as_sha256_digest(value: &str) -> Option<String> {
1206    let is_digest = value.len() == SHA256_HEX_LEN
1207        && value
1208            .chars()
1209            .all(|c| c.is_ascii_digit() || c.is_ascii_lowercase() && c.is_ascii_hexdigit());
1210
1211    is_digest.then(|| format!("sha256:{}…", &value[..DIGEST_PREFIX_LEN]))
1212}
1213
1214/// The most `.`-separated segments a JSON Web Token has: header, payload,
1215/// signature.
1216const JWT_SEGMENTS: usize = 3;
1217
1218/// Whether a value is a JSON Web Token.
1219///
1220/// The opaque-run rule cannot see one. [`is_opaque_char`] excludes `.`, and a
1221/// JWT is base64url runs joined by two of them, so a 100-character credential
1222/// arrives as three runs that are each under the threshold and prints
1223/// verbatim. `Authorization: Bearer <jwt>` is caught by the scheme-word rule
1224/// and a credential-keyed one by the key rule, so this only ever bit a token
1225/// logged bare or under a name nobody listed — which is what a GitHub App
1226/// installation assertion is when it reaches a log at all.
1227///
1228/// Narrow deliberately, because the obvious fix is the wrong one: adding `.`
1229/// to [`is_opaque_char`] would swallow every long dotted word there is —
1230/// package names, dated filenames, dotted identifiers. A JWT header is
1231/// base64url of `{"alg":…`, which always begins `eyJ`, and that is a
1232/// discriminator ordinary text does not have.
1233fn looks_like_jwt(value: &str) -> bool {
1234    if value.len() < OPAQUE_RUN_THRESHOLD || !value.starts_with("eyJ") {
1235        return false;
1236    }
1237
1238    let mut segments = 0usize;
1239    for segment in value.split('.') {
1240        segments += 1;
1241        if !segment.chars().all(is_opaque_char) {
1242            return false;
1243        }
1244    }
1245
1246    // Two segments as well as three: an unsigned token has an empty signature,
1247    // and the trailing `.` that would have made the third is trimmed as
1248    // wrapping punctuation before this is reached.
1249    (2..=JWT_SEGMENTS).contains(&segments)
1250}
1251
1252fn looks_like_path(value: &str) -> bool {
1253    let bytes = value.as_bytes();
1254
1255    // `C:\Users\…` and `C:/Users/…`.
1256    if bytes.len() >= 3
1257        && bytes[0].is_ascii_alphabetic()
1258        && bytes[1] == b':'
1259        && (bytes[2] == b'\\' || bytes[2] == b'/')
1260    {
1261        return true;
1262    }
1263
1264    // `\\server\share\…`.
1265    if value.starts_with("\\\\") {
1266        return true;
1267    }
1268
1269    // `~/…`.
1270    if value.starts_with("~/") || value.starts_with("~\\") {
1271        return true;
1272    }
1273
1274    // `/var/lib/…`. Two segments required, so a lone `/` and a bare `/tmp`
1275    // stay readable; see the module documentation for why a URL path is caught
1276    // by this too.
1277    if let Some(rest) = value.strip_prefix('/') {
1278        return rest.contains('/') && !rest.is_empty();
1279    }
1280
1281    false
1282}
1283
1284// ---------------------------------------------------------------------------
1285// The tracing layer
1286// ---------------------------------------------------------------------------
1287
1288/// The redacted fields recorded on a span, stashed in the span's extensions so
1289/// that an event inside the span can carry them.
1290#[derive(Debug, Clone)]
1291struct SpanFields(Map<String, Value>);
1292
1293/// Collects a `tracing` record into JSON, applying the allowlist and the
1294/// scrubber as it goes.
1295#[derive(Debug, Default)]
1296struct RedactingVisitor {
1297    fields: Map<String, Value>,
1298}
1299
1300impl RedactingVisitor {
1301    fn put_str(&mut self, name: &str, value: &str) {
1302        let rendered = if is_field_allowed(name) {
1303            redact(value)
1304        } else {
1305            REDACTION.to_string()
1306        };
1307        self.fields
1308            .insert(name.to_string(), Value::String(rendered));
1309    }
1310
1311    fn put_value(&mut self, name: &str, value: Value) {
1312        if is_field_allowed(name) {
1313            self.fields.insert(name.to_string(), value);
1314        } else {
1315            // Numbers and booleans go the same way as strings. A credential is
1316            // never an `i64`, but the rule that makes this sink trustworthy is
1317            // that it has *no* exception for a name nobody listed.
1318            self.fields
1319                .insert(name.to_string(), Value::String(REDACTION.to_string()));
1320        }
1321    }
1322}
1323
1324impl Visit for RedactingVisitor {
1325    fn record_str(&mut self, field: &Field, value: &str) {
1326        self.put_str(field.name(), value);
1327    }
1328
1329    fn record_debug(&mut self, field: &Field, value: &dyn fmt::Debug) {
1330        // The event message arrives here, as `format_args!` output.
1331        self.put_str(field.name(), &format!("{value:?}"));
1332    }
1333
1334    fn record_error(&mut self, field: &Field, value: &(dyn std::error::Error + 'static)) {
1335        self.put_str(field.name(), &value.to_string());
1336    }
1337
1338    fn record_bool(&mut self, field: &Field, value: bool) {
1339        self.put_value(field.name(), Value::Bool(value));
1340    }
1341
1342    fn record_i64(&mut self, field: &Field, value: i64) {
1343        self.put_value(field.name(), Value::from(value));
1344    }
1345
1346    fn record_u64(&mut self, field: &Field, value: u64) {
1347        self.put_value(field.name(), Value::from(value));
1348    }
1349
1350    fn record_f64(&mut self, field: &Field, value: f64) {
1351        self.put_value(field.name(), Value::from(value));
1352    }
1353
1354    fn record_i128(&mut self, field: &Field, value: i128) {
1355        self.put_str(field.name(), &value.to_string());
1356    }
1357
1358    fn record_u128(&mut self, field: &Field, value: u128) {
1359        self.put_str(field.name(), &value.to_string());
1360    }
1361}
1362
1363/// A `tracing` layer that writes one redacted JSON object per event.
1364///
1365/// Written here rather than assembled from `tracing_subscriber::fmt` because
1366/// redaction has to happen *before* formatting, and a `tracing::Event`'s fields
1367/// cannot be rewritten for a downstream formatter to pick up. Owning the
1368/// formatting is what makes "unconditional" true: there is no path from an
1369/// event to the output that does not go through
1370/// [`RedactingVisitor`].
1371#[derive(Debug, Clone)]
1372pub struct RedactingLayer<W> {
1373    writer: W,
1374}
1375
1376impl<W> RedactingLayer<W> {
1377    /// Wraps a writer factory — a file appender, a capture buffer, or
1378    /// `std::io::stderr`.
1379    pub const fn new(writer: W) -> Self {
1380        Self { writer }
1381    }
1382}
1383
1384impl<S, W> Layer<S> for RedactingLayer<W>
1385where
1386    S: Subscriber + for<'a> LookupSpan<'a>,
1387    W: for<'a> tracing_subscriber::fmt::MakeWriter<'a> + 'static,
1388{
1389    fn on_new_span(
1390        &self,
1391        attrs: &tracing::span::Attributes<'_>,
1392        id: &tracing::span::Id,
1393        ctx: Context<'_, S>,
1394    ) {
1395        let mut visitor = RedactingVisitor::default();
1396        attrs.record(&mut visitor);
1397        if let Some(span) = ctx.span(id) {
1398            span.extensions_mut().insert(SpanFields(visitor.fields));
1399        }
1400    }
1401
1402    fn on_record(
1403        &self,
1404        id: &tracing::span::Id,
1405        values: &tracing::span::Record<'_>,
1406        ctx: Context<'_, S>,
1407    ) {
1408        let mut visitor = RedactingVisitor::default();
1409        values.record(&mut visitor);
1410        if let Some(span) = ctx.span(id) {
1411            let mut extensions = span.extensions_mut();
1412            if let Some(existing) = extensions.get_mut::<SpanFields>() {
1413                existing.0.extend(visitor.fields);
1414            } else {
1415                extensions.insert(SpanFields(visitor.fields));
1416            }
1417        }
1418    }
1419
1420    fn on_event(&self, event: &Event<'_>, ctx: Context<'_, S>) {
1421        let mut visitor = RedactingVisitor::default();
1422        event.record(&mut visitor);
1423
1424        let mut record = Map::new();
1425        record.insert(
1426            "timestamp".to_string(),
1427            Value::String(chrono::Utc::now().to_rfc3339()),
1428        );
1429        record.insert(
1430            "level".to_string(),
1431            Value::String(event.metadata().level().to_string()),
1432        );
1433        // Named `logger` and not `target`: `target` is also a domain field —
1434        // the repository or organization a policy points at — and two different
1435        // things under one key is how a diagnostic becomes a puzzle.
1436        record.insert(
1437            "logger".to_string(),
1438            Value::String(event.metadata().target().to_string()),
1439        );
1440        record.insert("fields".to_string(), Value::Object(visitor.fields));
1441
1442        let spans: Vec<Value> = ctx
1443            .event_scope(event)
1444            .into_iter()
1445            .flat_map(tracing_subscriber::registry::Scope::from_root)
1446            .map(|span| {
1447                let mut entry = Map::new();
1448                entry.insert("name".to_string(), Value::String(span.name().to_string()));
1449                if let Some(fields) = span.extensions().get::<SpanFields>() {
1450                    entry.insert("fields".to_string(), Value::Object(fields.0.clone()));
1451                }
1452                Value::Object(entry)
1453            })
1454            .collect();
1455        if !spans.is_empty() {
1456            record.insert("spans".to_string(), Value::Array(spans));
1457        }
1458
1459        let mut line = serde_json::to_string(&Value::Object(record)).unwrap_or_else(|_| {
1460            String::from(
1461                r#"{"level":"ERROR","fields":{"message":"a log record could not be encoded"}}"#,
1462            )
1463        });
1464        line.push('\n');
1465
1466        // A failed write to the diagnostics file must not take the agent down,
1467        // and must not be reported through `tracing` either — that would
1468        // recurse straight back into this method.
1469        let mut writer = self.writer.make_writer();
1470        let _ = writer.write_all(line.as_bytes());
1471        let _ = writer.flush();
1472    }
1473}
1474
1475// ---------------------------------------------------------------------------
1476// Installing the sink
1477// ---------------------------------------------------------------------------
1478
1479/// Whose diagnostics these are, and therefore which file they go in.
1480///
1481/// # Why the daemon does not share the operator's file
1482///
1483/// On the two Unixes a boot-mode registration runs the daemon as `root` while
1484/// the four application-data directories stay in the operator's profile —
1485/// `05-infrastructure.md` puts them there and `service install` records those
1486/// paths into the plist. So two accounts write into one `logs/` directory, and
1487/// the appender creates its file with the umask default, `0644`. Whichever
1488/// account opened today's file first owns it, and if that was `root` the
1489/// operator's own `runner-manager status` can no longer append to it.
1490///
1491/// That was not a degraded log. `tracing_appender::rolling::daily` **panics**
1492/// when it cannot open the file, so every CLI command on such a host died with
1493/// a backtrace before it did anything — reported on 0.1.17, on a host whose
1494/// daemon had rolled the file over at midnight as `root`:
1495///
1496/// ```text
1497/// thread 'main' panicked at rolling.rs:156:14:
1498/// initializing rolling file appender failed: InitError { context: "failed to
1499/// create log file", source: Os { code: 13, kind: PermissionDenied } }
1500/// ```
1501///
1502/// [`install`] no longer panics — see there — but not panicking would only have
1503/// turned a crash into an operator who never gets diagnostics again. The two
1504/// writers are separated instead, so neither can take the other's file: the
1505/// account that runs the daemon owns [`SERVICE_LOG_STEM`] and the operator owns
1506/// [`OPERATOR_LOG_STEM`], for as long as the registration lives.
1507#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
1508pub enum LogRole {
1509    /// A command the operator ran, or a daemon they started in the foreground.
1510    Operator,
1511    /// The daemon a service manager started, which on a boot-mode host is a
1512    /// different account from the operator's.
1513    Service,
1514}
1515
1516/// The file stem [`LogRole::Operator`] writes.
1517pub const OPERATOR_LOG_STEM: &str = "runner-manager.log";
1518/// The file stem [`LogRole::Service`] writes, and what `service status`
1519/// reports as the daemon's log.
1520pub const SERVICE_LOG_STEM: &str = "runner-manager.service.log";
1521
1522impl LogRole {
1523    /// The file stem this role writes, before the appender's date suffix.
1524    #[must_use]
1525    pub const fn file_stem(self) -> &'static str {
1526        match self {
1527            Self::Operator => OPERATOR_LOG_STEM,
1528            Self::Service => SERVICE_LOG_STEM,
1529        }
1530    }
1531}
1532
1533impl fmt::Display for LogRole {
1534    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
1535        f.write_str(match self {
1536            Self::Operator => "operator",
1537            Self::Service => "service",
1538        })
1539    }
1540}
1541
1542/// The sink could not be installed.
1543#[derive(Debug, thiserror::Error)]
1544pub enum LoggingError {
1545    /// The application-data directories could not be created.
1546    ///
1547    /// Carries a [`crate::paths::PathsError`] rather than a bare
1548    /// [`std::io::Error`] because [`install`] creates `logs/` through
1549    /// [`crate::paths::AppPaths::create_all`], which is what applies the `0700`
1550    /// restriction; the source therefore names whichever of the four
1551    /// directories actually failed, and that need not be `logs/`.
1552    #[error("cannot create the log directory {}: {source}", directory.display())]
1553    Directory {
1554        /// The diagnostics directory [`install`] was asked to write into.
1555        directory: PathBuf,
1556        /// The underlying error.
1557        #[source]
1558        source: crate::paths::PathsError,
1559    },
1560
1561    /// No diagnostics file in `logs/` could be opened for appending.
1562    ///
1563    /// Both stems are named because [`install`] tries two — the role's own, and
1564    /// then one qualified by the account — and an operator who is told only
1565    /// about the second would go looking for a file this process never reached
1566    /// for first.
1567    ///
1568    /// The two attempts are composed into one string rather than carried as
1569    /// four fields. As four this variant reached 128 bytes on Windows — where
1570    /// `PathBuf` is wider than on the Unixes — and tripped
1571    /// `clippy::result_large_err`, which is a lint about every *success* path
1572    /// paying for the size of a failure. The message is the same either way,
1573    /// and only this error is ever built.
1574    #[error("cannot open a diagnostics file in {}: {attempts}", directory.display())]
1575    Appender {
1576        /// The diagnostics directory.
1577        directory: PathBuf,
1578        /// Both stems that were tried, and why each was refused.
1579        attempts: String,
1580    },
1581
1582    /// A global subscriber was already installed.
1583    #[error("a tracing subscriber is already installed for this process: {message}")]
1584    AlreadyInstalled {
1585        /// What `tracing_subscriber` said.
1586        message: String,
1587    },
1588}
1589
1590/// Keeps the background log-writing thread alive.
1591///
1592/// Dropping it flushes and stops that thread, so the value must be held for as
1593/// long as the program expects its diagnostics to be written. Losing it is a
1594/// silent, total loss of logs, which is why it is `#[must_use]`.
1595#[must_use = "dropping the guard stops the log writer and silently discards later diagnostics"]
1596#[derive(Debug)]
1597pub struct LoggingGuard {
1598    _worker: tracing_appender::non_blocking::WorkerGuard,
1599}
1600
1601/// Installs the redacting sink as this process's global subscriber, writing
1602/// daily-rotating files into `logs/`.
1603///
1604/// `role` decides the file, for the reason [`LogRole`] gives. `default_filter`
1605/// applies when `RUST_LOG` is unset or unparseable.
1606///
1607/// # Errors
1608///
1609/// [`LoggingError::Directory`], [`LoggingError::Appender`] and
1610/// [`LoggingError::AlreadyInstalled`]. **None of them is fatal to the caller**,
1611/// and the CLI treats all three as a warning: a `host show` that refused to
1612/// print a capacity because a log file could not be opened would be a worse
1613/// failure than the one it was reporting.
1614pub fn install(
1615    paths: &crate::paths::AppPaths,
1616    role: LogRole,
1617    default_filter: &str,
1618) -> Result<LoggingGuard, LoggingError> {
1619    use tracing_subscriber::layer::SubscriberExt as _;
1620    use tracing_subscriber::util::SubscriberInitExt as _;
1621
1622    // `AppPaths::create_all`, not `create_dir_all`. The claim `create_all`
1623    // documents — that a diagnostics file is not readable by other local
1624    // accounts — is only true if `logs/` is created at `0700`, and this is the
1625    // path a running daemon actually takes. Creating the directory here with a
1626    // bare `create_dir_all` left it at the umask default whenever `install`
1627    // won the race to create it, and `tracing_appender` then wrote 0644 files
1628    // into it: the invariant held in the test that asserts it and nowhere else.
1629    let directory = paths.logs_dir().to_path_buf();
1630    paths
1631        .create_all()
1632        .map_err(|source| LoggingError::Directory {
1633            directory: directory.clone(),
1634            source,
1635        })?;
1636
1637    let appender = open_appender(&directory, role)?;
1638    let (writer, worker) = tracing_appender::non_blocking(appender);
1639
1640    let filter = tracing_subscriber::EnvFilter::try_from_default_env()
1641        .unwrap_or_else(|_| tracing_subscriber::EnvFilter::new(default_filter));
1642
1643    tracing_subscriber::registry()
1644        .with(filter)
1645        .with(RedactingLayer::new(writer))
1646        .try_init()
1647        .map_err(|error| LoggingError::AlreadyInstalled {
1648            message: error.to_string(),
1649        })?;
1650
1651    Ok(LoggingGuard { _worker: worker })
1652}
1653
1654/// Opens today's diagnostics file, falling back to one only this account can
1655/// have created.
1656///
1657/// # Why there is a second attempt at all
1658///
1659/// [`LogRole`] keeps the daemon and the operator apart, which is enough on a
1660/// host where nothing else ever writes here. Two things still put a file the
1661/// caller cannot append to under the stem it wants:
1662///
1663/// 1. **A privileged command.** Signing in to the machine-scoped store is
1664///    `sudo runner-manager auth login`, and that run writes the *operator's*
1665///    stem as `root`. Every unprivileged command for the rest of that day then
1666///    finds a `root`-owned file under it.
1667/// 2. **A host upgraded into this change.** Files written before the roles were
1668///    split are all under [`OPERATOR_LOG_STEM`], and on a boot-mode host the
1669///    ones from the last few days belong to `root`.
1670///
1671/// Neither is worth losing diagnostics over, and neither can be repaired from
1672/// inside an unprivileged process — it may not chown the file, may not change
1673/// its mode, and must not delete an existing log. So it writes beside it, under
1674/// a stem carrying this account's identity, which no other account will choose.
1675///
1676/// The fallback is deliberately *not* the first choice. A file named after
1677/// whoever happened to open it first is a file an operator has to go looking
1678/// for; the plain stem stays the plain stem, and the qualified one appears only
1679/// on a host that has the collision.
1680fn open_appender(
1681    directory: &Path,
1682    role: LogRole,
1683) -> Result<tracing_appender::rolling::RollingFileAppender, LoggingError> {
1684    let stem = role.file_stem();
1685    let first = match build_appender(directory, stem) {
1686        Ok(appender) => return Ok(appender),
1687        Err(error) => error,
1688    };
1689
1690    let qualified = account_qualified_stem(stem);
1691    match build_appender(directory, &qualified) {
1692        Ok(appender) => Ok(appender),
1693        // Both stems are named. An operator told only about the second would go
1694        // looking for a file this process never reached for first.
1695        Err(second) => Err(LoggingError::Appender {
1696            directory: directory.to_path_buf(),
1697            attempts: format!(
1698                "neither {stem}.<date> ({first}) nor {qualified}.<date> ({second}) could be \
1699                 appended to"
1700            ),
1701        }),
1702    }
1703}
1704
1705/// One attempt, through the constructor that **returns** its failure.
1706///
1707/// `tracing_appender::rolling::daily` is the same thing with an `.expect` on
1708/// the end, and that `.expect` is a panic in the middle of an operator's `status`
1709/// command. The whole reason this function exists is that the builder hands the
1710/// error back instead.
1711fn build_appender(
1712    directory: &Path,
1713    stem: &str,
1714) -> Result<tracing_appender::rolling::RollingFileAppender, String> {
1715    tracing_appender::rolling::RollingFileAppender::builder()
1716        .rotation(tracing_appender::rolling::Rotation::DAILY)
1717        .filename_prefix(stem)
1718        .build(directory)
1719        .map_err(|error| error.to_string())
1720}
1721
1722/// `runner-manager.log` becomes `runner-manager.<account>.log`.
1723///
1724/// The stem carries its own `.log`, so the account goes before it rather than
1725/// after: the appender appends `.<date>`, and `runner-manager.log.uid-501` would
1726/// read as a rotated file rather than as another account's.
1727fn account_qualified_stem(stem: &str) -> String {
1728    let account = account_tag();
1729    match stem.rsplit_once('.') {
1730        Some((head, tail)) => format!("{head}.{account}.{tail}"),
1731        None => format!("{stem}.{account}"),
1732    }
1733}
1734
1735/// Something stable, filename-safe, and different for every local account.
1736///
1737/// The numeric effective user id rather than a name: it needs no lookup, cannot
1738/// contain a path separator, and is the identity the filesystem actually
1739/// compared when it refused the open.
1740#[cfg(unix)]
1741fn account_tag() -> String {
1742    // SAFETY: `geteuid` takes no argument, touches no memory this process owns,
1743    // and is documented never to fail.
1744    let uid = unsafe { libc::geteuid() };
1745    format!("uid-{uid}")
1746}
1747
1748/// As the Unix half, from the one identity Windows exposes without a lookup.
1749///
1750/// Sanitized rather than trusted: this becomes a file name, and `USERNAME` is
1751/// an ordinary environment variable that a caller may set to anything at all,
1752/// including something holding a path separator.
1753#[cfg(windows)]
1754fn account_tag() -> String {
1755    let raw = std::env::var("USERNAME").unwrap_or_default();
1756    let safe: String = raw
1757        .chars()
1758        .filter(|c| c.is_ascii_alphanumeric() || *c == '-' || *c == '_')
1759        .take(32)
1760        .collect();
1761    if safe.is_empty() {
1762        "other-account".to_string()
1763    } else {
1764        format!("user-{safe}")
1765    }
1766}
1767
1768#[cfg(test)]
1769mod tests {
1770    use super::*;
1771
1772    use std::sync::{Arc, Mutex};
1773
1774    use tracing_subscriber::layer::SubscriberExt as _;
1775
1776    // -----------------------------------------------------------------------
1777    // Capturing what the sink actually wrote
1778    // -----------------------------------------------------------------------
1779
1780    #[derive(Clone, Default)]
1781    struct Capture(Arc<Mutex<Vec<u8>>>);
1782
1783    impl Capture {
1784        fn text(&self) -> String {
1785            String::from_utf8_lossy(&self.0.lock().expect("not poisoned")).into_owned()
1786        }
1787    }
1788
1789    struct CaptureWriter(Arc<Mutex<Vec<u8>>>);
1790
1791    impl Write for CaptureWriter {
1792        fn write(&mut self, buf: &[u8]) -> std::io::Result<usize> {
1793            self.0.lock().expect("not poisoned").extend_from_slice(buf);
1794            Ok(buf.len())
1795        }
1796
1797        fn flush(&mut self) -> std::io::Result<()> {
1798            Ok(())
1799        }
1800    }
1801
1802    impl<'a> tracing_subscriber::fmt::MakeWriter<'a> for Capture {
1803        type Writer = CaptureWriter;
1804
1805        fn make_writer(&'a self) -> Self::Writer {
1806            CaptureWriter(Arc::clone(&self.0))
1807        }
1808    }
1809
1810    /// A sink with the same plumbing and no redaction at all.
1811    ///
1812    /// Exists so that `the_scan_catches_a_sink_that_does_not_redact` can prove
1813    /// the secret-injection scan below is capable of failing. Without it, a scan
1814    /// that found nothing would be indistinguishable from a scan that looked
1815    /// nowhere.
1816    #[derive(Debug, Clone)]
1817    struct PassthroughLayer<W>(W);
1818
1819    #[derive(Default)]
1820    struct PassthroughVisitor(Map<String, Value>);
1821
1822    impl Visit for PassthroughVisitor {
1823        fn record_str(&mut self, field: &Field, value: &str) {
1824            self.0
1825                .insert(field.name().to_string(), Value::String(value.to_string()));
1826        }
1827
1828        fn record_debug(&mut self, field: &Field, value: &dyn fmt::Debug) {
1829            self.0.insert(
1830                field.name().to_string(),
1831                Value::String(format!("{value:?}")),
1832            );
1833        }
1834    }
1835
1836    impl<S, W> Layer<S> for PassthroughLayer<W>
1837    where
1838        S: Subscriber + for<'a> LookupSpan<'a>,
1839        W: for<'a> tracing_subscriber::fmt::MakeWriter<'a> + 'static,
1840    {
1841        fn on_event(&self, event: &Event<'_>, _ctx: Context<'_, S>) {
1842            let mut visitor = PassthroughVisitor::default();
1843            event.record(&mut visitor);
1844            let mut line = serde_json::to_string(&Value::Object(visitor.0)).unwrap_or_default();
1845            line.push('\n');
1846            let _ = self.0.make_writer().write_all(line.as_bytes());
1847        }
1848    }
1849
1850    // -----------------------------------------------------------------------
1851    // The secret-injection log scan (`07-security.md`, security gate)
1852    // -----------------------------------------------------------------------
1853
1854    /// A user access token, in GitHub's user-to-server format.
1855    const USER_TOKEN: &str = "ghu_16C7e42F292c6912E7710c838347Ae178B4a";
1856    /// An installation-style token, to prove the prefix rule is not one-off.
1857    const SERVER_TOKEN: &str = "ghs_1CGGYnBAtn5ov3M0aTHhP7l3ZKuMhIB3pnPd";
1858    /// A fine-grained personal access token.
1859    const FINE_GRAINED: &str =
1860        "github_pat_11ABCDEFG0abcdefghijkl_ZYXWVUTSRQPONMLKJIHGFEDCBA9876543210zyxwv";
1861    /// Stands in for an encoded JIT configuration: base64, and long.
1862    const JIT_BLOB: &str = "eyJhZ2VudE5hbWUiOiJydW5uZXItbWFuYWdlciIsImVuY29kZWQiOiJhYmNkZWZnaGlqa2xtbm9wcXJzdHV2d3h5ejAxMjM0NTY3ODkrLz09In0=";
1863    /// A workspace path, which `07-security.md` also requires redacted.
1864    const WORKSPACE: &str = "/var/lib/runner-manager/runtime/9f2c/attempt-1";
1865    /// The same, on the platform the persona is least likely to be using but
1866    /// the CI matrix definitely is.
1867    const WINDOWS_WORKSPACE: &str = r"C:\Users\operator\AppData\Local\runner-manager\runtime\9f2c";
1868    /// A GitHub App installation assertion: a JSON Web Token.
1869    ///
1870    /// Its two `.` separators are the point. `is_opaque_char` excludes `.`, so
1871    /// the opaque-run rule sees three short runs rather than one long one, and
1872    /// every other secret here is caught by that rule when it stands alone.
1873    const JWT: &str = "eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.\
1874                       eyJpc3MiOiIxMjM0NTYiLCJpYXQiOjE3MDAwMDAwMDAsImV4cCI6MTcwMDAwMDYwMH0.\
1875                       c2lnbmF0dXJlLXRoYXQtaXMtb3BhcXVlLWFuZC1sb25nLWVub3VnaC10by1tYXR0ZXI";
1876
1877    /// An error whose `Debug` carries the HTTP body it was given.
1878    ///
1879    /// This is the shape `d2`'s secret store and `d3`'s installer will hand to
1880    /// `tracing::error!(reason = ?err)`, and it is not the same shape as any of
1881    /// the JSON above: `Debug` on a `String` escapes the quotes inside it, so
1882    /// the keys arrive spelled `\"password\"` rather than `"password"`.
1883    #[derive(Debug)]
1884    struct StoreError {
1885        body: String,
1886    }
1887
1888    fn secrets() -> Vec<&'static str> {
1889        vec![
1890            USER_TOKEN,
1891            SERVER_TOKEN,
1892            FINE_GRAINED,
1893            JIT_BLOB,
1894            JWT,
1895            WORKSPACE,
1896            WINDOWS_WORKSPACE,
1897        ]
1898    }
1899
1900    /// Routes every secret through `sink` in every shape a caller might use,
1901    /// and reports the first one that came out the other side.
1902    ///
1903    /// A helper returning `Result` rather than a test body, because the same
1904    /// injection has to be run against a deliberately non-redacting sink to
1905    /// show that it is capable of finding anything at all.
1906    fn scan_for_leaks<L>(layer: L, capture: &Capture) -> Result<String, String>
1907    where
1908        L: Layer<tracing_subscriber::Registry> + Send + Sync + 'static,
1909    {
1910        let subscriber = tracing_subscriber::registry().with(layer);
1911
1912        tracing::subscriber::with_default(subscriber, || {
1913            for secret in secrets() {
1914                // 1. In the message body, which is the one field that must be
1915                //    allowed and therefore cannot be protected by the allowlist.
1916                tracing::info!("starting runner with {secret}");
1917
1918                // 2. As a credential header in the message body, scheme and all.
1919                tracing::info!("request failed; Authorization: Bearer {secret}");
1920                tracing::warn!("retrying with x-api-key={secret}");
1921
1922                // 3. In a field nobody listed — the case that must be safe with
1923                //    no edit to this file.
1924                tracing::info!(runner_token = %secret, "registered");
1925
1926                // 4. In a field that *is* listed, which is where the allowlist
1927                //    alone would not save anything.
1928                tracing::info!(outcome = %secret, event = "started", "registered");
1929                tracing::info!(message = %secret, "ignored");
1930
1931                // 5. Through `Debug` rather than `Display`.
1932                tracing::info!(?secret, "debug shaped");
1933
1934                // 6. Embedded in a structured value rather than standing
1935                //    alone. Every shape above presents the secret as its
1936                //    own whitespace-delimited word, which is the one shape
1937                //    the opaque-run rule catches for free -- so the scan
1938                //    could pass while the sink leaked anything embedded.
1939                //    Compact JSON is what `serde_json::to_string` emits and
1940                //    what an HTTP error body arrives as, and it was emitted
1941                //    verbatim: `redact_core` recursed into the value for
1942                //    `=` only, and the JSON quote left the key spelled
1943                //    `encoded_jit_config"`, which is not on CREDENTIAL_KEYS.
1944                for key in ["encoded_jit_config", "runner_token", "pat"] {
1945                    tracing::error!("registration failed: {{\"{key}\":\"{secret}\"}}");
1946                    tracing::error!("registration failed: {{ \"{key}\": \"{secret}\" }}");
1947                    tracing::error!("registration failed: {key}:{secret}");
1948                }
1949
1950                // 7. An `Authorization` header value, in the same three
1951                //    embedded shapes.
1952                tracing::error!("response: {{\"authorization\":\"Bearer {secret}\"}}");
1953                tracing::error!("response: {{ \"authorization\": \"Bearer {secret}\" }}");
1954                tracing::error!("response: authorization:Bearer {secret}");
1955
1956                // 9. A compact object with **more than one field**, the
1957                //    credential key second. Shape 6 above differs from this by
1958                //    field order and by nothing else, and `serde_json::to_string`
1959                //    is what chooses the order -- so shape 6 could pass while an
1960                //    error body from a struct with two fields leaked. The rules
1961                //    split a word once, on the first separator they find, so
1962                //    only `runner_id` was ever examined.
1963                for key in ["encoded_jit_config", "runner_token", "access_token"] {
1964                    tracing::error!(
1965                        "registration failed: {{\"runner_id\":42,\"{key}\":\"{secret}\"}}"
1966                    );
1967                    tracing::error!(
1968                        "registration failed: {{\"status\":422,\"message\":\"bad\",\"{key}\":\"{secret}\"}}"
1969                    );
1970                    // The spaced spelling of the same thing, which was already
1971                    // safe -- by accident, because the value is its own word.
1972                    tracing::error!(
1973                        "registration failed: {{ \"runner_id\": 42, \"{key}\": \"{secret}\" }}"
1974                    );
1975                }
1976
1977                // 10. Nested, one level and two. The value recursion goes to
1978                //     `redact_value`, which is a leaf and never re-enters
1979                //     `redact_core`, so an object inside an object was emitted
1980                //     whole.
1981                tracing::error!("response: {{\"body\":{{\"runner_token\":\"{secret}\"}}}}");
1982                tracing::error!(
1983                    "response: {{\"error\":{{\"status\":422,\"body\":{{\"encoded_jit_config\":\"{secret}\"}}}}}}"
1984                );
1985
1986                // 11. A form-encoded body with the credential parameter second.
1987                //     `&` was not a separator this module knew, so the whole
1988                //     body was one word and only `scope` was ever judged.
1989                tracing::error!("token exchange failed: scope=repo&access_token={secret}");
1990                tracing::error!(
1991                    "token exchange failed: grant_type=refresh&refresh_token={secret}&scope=repo"
1992                );
1993
1994                // 12. Backslash-escaped JSON, through `Debug` rather than
1995                //     `Display`. Shape 5 is also `Debug`, but it puts the
1996                //     secret on an *unlisted* field, which is dropped wholesale
1997                //     -- so the `Debug` lens and the compact-JSON lens were both
1998                //     here and had never been crossed. `reason` is allowed, so
1999                //     this one reaches the scrubber, with its keys spelled
2000                //     `\"runner_token\"`.
2001                let failure = StoreError {
2002                    body: format!("{{\"runner_token\":\"{secret}\"}}"),
2003                };
2004                tracing::error!(reason = ?failure, "the secret store rejected the request");
2005
2006                // 13. A URL earlier in the same word. `redact_core` split on the
2007                //     first `://` and handed everything before it to
2008                //     `redact_url` as a "scheme" and everything after it as
2009                //     the URL -- and the terminal arm of `redact_url` echoes
2010                //     both verbatim. So one URL anywhere in a word put the
2011                //     whole of the rest of that word beyond every rule below.
2012                //     `documentation_url` is in essentially every GitHub REST
2013                //     error body, which makes that "any error body logged
2014                //     alongside a credential".
2015                tracing::error!(
2016                    "api failed: {{\"message\":\"Bad credentials\",\"documentation_url\":\"https://docs.github.com/rest\",\"token\":\"{secret}\"}}"
2017                );
2018                // The reverse order leaked for the mirror reason: everything
2019                // before the `://` became the scheme, and was echoed.
2020                tracing::error!(
2021                    "api failed: {{\"token\":\"{secret}\",\"documentation_url\":\"https://docs.github.com/rest\"}}"
2022                );
2023                // A nested object behind a URL, which is the shape a clone
2024                // failure arrives in.
2025                tracing::error!(
2026                    "clone failed: {{\"remote\":\"https://github.com/o/r.git\",\"body\":{{\"password\":\"{secret}\"}}}}"
2027                );
2028
2029                // 14. A value with no key of its own: an array element. The
2030                //     structural cut recursed on the *raw* fragment, and the
2031                //     terminal fallback then handed it to `redact_value` with
2032                //     its quote still attached -- so `"ghu_…` failed
2033                //     `starts_with("ghu_")`, failed `is_opaque_char` on the
2034                //     quote, failed the `eyJ` test, and failed
2035                //     `looks_like_path`. This is shape 9's defect one step
2036                //     down: "only the first pair is examined" became "a value
2037                //     with no key of its own is never examined", and it
2038                //     survived because there was no array among the twelve
2039                //     shapes above.
2040                tracing::error!("registration failed: {{\"tokens\":[\"{secret}\",\"x\"]}}");
2041                tracing::error!("registration failed: {{\"tokens\":[\"x\",\"{secret}\"]}}");
2042                tracing::error!("registration failed: [\"x\",\"{secret}\"]");
2043                tracing::error!("registration failed: [{{\"id\":1}},[\"{secret}\"]]");
2044                let listed = StoreError {
2045                    body: format!("{{\"tokens\":[\"x\",\"{secret}\"]}}"),
2046                };
2047                tracing::error!(reason = ?listed, "the secret store rejected the list");
2048
2049                // 15. `;` as the separator. It is in `WRAPPERS`, so it was
2050                //     recognised as punctuation, and it was not in
2051                //     `STRUCTURAL`, so it never cut a word. It is what
2052                //     separates the pairs of a Windows connection string, of a
2053                //     credential string, and of a cookie header written
2054                //     without a space -- `d2`'s territory exactly.
2055                tracing::error!("store rejected: Server=host;Database=x;Password={secret};");
2056                tracing::error!("store rejected: user=operator;password={secret}");
2057                tracing::error!("store rejected: theme=dark;session={secret}");
2058
2059                // 16. Wrapped in an element rather than in a quote.
2060                //     `split_wrappers` strips only the *outermost* `<` and
2061                //     `>`, so `<string>…</string>` arrived as
2062                //     `string>…</string`, which matches no rule at all. `d3`'s
2063                //     installers handle launchd plists, which is where this
2064                //     shape comes from.
2065                tracing::error!("plist rejected: <string>{secret}</string>");
2066                tracing::error!("plist rejected: <key>token</key><string>{secret}</string>");
2067
2068                // 17. A `;` or an `&` *after a URL in the same word*. This is
2069                //     shape 15 and shape 11 with one thing changed: a URL
2070                //     earlier in the word. `STRUCTURAL` has nine characters and
2071                //     `is_url_terminator` recognised seven of them -- `;` and
2072                //     `&` were missing -- and the URL branch runs *ahead* of
2073                //     the structural cut. So `split_url` swallowed everything
2074                //     to the next terminator, including the separator that
2075                //     would have cut the word, and `redact_url`'s terminal arm
2076                //     echoed it verbatim.
2077                //
2078                //     The L3 fix (bounding the URL) and the L1 fix (cutting on
2079                //     `;` and `&`) each work alone and did not compose: this
2080                //     commit added `;` to `STRUCTURAL` *for connection strings*
2081                //     while leaving the path a URL opens straight through it.
2082                //     A connection string whose `Server=` is a URL is exactly
2083                //     `d2`'s shape, and a systemd `Environment=` line is
2084                //     `d3`'s.
2085                tracing::error!(
2086                    "store rejected: Server=https://vault.local/api;Password={secret};"
2087                );
2088                tracing::error!("keychain error: url=https://kc.local;secret={secret}");
2089                tracing::error!("unit rejected: Environment=API=https://a.com/v1;TOKEN={secret}");
2090                tracing::error!("token exchange failed: cb=https://a.com/x&access_token={secret}");
2091                tracing::error!(
2092                    "dsn rejected: dsn=https://sentry.local/1;password={secret};user=x"
2093                );
2094                tracing::error!(
2095                    "callback failed: redirect=https://a.com/cb&state=1&access_token={secret}"
2096                );
2097                let routed = StoreError {
2098                    body: format!("Server=https://vault.local/api;Password={secret};"),
2099                };
2100                tracing::error!(reason = ?routed, "the secret store rejected the connection string");
2101
2102                // 18. A credential key whose value is not in its own fragment.
2103                //     The empty-value skip is right, but its justification --
2104                //     "the value is in the next fragment, where the structural
2105                //     cut reaches it" -- was false: `redact_fragment` carried
2106                //     nothing across the cut, so nothing ever reached it. An
2107                //     array, a nested object, an empty pair and a plist
2108                //     key/value pair are four spellings of the same gap.
2109                tracing::error!("store rejected: {{\"password\":[\"{secret}\"]}}");
2110                tracing::error!("store rejected: {{\"password\":{{\"v\":\"{secret}\"}}}}");
2111                tracing::error!("store rejected: Password=;{secret}");
2112                tracing::error!("plist rejected: <key>password</key><string>{secret}</string>");
2113
2114                // 19. A structural character *inside* a credential value. `,`
2115                //     and `&` could already split a secret; this commit added
2116                //     `;`, `<` and `>`, so each of those became a new character
2117                //     that can cut a punctuated password in half and leave the
2118                //     tail standing.
2119                tracing::error!("store rejected: {{\"password\":\"a<{secret}>b\"}}");
2120                tracing::error!("store rejected: {{\"password\":\"a&{secret},b\"}}");
2121
2122                // 20. A secret in a URL *path*. `redact_url` applies no shape
2123                //     rule to the path at all, so a token that reaches one is
2124                //     echoed whole -- and the token-prefix rule is documented
2125                //     as a belt that catches a credential anywhere.
2126                //
2127                //     The two workspace paths are excluded, and the exclusion
2128                //     is a real limitation rather than a convenience. A path is
2129                //     judged one segment at a time, because judging the whole
2130                //     of a URL path with `looks_like_path` would redact every
2131                //     URL with two segments in it -- which is the
2132                //     over-redaction the module documentation explicitly
2133                //     promises a full URL escapes. A filesystem path pasted
2134                //     into a URL path is therefore indistinguishable from an
2135                //     ordinary deep URL path: `/var/lib/…` and `/repos/o/r/…`
2136                //     are the same shape, segment by segment. A *credential* in
2137                //     a path is caught, because a credential has a shape of its
2138                //     own; a path in a path is not.
2139                if !looks_like_path(secret) {
2140                    tracing::error!("download failed: https://github.com/o/r/raw/{secret}/f");
2141                    tracing::error!("download failed: https://github.com/o/r/raw/{secret}");
2142                }
2143
2144                // 8. Carried on a span rather than on the event.
2145                let span = tracing::info_span!("attempt", jit_config = %secret);
2146                let _entered = span.enter();
2147                tracing::info!(event = "inside_span", "in a span");
2148            }
2149        });
2150
2151        let output = capture.text();
2152        for secret in secrets() {
2153            for needle in needles(secret) {
2154                if output.contains(&needle) {
2155                    return Err(format!(
2156                        "the sink emitted a secret verbatim: {secret}\n\
2157                         (found as: {needle})\n--- output ---\n{output}"
2158                    ));
2159                }
2160            }
2161        }
2162        Ok(output)
2163    }
2164
2165    /// Every spelling a secret can have in the sink's output.
2166    ///
2167    /// The sink writes JSON, so a secret containing a character `serde_json`
2168    /// escapes never appears in the output as it was written. `WINDOWS_WORKSPACE`
2169    /// is a raw string with single backslashes and is emitted with doubled ones,
2170    /// so scanning for the literal could not have matched it — not even against
2171    /// `PassthroughLayer`, which redacts nothing at all. The path really is
2172    /// redacted, so this was a hole in the coverage rather than a leak, but a
2173    /// needle that cannot match is a check that cannot fail.
2174    fn needles(secret: &str) -> Vec<String> {
2175        // Strip the quotes `to_string` adds; what is left is the text exactly
2176        // as it appears inside a JSON string literal.
2177        let escape = |text: &str| {
2178            let json = serde_json::to_string(text).expect("a string is serialisable");
2179            json[1..json.len() - 1].to_string()
2180        };
2181
2182        let once = escape(secret);
2183        // Twice, because a secret can be escaped twice on the way out, and
2184        // shape 12 is where that happens: `Debug` on a `String` escapes the
2185        // quotes *and* the backslashes inside it, and the sink then
2186        // JSON-encodes what `Debug` produced. A Windows path arrives in that
2187        // line with four backslashes where the secret has one, so the
2188        // once-escaped needle cannot match it there -- not even against
2189        // `PassthroughLayer`, which redacts nothing at all. That cell of the
2190        // matrix was a check incapable of failing, which is the same defect
2191        // this helper was written to fix one level down.
2192        let twice = escape(&once);
2193
2194        let mut spellings = vec![secret.to_string(), once, twice];
2195        // Consecutive-only is enough: each level of escaping is a superset of
2196        // the last, so equal spellings are always adjacent.
2197        spellings.dedup();
2198        spellings
2199    }
2200
2201    #[test]
2202    fn the_scan_looks_for_a_needle_that_can_actually_occur() {
2203        // Guards the helper above: if `needles` ever stops producing the
2204        // JSON-escaped spelling, the Windows workspace silently stops being
2205        // scanned for and every assertion about it becomes vacuous.
2206        let windows = needles(WINDOWS_WORKSPACE);
2207        assert_eq!(
2208            windows.len(),
2209            3,
2210            "a backslash path has three spellings, one per level of escaping it \
2211             can pass through on the way out: {windows:?}"
2212        );
2213        assert!(
2214            windows[1].contains(r"\\Users\\operator"),
2215            "the once-escaped spelling is what an ordinary JSON line contains: {windows:?}"
2216        );
2217        assert!(
2218            windows[2].contains(r"\\\\Users\\\\operator"),
2219            "the twice-escaped spelling is what a Debug rendering inside a JSON \
2220             line contains: {windows:?}"
2221        );
2222
2223        // The third spelling is not hypothetical: it is exactly what the sink
2224        // writes for shape 12, and without it that cell of the scan was a
2225        // check that could not fail.
2226        let shape_twelve = serde_json::to_string(&Value::String(format!(
2227            "{:?}",
2228            StoreError {
2229                body: format!("{{\"runner_token\":\"{WINDOWS_WORKSPACE}\"}}"),
2230            }
2231        )))
2232        .expect("serialisable");
2233        assert!(
2234            shape_twelve.contains(&windows[2]),
2235            "the twice-escaped needle must be findable in what shape 12 emits: {shape_twelve}"
2236        );
2237        assert!(
2238            !shape_twelve.contains(&windows[1]),
2239            "and the once-escaped one must not be, or the gap was never there: {shape_twelve}"
2240        );
2241
2242        // Confirms the premise: the raw spelling genuinely cannot occur in the
2243        // sink's output, which is why scanning only for it proved nothing.
2244        let rendered = serde_json::to_string(&Value::String(WINDOWS_WORKSPACE.to_string()))
2245            .expect("serialisable");
2246        assert!(
2247            !rendered.contains(WINDOWS_WORKSPACE),
2248            "if this ever contains the raw path, the original scan was fine after all: {rendered}"
2249        );
2250        assert!(rendered.contains(&windows[1]), "{rendered}");
2251
2252        // A secret with nothing to escape has exactly one spelling.
2253        assert_eq!(needles(USER_TOKEN), vec![USER_TOKEN.to_string()]);
2254    }
2255
2256    #[test]
2257    fn the_secret_injection_scan_finds_nothing() {
2258        let capture = Capture::default();
2259        let output = scan_for_leaks(RedactingLayer::new(capture.clone()), &capture)
2260            .unwrap_or_else(|complaint| panic!("{complaint}"));
2261
2262        // Something must have been written, or "no secrets found" is trivially
2263        // true and means nothing.
2264        assert!(!output.trim().is_empty(), "the sink wrote nothing at all");
2265        assert!(
2266            output.contains(REDACTION),
2267            "nothing was redacted, so nothing was routed through the sink:\n{output}"
2268        );
2269    }
2270
2271    #[test]
2272    fn the_scan_catches_a_sink_that_does_not_redact() {
2273        let capture = Capture::default();
2274        let complaint = scan_for_leaks(PassthroughLayer(capture.clone()), &capture)
2275            .expect_err("a sink with no redaction must be caught");
2276        assert!(
2277            complaint.contains("emitted a secret verbatim"),
2278            "the complaint must name the failure mode: {complaint}"
2279        );
2280    }
2281
2282    // -----------------------------------------------------------------------
2283    // The allowlist
2284    // -----------------------------------------------------------------------
2285
2286    fn emit(capture: &Capture, body: impl FnOnce()) -> String {
2287        let subscriber = tracing_subscriber::registry().with(RedactingLayer::new(capture.clone()));
2288        tracing::subscriber::with_default(subscriber, body);
2289        capture.text()
2290    }
2291
2292    #[test]
2293    fn a_field_nobody_listed_is_redacted_by_default() {
2294        let capture = Capture::default();
2295        let output = emit(&capture, || {
2296            tracing::info!(
2297                a_field_added_by_a_later_task = "supersecret-value",
2298                "an event"
2299            );
2300        });
2301
2302        assert!(
2303            !output.contains("supersecret-value"),
2304            "an unlisted field leaked its value:\n{output}"
2305        );
2306        assert!(output.contains(REDACTION), "{output}");
2307        // The name survives: knowing the field was there is useful, and a field
2308        // name is not a credential.
2309        assert!(
2310            output.contains("a_field_added_by_a_later_task"),
2311            "the field name should be kept:\n{output}"
2312        );
2313    }
2314
2315    #[test]
2316    fn an_unlisted_numeric_field_is_redacted_too() {
2317        // The rule has no exception for a type, because "a credential is never
2318        // an integer" is exactly the kind of reasoning that stops being true
2319        // once somebody logs an installation-scoped identifier they should not.
2320        let capture = Capture::default();
2321        let output = emit(&capture, || {
2322            tracing::info!(unlisted_number = 8_675_309_u64, "an event");
2323        });
2324        assert!(!output.contains("8675309"), "{output}");
2325        assert!(output.contains(REDACTION), "{output}");
2326    }
2327
2328    #[test]
2329    fn listed_fields_survive() {
2330        let capture = Capture::default();
2331        let output = emit(&capture, || {
2332            tracing::info!(
2333                event = "reconciled",
2334                policy_id = "9f2c1a44-0000-4000-8000-000000000001",
2335                count = 3,
2336                outcome = "started",
2337                "reconciliation finished"
2338            );
2339        });
2340
2341        for expected in [
2342            "reconciled",
2343            "9f2c1a44-0000-4000-8000-000000000001",
2344            "started",
2345            "reconciliation finished",
2346        ] {
2347            assert!(
2348                output.contains(expected),
2349                "{expected} missing from:\n{output}"
2350            );
2351        }
2352        assert!(output.contains("\"count\":3"), "{output}");
2353    }
2354
2355    #[test]
2356    fn the_allowlist_is_sorted_and_has_no_duplicates() {
2357        // `is_field_allowed` uses a binary search, so an unsorted list would not
2358        // merely be untidy — it would silently start redacting fields that are
2359        // on it.
2360        let mut sorted = ALLOWED_FIELDS.to_vec();
2361        sorted.sort_unstable();
2362        assert_eq!(
2363            ALLOWED_FIELDS,
2364            &sorted[..],
2365            "ALLOWED_FIELDS must stay sorted"
2366        );
2367
2368        let mut deduped = sorted.clone();
2369        deduped.dedup();
2370        assert_eq!(sorted.len(), deduped.len(), "ALLOWED_FIELDS has duplicates");
2371
2372        for name in ALLOWED_FIELDS {
2373            assert!(is_field_allowed(name), "{name} is listed but not allowed");
2374        }
2375        assert!(!is_field_allowed("authorization"));
2376        assert!(!is_field_allowed("runner_token"));
2377    }
2378
2379    #[test]
2380    fn every_record_is_one_parseable_json_object_per_line() {
2381        let capture = Capture::default();
2382        let output = emit(&capture, || {
2383            tracing::info!(event = "one", "first");
2384            tracing::warn!(event = "two", "second");
2385        });
2386
2387        let lines: Vec<&str> = output
2388            .lines()
2389            .filter(|line| !line.trim().is_empty())
2390            .collect();
2391        assert_eq!(lines.len(), 2, "{output}");
2392        for line in lines {
2393            let value: Value = serde_json::from_str(line)
2394                .unwrap_or_else(|error| panic!("not JSON: {error}\n{line}"));
2395            assert!(value.get("timestamp").is_some(), "{line}");
2396            assert!(value.get("level").is_some(), "{line}");
2397            assert!(value.get("logger").is_some(), "{line}");
2398            assert!(value.get("fields").is_some(), "{line}");
2399        }
2400    }
2401
2402    #[test]
2403    fn span_fields_are_redacted_and_carried() {
2404        let capture = Capture::default();
2405        let output = emit(&capture, || {
2406            let span = tracing::info_span!("attempt", attempt_id = "abc-123", jit = %JIT_BLOB);
2407            let _entered = span.enter();
2408            tracing::info!(event = "inside", "in the span");
2409        });
2410
2411        assert!(output.contains("\"name\":\"attempt\""), "{output}");
2412        assert!(
2413            output.contains("abc-123"),
2414            "the listed span field survives:\n{output}"
2415        );
2416        assert!(!output.contains(JIT_BLOB), "a span field leaked:\n{output}");
2417    }
2418
2419    /// Asserts that [`install`] created its directories the restrictive way.
2420    ///
2421    /// `install` used to call `std::fs::create_dir_all(logs_dir)` directly.
2422    /// That is invisible on Windows and *nearly* invisible on Unix — the
2423    /// directory exists either way, and `AppPaths::create_all`'s own test still
2424    /// passed, because it tests `create_all` rather than the path a running
2425    /// daemon actually takes. So this checks the two things that distinguish
2426    /// them:
2427    ///
2428    /// 1. **All four directories exist.** `create_dir_all(logs_dir)` makes
2429    ///    exactly one. This half runs on every platform, which matters because
2430    ///    Windows is the leg most likely to be run locally.
2431    /// 2. **On Unix the mode is `0700`.** This is the invariant `create_all`
2432    ///    documents — that a diagnostics file is not readable by other local
2433    ///    accounts — and `tracing_appender` writes 0644 files into whatever
2434    ///    directory it is given, so a `0755` `logs/` defeats it entirely.
2435    fn assert_install_created_restricted_directories(paths: &crate::paths::AppPaths) {
2436        for (purpose, path) in paths.all() {
2437            assert!(
2438                path.is_dir(),
2439                "install must create the {purpose} directory too: going through \
2440                 AppPaths::create_all is what applies the restriction, and creating only \
2441                 logs/ is the bug this asserts against ({})",
2442                path.display()
2443            );
2444
2445            #[cfg(unix)]
2446            {
2447                use std::os::unix::fs::PermissionsExt as _;
2448
2449                let mode = std::fs::metadata(path)
2450                    .expect("the directory exists")
2451                    .permissions()
2452                    .mode()
2453                    & 0o777;
2454                assert_eq!(
2455                    mode, 0o700,
2456                    "the {purpose} directory is mode {mode:04o}; a diagnostics file under a \
2457                     group- or world-readable directory is readable by other local accounts"
2458                );
2459            }
2460        }
2461    }
2462
2463    /// The one test that exercises [`install`] rather than assembling a
2464    /// subscriber by hand: the rolling file appender, the non-blocking writer,
2465    /// and the guard that has to be held for any of it to reach disk.
2466    ///
2467    /// It installs a *global* subscriber, which a process can only do once, so
2468    /// it is the only test here that may. Everything else uses
2469    /// `with_default`, which is thread-local and does not conflict — including
2470    /// on threads that run after this one.
2471    #[test]
2472    #[serial_test::serial(global_subscriber)]
2473    fn install_writes_redacted_json_into_the_logs_directory() {
2474        // `install` honours `RUST_LOG`, and a developer who has set it to
2475        // something restrictive would otherwise see this fail for a reason that
2476        // has nothing to do with the code. CI sets no such variable.
2477        if std::env::var_os("RUST_LOG").is_some() {
2478            return;
2479        }
2480
2481        let root = tempfile::tempdir().expect("a temporary directory");
2482        let paths = crate::paths::AppPaths::rooted_at(root.path());
2483
2484        let outcome = install(&paths, LogRole::Operator, "trace");
2485
2486        // Asserted before the early return below, because `install` creates the
2487        // directories before it touches the global subscriber: this holds
2488        // whether or not this test won the race to install one.
2489        assert_install_created_restricted_directories(&paths);
2490
2491        let Ok(guard) = outcome else {
2492            // Another test binary in the same process already installed one.
2493            // Not this test's failure, and not worth making the suite
2494            // order-dependent over.
2495            return;
2496        };
2497
2498        tracing::info!(event = "installed", runner_token = %USER_TOKEN, "hello from the sink");
2499
2500        // Dropping the guard flushes and stops the background writer, which is
2501        // the documented way to be sure the line reached disk.
2502        drop(guard);
2503
2504        let written: Vec<PathBuf> = std::fs::read_dir(paths.logs_dir())
2505            .expect("the log directory was created")
2506            .filter_map(Result::ok)
2507            .map(|entry| entry.path())
2508            .collect();
2509        assert_eq!(
2510            written.len(),
2511            1,
2512            "expected one rotating log file: {written:?}"
2513        );
2514
2515        let contents = std::fs::read_to_string(&written[0]).expect("readable");
2516        assert!(contents.contains("hello from the sink"), "{contents}");
2517        assert!(
2518            !contents.contains(USER_TOKEN),
2519            "the file sink must redact exactly like the in-memory one:\n{contents}"
2520        );
2521        assert!(contents.contains(REDACTION), "{contents}");
2522        serde_json::from_str::<Value>(contents.lines().next().expect("a line"))
2523            .expect("each line is a JSON object");
2524    }
2525
2526    // -----------------------------------------------------------------------
2527    // Two accounts, one logs/ directory
2528    // -----------------------------------------------------------------------
2529
2530    /// The daemon and the operator do not reach for the same file.
2531    ///
2532    /// Cheap, and it is the whole of the separation: everything else about
2533    /// [`LogRole`] follows from these two names being different.
2534    #[test]
2535    fn the_daemon_and_the_operator_write_different_files() {
2536        assert_ne!(
2537            LogRole::Service.file_stem(),
2538            LogRole::Operator.file_stem(),
2539            "a boot-mode daemon runs as another account and creates its file 0644; sharing a \
2540             stem hands whichever of them opened it first the day's file and locks the other out"
2541        );
2542        assert_eq!(LogRole::Operator.file_stem(), OPERATOR_LOG_STEM);
2543        assert_eq!(LogRole::Service.file_stem(), SERVICE_LOG_STEM);
2544    }
2545
2546    /// A file this account may not append to is written *beside*, not panicked
2547    /// on.
2548    ///
2549    /// This is the reported 0.1.17 crash, reproduced at the mode that caused
2550    /// it. On the host it was ownership — `root` had rolled the file over at
2551    /// midnight — and an unprivileged test cannot make a `root`-owned file, so
2552    /// it makes an unwritable one instead: the appender's `OpenOptions` refuse
2553    /// both with the same `EACCES`, and refusing was what
2554    /// `tracing_appender::rolling::daily` turned into a panic.
2555    #[cfg(unix)]
2556    #[test]
2557    fn a_log_file_this_account_cannot_append_to_is_written_beside() {
2558        use std::os::unix::fs::PermissionsExt as _;
2559
2560        // `root` is not subject to the mode bits, so it would sail through the
2561        // very open this test needs to fail.
2562        if account_tag() == "uid-0" {
2563            return;
2564        }
2565
2566        let root = tempfile::tempdir().expect("a temporary directory");
2567        let paths = crate::paths::AppPaths::rooted_at(root.path());
2568        paths
2569            .create_all()
2570            .expect("the four directories are created");
2571
2572        // Every date the appender might choose, so the test cannot flake on a
2573        // run that straddles midnight UTC.
2574        let logs = paths.logs_dir().to_path_buf();
2575        let today = chrono::Utc::now();
2576        for date in [
2577            (today - chrono::Duration::days(1))
2578                .format("%Y-%m-%d")
2579                .to_string(),
2580            today.format("%Y-%m-%d").to_string(),
2581            (today + chrono::Duration::days(1))
2582                .format("%Y-%m-%d")
2583                .to_string(),
2584        ] {
2585            let taken = logs.join(format!("{OPERATOR_LOG_STEM}.{date}"));
2586            std::fs::write(&taken, b"another account's diagnostics").expect("writable");
2587            std::fs::set_permissions(&taken, std::fs::Permissions::from_mode(0o444))
2588                .expect("the mode is applied");
2589        }
2590
2591        let mut appender = open_appender(&logs, LogRole::Operator)
2592            .expect("a diagnostics file is opened beside the one this account may not append to");
2593        appender
2594            .write_all(b"a line\n")
2595            .expect("the line is written");
2596        appender.flush().expect("the line is flushed");
2597
2598        let qualified = account_qualified_stem(OPERATOR_LOG_STEM);
2599        let written: Vec<String> = std::fs::read_dir(&logs)
2600            .expect("the directory is readable")
2601            .flatten()
2602            .map(|entry| entry.file_name().to_string_lossy().into_owned())
2603            .collect();
2604        assert!(
2605            written.iter().any(|name| name.starts_with(&qualified)),
2606            "expected a file under {qualified:?} beside the unwritable ones, and found {written:?}"
2607        );
2608        for name in &written {
2609            if name.starts_with(&qualified) {
2610                continue;
2611            }
2612            assert_eq!(
2613                std::fs::read(logs.join(name)).expect("readable"),
2614                b"another account's diagnostics",
2615                "{name} belongs to another account and must not have been touched"
2616            );
2617        }
2618    }
2619
2620    // -----------------------------------------------------------------------
2621    // The scrubber, in isolation
2622    // -----------------------------------------------------------------------
2623
2624    #[test]
2625    fn tokens_are_redacted_wherever_they_appear() {
2626        for token in [USER_TOKEN, SERVER_TOKEN, FINE_GRAINED] {
2627            for shape in [
2628                token.to_string(),
2629                format!("using {token} now"),
2630                format!("(\"{token}\")"),
2631                format!("token={token}"),
2632                format!("Authorization: Bearer {token}"),
2633                format!("authorization={token}"),
2634                // A URL's userinfo. This is the canonical token-authenticated
2635                // git remote, so it is what a clone or fetch error carries, and
2636                // it is the shape the `://` branch used to hand back verbatim.
2637                format!("https://x-access-token:{token}@github.com/owner/repo.git"),
2638                format!("fatal: could not read from https://{token}@github.com/o/r"),
2639                // Userinfo with no password half.
2640                format!("https://{token}@github.com/o/r.git"),
2641                // A fragment, which used to be stripped only when it was a
2642                // query string.
2643                format!("https://github.com/login/oauth#access_token={token}"),
2644                format!("https://github.com/x?a=1#token={token}"),
2645            ] {
2646                let redacted = redact(&shape);
2647                assert!(
2648                    !redacted.contains(token),
2649                    "{shape:?} survived redaction as {redacted:?}"
2650                );
2651            }
2652        }
2653    }
2654
2655    #[test]
2656    fn a_url_keeps_what_diagnoses_it_and_loses_what_authenticates_it() {
2657        // The host and the path are the diagnosable part and must survive, or
2658        // nobody keeps this redaction.
2659        assert_eq!(
2660            redact(
2661                "https://x-access-token:ghu_16C7e42F292c6912E7710c838347Ae178B4a@github.com/owner/repo.git"
2662            ),
2663            format!("https://{REDACTION}@github.com/owner/repo.git")
2664        );
2665        assert_eq!(
2666            redact("https://user:hunter2@api.github.com/repos/o/r?page=2"),
2667            format!("https://{REDACTION}@api.github.com/repos/o/r?{REDACTION}")
2668        );
2669        // A password containing an `@`: the host is what follows the last one.
2670        assert_eq!(
2671            redact("https://user:p@ss@github.com/o/r"),
2672            format!("https://{REDACTION}@github.com/o/r")
2673        );
2674        // Userinfo on a bare authority, with no path at all.
2675        assert_eq!(
2676            redact("https://token@github.com"),
2677            format!("https://{REDACTION}@github.com")
2678        );
2679        // An `@` after the first `/` is part of the path, not userinfo.
2680        assert_eq!(
2681            redact("https://github.com/@owner/repo"),
2682            "https://github.com/@owner/repo"
2683        );
2684    }
2685
2686    #[test]
2687    fn an_encoded_jit_configuration_is_redacted() {
2688        assert_eq!(redact(JIT_BLOB), REDACTION);
2689        let sentence = format!("handing off {JIT_BLOB} to the runner");
2690        let redacted = redact(&sentence);
2691        assert!(!redacted.contains(JIT_BLOB), "{redacted}");
2692        assert!(redacted.starts_with("handing off "), "{redacted}");
2693        assert!(redacted.ends_with(" to the runner"), "{redacted}");
2694    }
2695
2696    #[test]
2697    fn a_secret_embedded_in_a_structured_value_is_redacted() {
2698        // Compact JSON is what `serde_json::to_string` emits and what an
2699        // HTTP error body arrives as; spaced JSON is the only shape that
2700        // used to be safe, and it was safe by accident -- the value is its
2701        // own word there, so the opaque-run rule caught it without the key
2702        // ever being recognised.
2703        for shape in [
2704            format!("{{\"encoded_jit_config\":\"{JIT_BLOB}\"}}"),
2705            format!("{{ \"encoded_jit_config\": \"{JIT_BLOB}\" }}"),
2706            format!("encoded_jit_config:{JIT_BLOB}"),
2707            format!("{{\"runner_token\":\"{USER_TOKEN}\"}}"),
2708            format!("runner_token:{USER_TOKEN}"),
2709            format!("pat:{USER_TOKEN}"),
2710            format!("{{\"authorization\":\"Bearer {USER_TOKEN}\"}}"),
2711        ] {
2712            let redacted = redact(&shape);
2713            assert!(
2714                !redacted.contains(JIT_BLOB) && !redacted.contains(USER_TOKEN),
2715                "leaked from {shape}:\n{redacted}"
2716            );
2717        }
2718
2719        // A credential short enough to clear the opaque-run threshold and
2720        // without a GitHub prefix has nothing but the key to give it away,
2721        // so it leaked from the spaced shape too: `\"password\":` left the
2722        // key spelled `password\"`, which is on no list.
2723        for shape in [
2724            "{\"password\":\"hunter2\"}",
2725            "{ \"password\": \"hunter2\" }",
2726        ] {
2727            let redacted = redact(shape);
2728            assert!(
2729                !redacted.contains("hunter2"),
2730                "leaked from {shape}: {redacted}"
2731            );
2732        }
2733
2734        // The neighbouring context still survives: this is a scrubber, not
2735        // a deleter.
2736        let body = format!("registration failed: {{\"encoded_jit_config\":\"{JIT_BLOB}\"}}");
2737        let redacted = redact(&body);
2738        assert!(redacted.starts_with("registration failed: "), "{redacted}");
2739        assert!(redacted.contains("encoded_jit_config"), "{redacted}");
2740    }
2741
2742    #[test]
2743    fn a_credential_key_is_found_wherever_it_sits_in_a_compact_structure() {
2744        // Every shape the test above covers puts the credential in the
2745        // *first* key/value pair, and the rules split a word once, on the
2746        // first separator they find. So redaction was a function of field
2747        // order -- and `serde_json::to_string` is what decides field order for
2748        // any struct with more than one field, which is what an error body is.
2749        //
2750        // Nesting and a form-encoded body are the same defect wearing
2751        // different punctuation: the value recursion ends at `redact_value`,
2752        // which is a leaf, and `&` was not a separator at all.
2753        for shape in [
2754            // The credential key second, in a compact object.
2755            format!("{{\"runner_id\":42,\"encoded_jit_config\":\"{JIT_BLOB}\"}}"),
2756            format!("{{\"status\":422,\"message\":\"bad\",\"runner_token\":\"{USER_TOKEN}\"}}"),
2757            // Nested one level, and two.
2758            format!("{{\"body\":{{\"runner_token\":\"{USER_TOKEN}\"}}}}"),
2759            format!(
2760                "{{\"error\":{{\"status\":422,\"body\":{{\"encoded_jit_config\":\"{JIT_BLOB}\"}}}}}}"
2761            ),
2762            // An array of objects, which is what a list endpoint returns.
2763            format!("[{{\"id\":1}},{{\"access_token\":\"{USER_TOKEN}\"}}]"),
2764            // Form-encoded, credential second and in the middle.
2765            format!("scope=repo&access_token={USER_TOKEN}"),
2766            format!("grant_type=refresh&refresh_token={USER_TOKEN}&scope=repo"),
2767        ] {
2768            let redacted = redact(&shape);
2769            assert!(
2770                !redacted.contains(JIT_BLOB) && !redacted.contains(USER_TOKEN),
2771                "leaked from {shape}:\n{redacted}"
2772            );
2773            assert!(redacted.contains(REDACTION), "{shape} -> {redacted}");
2774        }
2775
2776        // The pairs around it stay legible. A body whose every field came back
2777        // `[redacted]` diagnoses nothing, and is how a redaction gets turned
2778        // off rather than fixed.
2779        let redacted = redact(&format!(
2780            "failed: {{\"runner_id\":42,\"encoded_jit_config\":\"{JIT_BLOB}\"}}"
2781        ));
2782        assert!(redacted.starts_with("failed: "), "{redacted}");
2783        assert!(redacted.contains("\"runner_id\":42"), "{redacted}");
2784        assert!(redacted.contains("encoded_jit_config"), "{redacted}");
2785
2786        // A credential short enough to clear the opaque-run threshold and
2787        // carrying no GitHub prefix has nothing but its key to give it away,
2788        // so it is the case the shape rules cannot rescue.
2789        for shape in [
2790            "{\"user\":\"operator\",\"password\":\"hunter2\"}",
2791            "{\"body\":{\"password\":\"hunter2\"}}",
2792            "user=operator&password=hunter2",
2793        ] {
2794            assert!(
2795                !redact(shape).contains("hunter2"),
2796                "leaked from {shape}: {}",
2797                redact(shape)
2798            );
2799        }
2800
2801        // Ordinary punctuation still survives the cut, or the structure that
2802        // makes a log line readable is gone with the secret.
2803        assert_eq!(
2804            redact("desired 3, active 1, headroom 2"),
2805            "desired 3, active 1, headroom 2"
2806        );
2807        assert_eq!(
2808            redact("labels=[linux,x64,self-hosted]"),
2809            "labels=[linux,x64,self-hosted]"
2810        );
2811    }
2812
2813    #[test]
2814    fn backslash_escaped_json_reaches_the_key_rules_and_the_value_rules() {
2815        // `tracing::error!(reason = ?err)` reaches this module through
2816        // `record_debug` and `format!("{:?}")`, and `Debug` on a `String`
2817        // escapes the quotes inside it. So an error whose `Debug` embeds an
2818        // HTTP body arrives with its keys spelled `\"password\"` -- and `\`
2819        // is not in `WRAPPERS`, so `trim_matches(WRAPPERS)` left it welded on
2820        // and no list contained the result. `reason` is on ALLOWED_FIELDS, so
2821        // this reaches the scrubber rather than being dropped.
2822        let failure = StoreError {
2823            body: "{\"password\":\"hunter2\"}".to_string(),
2824        };
2825        // The fixture carries the secret before anything has looked at it. A
2826        // premise, asserted rather than assumed: a fixture that quietly
2827        // stopped carrying one would make every assertion below vacuously
2828        // true.
2829        assert!(failure.body.contains("hunter2"), "{}", failure.body);
2830        let rendered = format!("{failure:?}");
2831        // The premise, asserted rather than assumed: `Debug` really does
2832        // produce the escaped spelling. If it ever stops, this test is
2833        // measuring something else.
2834        assert!(
2835            rendered.contains("\\\"password\\\""),
2836            "the premise is that Debug escapes the quotes: {rendered}"
2837        );
2838        assert!(
2839            !redact(&rendered).contains("hunter2"),
2840            "leaked: {}",
2841            redact(&rendered)
2842        );
2843
2844        for shape in [
2845            "{\\\"password\\\":\\\"hunter2\\\"}",
2846            "Error { body: \"{\\\"access_token\\\":\\\"hunter2\\\"}\" }",
2847        ] {
2848            assert!(
2849                !redact(shape).contains("hunter2"),
2850                "leaked from {shape}: {}",
2851                redact(shape)
2852            );
2853        }
2854
2855        // The escaped spelling of a long secret leaked too, and for the same
2856        // reason: the key was unrecognised, so nothing below it ever ran.
2857        let escaped = format!("{{\\\"encoded_jit_config\\\":\\\"{JIT_BLOB}\\\"}}");
2858        assert!(!redact(&escaped).contains(JIT_BLOB), "{}", redact(&escaped));
2859
2860        // Trimming the key is only half of it, and it is the half that is
2861        // easy to mistake for the whole. `runner_token` is deliberately *not*
2862        // on CREDENTIAL_KEYS -- a field name is not what makes a value a
2863        // secret, and this module's own documentation says so -- so that pair
2864        // is caught by the token-prefix rule reading its *value*, or it is not
2865        // caught at all. The escaped quote hid the value from that rule
2866        // exactly as it hid the key from the key rules, and a key-only fix
2867        // leaves this one leaking through the sink.
2868        assert!(
2869            !is_credential_key("runner_token"),
2870            "the premise of this case is an unlisted key; once it is listed, \
2871             this stops exercising the value side at all"
2872        );
2873        let escaped_value = format!("{{\\\"runner_token\\\":\\\"{USER_TOKEN}\\\"}}");
2874        assert!(
2875            !redact(&escaped_value).contains(USER_TOKEN),
2876            "the value side leaked: {}",
2877            redact(&escaped_value)
2878        );
2879
2880        // The trim is unconditional on a key and conditional on a value: a
2881        // backslash comes off a value only where it is escaping punctuation.
2882        // `split_wrappers` runs before `looks_like_path`, so putting `\` in
2883        // `WRAPPERS` would trim away the leading `\\` that UNC detection keys
2884        // on and turn a share path back into ordinary text.
2885        assert_eq!(redact(r"\\fileserver\share\jit"), PATH_REDACTION);
2886        assert_eq!(redact(r"\\?\C:\Users\operator\runtime"), PATH_REDACTION);
2887        // And the escaped spelling of a share path is still a share path:
2888        // `\\` escapes nothing, so it survives the trim that `\"` does not.
2889        assert_eq!(redact(r"\\\\fileserver\\share\\jit"), PATH_REDACTION);
2890        // A path ending in its own separator is still a path without it. The
2891        // separator is punctuation, so it is trimmed, judged, and put back.
2892        let trailing = redact("C:\\Users\\operator\\runtime\\");
2893        assert!(trailing.starts_with(PATH_REDACTION), "{trailing}");
2894        assert!(!trailing.contains("operator"), "{trailing}");
2895    }
2896
2897    #[test]
2898    fn a_url_does_not_make_the_rest_of_its_word_unredactable() {
2899        // `redact_core` split on the first `://` and handed *everything*
2900        // before it to `redact_url` as a scheme and everything after it as a
2901        // URL. The terminal arm of `redact_url` echoes both verbatim, so a
2902        // single URL anywhere in a word turned the whole of the rest of that
2903        // word into text no rule below could reach. The only thing that saved
2904        // the shape was a `?` or `#` inside the URL, which made `redact_url`
2905        // replace the tail.
2906        //
2907        // `documentation_url` is in essentially every GitHub REST error body,
2908        // so this was any such body logged alongside a credential.
2909        let body = format!(
2910            "{{\"message\":\"Bad credentials\",\"documentation_url\":\"https://docs.github.com/rest\",\"token\":\"{USER_TOKEN}\"}}"
2911        );
2912        let redacted = redact(&body);
2913        assert!(!redacted.contains(USER_TOKEN), "leaked: {redacted}");
2914        // The URL is the diagnosable part and must survive, or nobody keeps
2915        // this redaction.
2916        assert!(
2917            redacted.contains("https://docs.github.com/rest"),
2918            "the URL should survive: {redacted}"
2919        );
2920
2921        // The mirror order: everything before the `://` became the "scheme".
2922        let reversed = format!(
2923            "{{\"token\":\"{USER_TOKEN}\",\"documentation_url\":\"https://docs.github.com/rest\"}}"
2924        );
2925        let redacted = redact(&reversed);
2926        assert!(!redacted.contains(USER_TOKEN), "leaked: {redacted}");
2927
2928        // A nested object behind a URL was missed even when the URL's own
2929        // userinfo was caught, because the miss and the catch happened in the
2930        // same call.
2931        let clone = format!(
2932            "{{\"remote\":\"https://x-access-token:{USER_TOKEN}@github.com/o/r.git\",\"body\":{{\"password\":\"hunter2\"}}}}"
2933        );
2934        let redacted = redact(&clone);
2935        assert!(!redacted.contains(USER_TOKEN), "leaked: {redacted}");
2936        assert!(!redacted.contains("hunter2"), "leaked: {redacted}");
2937        assert!(
2938            redacted.contains("@github.com/o/r.git"),
2939            "the remote should stay diagnosable: {redacted}"
2940        );
2941
2942        // A URL still owns its own query string, which is why the URL check
2943        // sits ahead of the structural cut: `?` and `#` are not structural
2944        // characters, and an OAuth response puts the token after one of them.
2945        assert_eq!(
2946            redact(&format!(
2947                "{{\"url\":\"https://api.github.com/x?token={USER_TOKEN}\"}}"
2948            )),
2949            format!("{{\"url\":\"https://api.github.com/x?{REDACTION}\"}}")
2950        );
2951        // And a URL standing on its own is untouched.
2952        assert_eq!(
2953            redact("GET https://api.github.com/repos/o/r/actions/runners"),
2954            "GET https://api.github.com/repos/o/r/actions/runners"
2955        );
2956
2957        // A URL that is a credential key's *own* value keeps its scheme, host
2958        // and path like every other URL -- a URL is not a token, and this
2959        // module keeps exactly that much of one everywhere else. What must not
2960        // happen is the key's empty value being redacted on the way past,
2961        // which put the URL out behind a `[redacted]` that had replaced
2962        // nothing: `token=[redacted]https://evil.example/x`. Bounding the URL
2963        // is what created that shape, and pass one declining an empty value is
2964        // what closes it.
2965        assert_eq!(
2966            redact("token=https://evil.example/x"),
2967            "token=https://evil.example/x"
2968        );
2969        assert_eq!(
2970            redact("{\"token\":\"https://evil.example/x\"}"),
2971            "{\"token\":\"https://evil.example/x\"}"
2972        );
2973        // An empty credential value is nothing to redact wherever it sits, and
2974        // saying otherwise reports a secret in a place none was.
2975        assert_eq!(redact("{\"password\":\"\"}"), "{\"password\":\"\"}");
2976    }
2977
2978    #[test]
2979    fn an_array_element_is_judged_without_the_quote_that_wraps_it() {
2980        // The structural cut recursed on the *raw* fragment, and the terminal
2981        // fallback then called `redact_value` with the wrappers still on. A
2982        // fragment carrying no `:` or `=` of its own -- which is exactly what
2983        // an array element is -- therefore reached the shape rules as
2984        // `"ghu_…`: `starts_with("ghu_")` fails, `is_opaque_char` fails on the
2985        // quote, `starts_with("eyJ")` fails, and `looks_like_path` never gets
2986        // a clean look at `"C:\Users\…`.
2987        //
2988        // This is the defect the structural cut was written to close, one step
2989        // further down: "only the first key/value pair is ever examined"
2990        // became "a value with no key of its own is never examined". It
2991        // survived a round because `scan_for_leaks` had no array among its
2992        // twelve shapes.
2993        for shape in [
2994            format!("{{\"tokens\":[\"{USER_TOKEN}\",\"x\"]}}"),
2995            format!("{{\"tokens\":[\"x\",\"{USER_TOKEN}\"]}}"),
2996            format!("[\"x\",\"{USER_TOKEN}\"]"),
2997            format!("[{{\"id\":1}},[\"{USER_TOKEN}\"]]"),
2998            // The `Debug`-escaped spelling of the same thing.
2999            format!("{{\\\"tokens\\\":[\\\"x\\\",\\\"{USER_TOKEN}\\\"]}}"),
3000        ] {
3001            let redacted = redact(&shape);
3002            assert!(
3003                !redacted.contains(USER_TOKEN),
3004                "leaked from {shape}: {redacted}"
3005            );
3006            assert!(redacted.contains(REDACTION), "{shape} -> {redacted}");
3007        }
3008
3009        // Every other shape rule was reachable the same way.
3010        let jit = format!("{{\"items\":[\"{JIT_BLOB}\"]}}");
3011        assert!(!redact(&jit).contains(JIT_BLOB), "{}", redact(&jit));
3012        let assertions = format!("{{\"assertions\":[\"{JWT}\"]}}");
3013        assert!(
3014            !redact(&assertions).contains(JWT),
3015            "{}",
3016            redact(&assertions)
3017        );
3018        let opaque = "ZYXWVUTSRQPONMLKJIHGFEDCBA9876543210zyxwvut";
3019        assert!(opaque.len() > OPAQUE_RUN_THRESHOLD, "{}", opaque.len());
3020        let listed = format!("[\"x\",\"{opaque}\"]");
3021        assert!(!redact(&listed).contains(opaque), "{}", redact(&listed));
3022
3023        // A path in an array. `07-security.md` requires paths redacted, and
3024        // only `looks_like_path` can see one -- with a clean look at the value
3025        // and not before.
3026        let roots = format!("{{\"roots\":[\"x\",\"{WINDOWS_WORKSPACE}\"]}}");
3027        let redacted = redact(&roots);
3028        assert!(!redacted.contains(WINDOWS_WORKSPACE), "leaked: {redacted}");
3029        assert!(redacted.contains(PATH_REDACTION), "{redacted}");
3030
3031        // The array survives being scrubbed: this is a scrubber, not a
3032        // deleter.
3033        assert_eq!(
3034            redact("labels=[\"linux\",\"x64\"]"),
3035            "labels=[\"linux\",\"x64\"]"
3036        );
3037    }
3038
3039    #[test]
3040    fn a_semicolon_cuts_a_word_the_way_a_comma_does() {
3041        // `;` was in `WRAPPERS` -- recognised as punctuation -- and not in
3042        // `STRUCTURAL`, so it never cut a word. It is the separator for a
3043        // Windows connection string, for a credential string, and for cookie
3044        // pairs written without a space, which is `d2`'s territory exactly.
3045        // `Set-Cookie: theme=dark; session=…` was safe only because of the
3046        // space after the `;`.
3047        for shape in [
3048            "store rejected: Server=host;Database=x;Password=hunter2;",
3049            "store rejected: user=operator;password=hunter2",
3050            "Server=host;Password=hunter2;Database=x",
3051        ] {
3052            let redacted = redact(shape);
3053            assert!(
3054                !redacted.contains("hunter2"),
3055                "leaked from {shape}: {redacted}"
3056            );
3057        }
3058
3059        let cookies = format!("theme=dark;session={USER_TOKEN}");
3060        let redacted = redact(&cookies);
3061        assert!(!redacted.contains(USER_TOKEN), "leaked: {redacted}");
3062
3063        // Separators go back verbatim, so ordinary prose is unaffected.
3064        assert_eq!(
3065            redact("started; then reconciled; then idled"),
3066            "started; then reconciled; then idled"
3067        );
3068        assert_eq!(
3069            redact("desired 3;active 1;headroom 2"),
3070            "desired 3;active 1;headroom 2"
3071        );
3072    }
3073
3074    #[test]
3075    fn an_element_wrapped_value_is_redacted() {
3076        // `split_wrappers` strips only the *outermost* `<` and `>`, so
3077        // `<string>ghu_…</string>` arrived as `string>ghu_…</string`, which
3078        // matches no rule at all. `d3`'s installers handle launchd plists,
3079        // which is where this shape comes from; a systemd unit line is the
3080        // `key=value` spelling and was already covered.
3081        for shape in [
3082            format!("<string>{USER_TOKEN}</string>"),
3083            format!("<key>token</key><string>{USER_TOKEN}</string>"),
3084            format!("<dict><key>Token</key><string>{USER_TOKEN}</string></dict>"),
3085        ] {
3086            let redacted = redact(&shape);
3087            assert!(
3088                !redacted.contains(USER_TOKEN),
3089                "leaked from {shape}: {redacted}"
3090            );
3091            assert!(redacted.contains(REDACTION), "{shape} -> {redacted}");
3092        }
3093
3094        // The element names survive: they are what says the line was about a
3095        // plist at all.
3096        let plist = format!("<key>token</key><string>{JIT_BLOB}</string>");
3097        let redacted = redact(&plist);
3098        assert!(!redacted.contains(JIT_BLOB), "leaked: {redacted}");
3099        assert!(redacted.contains("<key>token</key>"), "{redacted}");
3100
3101        // An angle bracket in ordinary text is re-emitted verbatim.
3102        assert_eq!(redact("Custom<Io>"), "Custom<Io>");
3103    }
3104
3105    #[test]
3106    fn a_redacted_value_keeps_the_punctuation_that_wrapped_it() {
3107        // Pass one returned `format!("{key}{separator}{REDACTION}")` and
3108        // dropped the value's trailing wrapper, so a redacted object came out
3109        // with an unbalanced quote: `{"password":[redacted]"}`. Pass two
3110        // already put `lead` and `trail` back.
3111        //
3112        // Worth fixing because this module argues, correctly, that the
3113        // structure around a redaction is what keeps a line diagnosable -- and
3114        // a reader who cannot parse the line cannot tell a redaction from a
3115        // truncation.
3116        assert_eq!(
3117            redact("{\"password\":\"hunter2\"}"),
3118            format!("{{\"password\":\"{REDACTION}\"}}")
3119        );
3120        assert_eq!(
3121            redact(&format!("{{\"access_token\":\"{USER_TOKEN}\"}}")),
3122            format!("{{\"access_token\":\"{REDACTION}\"}}")
3123        );
3124        assert_eq!(
3125            redact(&format!(
3126                "[{{\"id\":1}},{{\"access_token\":\"{USER_TOKEN}\"}}]"
3127            )),
3128            format!("[{{\"id\":1}},{{\"access_token\":\"{REDACTION}\"}}]")
3129        );
3130        // The `Debug`-escaped spelling keeps its escaped quotes.
3131        assert_eq!(
3132            redact("{\\\"password\\\":\\\"hunter2\\\"}"),
3133            format!("{{\\\"password\\\":\\\"{REDACTION}\\\"}}")
3134        );
3135
3136        // The point of all of it: what the sink writes is still the JSON it
3137        // was handed, minus the secret.
3138        let line = redact(&format!(
3139            "{{\"runner_id\":42,\"access_token\":\"{USER_TOKEN}\"}}"
3140        ));
3141        serde_json::from_str::<Value>(&line)
3142            .unwrap_or_else(|error| panic!("a redacted body must still parse: {line} ({error})"));
3143    }
3144
3145    #[test]
3146    fn a_bare_jwt_is_redacted_despite_its_dots() {
3147        // `is_opaque_char` excludes `.`, so a JWT is three opaque runs rather
3148        // than one, each under the threshold, and a 100-character credential
3149        // printed verbatim. `Authorization: Bearer <jwt>` is caught by the
3150        // scheme rule and a credential-keyed one by the key rule, so this bit
3151        // only where a token stood alone or under a name nobody listed.
3152        assert!(
3153            JWT.len() > 100,
3154            "the premise is a long token: {}",
3155            JWT.len()
3156        );
3157        assert_eq!(redact(JWT), REDACTION);
3158        assert_eq!(
3159            redact(&format!("minted {JWT} for the installation")),
3160            format!("minted {REDACTION} for the installation")
3161        );
3162        // Sentence-final punctuation is trimmed as a wrapper first, so the
3163        // token is still recognised.
3164        assert!(!redact(&format!("minted {JWT}.")).contains(JWT));
3165        // Under a key nobody listed.
3166        assert!(!redact(&format!("assertion={JWT}")).contains(JWT));
3167        assert!(!redact(&format!("{{\"assertion\":\"{JWT}\"}}")).contains(JWT));
3168
3169        // Narrow on purpose. Adding `.` to `is_opaque_char` is the obvious fix
3170        // and the wrong one: it swallows every long dotted word there is.
3171        for ordinary in [
3172            "com.example.runner.manager.platform.process.identity.token",
3173            "runner-manager.2026.08.21.log",
3174            "api.github.com",
3175            "9f2c1a44-0000-4000-8000-000000000001.attempt.json",
3176        ] {
3177            assert_eq!(redact(ordinary), ordinary, "over-redacted {ordinary}");
3178        }
3179    }
3180
3181    #[test]
3182    fn the_sink_redacts_a_short_credential_that_only_its_key_gives_away() {
3183        // `scan_for_leaks` cannot carry this case. A credential short enough
3184        // to clear the opaque-run threshold and carrying no GitHub prefix is
3185        // indistinguishable from an ordinary word when it is logged bare, so
3186        // injecting it through shape 1 would demand a redaction no shape rule
3187        // can deliver. Here the key is always present, which is the situation
3188        // `d2`'s secret store is actually in -- and it is through the sink,
3189        // not through `redact` alone, because that distinction is why these
3190        // shapes survived a round.
3191        let capture = Capture::default();
3192        let output = emit(&capture, || {
3193            tracing::error!("store rejected: {{\"user\":\"operator\",\"password\":\"hunter2\"}}");
3194            tracing::error!("store rejected: {{\"body\":{{\"password\":\"hunter2\"}}}}");
3195            tracing::error!("store rejected: user=operator&password=hunter2");
3196            let failure = StoreError {
3197                body: "{\"password\":\"hunter2\"}".to_string(),
3198            };
3199            tracing::error!(reason = ?failure, "store rejected the request");
3200
3201            // A `;` or an `&` after a URL in the same word. The URL branch runs
3202            // ahead of the structural cut and `split_url` did not stop at
3203            // either character, so the separator that would have cut the word
3204            // was swallowed into the URL and echoed.
3205            tracing::error!("store rejected: Server=https://vault.local/api;Password=hunter2;");
3206            tracing::error!("keychain error: url=https://kc.local;password=hunter2");
3207            tracing::error!("unit rejected: Environment=API=https://a.com/v1;PASSWORD=hunter2");
3208            tracing::error!("token exchange failed: cb=https://a.com/x&password=hunter2");
3209
3210            // A credential key whose value is in a *later* fragment. The
3211            // empty-value skip is right; the claim that the structural cut
3212            // reaches such a value was not, because nothing carried the key
3213            // across the cut.
3214            tracing::error!("store rejected: {{\"password\":[\"hunter2\"]}}");
3215            tracing::error!("store rejected: {{\"password\":{{\"v\":\"hunter2\"}}}}");
3216            tracing::error!("store rejected: Password=;hunter2");
3217            tracing::error!("plist rejected: <key>password</key><string>hunter2</string>");
3218            tracing::error!(
3219                "plist rejected: <dict><key>password</key><string>hunter2</string></dict>"
3220            );
3221
3222            // A structural character inside the value. `,` and `&` could
3223            // already split a secret; this commit added `;`, `<` and `>`.
3224            //
3225            // The pieces are spelled so that none of them is a substring of
3226            // anything the line legitimately keeps -- `ss` would have "found" a
3227            // leak in the surviving key `password`, which is a check that fails
3228            // for the wrong reason and is no better than one that cannot fail.
3229            tracing::error!("store rejected: {{\"password\":\"qq<vv>xx\"}}");
3230            tracing::error!("store rejected: {{\"password\":\"j1&k2,m3\"}}");
3231            tracing::error!("store rejected: {{\"password\":\"n4;p5<q6>r7,t8\"}}");
3232        });
3233
3234        assert!(
3235            !output.contains("hunter2"),
3236            "a short credential leaked through the sink:\n{output}"
3237        );
3238        // The punctuated passwords: every piece a structural character can cut
3239        // one into has to go, not just the piece that kept the key company.
3240        for piece in [
3241            "qq", "vv", "xx", "j1", "k2", "m3", "n4", "p5", "q6", "r7", "t8",
3242        ] {
3243            assert!(
3244                !output.contains(piece),
3245                "a structural character split a secret and the tail survived \
3246                 ({piece}):\n{output}"
3247            );
3248        }
3249        assert!(output.contains(REDACTION), "{output}");
3250        // Sixteen lines, or the sink was never reached and "no secret found" is
3251        // true of an empty string.
3252        assert_eq!(
3253            output
3254                .lines()
3255                .filter(|line| !line.trim().is_empty())
3256                .count(),
3257            16,
3258            "{output}"
3259        );
3260    }
3261
3262    /// The multi-word plist spelling, which is worse than the one-word one:
3263    /// the credential key and the value are in the same word, but the value
3264    /// itself is two words, so closing it needs the fragment carry to reach the
3265    /// word-level follow-on rule.
3266    #[test]
3267    fn a_credential_value_that_runs_past_its_own_word_is_still_redacted() {
3268        let capture = Capture::default();
3269        let output = emit(&capture, || {
3270            tracing::error!("plist rejected: <key>password</key><string>correct horse</string>");
3271            tracing::error!("plist rejected: <key>password</key> <string>correct horse</string>");
3272        });
3273        for piece in ["correct", "horse"] {
3274            assert!(
3275                !output.contains(piece),
3276                "a multi-word credential value leaked ({piece}):\n{output}"
3277            );
3278        }
3279    }
3280
3281    /// The other half of the fragment carry: it must not swallow the ordinary,
3282    /// *closed* pairs sitting next to a credential.
3283    ///
3284    /// Every shape here is one the carry has an opportunity to over-redact —
3285    /// a closed empty value, a neighbouring key, the pairs of a connection
3286    /// string, and the element names of a plist. Redaction that eats the
3287    /// diagnostics is redaction nobody keeps, and a carry with no terminator
3288    /// is exactly how that happens.
3289    #[test]
3290    fn the_fragment_carry_stops_where_the_value_does() {
3291        for (input, expected) in [
3292            (
3293                "{\"password\":\"abc\",\"user\":\"bob\"}",
3294                format!("{{\"password\":\"{REDACTION}\",\"user\":\"bob\"}}"),
3295            ),
3296            (
3297                "Server=host;Password=hunter2;User=bob",
3298                format!("Server=host;Password={REDACTION};User=bob"),
3299            ),
3300            (
3301                "{\"password\":\"\",\"user\":\"bob\"}",
3302                "{\"password\":\"\",\"user\":\"bob\"}".to_string(),
3303            ),
3304            (
3305                "<dict><key>password</key><string>hunter2</string></dict>",
3306                format!("<dict><key>password</key><string>{REDACTION}</string></dict>"),
3307            ),
3308            (
3309                "{\"password\":\"qq<vv>xx\"}",
3310                format!("{{\"password\":\"{REDACTION}<{REDACTION}>{REDACTION}\"}}"),
3311            ),
3312        ] {
3313            assert_eq!(redact(input), expected, "from {input}");
3314        }
3315
3316        // A closed value ends the claim, so the word after it is ordinary text.
3317        assert_eq!(
3318            redact("{\"password\":\"x\"} ok"),
3319            format!("{{\"password\":\"{REDACTION}\"}} ok")
3320        );
3321        assert_eq!(
3322            redact("Server=host;Password=hunter2; ok"),
3323            format!("Server=host;Password={REDACTION}; ok")
3324        );
3325
3326        // `{"password":""} ok` is *not* on that list, and the reason is worth
3327        // recording rather than asserting away. `split_wrappers` strips the
3328        // empty value with the rest of the trailing punctuation, so the word
3329        // reaches `redact_word` as the core `password":` — a bare credential
3330        // key, which the stem rule has always read as "the value is the next
3331        // word". That rule predates the carry, is untouched by it, and fires
3332        // first; the empty-value skip below it never gets a look. So the `ok`
3333        // is redacted, exactly as it was before any of this.
3334        assert_eq!(
3335            redact("{\"password\":\"\"} ok"),
3336            format!("{{\"password\":\"\"}} {REDACTION}")
3337        );
3338        // What the empty-value skip does still guarantee is the thing it was
3339        // written for: no `[redacted]` is invented *inside* the pair itself.
3340        assert_eq!(redact("{\"password\":\"\"}"), "{\"password\":\"\"}");
3341
3342        // Ordinary prose is untouched: the carry is only ever armed by a
3343        // credential key.
3344        assert_eq!(
3345            redact("started; then reconciled; then idled"),
3346            "started; then reconciled; then idled"
3347        );
3348        assert_eq!(redact("Custom<Io>"), "Custom<Io>");
3349    }
3350
3351    #[test]
3352    fn paths_are_redacted_on_both_families() {
3353        assert_eq!(redact(WORKSPACE), PATH_REDACTION);
3354        assert_eq!(redact(WINDOWS_WORKSPACE), PATH_REDACTION);
3355        assert_eq!(redact("\\\\fileserver\\share\\jit"), PATH_REDACTION);
3356        assert_eq!(
3357            redact("~/Library/Application Support/x"),
3358            format!("{PATH_REDACTION} Support/x")
3359        );
3360        assert_eq!(
3361            redact("runtime=/var/lib/runner-manager/runtime"),
3362            format!("runtime={PATH_REDACTION}")
3363        );
3364    }
3365
3366    #[test]
3367    fn a_url_survives_but_its_query_string_and_fragment_do_not() {
3368        assert_eq!(
3369            redact("GET https://api.github.com/repos/owner/repo/actions/runners"),
3370            "GET https://api.github.com/repos/owner/repo/actions/runners"
3371        );
3372        assert_eq!(
3373            redact("https://api.github.com/x?access_token=ghu_secret"),
3374            format!("https://api.github.com/x?{REDACTION}")
3375        );
3376        // A fragment carries a token in an OAuth implicit-flow response, and
3377        // is never needed to diagnose an HTTP call.
3378        assert_eq!(
3379            redact("https://api.github.com/x#access_token=ghu_secret"),
3380            format!("https://api.github.com/x#{REDACTION}")
3381        );
3382        // Whichever comes first ends the diagnosable part.
3383        assert_eq!(
3384            redact("https://api.github.com/x?page=2#access_token=ghu_secret"),
3385            format!("https://api.github.com/x?{REDACTION}")
3386        );
3387    }
3388
3389    /// `;` and `&` end a URL, but only ahead of its query string.
3390    ///
3391    /// `STRUCTURAL` has nine characters and `is_url_terminator` recognised
3392    /// seven of them. Because the URL branch runs *ahead* of the structural
3393    /// cut, a URL earlier in a word made `split_url` swallow everything to the
3394    /// next terminator — including the `;` or `&` that would have cut the word
3395    /// — and `redact_url`'s terminal arm echoes what it is given. So the L3 fix
3396    /// (bounding a URL) and the L1 fix (cutting on `;` and `&`) did not
3397    /// compose, and this commit added `;` to `STRUCTURAL` for connection
3398    /// strings while leaving the path a URL opens straight through it.
3399    #[test]
3400    fn a_separator_after_a_url_still_cuts_the_word() {
3401        for shape in [
3402            format!("store rejected: Server=https://vault.local/api;Password={USER_TOKEN};"),
3403            format!("keychain error: url=https://kc.local;secret={USER_TOKEN}"),
3404            format!("unit rejected: Environment=API=https://a.com/v1;TOKEN={USER_TOKEN}"),
3405            format!("token exchange failed: cb=https://a.com/x&access_token={USER_TOKEN}"),
3406            format!("dsn rejected: dsn=https://sentry.local/1;password={USER_TOKEN};user=x"),
3407        ] {
3408            let redacted = redact(&shape);
3409            assert!(
3410                !redacted.contains(USER_TOKEN),
3411                "leaked from {shape}:\n{redacted}"
3412            );
3413        }
3414
3415        // The URL itself still survives, or nobody keeps this redaction.
3416        assert_eq!(
3417            redact("store rejected: Server=https://vault.local/api;Password=hunter2;"),
3418            format!("store rejected: Server=https://vault.local/api;Password={REDACTION};")
3419        );
3420
3421        // The obvious fix is the wrong one, and this is what it costs. Making
3422        // `;` and `&` terminators everywhere ends a URL *inside* its own query
3423        // string, and `redact_url` replaces a query wholesale precisely because
3424        // a token can be in any parameter and this module does not guess which:
3425        // `?[redacted]` would become `?[redacted]&state=hunter2`. So the cut is
3426        // restricted to the part of the URL ahead of the first `?` or `#`.
3427        assert_eq!(
3428            redact("https://a.com/cb?code=1&state=hunter2"),
3429            format!("https://a.com/cb?{REDACTION}")
3430        );
3431        assert_eq!(
3432            redact("https://a.com/cb#code=1&state=hunter2"),
3433            format!("https://a.com/cb#{REDACTION}")
3434        );
3435        assert_eq!(
3436            redact(&format!(
3437                "https://a.com/cb?a=1;b=2&access_token={USER_TOKEN}"
3438            )),
3439            format!("https://a.com/cb?{REDACTION}")
3440        );
3441
3442        // And leaving `\` out of the terminators is still right: a Windows
3443        // path used as a URL password puts the `@` that identifies the
3444        // userinfo behind the first backslash, and `redact_url` echoes what it
3445        // is given.
3446        assert_eq!(
3447            redact(r"https://x-access-token:C:\Users\op\p@github.com/o/r.git"),
3448            format!("https://{REDACTION}@github.com/o/r.git")
3449        );
3450    }
3451
3452    /// A secret in a URL *path* is echoed verbatim.
3453    ///
3454    /// `redact_url` applied no shape rule to the path at all. The design
3455    /// intends a path to be diagnosable — and it still is — but the
3456    /// token-prefix rule is documented as a belt that catches a credential
3457    /// *anywhere*, and a path was the one place it never ran.
3458    #[test]
3459    fn a_secret_in_a_url_path_is_redacted_and_the_rest_of_the_path_is_not() {
3460        for shape in [
3461            format!("https://github.com/o/r/raw/{USER_TOKEN}/f"),
3462            format!("https://api.github.com/{JIT_BLOB}"),
3463            format!("https://a.com/x/{JWT}?page=2"),
3464        ] {
3465            let redacted = redact(&shape);
3466            assert!(
3467                !redacted.contains(USER_TOKEN)
3468                    && !redacted.contains(JIT_BLOB)
3469                    && !redacted.contains(JWT),
3470                "leaked from {shape}:\n{redacted}"
3471            );
3472        }
3473
3474        assert_eq!(
3475            redact(&format!("https://github.com/o/r/raw/{USER_TOKEN}/f")),
3476            format!("https://github.com/o/r/raw/{REDACTION}/f")
3477        );
3478
3479        // The rest of the path is exactly as diagnosable as it was: a segment
3480        // is judged on its own, and an ordinary one is not a secret.
3481        for url in [
3482            "https://api.github.com/repos/owner/repo/actions/runners",
3483            "https://github.com/actions/runner/releases/download/v2.330.0/actions-runner-linux-x64-2.330.0.tar.gz",
3484            "https://github.com/o/r.git",
3485            "https://github.com/@owner/repo",
3486            "https://api.github.com/repos/o/r/actions/runners/42",
3487        ] {
3488            assert_eq!(redact(url), url, "over-redacted a diagnosable path: {url}");
3489        }
3490    }
3491
3492    /// A large message must not take the process down.
3493    ///
3494    /// The URL branch recursed into `redact_core` on both sides of the cut. The
3495    /// termination argument was sound — every call is on a strictly shorter
3496    /// slice — and it was silent on *depth*, which was linear in the length of
3497    /// the input. A message of ~2000 URL-carrying items, about 86 KB, exited
3498    /// `0xc00000fd STATUS_STACK_OVERFLOW`.
3499    ///
3500    /// **A stack overflow is not catchable — it aborts the process.** A large
3501    /// HTTP error body is attacker-influenceable content, so this was a way to
3502    /// kill the agent from inside its own log sink, which is worse than any
3503    /// leak: a sink that is not running redacts nothing.
3504    ///
3505    /// Note what this test can and cannot do. An overflow aborts the test
3506    /// binary rather than failing an assertion, so the value of this test is
3507    /// that **the suite completes at all**; the assertions below only confirm
3508    /// it did the work rather than skipping it.
3509    #[test]
3510    fn a_large_message_does_not_overflow_the_stack() {
3511        // 20000 items is ~860 KB and an order of magnitude past the ~2000 that
3512        // was measured to abort. It is also what the commit *before* the URL
3513        // branch survived, because that branch returned without recursing --
3514        // so this is a regression guard with the margin the regression had.
3515        const ITEMS: usize = 20_000;
3516        let body = r#"{"url":"https://api.github.com/repos/o/r"},"#.repeat(ITEMS);
3517        assert!(body.len() > 800_000, "the premise is a large body");
3518
3519        let redacted = redact(&body);
3520        assert!(
3521            redacted.contains("https://api.github.com/repos/o/r"),
3522            "the URLs are the diagnosable part and must survive"
3523        );
3524
3525        // The same depth, reached through the structural cut and through a
3526        // credential value rather than through a bare URL.
3527        let nested = format!("{{\"error\":{{\"body\":{{\"list\":[{}]}}}}}}", body);
3528        assert!(!redact(&nested).is_empty());
3529
3530        // A long run with no URL in it at all, so the structural loop is
3531        // exercised on its own.
3532        let flat = "{\"a\":1},".repeat(ITEMS);
3533        assert!(!redact(&flat).is_empty());
3534
3535        // And one long unbroken URL, where the *path* loop is what iterates.
3536        let long_path = format!("https://a.com/{}", "seg/".repeat(ITEMS));
3537        assert!(!redact(&long_path).is_empty());
3538    }
3539
3540    #[test]
3541    fn a_credential_header_loses_its_scheme_and_its_value() {
3542        assert_eq!(
3543            redact("Authorization: Bearer abc123"),
3544            format!("Authorization: {REDACTION} {REDACTION}")
3545        );
3546        assert_eq!(
3547            redact("Authorization: Bearer abc123, Accept: application/json"),
3548            // The comma ends the header value, so the next header's name is not
3549            // swallowed.
3550            format!("Authorization: {REDACTION} {REDACTION} Accept: application/json")
3551        );
3552        assert_eq!(
3553            redact("x-api-key=hunter2"),
3554            format!("x-api-key={REDACTION}")
3555        );
3556        assert_eq!(
3557            redact("Cookie: session=abc"),
3558            format!("Cookie: {REDACTION}")
3559        );
3560
3561        // A webhook signature is not a credential the way a token is -- it is
3562        // derived, and it verifies rather than authenticates -- but it has a
3563        // SHA-256's exact shape, so the digest carve-out rendered it as a
3564        // labelled 12-character prefix rather than redacting it. Twelve hex
3565        // characters of an HMAC are of no use to anybody, which is why the
3566        // carve-out is safe in general; the header is on the list anyway,
3567        // because there is no diagnostic worth having in a signature and one
3568        // entry is cheaper than the argument.
3569        let hmac = "9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08";
3570        assert_eq!(
3571            redact(&format!("X-Hub-Signature-256: sha256={hmac}")),
3572            format!("X-Hub-Signature-256: {REDACTION}")
3573        );
3574        // Pass one puts the value's wrapping quote back, so what comes out is
3575        // still the object that went in. It used to emit
3576        // `{"x-hub-signature-256":[redacted]"}` -- an unbalanced quote in a
3577        // line this module argues has to stay diagnosable.
3578        //
3579        // This case is also `redact_core`'s two-pass witness, and the comment
3580        // there names it: the `=` comes first and names nothing, but what
3581        // follows it is a whole HMAC -- non-empty -- so a merged pass inspects
3582        // that value and returns before ever reaching the `:` that names
3583        // `x-hub-signature-256`.
3584        assert_eq!(
3585            redact(&format!("{{\"x-hub-signature-256\":\"sha256={hmac}\"}}")),
3586            format!("{{\"x-hub-signature-256\":\"{REDACTION}\"}}")
3587        );
3588        // The SHA-1 spelling GitHub still sends alongside it.
3589        assert_eq!(
3590            redact(&format!("X-Hub-Signature: sha1={hmac}")),
3591            format!("X-Hub-Signature: {REDACTION}")
3592        );
3593        // The carve-out itself is untouched: a bare digest still renders as a
3594        // prefix, because a checksum gate that cannot say which digest it got
3595        // is a gate nobody can act on.
3596        assert_eq!(redact(hmac), "sha256:9f86d081884c…");
3597    }
3598
3599    #[test]
3600    fn ordinary_text_survives_intact() {
3601        // Redaction that eats the diagnostics is redaction nobody keeps. These
3602        // are the shapes this product's own log lines actually take.
3603        for text in [
3604            "reconciliation finished",
3605            "runner exited idle without work",
3606            "desired 3, active 1, headroom 2",
3607            "attempt 9f2c1a44-0000-4000-8000-000000000001 is busy",
3608            "http 403 after 2 retries",
3609            "windows/x64 is a documented pair",
3610        ] {
3611            assert_eq!(redact(text), text, "redaction damaged an ordinary message");
3612        }
3613    }
3614
3615    /// A SHA-256 is 64 opaque characters and would otherwise be swallowed by
3616    /// the opaque-run rule. `07-security.md` makes checksum verification a
3617    /// security gate, and a gate that cannot say *which* digest it got is a
3618    /// gate nobody can act on, so a digest renders as a labelled prefix.
3619    #[test]
3620    fn a_digest_renders_as_a_prefix_rather_than_disappearing() {
3621        let digest = "9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08";
3622
3623        assert_eq!(redact(digest), "sha256:9f86d081884c…");
3624        // The point of the change: expected-versus-actual stays legible.
3625        let other = "60303ae22b998861bce3b28f33eec1be758a213c86c93c076dbe9f558c11c752";
3626        assert_ne!(
3627            redact(digest),
3628            redact(other),
3629            "two digests must stay distinguishable"
3630        );
3631
3632        // In the sentence shape `e2` will actually write.
3633        assert_eq!(
3634            redact(&format!("checksum mismatch: expected {digest} got {other}")),
3635            "checksum mismatch: expected sha256:9f86d081884c… got sha256:60303ae22b99…"
3636        );
3637
3638        // A prefix a caller logged itself is below the threshold and untouched.
3639        assert_eq!(redact(&digest[..12]), digest[..12].to_string());
3640
3641        // An already-labelled digest is labelled once, not twice, and is
3642        // truncated like a bare one.
3643        //
3644        // Before `redact_core` recursed into a `:` value this arrived at
3645        // `redact_value` whole, matched nothing there -- a `:` is not an
3646        // opaque character -- and was printed in full. Recursing hands
3647        // `as_sha256_digest` the digest on its own, which re-attaches its own
3648        // label, so the redundant one is dropped rather than doubled.
3649        //
3650        // The truncation is the part worth having: a 64-character lowercase
3651        // hex run is only *usually* a digest, an HMAC-SHA256 signature has
3652        // the same shape, and `sha256:<hmac>` used to be emitted whole.
3653        assert_eq!(redact(&format!("sha256:{digest}")), "sha256:9f86d081884c…");
3654        // The `=` spelling had the same doubling and is fixed with it; the
3655        // caller's own separator survives either way.
3656        assert_eq!(redact(&format!("sha256={digest}")), "sha256=9f86d081884c…");
3657    }
3658
3659    #[test]
3660    fn the_digest_exception_is_exactly_sixty_four_lowercase_hex_and_nothing_else() {
3661        // The narrower this carve-out is, the less there is to reason about.
3662        // Anything that is not precisely a lowercase SHA-256 stays redacted by
3663        // the opaque-run rule.
3664        let digest = "9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08";
3665
3666        assert_eq!(
3667            redact(&digest.to_ascii_uppercase()),
3668            REDACTION,
3669            "uppercase hex is not the shape this exception recognises"
3670        );
3671        assert_eq!(
3672            redact(&format!("{digest}0")),
3673            REDACTION,
3674            "65 characters is not a SHA-256"
3675        );
3676        // 64 characters of base64, which is what an encoded secret of that
3677        // length looks like: not hex, so not exempt.
3678        let base64ish = "ZYXWVUTSRQPONMLKJIHGFEDCBA9876543210zyxwvutsrqponmlkjihgfedcba98";
3679        assert_eq!(base64ish.len(), 64);
3680        assert_eq!(redact(base64ish), REDACTION);
3681
3682        // And the JIT blob, which is the thing the opaque-run rule exists for,
3683        // must not have been weakened by any of this.
3684        assert_eq!(redact(JIT_BLOB), REDACTION);
3685    }
3686
3687    #[test]
3688    fn a_uuid_survives() {
3689        // 36 characters, below the opaque-run threshold on purpose: every
3690        // identifier in this product's domain model is a UUID, and redacting
3691        // them would make the logs useless.
3692        let id = "9f2c1a44-0000-4000-8000-000000000001";
3693        assert_eq!(redact(id), id);
3694        assert!(id.len() < OPAQUE_RUN_THRESHOLD);
3695    }
3696
3697    #[test]
3698    fn redaction_preserves_whitespace_and_line_structure() {
3699        let input = "line one\nAuthorization: Bearer x\nline three";
3700        let redacted = redact(input);
3701        assert_eq!(redacted.lines().count(), 3, "{redacted}");
3702        assert!(redacted.starts_with("line one\n"), "{redacted}");
3703        assert!(redacted.ends_with("\nline three"), "{redacted}");
3704    }
3705}