Skip to main content

runner_manager_platform/
logging.rs

1// owner: d1-platform-core
2
3//! The structured log sink, and the redaction that is not optional.
4//!
5//! `07-security.md`'s threat table, on *"Logs disclose repository secrets"*:
6//! **"Structured allowlist logging with unconditional redaction of tokens,
7//! headers, JIT blobs, and paths."** Its security gate is a *secret-injection
8//! log scan*, and its release gate is that *"the user access token and the
9//! encoded JIT configuration are absent from logs, databases, snapshots, crash
10//! reports, and CLI output"*.
11//!
12//! # Allowlist, not denylist — and why that is the whole design
13//!
14//! A denylist redacts the fields somebody remembered to name. It is correct on
15//! the day it is written and wrong on the day a later task adds a field, which
16//! is the day it matters. So this sink inverts the default: **a field whose
17//! name is not in [`ALLOWED_FIELDS`] has its value replaced outright.** A task
18//! that adds `runner_token` to a log call gets `[redacted]` with no review, no
19//! ceremony, and no leak. Adding a field to the allowlist is a deliberate edit
20//! to this file that shows up in a diff.
21//!
22//! The field name itself is kept. Knowing that an event carried a
23//! `runner_token` field is useful, and the name is not the secret.
24//!
25//! # Two layers, because one is not enough
26//!
27//! The allowlist protects *fields*. It cannot protect the message body, which
28//! has to be allowed or the logs say nothing — and a message body is exactly
29//! where a secret ends up when somebody writes
30//! `info!("failed with {authorization}")`. So every string that survives the
31//! allowlist is then scrubbed by value shape ([`redact`]): GitHub token
32//! prefixes, credential header names, long opaque runs such as an encoded JIT
33//! configuration, and filesystem paths.
34//!
35//! Scrubbing by shape over-redacts, deliberately. Some of that is worth knowing
36//! about before it surprises somebody:
37//!
38//! - A slash-rooted word is treated as a filesystem path, and a URL *path* on
39//!   its own looks exactly like one. Log a full URL — `https://api.github.com/…`
40//!   — and it survives intact; log a bare `/repos/owner/repo` and it becomes
41//!   `[path]`.
42//! - Any unbroken run of 40 or more base64, base64url, or hex characters is
43//!   treated as an opaque secret. There is exactly one carve-out: a value that
44//!   is precisely 64 lowercase hex characters is a SHA-256 digest, and renders
45//!   as a labelled 12-character prefix rather than disappearing. A digest is
46//!   not a secret, and `07-security.md` makes checksum verification a security
47//!   gate whose most useful diagnostic is expected-versus-actual.
48//! - A JSON Web Token is redacted whole. Its `.` separators split it into runs
49//!   the opaque-run rule is too short-sighted to catch, so it is recognised by
50//!   shape instead: two or three base64url segments whose header begins `eyJ`.
51//! - A URL keeps its scheme, host and path and loses everything that
52//!   authenticates it: the `user:password@` userinfo, the query string, and the
53//!   fragment. The userinfo matters more than it looks — a
54//!   token-authenticated git remote is
55//!   `https://x-access-token:ghu_…@github.com/owner/repo.git`, so it is the
56//!   shape a clone or fetch failure arrives in. The path stays diagnosable but
57//!   is not exempt: each segment goes through the same shape rules on its own,
58//!   because a token in `…/raw/ghu_…/f` is a token, and the alternative was the
59//!   one place the belt never ran. A 40-character git object name is opaque
60//!   enough to go with it.
61//! - A word ending in `:` or `=` whose stem is a credential header name causes
62//!   the next two words to be redacted, so `Authorization: Bearer ghu_…` loses
63//!   both the scheme and the token.
64//!
65//! A word is cut on structural punctuation — `,`, `;`, `{`, `}`, `[`, `]`,
66//! `<`, `>` and `&` — before any of that runs, and each fragment is then judged
67//! the way a whole word is: unwrapped, judged on its core, and re-emitted with
68//! its punctuation put back. Without that cut only the *first* key/value pair
69//! in a compact structure is ever examined, and redaction becomes a function of
70//! field order: `{"encoded_jit_config":"…"}` was caught and
71//! `{"runner_id":42,"encoded_jit_config":"…"}` was not, while
72//! `serde_json::to_string` is what decides which of the two an error body is.
73//! Nesting, a form-encoded body, a `;`-separated connection string and a plist
74//! element are all that same defect in different punctuation. So is an array
75//! element, one step further down — a fragment judged *with* its quote still
76//! attached matches no shape rule at all, and an array element is the one
77//! fragment that has no key of its own to give it away.
78//!
79//! Because the cut is flat, a credential's value does not have to be in the
80//! same fragment as the key that names it — `{"password":["hunter2"]}`,
81//! `{"password":{"v":"hunter2"}}` and a plist's
82//! `<key>password</key><string>hunter2</string>` all put it one or more
83//! fragments away — so a *carry* is threaded along the fragments to say that a
84//! key is still waiting for its value, or that a quoted value was cut before
85//! its closing quote. It steps over element names, because markup is not the
86//! value a key named, and it stops at the punctuation that visibly closes the
87//! value, because a redaction reported where no secret was is a false signal in
88//! the one log a reader consults to find out whether anything leaked.
89//!
90//! A URL is cut out of the text around it rather than being allowed to own the
91//! rest of the word. Its scheme is the run of scheme characters immediately
92//! before the `://`, and it ends at the first character that cannot appear in a
93//! URL — plus, *ahead of its query string only*, at a `;` or an `&`, which are
94//! structural characters everywhere else and are query syntax after the `?`.
95//! Everything on either side goes back through the rules. So
96//! `{"documentation_url":"https://…","token":"ghu_…"}` — which is what a GitHub
97//! REST error body looks like — keeps the URL and redacts the token, rather
98//! than the URL swallowing the token, and
99//! `Server=https://vault.local/api;Password=…` keeps the URL and redacts the
100//! password rather than the URL swallowing the `;` that separates them.
101//!
102//! **The text after a URL is iterated over, not recursed into.** That is a
103//! memory-safety property, not a style: recursing there made stack depth linear
104//! in the length of the message, and ~86 KB of URL-carrying JSON exited
105//! `STATUS_STACK_OVERFLOW`. A stack overflow is not catchable and takes the
106//! process with it, so an attacker-influenceable error body could kill the
107//! agent from inside its own log sink — and a sink that is not running redacts
108//! nothing at all. For the same reason every search in `split_url` is bounded
109//! by the URL's own end: a pass that never returns is as effective a denial as
110//! one that overflows.
111//!
112//! A key is also trimmed of backslashes, which a value never is: `Debug` on a
113//! `String` escapes the quotes inside it, so a body reached through
114//! `error!(reason = ?err)` spells its keys `\"password\"`. The `trim_key`
115//! function documents why the same trim must not be applied to a value; it is
116//! named rather than linked because it is private and this module's
117//! documentation is not.
118//!
119//! Every one of those is a case where being wrong costs a slightly less
120//! readable log line, against a case where being wrong the other way costs a
121//! disclosed credential.
122//!
123//! # What this does not do
124//!
125//! It does not stop a caller printing to standard output, and it does not
126//! redact a `Debug` derive somewhere else in the program. Those are held by
127//! different controls: `secrecy::SecretString` for the two values that matter
128//! (`07-security.md`'s credential inventory), and
129//! [`crate::process::SpawnSpec::spawn_with_handoff`] for the command line.
130
131use std::fmt;
132use std::io::Write;
133use std::path::{Path, PathBuf};
134
135use serde_json::{Map, Value};
136use tracing::field::{Field, Visit};
137use tracing::{Event, Subscriber};
138use tracing_subscriber::layer::{Context, Layer};
139use tracing_subscriber::registry::LookupSpan;
140
141/// What replaces a value this sink will not emit.
142pub const REDACTION: &str = "[redacted]";
143
144/// What replaces something that was recognisably a filesystem path. Distinct
145/// from [`REDACTION`] so a reader can tell "a path was here" from "a secret was
146/// here" without either being disclosed.
147pub const PATH_REDACTION: &str = "[path]";
148
149/// The field names this sink emits verbatim. Everything else is replaced with
150/// [`REDACTION`].
151///
152/// ---
153///
154/// **Read this before adding a name.** A field on this list still passes
155/// through [`redact`], so adding one does not switch redaction off — but it
156/// does mean the field's value reaches the scrubber instead of being discarded,
157/// and the scrubber only recognises shapes it was taught. Add a name only when
158/// the value is structurally incapable of carrying a credential: an
159/// identifier, an enumerated state, a count, a duration. Never add a name whose
160/// value is free text supplied by GitHub or by a workflow.
161///
162/// Kept sorted, and a test enforces that, so an addition is one line in a diff
163/// rather than a name buried in the middle of a list.
164pub const ALLOWED_FIELDS: &[&str] = &[
165    "arch",
166    "attempt",
167    "attempt_id",
168    "attempt_state",
169    "capacity",
170    "count",
171    "demand",
172    "desired",
173    "duration_ms",
174    "elapsed_ms",
175    "error_kind",
176    "event",
177    "exit_code",
178    "headroom",
179    "host_id",
180    "http_status",
181    "installation_id",
182    "job_id",
183    "label",
184    "lock",
185    "materialize_ms",
186    "message",
187    "mode",
188    "os",
189    "outcome",
190    "pid",
191    "policy_id",
192    "policy_state",
193    "reason",
194    "register_ms",
195    "retry_in_ms",
196    "runner_id",
197    "scope",
198    "spawn_ms",
199    "start_mode",
200    "state",
201    "target",
202    "version",
203];
204
205/// Header and parameter names whose value is a credential.
206///
207/// Compared case-insensitively with `-` and `_` treated as the same character,
208/// because the same header arrives as `Authorization`, `authorization`, and
209/// `x_api_key` depending on who wrote the line.
210const CREDENTIAL_KEYS: &[&str] = &[
211    "access.token",
212    "api.key",
213    "apikey",
214    "auth",
215    "authorization",
216    "client.secret",
217    "cookie",
218    "credential",
219    "encoded.jit.config",
220    "jit",
221    "jit.config",
222    "jitconfig",
223    "password",
224    "private.token",
225    "proxy.authorization",
226    "refresh.token",
227    "secret",
228    "set.cookie",
229    "token",
230    "www.authenticate",
231    "x.api.key",
232    "x.auth.token",
233    "x.github.token",
234    "x.hub.signature",
235    "x.hub.signature.256",
236];
237
238/// Authentication scheme words. Whatever follows one of these is the
239/// credential.
240const SCHEME_WORDS: &[&str] = &["basic", "bearer", "digest", "negotiate", "token"];
241
242/// Prefixes GitHub gives its credentials. Present as a belt on top of the
243/// opaque-run rule below, because a short-lived or truncated token can be under
244/// the length threshold while still being a live credential.
245const TOKEN_PREFIXES: &[&str] = &["ghp_", "gho_", "ghu_", "ghs_", "ghr_", "github_pat_", "gh_"];
246
247/// How long an unbroken run of opaque characters has to be before it is assumed
248/// to be a secret.
249///
250/// 40 rather than something shorter so that ordinary identifiers, hyphenated
251/// words, and UUIDs (36 characters, and not secret) survive; short enough that
252/// every GitHub credential format and every encoded JIT configuration is well
253/// past it.
254const OPAQUE_RUN_THRESHOLD: usize = 40;
255
256/// Characters that make up base64, base64url, and hex.
257fn is_opaque_char(c: char) -> bool {
258    c.is_ascii_alphanumeric() || matches!(c, '+' | '/' | '=' | '_' | '-')
259}
260
261/// Punctuation that may wrap a value without being part of it.
262const WRAPPERS: &[char] = &[
263    '"', '\'', '`', '(', ')', '[', ']', '{', '}', '<', '>', ',', ';', '.', '!', '?',
264];
265
266/// Punctuation that separates one key/value pair from the next *inside* a
267/// single whitespace-delimited word.
268///
269/// [`redact_core`] judges a fragment by splitting it once, on the first `=` or
270/// `:` it finds. That is enough for `key=value` and for a one-field object,
271/// and it is why every shape this module was tested against put the credential
272/// in the first pair. It is not enough for anything `serde_json::to_string`
273/// actually emits: a struct with two fields becomes
274/// `{"runner_id":42,"encoded_jit_config":"…"}`, where the only pair ever
275/// examined is `runner_id`. Nesting is the same defect one level down, and a
276/// form-encoded body is the same defect spelled with `&`.
277///
278/// So a word is cut on these before any of that runs, and each fragment is
279/// judged on its own, by [`redact_fragment`]. The separators go back verbatim,
280/// because the structure around a redaction is what keeps the line diagnosable.
281///
282/// `;` is here for the reason `&` is: it separates the pairs of a Windows
283/// connection string (`Server=host;Database=x;Password=…`), of a credential
284/// string, and of a cookie header written without a space after the separator.
285/// It was already in [`WRAPPERS`] — recognised as punctuation, and so never
286/// used to cut — which left `Set-Cookie: theme=dark; session=…` safe only
287/// because of the space. `d2` logs keychain, DPAPI and libsecret failures, and
288/// that is the shape they arrive in.
289///
290/// `<` and `>` are here because [`split_wrappers`] strips only the *outermost*
291/// pair, so `<string>ghu_…</string>` reached the rules as
292/// `string>ghu_…</string`, which is on no list and matches no shape. `d3`'s
293/// installers handle launchd plists, which is where that shape comes from.
294const STRUCTURAL: &[char] = &[',', ';', '{', '}', '[', ']', '<', '>', '&'];
295
296/// Whether this sink will emit a field's value rather than replacing it.
297#[must_use]
298pub fn is_field_allowed(name: &str) -> bool {
299    ALLOWED_FIELDS.binary_search(&name).is_ok()
300}
301
302/// Trims the punctuation a *key* can arrive wrapped in: [`WRAPPERS`], plus the
303/// backslash.
304///
305/// The backslash is here rather than in [`WRAPPERS`] on purpose, and the
306/// distinction is load-bearing in both directions.
307///
308/// It has to be trimmed somewhere. `tracing::error!(reason = ?err)` reaches
309/// this module through `record_debug` and `format!("{:?}")`, and `Debug` on a
310/// `String` escapes the quotes inside it — so an error whose `Debug` embeds an
311/// HTTP body arrives with its keys spelled `\"password\"`. Trimming only
312/// [`WRAPPERS`] leaves the backslash welded on, and `\"password\` is on no
313/// list. That is `d2`'s shape: a secret-store failure carrying the body it was
314/// handed.
315///
316/// It must not be trimmed unconditionally. [`split_wrappers`] runs before
317/// [`looks_like_path`], so a `\` in [`WRAPPERS`] would strip the leading
318/// `\\` that UNC detection keys on, and `\\server\share` would stop being
319/// recognised as a path. A key is the one place a backslash can never be part
320/// of the value, so it is the one place the trim is unconditional;
321/// [`trim_start_wrappers`] takes the same backslash off a *value* only when it
322/// is escaping punctuation.
323fn trim_key(key: &str) -> &str {
324    key.trim()
325        .trim_matches(|c: char| c == '\\' || WRAPPERS.contains(&c))
326}
327
328fn normalise_key(key: &str) -> String {
329    trim_key(key).to_ascii_lowercase().replace(['-', '_'], ".")
330}
331
332fn is_credential_key(key: &str) -> bool {
333    let key = normalise_key(key);
334    CREDENTIAL_KEYS.contains(&key.as_str())
335}
336
337fn is_scheme_word(word: &str) -> bool {
338    SCHEME_WORDS.contains(&word.to_ascii_lowercase().as_str())
339}
340
341/// Scrubs a string of anything that looks like a credential, an encoded JIT
342/// configuration, or a filesystem path.
343///
344/// Applied to every string this sink emits, including the ones whose field name
345/// is on [`ALLOWED_FIELDS`]. Also public so that the TUI and the CLI can put a
346/// value through the same rules before showing it, rather than inventing a
347/// second, differently wrong set.
348#[must_use]
349pub fn redact(text: &str) -> String {
350    let mut out = String::with_capacity(text.len());
351    // How many upcoming words to replace outright, because a credential key or
352    // an authentication scheme word said the value comes next.
353    let mut pending: u32 = 0;
354
355    for chunk in text.split_inclusive(char::is_whitespace) {
356        let (word, whitespace) = split_trailing_whitespace(chunk);
357
358        if word.is_empty() {
359            out.push_str(whitespace);
360            continue;
361        }
362
363        if pending > 0 {
364            pending -= 1;
365            out.push_str(REDACTION);
366            out.push_str(whitespace);
367            // A header value ends at a comma or semicolon; anything after that
368            // is the next header's name and is not a secret.
369            if word.ends_with([',', ';']) {
370                pending = 0;
371            }
372            continue;
373        }
374
375        let (rendered, follow_on) = redact_word(word);
376        out.push_str(&rendered);
377        out.push_str(whitespace);
378        pending = follow_on;
379    }
380
381    out
382}
383
384/// Splits a chunk produced by `split_inclusive` into its word and the single
385/// whitespace character that terminated it, if any.
386fn split_trailing_whitespace(chunk: &str) -> (&str, &str) {
387    match chunk.char_indices().next_back() {
388        Some((index, last)) if last.is_whitespace() => chunk.split_at(index),
389        _ => (chunk, ""),
390    }
391}
392
393/// Splits a fragment into its leading wrapping punctuation, its core, and its
394/// trailing wrapping punctuation.
395///
396/// Shared by [`redact_word`] and [`redact_core`], because both have to judge a
397/// fragment on its core while emitting it with its punctuation intact.
398///
399/// [`WRAPPERS`], and an *escaped* quote as well. Escaped text is exactly what
400/// a `Debug` rendering of a string is, and in it every quote arrives with a
401/// backslash welded on — so a value spelled `\"ghu_…\"` is wrapped in the
402/// same way `"ghu_…"` is, and none of the shape rules can see the token until
403/// the wrapper comes off. Trimming the key alone is not enough: `runner_token`
404/// is deliberately *not* on [`CREDENTIAL_KEYS`], because a name is not what
405/// makes a value a secret, so that pair is caught by the token-prefix rule
406/// reading its value or it is not caught at all.
407fn split_wrappers(fragment: &str) -> (&str, &str, &str) {
408    let leading = fragment.len() - trim_start_wrappers(fragment).len();
409    let (prefix, rest) = fragment.split_at(leading);
410    let core_len = trim_end_wrappers(rest).len();
411    let (core, suffix) = rest.split_at(core_len);
412    (prefix, core, suffix)
413}
414
415/// Trims wrapping punctuation from the front of a fragment.
416///
417/// A backslash counts as punctuation **only when it escapes punctuation**, and
418/// that restriction is the whole of what keeps this safe. `\"value` is a
419/// quoted value spelled the way an escaped rendering spells it, and the
420/// backslash is not part of the value. `\\server\share` is a UNC path whose
421/// leading backslashes escape nothing and *are* the value — trimming them is
422/// exactly what would stop [`looks_like_path`] recognising it, and this runs
423/// before [`looks_like_path`] does. `\\` is not a backslash-escaping-a-
424/// wrapper, so the two cases separate cleanly.
425fn trim_start_wrappers(fragment: &str) -> &str {
426    let mut rest = fragment;
427    loop {
428        let trimmed = rest.trim_start_matches(WRAPPERS);
429        let trimmed = match trimmed.strip_prefix('\\') {
430            Some(after) if after.starts_with(WRAPPERS) => after,
431            _ => trimmed,
432        };
433        if trimmed.len() == rest.len() {
434            return rest;
435        }
436        rest = trimmed;
437    }
438}
439
440/// Trims wrapping punctuation from the end of a fragment.
441///
442/// A trailing backslash goes with it, and needs no adjacency test: trimming
443/// runs right to left, so a backslash that has reached the end is one whose
444/// quote has already been taken off. A path that ends in a separator is still
445/// a path without it.
446fn trim_end_wrappers(fragment: &str) -> &str {
447    let mut rest = fragment;
448    loop {
449        let trimmed = rest.trim_end_matches(WRAPPERS).trim_end_matches('\\');
450        if trimmed.len() == rest.len() {
451            return rest;
452        }
453        rest = trimmed;
454    }
455}
456
457/// What a credential key left outstanding at the end of the text that named it.
458///
459/// The structural cut is flat and a credential's value does not have to be in
460/// the same fragment as the key that names it, so something has to carry the
461/// key across the cut. Nothing did, which is [`redact_core`]'s empty-value
462/// case: the comment there claimed the value would be reached "in the next
463/// fragment, where the structural cut reaches it", and no such thing happened.
464///
465/// The two variants are not the same claim, and collapsing them over-redacts:
466///
467/// - [`Carry::Expecting`] is *"a key named a value that has not appeared yet"*.
468///   It has to step over markup, because `<key>password</key><string>…</string>`
469///   puts `/key` and `string` between the key and its value.
470/// - [`Carry::Unclosed`] is *"a quoted value was redacted and its closing quote
471///   is not in this fragment"*, so the value continues past the cut. Inside an
472///   open quote `<` and `>` are literal text rather than markup, so this one
473///   must *not* step over anything.
474#[derive(Clone, Copy, PartialEq, Eq, Debug)]
475enum Carry {
476    /// Nothing outstanding.
477    None,
478    /// A credential key supplied no value of its own.
479    Expecting,
480    /// A redacted credential value was cut before its closing quote.
481    Unclosed,
482}
483
484/// Whether wrapping punctuation closes the value it followed.
485///
486/// A quote closes a string and a bracket closes a structure, so either one
487/// means the credential value ended here and a carry must stop. `,` and `;` are
488/// on the list for the reason [`redact`] already stops its word-level follow-on
489/// at them: they end a header value, and what comes after is the next header's
490/// name.
491///
492/// `>` is deliberately absent. It closes a *tag*, not a value —
493/// `<key>password</key>` ends in one with the credential's value still to come
494/// — and treating it as a terminator is what would leave the multi-word plist
495/// spelling leaking.
496fn closes_a_value(wrappers: &str) -> bool {
497    wrappers.contains(['"', '\'', '`', ',', ';', ']', '}', ')'])
498}
499
500/// Whether a value opened a quote that its own fragment did not close.
501fn opens_an_unclosed_quote(lead: &str, trail: &str) -> bool {
502    const QUOTES: [char; 3] = ['"', '\'', '`'];
503    lead.contains(QUOTES) && !trail.contains(QUOTES)
504}
505
506/// Whether a fragment sits where an element *name* goes rather than where its
507/// content does.
508///
509/// `<` and `>` are in [`STRUCTURAL`] because `<string>ghu_…</string>` reached
510/// the rules as `string>ghu_…</string` and matched nothing. Having cut on them,
511/// the loop knows which side of a tag it is on, and a [`Carry::Expecting`] must
512/// not be spent on `/key` or `string` when the value it is waiting for is the
513/// element's content. A tag name is still judged by every other rule — this
514/// says only that it is not the value a credential key named.
515fn is_tag_name(preceding: Option<char>, following: Option<char>) -> bool {
516    preceding == Some('<') && following == Some('>')
517}
518
519/// Redacts one whitespace-delimited word, and says how many following words the
520/// word implicates.
521fn redact_word(word: &str) -> (String, u32) {
522    // Wrapping punctuation is kept so that JSON-ish and prose context survives
523    // — `("ghu_…")` should become `("[redacted]")`, not `[redacted]`.
524    let (prefix, core, suffix) = split_wrappers(word);
525
526    if core.is_empty() {
527        return (word.to_string(), 0);
528    }
529
530    // A bare credential key or scheme word: the value is the next word.
531    //
532    // The stem is unwrapped before it is judged: spaced JSON writes the key as
533    // `"password":`, which leaves `password"` welded to a quote, and that is on
534    // no list. A long value survived this anyway by way of the opaque-run rule,
535    // so the gap only ever showed on a short one.
536    let stem = core.trim_end_matches([':', '=']);
537    if stem.len() < core.len() && is_credential_key(stem) {
538        // Two, so that `Authorization: Bearer <token>` loses the scheme and the
539        // token rather than only the scheme.
540        return (word.to_string(), 2);
541    }
542    if is_scheme_word(core) {
543        return (word.to_string(), 1);
544    }
545
546    let (rendered, carry) = redact_core(core, suffix);
547
548    // A value the word did not finish is the *next* word's problem, which is
549    // the follow-on rule the word level already has, reached from one level
550    // down. A pretty-printed plist is why it is needed: `<key>password</key>`
551    // and `<string>hunter2</string>` are on separate lines, so the key and its
552    // value are not even in the same word.
553    //
554    // The two carries ask for different amounts, for the same reason the stem
555    // rule above asks for two words and not one:
556    //
557    // - [`Carry::Expecting`] means the value has not been seen *at all*, so it
558    //   may be introduced by a word of its own — `<string>correct` and then
559    //   `horse</string>`. Two, exactly as `Authorization: Bearer <token>` needs
560    //   two.
561    // - [`Carry::Unclosed`] means the value has already started and was cut, so
562    //   only its remainder is outstanding. One.
563    //
564    // Trailing punctuation that closes the value withdraws the claim, for the
565    // same reason pass one declines an empty value: `{"password":""}` names a
566    // credential and supplies nothing, and redacting the next word there would
567    // report a secret where none was.
568    let follow_on = if closes_a_value(suffix) {
569        0
570    } else {
571        match carry {
572            Carry::None => 0,
573            Carry::Unclosed => 1,
574            Carry::Expecting => 2,
575        }
576    };
577    (format!("{prefix}{rendered}{suffix}"), follow_on)
578}
579
580/// Redacts one fragment of a cut-up word, judging it on its core.
581///
582/// [`redact_core`] used to recurse on the *raw* fragment, and its terminal
583/// fallback then handed that fragment to [`redact_value`] with its wrapping
584/// punctuation still attached. A fragment carrying no `:` or `=` of its own —
585/// which is exactly what an array element is — therefore reached the shape
586/// rules as `"ghu_…`, where `starts_with("ghu_")` fails, [`is_opaque_char`]
587/// fails on the quote, the `eyJ` test fails, and [`looks_like_path`] never gets
588/// a clean look at `"C:\Users\…`.
589///
590/// That is the defect the structural cut was written to close, one step further
591/// down: *"only the first key/value pair is ever examined"* became *"a value
592/// with no key of its own is never examined"*. It survived a round because the
593/// secret-injection scan had no array among its shapes, which is the same
594/// lesson in a different place — a check that cannot fail proves nothing.
595///
596/// So a fragment is treated the way [`redact_word`] treats a word: split,
597/// judge the core, put the punctuation back.
598///
599/// The *word-level* follow-on rule is still not repeated here, and the reason
600/// stands: *"the value is the next word"* is a statement about whitespace, and
601/// there is no next word inside a fragment. What that argument does not cover —
602/// and what an earlier spelling of it wrongly took itself to have settled — is
603/// the *fragment-level* claim: **the value is the next fragment**. That one is
604/// a statement about the structural cut, it is true, and nothing implemented
605/// it, which is why `{"password":["hunter2"]}` and
606/// `<key>password</key><string>hunter2</string>` went out whole. [`Carry`] is
607/// that claim, threaded through the loop in [`redact_core`].
608///
609/// `trailing` says this is the last fragment of its core, and it is what keeps
610/// the carry from escaping a word it has already been spent inside: consuming
611/// a claim in the middle of a core settles it, while consuming it at the end
612/// leaves open the possibility that the value runs on into the next word.
613fn redact_fragment(
614    fragment: &str,
615    carry: Carry,
616    tag_name: bool,
617    trailing: bool,
618) -> (String, Carry) {
619    let (prefix, core, suffix) = split_wrappers(fragment);
620    if core.is_empty() {
621        return (fragment.to_string(), carry);
622    }
623
624    let claimed = match carry {
625        Carry::None => false,
626        // Markup is not the value a credential key named.
627        Carry::Expecting => !tag_name,
628        // Inside an open quote there is no markup, only text.
629        Carry::Unclosed => true,
630    };
631
632    if claimed {
633        // Two different reasons to think the value has more to come, and one
634        // answer: it is `Unclosed` from here, never the `Expecting` it may have
635        // arrived as, because the value has now started.
636        //
637        // - `carry == Unclosed` — still inside an open quote, so the value runs
638        //   into the next fragment as well.
639        // - `trailing` — a claim spent on the *last* fragment of a core is one
640        //   whose value reached the end of the word with nothing closing it, so
641        //   it may continue into the next word.
642        //   `<key>password</key><string>` + `correct horse</string>` is that
643        //   shape.
644        //
645        // Anything else was spent in the middle of a core, where the value was
646        // and is done — and punctuation that closes a string or a structure
647        // settles it either way.
648        let next = if !closes_a_value(suffix) && (carry == Carry::Unclosed || trailing) {
649            Carry::Unclosed
650        } else {
651            Carry::None
652        };
653        return (format!("{prefix}{REDACTION}{suffix}"), next);
654    }
655
656    let (rendered, next) = redact_core(core, suffix);
657    let next = if closes_a_value(suffix) {
658        Carry::None
659    } else if next == Carry::None {
660        // **An unclaimed fragment preserves the claim rather than clearing
661        // it.** This is the whole of what lets a plist work: `/key` and
662        // `string` sit between `<key>password</key>` and the `<string>` content
663        // that is the value, they are tag names so they do not spend the claim,
664        // and a fragment that neither spends nor arms one must leave it exactly
665        // as it found it. Overwriting with this fragment's own (empty) result
666        // is how `<key>password</key><string>hunter2</string>` kept leaking
667        // after the carry existed.
668        carry
669    } else {
670        next
671    };
672    (format!("{prefix}{rendered}{suffix}"), next)
673}
674
675/// Redacts one word with its wrapping punctuation already removed, and reports
676/// what it left outstanding for whatever follows it.
677///
678/// `closing` is the wrapping punctuation the caller stripped off the end of
679/// this core. It is not emitted here — the caller puts it back — but the loop
680/// below has to see it, because the character that follows the *last* fragment
681/// is what says whether that fragment is an element name or an element's
682/// content.
683fn redact_core(core: &str, closing: &str) -> (String, Carry) {
684    // A URL survives, minus everything on it that carries a credential. The
685    // scheme, host and path are what makes a log line diagnosable.
686    //
687    // The URL is *cut out* of the text around it rather than being allowed to
688    // own the rest of the fragment, and both sides of the cut come back through
689    // here. [`split_url`] documents what the old unbounded `split_once("://")`
690    // cost. This still runs ahead of the structural cut, because a query string
691    // is `redact_url`'s to own: `?` and `#` are not structural characters, and
692    // an OAuth implicit-flow response puts the token after one of them.
693    //
694    // **The tail is iterated, not recursed into, and that is a memory-safety
695    // property rather than a matter of taste.** An earlier spelling called
696    // `redact_core` on the remainder, and argued only that the recursion
697    // *terminates*: every call is on a strictly shorter slice, which is true
698    // and is not enough. Termination says nothing about **depth**, and depth
699    // was linear in the length of the input — one frame per URL. A message of
700    // ~2000 `{"url":"https://…"},` items, about 86 KB, exited
701    // `0xc00000fd STATUS_STACK_OVERFLOW`.
702    //
703    // A stack overflow is not a `panic`: it is not catchable, it does not
704    // unwind, and it takes the process with it. A large HTTP error body is
705    // content an attacker can influence, so that was a way to kill the agent
706    // from inside its own log sink — and a sink that is not running redacts
707    // nothing at all, which is worse than any single leak.
708    //
709    // With the loop, every remaining re-entry is depth-bounded by construction
710    // rather than by argument:
711    //
712    // - `prefix` holds everything before the *first* `://`, so it contains no
713    //   `://` and cannot reach this branch again.
714    // - `rest` is what is left when the loop finds no further `://`, for the
715    //   same reason.
716    // - the structural branch below calls [`redact_fragment`], whose fragments
717    //   contain no structural character by construction and so cannot reach it
718    //   again either.
719    //
720    // Three levels, whatever the input is. Anything added here that recurses on
721    // a slice whose length is not bounded by a constant re-opens this.
722    if let Some((prefix, scheme, url, remainder)) = split_url(core) {
723        let mut out = String::with_capacity(core.len());
724        if !prefix.is_empty() {
725            out.push_str(&redact_core(prefix, "").0);
726        }
727        out.push_str(&redact_url(scheme, url));
728
729        let mut rest = remainder;
730        while let Some((prefix, scheme, url, remainder)) = split_url(rest) {
731            if !prefix.is_empty() {
732                out.push_str(&redact_core(prefix, "").0);
733            }
734            out.push_str(&redact_url(scheme, url));
735            rest = remainder;
736        }
737        if !rest.is_empty() {
738            out.push_str(&redact_core(rest, "").0);
739        }
740
741        // A URL is a complete value: it carries its own credentials in its own
742        // places, and `redact_url` has already dealt with them. Nothing is
743        // outstanding, which is also what keeps `{"token":"https://…"}` from
744        // arming a claim against whatever follows the URL.
745        return (out, Carry::None);
746    }
747
748    // A Windows drive path is `key:value`-shaped by accident, and
749    // `looks_like_path` only recognises the drive letter at position zero.
750    // Splitting such a path on its colon would hand the tail to a rule that
751    // matches nothing, so it has to be judged whole, before the separators
752    // get at it.
753    if looks_like_path(core) {
754        return (PATH_REDACTION.to_string(), Carry::None);
755    }
756
757    // A compact structure holds more than one key/value pair, and the rules
758    // below examine exactly one of them: `split_once` stops at the first
759    // separator it finds. So `{"encoded_jit_config":"…"}` was caught and
760    // `{"runner_id":42,"encoded_jit_config":"…"}` was not — the two differ by
761    // field order and by nothing else, and `serde_json::to_string` is what
762    // chooses the order. Nesting was the same defect: the value recursion
763    // below ends at `redact_value`, which is a leaf and never comes back
764    // here, so an object inside an object went out whole. A form-encoded body
765    // was the same defect again, with `&` as a separator nothing knew.
766    //
767    // Cutting on [`STRUCTURAL`] first turns all three into the shape the rules
768    // already handle, and does it at any depth: the cut is flat, so
769    // `{"a":{"b":{"token":"…"}}}` yields the same fragments a one-level object
770    // would.
771    //
772    // Each fragment goes through [`redact_fragment`] rather than straight back
773    // in here, because a fragment carries its own wrapping punctuation and a
774    // value judged with its quote attached matches nothing at all.
775    //
776    // The recursion terminates *and is depth-bounded*: a fragment contains no
777    // structural character by construction, so `redact_fragment` cannot reach
778    // this branch again, and the URL branch above iterates over its tail rather
779    // than recursing into it. The loop here is a loop for the same reason — a
780    // fragment per separator, at whatever length the input has.
781    //
782    // A [`Carry`] is threaded along it because the cut is flat and a
783    // credential's value need not be in the same fragment as its key. Without
784    // it the empty-value skip in pass one was a promise nothing kept: the value
785    // was said to be reachable "in the next fragment", and no next fragment
786    // ever heard about the key.
787    if core.contains(STRUCTURAL) {
788        let mut out = String::with_capacity(core.len());
789        let mut carry = Carry::None;
790        let mut rest = core;
791        // The separator on each side of a fragment is what says whether it is
792        // an element name or an element's content.
793        let mut preceding: Option<char> = None;
794        while let Some(index) = rest.find(STRUCTURAL) {
795            let (fragment, tail) = rest.split_at(index);
796            // Every character in `STRUCTURAL` is ASCII, so the separator is
797            // one byte, this cannot split a code point, and the byte is the
798            // whole character.
799            let separator = char::from(tail.as_bytes()[0]);
800            if !fragment.is_empty() {
801                let (rendered, next) = redact_fragment(
802                    fragment,
803                    carry,
804                    is_tag_name(preceding, Some(separator)),
805                    false,
806                );
807                out.push_str(&rendered);
808                carry = next;
809            }
810            out.push_str(&tail[..1]);
811            preceding = Some(separator);
812            rest = &tail[1..];
813        }
814        if !rest.is_empty() {
815            // The last fragment's *following* character is the caller's closing
816            // punctuation: in `<key>password</key>` the final `/key` is a tag
817            // name only because the `>` that ends it was stripped as a wrapper
818            // before this ran.
819            let following = closing.chars().next();
820            let (rendered, next) =
821                redact_fragment(rest, carry, is_tag_name(preceding, following), true);
822            out.push_str(&rendered);
823            carry = next;
824        }
825        return (out, carry);
826    }
827
828    // `key=value` and `key:value` in a single fragment.
829    //
830    // Pass one asks only whether either separator names a credential, and it
831    // runs to completion before any value is inspected, because the *first*
832    // separator in a fragment is not necessarily the one that names the key.
833    //
834    // The witness is a webhook signature header in its compact-JSON spelling,
835    // `{"x-hub-signature-256":"sha256=<hmac>"}`, and
836    // `a_credential_header_loses_its_scheme_and_its_value` holds it. The `=`
837    // comes first and names nothing, but what follows it is a whole HMAC —
838    // *non-empty*, so a merged pass inspects that value, renders it as a
839    // digest prefix, and returns without ever reaching the `:` that names
840    // `x-hub-signature-256`. `{"password":"a=b"}` is the same witness with
841    // nothing else going on: merged, it comes out whole.
842    //
843    // Two earlier spellings of this comment named
844    // `{"encoded_jit_config":"eyJ…In0="}` and then
845    // `{"encoded_jit_config":"eyJ…In0=","runner_id":42}`, and neither
846    // demonstrates the invariant: base64 padding is trailing-only, so the `=`
847    // split yields a value that `split_wrappers` trims to empty, and the
848    // `value.is_empty()` guard in pass two falls through to the `:` anyway. A
849    // comment naming a case that does not demonstrate its own invariant is how
850    // the invariant gets deleted by the next person, so the witness above is
851    // one that reds a named test when the two passes are merged.
852    //
853    // Both halves are judged through `trim_key`: compact JSON welds a quote to
854    // each, so the key arrives as `encoded_jit_config"`, and a `Debug`
855    // rendering welds a backslash as well.
856    // Whether pass one saw a credential key and was given nothing to redact.
857    let mut names_a_credential = false;
858
859    for separator in ['=', ':'] {
860        if let Some((key, raw_value)) = core.split_once(separator)
861            && is_credential_key(key)
862        {
863            // The value's own wrapping punctuation goes back, exactly as pass
864            // two puts it back. Dropping it emitted `{"password":[redacted]"}`
865            // — an unbalanced quote in a line this module argues, correctly,
866            // has to stay diagnosable, and a reader who cannot parse the line
867            // cannot tell a redaction from a truncation.
868            let (lead, value, trail) = split_wrappers(raw_value);
869
870            // An empty value is nothing to redact, and pass two declines one
871            // for the same reason. Claiming `[redacted]` here says a secret was
872            // somewhere none was, which is a false signal in the one log a
873            // reader consults to find out whether anything leaked.
874            //
875            // It is also load-bearing for the URL branch above, which sends
876            // the text before a URL back through here: `{"token":"https://…"}`
877            // leaves this pass a key of `token` and a value of nothing, and
878            // redacting that emitted `token":"[redacted]https://…` — the URL
879            // still standing behind the redaction meant to have replaced it.
880            //
881            // **What it is not is a reason to forget the key.** An earlier
882            // spelling of this comment said the value would be found "in the
883            // next fragment or the next word, where the structural cut and the
884            // follow-on rule reach it", and half of that was false: the
885            // follow-on rule does reach the next *word*, and nothing whatever
886            // reached the next *fragment*, because `redact_fragment`
887            // deliberately carries no follow-on. So `{"password":["hunter2"]}`,
888            // `{"password":{"v":"hunter2"}}` and `Password=;hunter2` — a
889            // credential key with its value one fragment away, which is what an
890            // array, a nested object and an empty pair all are — went out
891            // whole, with the key sitting in plain sight next to them.
892            //
893            // That is the failure this module warns about two comments down: a
894            // comment naming a case that does not demonstrate its own
895            // invariant. The claim is now true because [`Carry::Expecting`]
896            // implements it, and `the_fragment_carry_stops_where_the_value_does`
897            // holds both halves — that the claim is made, and that it is
898            // withdrawn where the value visibly ended.
899            if value.is_empty() {
900                names_a_credential = true;
901                continue;
902            }
903
904            // A quoted value whose closing quote is not in this fragment is a
905            // value the cut ran through the middle of: `{"password":"p&ss"}`
906            // reaches here as `password":"p`, and `ss` is the rest of the
907            // secret. Adding `;`, `<` and `>` to `STRUCTURAL` this round made
908            // three more characters able to do that, and punctuated passwords
909            // are ordinary.
910            let carry = if opens_an_unclosed_quote(lead, trail) {
911                Carry::Unclosed
912            } else {
913                Carry::None
914            };
915            return (format!("{key}{separator}{lead}{REDACTION}{trail}"), carry);
916        }
917    }
918
919    // Pass two: not a credential key, but the value may still be a path or a
920    // token: `runtime=/var/lib/runner-manager/…`. Applied to `:` as well as
921    // to `=`, because `:` is what compact JSON and a bare `key:value` use,
922    // and recursing for `=` alone is what let an encoded JIT configuration
923    // and a `runner_token:ghu_…` pair through verbatim.
924    for separator in ['=', ':'] {
925        if let Some((key, raw_value)) = core.split_once(separator) {
926            let (lead, value, trail) = split_wrappers(raw_value);
927            if value.is_empty() {
928                continue;
929            }
930            let redacted = redact_value(value);
931            // `as_sha256_digest` re-attaches its own `sha256:` label, so an
932            // already-labelled digest would come back doubled as
933            // `sha256:sha256:9f86d081884c…`. Dropping the redundant label
934            // keeps the caller's own key and separator, and leaves the digest
935            // truncated -- which the unrecursed `:` path never did, and which
936            // is worth having, because an HMAC-SHA256 signature has exactly a
937            // digest's shape and was previously printed in full.
938            if trim_key(key).eq_ignore_ascii_case("sha256")
939                && let Some(bare) = redacted.strip_prefix("sha256:")
940            {
941                return (format!("{key}{separator}{lead}{bare}{trail}"), Carry::None);
942            }
943            return (
944                format!("{key}{separator}{lead}{redacted}{trail}"),
945                Carry::None,
946            );
947        }
948    }
949
950    // A credential key with no value at all — either `password:` with nothing
951    // after it, or a bare `password` between a plist's tags. Both name a value
952    // that is somewhere else, and the caller is the only one who can see where.
953    let carry = if names_a_credential || is_credential_key(core) {
954        Carry::Expecting
955    } else {
956        Carry::None
957    };
958    (redact_value(core), carry)
959}
960
961/// Characters a URL scheme is made of: RFC 3986 allows a letter followed by
962/// letters, digits, `+`, `-` and `.`.
963fn is_scheme_byte(byte: u8) -> bool {
964    byte.is_ascii_alphanumeric() || matches!(byte, b'+' | b'-' | b'.')
965}
966
967/// Characters that cannot appear inside a URL, and therefore end one.
968///
969/// `\` is deliberately absent, even though it cannot appear in a URL either.
970/// The shape it would have been there for is a `Debug`-escaped body, which
971/// spells its quotes `\"` — and the quote already ends the URL. Making the
972/// backslash a terminator as well costs a real redaction: a Windows path used
973/// as a URL password, `https://x-access-token:C:\Users\…@github.com/o/r.git`,
974/// would then end its URL at the first `\`, which puts the `@` that identifies
975/// the userinfo *outside* the span, and [`redact_url`] echoes what it is given.
976/// Measured: with `\` a terminator, that line emits the path verbatim; without
977/// it, the userinfo is replaced as it should be.
978fn is_url_terminator(c: char) -> bool {
979    c.is_whitespace() || matches!(c, '"' | ',' | '{' | '}' | '[' | ']' | '<' | '>')
980}
981
982/// The same, plus the two [`STRUCTURAL`] characters that end a URL only *ahead
983/// of its query string*.
984///
985/// [`STRUCTURAL`] has nine characters and [`is_url_terminator`] recognises seven
986/// of them; `;` and `&` were missing. Because the URL branch runs ahead of the
987/// structural cut, a URL earlier in a word made [`split_url`] swallow
988/// everything to the next terminator — including the separator that would have
989/// cut the word — and [`redact_url`]'s terminal arm echoes what it is given. So
990/// `Server=https://vault.local/api;Password=hunter2` and
991/// `cb=https://a.com/x&access_token=ghu_…` went out whole: the fix that bounds
992/// a URL and the fix that cuts on `;` and `&` each worked alone and did not
993/// compose.
994///
995/// **Adding them to [`is_url_terminator`] outright is the obvious fix and the
996/// wrong one**, which is why there are two predicates rather than one extended
997/// list. `&` is what separates the parameters *inside* a query string, and
998/// [`redact_url`] replaces a query wholesale precisely because a token can be
999/// in any parameter and this module does not guess which. Measured:
1000/// `https://a.com/cb?code=1&state=hunter2` goes from `?[redacted]` to
1001/// `?[redacted]&state=hunter2`. `;` is the same story — it is a legal query
1002/// separator too.
1003///
1004/// Ahead of the `?` there is no such conflict: a `;` or an `&` in an authority
1005/// or a path is not URL syntax this module needs to keep, and something else in
1006/// the word is much the likelier reading.
1007fn is_url_head_terminator(c: char) -> bool {
1008    is_url_terminator(c) || matches!(c, ';' | '&')
1009}
1010
1011/// Finds the first URL in a fragment and cuts it out of the text around it.
1012///
1013/// Returns the text before the URL, its scheme, the URL itself with the `://`
1014/// removed, and the text after it — or `None` when the fragment holds no URL.
1015///
1016/// **Both bounds are the point.** `split_once("://")` treated *everything*
1017/// before the separator as the scheme and everything after it as the URL, and
1018/// the terminal arm of [`redact_url`] echoes both verbatim — so a single URL
1019/// anywhere in a word put the whole of the rest of that word beyond every rule
1020/// below it. `documentation_url` is in essentially every GitHub REST error
1021/// body, so that was *any* such body logged alongside a credential:
1022///
1023/// ```text
1024/// {"documentation_url":"https://docs.github.com/rest","token":"ghu_…"}
1025/// ```
1026///
1027/// came out intact. The reverse order leaked for the mirror reason —
1028/// everything before the `://` became the "scheme" — and a nested object behind
1029/// a URL was missed even when the URL's own userinfo was caught, because the
1030/// miss and the catch happened in the same call. The only thing that saved the
1031/// shape at all was a `?` or `#` *inside* the URL, which made [`redact_url`]
1032/// replace the tail.
1033///
1034/// An empty scheme run means the `://` is not introducing a URL, and the
1035/// fragment is left to the rules below rather than handed over as a URL with no
1036/// scheme.
1037fn split_url(fragment: &str) -> Option<(&str, &str, &str, &str)> {
1038    let separator = fragment.find("://")?;
1039    let before = &fragment[..separator];
1040
1041    // Scheme characters are ASCII, so a byte count is a character boundary.
1042    let scheme_len = before
1043        .bytes()
1044        .rev()
1045        .take_while(|byte| is_scheme_byte(*byte))
1046        .count();
1047    if scheme_len == 0 {
1048        return None;
1049    }
1050    let (prefix, scheme) = before.split_at(before.len() - scheme_len);
1051
1052    let after = &fragment[separator + "://".len()..];
1053
1054    // Where the URL ends no matter what, and **the bound every other search
1055    // here is taken inside**. Searching the rest of the fragment first is a
1056    // correct answer computed quadratically: a body of *n* URLs would scan the
1057    // whole remaining text once per URL looking for a `?` that is not there,
1058    // which on the 860 KB regression case in
1059    // `a_large_message_does_not_overflow_the_stack` is 17 billion character
1060    // comparisons. That test is a guard against this module taking the process
1061    // down, and a redaction pass that never returns takes it down just as
1062    // effectively as an overflow does.
1063    let end = after.find(is_url_terminator).unwrap_or(after.len());
1064    let url = &after[..end];
1065
1066    // The head is the authority and path — everything ahead of the first `?` or
1067    // `#`. Only there do `;` and `&` end the URL; inside a query string they
1068    // are query syntax, and the query is `redact_url`'s to replace wholesale.
1069    let query = url.find(['?', '#']).unwrap_or(url.len());
1070    let end = url[..query].find(is_url_head_terminator).unwrap_or(end);
1071    let (url, remainder) = after.split_at(end);
1072
1073    Some((prefix, scheme, url, remainder))
1074}
1075
1076/// Redacts the three places a URL can carry a credential, keeping the rest.
1077///
1078/// `scheme` is everything before `://` and `rest` everything after it.
1079///
1080/// The **userinfo** is the one this module used to miss, and it is not an
1081/// exotic shape: `https://x-access-token:ghu_…@github.com/owner/repo.git` is
1082/// the canonical token-authenticated git remote, so it is what `e2` and `e3`
1083/// will have in hand when a clone or a download fails and they log the error
1084/// they were given. The old `://` branch returned before the token-prefix and
1085/// opaque-run rules could run, so that URL came out intact.
1086///
1087/// A **fragment** is stripped for the same reason a query string is: an OAuth
1088/// implicit-flow response puts the token after the `#`, and the fragment is
1089/// never load-bearing for diagnosing an HTTP call.
1090fn redact_url(scheme: &str, rest: &str) -> String {
1091    // The authority runs to the first `/`, `?` or `#`.
1092    let authority_end = rest.find(['/', '?', '#']).unwrap_or(rest.len());
1093    let (authority, tail) = rest.split_at(authority_end);
1094
1095    // `rsplit_once`, not `split_once`: a password may itself contain an `@`,
1096    // and the host is what follows the *last* one.
1097    let host = match authority.rsplit_once('@') {
1098        // Replaced rather than deleted. That the request carried credentials at
1099        // all is diagnostic — it is often the answer to "why did this 401?" —
1100        // and it is the credential, not its existence, that must not be here.
1101        Some((_userinfo, host)) => format!("{REDACTION}@{host}"),
1102        None => authority.to_string(),
1103    };
1104
1105    // Whichever of `?` and `#` comes first ends the diagnosable part; anything
1106    // after it is replaced wholesale rather than parsed, because a token can be
1107    // in any parameter and this module does not guess which.
1108    match tail.find(['?', '#']) {
1109        Some(cut) => {
1110            let (path, query) = tail.split_at(cut);
1111            let separator = &query[..1];
1112            format!(
1113                "{scheme}://{host}{}{separator}{REDACTION}",
1114                redact_path(path)
1115            )
1116        }
1117        None => format!("{scheme}://{host}{}", redact_path(tail)),
1118    }
1119}
1120
1121/// Applies the shape rules to a URL path, one segment at a time.
1122///
1123/// The path is the diagnosable part of a URL and stays that way: a segment is
1124/// judged on its own, and an ordinary one — `repos`, `owner`, `actions-runner-
1125/// linux-x64-2.330.0.tar.gz` — is not a secret and is not touched. What this
1126/// closes is that [`redact_url`] previously applied *no* rule to a path at all,
1127/// so `https://github.com/o/r/raw/ghu_…/f` was echoed whole while the identical
1128/// token one character to the left of the `/` would have been replaced. The
1129/// token-prefix rule is documented as a belt that catches a credential
1130/// *anywhere*, and a path was the one place it never ran.
1131///
1132/// Segment at a time rather than whole, because [`redact_value`] would
1133/// otherwise see the `/` and hand the whole path to [`looks_like_path`], which
1134/// is exactly the over-redaction the module documentation promises a full URL
1135/// escapes.
1136///
1137/// Worth knowing: [`OPAQUE_RUN_THRESHOLD`] is 40, and a git object name written
1138/// as hex is exactly 40 characters, so a `…/raw/<sha1>/f` URL loses its commit
1139/// to `[redacted]`. That is the module's standing trade — a less readable line
1140/// against a disclosed credential — and it is called out here because a commit
1141/// SHA is the one path segment somebody may miss.
1142fn redact_path(path: &str) -> String {
1143    let mut out = String::with_capacity(path.len());
1144    for (index, segment) in path.split('/').enumerate() {
1145        if index > 0 {
1146            out.push('/');
1147        }
1148        if !segment.is_empty() {
1149            out.push_str(&redact_value(segment));
1150        }
1151    }
1152    out
1153}
1154
1155/// The shape rules, applied to a bare value.
1156fn redact_value(value: &str) -> String {
1157    let lower = value.to_ascii_lowercase();
1158    if TOKEN_PREFIXES
1159        .iter()
1160        .any(|prefix| lower.starts_with(prefix))
1161    {
1162        return REDACTION.to_string();
1163    }
1164
1165    if looks_like_path(value) {
1166        return PATH_REDACTION.to_string();
1167    }
1168
1169    // A digest is not a secret, and `07-security.md` makes checksum
1170    // verification a security gate. The most useful thing `e2` can write when
1171    // that gate fails is expected-versus-actual, and until this carve-out
1172    // existed both sides came out as `[redacted]` — a gate that reports it
1173    // failed and refuses to say how.
1174    if let Some(digest) = as_sha256_digest(value) {
1175        return digest;
1176    }
1177
1178    if looks_like_jwt(value) {
1179        return REDACTION.to_string();
1180    }
1181
1182    if value.len() >= OPAQUE_RUN_THRESHOLD && value.chars().all(is_opaque_char) {
1183        return REDACTION.to_string();
1184    }
1185
1186    value.to_string()
1187}
1188
1189/// The length of a SHA-256 digest written as lowercase hex.
1190const SHA256_HEX_LEN: usize = 64;
1191
1192/// How much of a digest is shown.
1193///
1194/// 12 is the short-digest convention git and the OCI tooling use: 48 bits, far
1195/// more than enough to tell an expected digest from the one that was actually
1196/// computed, which is the only comparison a checksum failure needs. Truncating
1197/// is also what makes this carve-out safe to have at all — a 64-character
1198/// lowercase hex run is *usually* a digest, but an HMAC-SHA256 signature has
1199/// the same shape, and 12 of its 64 characters are of no use to anybody.
1200const DIGEST_PREFIX_LEN: usize = 12;
1201
1202/// Renders a value that is exactly a lowercase SHA-256 digest as a labelled
1203/// prefix, or `None` when it is not one.
1204///
1205/// Deliberately strict: exactly 64 characters, and lowercase only. An uppercase
1206/// or mixed-case run falls through to the opaque-run rule and is redacted,
1207/// because the narrower this exception is, the less there is to reason about.
1208fn as_sha256_digest(value: &str) -> Option<String> {
1209    let is_digest = value.len() == SHA256_HEX_LEN
1210        && value
1211            .chars()
1212            .all(|c| c.is_ascii_digit() || c.is_ascii_lowercase() && c.is_ascii_hexdigit());
1213
1214    is_digest.then(|| format!("sha256:{}…", &value[..DIGEST_PREFIX_LEN]))
1215}
1216
1217/// The most `.`-separated segments a JSON Web Token has: header, payload,
1218/// signature.
1219const JWT_SEGMENTS: usize = 3;
1220
1221/// Whether a value is a JSON Web Token.
1222///
1223/// The opaque-run rule cannot see one. [`is_opaque_char`] excludes `.`, and a
1224/// JWT is base64url runs joined by two of them, so a 100-character credential
1225/// arrives as three runs that are each under the threshold and prints
1226/// verbatim. `Authorization: Bearer <jwt>` is caught by the scheme-word rule
1227/// and a credential-keyed one by the key rule, so this only ever bit a token
1228/// logged bare or under a name nobody listed — which is what a GitHub App
1229/// installation assertion is when it reaches a log at all.
1230///
1231/// Narrow deliberately, because the obvious fix is the wrong one: adding `.`
1232/// to [`is_opaque_char`] would swallow every long dotted word there is —
1233/// package names, dated filenames, dotted identifiers. A JWT header is
1234/// base64url of `{"alg":…`, which always begins `eyJ`, and that is a
1235/// discriminator ordinary text does not have.
1236fn looks_like_jwt(value: &str) -> bool {
1237    if value.len() < OPAQUE_RUN_THRESHOLD || !value.starts_with("eyJ") {
1238        return false;
1239    }
1240
1241    let mut segments = 0usize;
1242    for segment in value.split('.') {
1243        segments += 1;
1244        if !segment.chars().all(is_opaque_char) {
1245            return false;
1246        }
1247    }
1248
1249    // Two segments as well as three: an unsigned token has an empty signature,
1250    // and the trailing `.` that would have made the third is trimmed as
1251    // wrapping punctuation before this is reached.
1252    (2..=JWT_SEGMENTS).contains(&segments)
1253}
1254
1255fn looks_like_path(value: &str) -> bool {
1256    let bytes = value.as_bytes();
1257
1258    // `C:\Users\…` and `C:/Users/…`.
1259    if bytes.len() >= 3
1260        && bytes[0].is_ascii_alphabetic()
1261        && bytes[1] == b':'
1262        && (bytes[2] == b'\\' || bytes[2] == b'/')
1263    {
1264        return true;
1265    }
1266
1267    // `\\server\share\…`.
1268    if value.starts_with("\\\\") {
1269        return true;
1270    }
1271
1272    // `~/…`.
1273    if value.starts_with("~/") || value.starts_with("~\\") {
1274        return true;
1275    }
1276
1277    // `/var/lib/…`. Two segments required, so a lone `/` and a bare `/tmp`
1278    // stay readable; see the module documentation for why a URL path is caught
1279    // by this too.
1280    if let Some(rest) = value.strip_prefix('/') {
1281        return rest.contains('/') && !rest.is_empty();
1282    }
1283
1284    false
1285}
1286
1287// ---------------------------------------------------------------------------
1288// The tracing layer
1289// ---------------------------------------------------------------------------
1290
1291/// The redacted fields recorded on a span, stashed in the span's extensions so
1292/// that an event inside the span can carry them.
1293#[derive(Debug, Clone)]
1294struct SpanFields(Map<String, Value>);
1295
1296/// Collects a `tracing` record into JSON, applying the allowlist and the
1297/// scrubber as it goes.
1298#[derive(Debug, Default)]
1299struct RedactingVisitor {
1300    fields: Map<String, Value>,
1301}
1302
1303impl RedactingVisitor {
1304    fn put_str(&mut self, name: &str, value: &str) {
1305        let rendered = if is_field_allowed(name) {
1306            redact(value)
1307        } else {
1308            REDACTION.to_string()
1309        };
1310        self.fields
1311            .insert(name.to_string(), Value::String(rendered));
1312    }
1313
1314    fn put_value(&mut self, name: &str, value: Value) {
1315        if is_field_allowed(name) {
1316            self.fields.insert(name.to_string(), value);
1317        } else {
1318            // Numbers and booleans go the same way as strings. A credential is
1319            // never an `i64`, but the rule that makes this sink trustworthy is
1320            // that it has *no* exception for a name nobody listed.
1321            self.fields
1322                .insert(name.to_string(), Value::String(REDACTION.to_string()));
1323        }
1324    }
1325}
1326
1327impl Visit for RedactingVisitor {
1328    fn record_str(&mut self, field: &Field, value: &str) {
1329        self.put_str(field.name(), value);
1330    }
1331
1332    fn record_debug(&mut self, field: &Field, value: &dyn fmt::Debug) {
1333        // The event message arrives here, as `format_args!` output.
1334        self.put_str(field.name(), &format!("{value:?}"));
1335    }
1336
1337    fn record_error(&mut self, field: &Field, value: &(dyn std::error::Error + 'static)) {
1338        self.put_str(field.name(), &value.to_string());
1339    }
1340
1341    fn record_bool(&mut self, field: &Field, value: bool) {
1342        self.put_value(field.name(), Value::Bool(value));
1343    }
1344
1345    fn record_i64(&mut self, field: &Field, value: i64) {
1346        self.put_value(field.name(), Value::from(value));
1347    }
1348
1349    fn record_u64(&mut self, field: &Field, value: u64) {
1350        self.put_value(field.name(), Value::from(value));
1351    }
1352
1353    fn record_f64(&mut self, field: &Field, value: f64) {
1354        self.put_value(field.name(), Value::from(value));
1355    }
1356
1357    fn record_i128(&mut self, field: &Field, value: i128) {
1358        self.put_str(field.name(), &value.to_string());
1359    }
1360
1361    fn record_u128(&mut self, field: &Field, value: u128) {
1362        self.put_str(field.name(), &value.to_string());
1363    }
1364}
1365
1366/// A `tracing` layer that writes one redacted JSON object per event.
1367///
1368/// Written here rather than assembled from `tracing_subscriber::fmt` because
1369/// redaction has to happen *before* formatting, and a `tracing::Event`'s fields
1370/// cannot be rewritten for a downstream formatter to pick up. Owning the
1371/// formatting is what makes "unconditional" true: there is no path from an
1372/// event to the output that does not go through
1373/// [`RedactingVisitor`].
1374#[derive(Debug, Clone)]
1375pub struct RedactingLayer<W> {
1376    writer: W,
1377}
1378
1379impl<W> RedactingLayer<W> {
1380    /// Wraps a writer factory — a file appender, a capture buffer, or
1381    /// `std::io::stderr`.
1382    pub const fn new(writer: W) -> Self {
1383        Self { writer }
1384    }
1385}
1386
1387impl<S, W> Layer<S> for RedactingLayer<W>
1388where
1389    S: Subscriber + for<'a> LookupSpan<'a>,
1390    W: for<'a> tracing_subscriber::fmt::MakeWriter<'a> + 'static,
1391{
1392    fn on_new_span(
1393        &self,
1394        attrs: &tracing::span::Attributes<'_>,
1395        id: &tracing::span::Id,
1396        ctx: Context<'_, S>,
1397    ) {
1398        let mut visitor = RedactingVisitor::default();
1399        attrs.record(&mut visitor);
1400        if let Some(span) = ctx.span(id) {
1401            span.extensions_mut().insert(SpanFields(visitor.fields));
1402        }
1403    }
1404
1405    fn on_record(
1406        &self,
1407        id: &tracing::span::Id,
1408        values: &tracing::span::Record<'_>,
1409        ctx: Context<'_, S>,
1410    ) {
1411        let mut visitor = RedactingVisitor::default();
1412        values.record(&mut visitor);
1413        if let Some(span) = ctx.span(id) {
1414            let mut extensions = span.extensions_mut();
1415            if let Some(existing) = extensions.get_mut::<SpanFields>() {
1416                existing.0.extend(visitor.fields);
1417            } else {
1418                extensions.insert(SpanFields(visitor.fields));
1419            }
1420        }
1421    }
1422
1423    fn on_event(&self, event: &Event<'_>, ctx: Context<'_, S>) {
1424        let mut visitor = RedactingVisitor::default();
1425        event.record(&mut visitor);
1426
1427        let mut record = Map::new();
1428        record.insert(
1429            "timestamp".to_string(),
1430            Value::String(chrono::Utc::now().to_rfc3339()),
1431        );
1432        record.insert(
1433            "level".to_string(),
1434            Value::String(event.metadata().level().to_string()),
1435        );
1436        // Named `logger` and not `target`: `target` is also a domain field —
1437        // the repository or organization a policy points at — and two different
1438        // things under one key is how a diagnostic becomes a puzzle.
1439        record.insert(
1440            "logger".to_string(),
1441            Value::String(event.metadata().target().to_string()),
1442        );
1443        record.insert("fields".to_string(), Value::Object(visitor.fields));
1444
1445        let spans: Vec<Value> = ctx
1446            .event_scope(event)
1447            .into_iter()
1448            .flat_map(tracing_subscriber::registry::Scope::from_root)
1449            .map(|span| {
1450                let mut entry = Map::new();
1451                entry.insert("name".to_string(), Value::String(span.name().to_string()));
1452                if let Some(fields) = span.extensions().get::<SpanFields>() {
1453                    entry.insert("fields".to_string(), Value::Object(fields.0.clone()));
1454                }
1455                Value::Object(entry)
1456            })
1457            .collect();
1458        if !spans.is_empty() {
1459            record.insert("spans".to_string(), Value::Array(spans));
1460        }
1461
1462        let mut line = serde_json::to_string(&Value::Object(record)).unwrap_or_else(|_| {
1463            String::from(
1464                r#"{"level":"ERROR","fields":{"message":"a log record could not be encoded"}}"#,
1465            )
1466        });
1467        line.push('\n');
1468
1469        // A failed write to the diagnostics file must not take the agent down,
1470        // and must not be reported through `tracing` either — that would
1471        // recurse straight back into this method.
1472        let mut writer = self.writer.make_writer();
1473        let _ = writer.write_all(line.as_bytes());
1474        let _ = writer.flush();
1475    }
1476}
1477
1478// ---------------------------------------------------------------------------
1479// Installing the sink
1480// ---------------------------------------------------------------------------
1481
1482/// Whose diagnostics these are, and therefore which file they go in.
1483///
1484/// # Why the daemon does not share the operator's file
1485///
1486/// On the two Unixes a boot-mode registration runs the daemon as `root` while
1487/// the four application-data directories stay in the operator's profile —
1488/// `05-infrastructure.md` puts them there and `service install` records those
1489/// paths into the plist. So two accounts write into one `logs/` directory, and
1490/// the appender creates its file with the umask default, `0644`. Whichever
1491/// account opened today's file first owns it, and if that was `root` the
1492/// operator's own `runner-manager status` can no longer append to it.
1493///
1494/// That was not a degraded log. `tracing_appender::rolling::daily` **panics**
1495/// when it cannot open the file, so every CLI command on such a host died with
1496/// a backtrace before it did anything — reported on 0.1.17, on a host whose
1497/// daemon had rolled the file over at midnight as `root`:
1498///
1499/// ```text
1500/// thread 'main' panicked at rolling.rs:156:14:
1501/// initializing rolling file appender failed: InitError { context: "failed to
1502/// create log file", source: Os { code: 13, kind: PermissionDenied } }
1503/// ```
1504///
1505/// [`install`] no longer panics — see there — but not panicking would only have
1506/// turned a crash into an operator who never gets diagnostics again. The two
1507/// writers are separated instead, so neither can take the other's file: the
1508/// account that runs the daemon owns [`SERVICE_LOG_STEM`] and the operator owns
1509/// [`OPERATOR_LOG_STEM`], for as long as the registration lives.
1510#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
1511pub enum LogRole {
1512    /// A command the operator ran, or a daemon they started in the foreground.
1513    Operator,
1514    /// The daemon a service manager started, which on a boot-mode host is a
1515    /// different account from the operator's.
1516    Service,
1517}
1518
1519/// The file stem [`LogRole::Operator`] writes.
1520pub const OPERATOR_LOG_STEM: &str = "runner-manager.log";
1521/// The file stem [`LogRole::Service`] writes, and what `service status`
1522/// reports as the daemon's log.
1523pub const SERVICE_LOG_STEM: &str = "runner-manager.service.log";
1524
1525impl LogRole {
1526    /// The file stem this role writes, before the appender's date suffix.
1527    #[must_use]
1528    pub const fn file_stem(self) -> &'static str {
1529        match self {
1530            Self::Operator => OPERATOR_LOG_STEM,
1531            Self::Service => SERVICE_LOG_STEM,
1532        }
1533    }
1534}
1535
1536impl fmt::Display for LogRole {
1537    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
1538        f.write_str(match self {
1539            Self::Operator => "operator",
1540            Self::Service => "service",
1541        })
1542    }
1543}
1544
1545/// The sink could not be installed.
1546#[derive(Debug, thiserror::Error)]
1547pub enum LoggingError {
1548    /// The application-data directories could not be created.
1549    ///
1550    /// Carries a [`crate::paths::PathsError`] rather than a bare
1551    /// [`std::io::Error`] because [`install`] creates `logs/` through
1552    /// [`crate::paths::AppPaths::create_all`], which is what applies the `0700`
1553    /// restriction; the source therefore names whichever of the four
1554    /// directories actually failed, and that need not be `logs/`.
1555    #[error("cannot create the log directory {}: {source}", directory.display())]
1556    Directory {
1557        /// The diagnostics directory [`install`] was asked to write into.
1558        directory: PathBuf,
1559        /// The underlying error.
1560        #[source]
1561        source: crate::paths::PathsError,
1562    },
1563
1564    /// No diagnostics file in `logs/` could be opened for appending.
1565    ///
1566    /// Both stems are named because [`install`] tries two — the role's own, and
1567    /// then one qualified by the account — and an operator who is told only
1568    /// about the second would go looking for a file this process never reached
1569    /// for first.
1570    ///
1571    /// The two attempts are composed into one string rather than carried as
1572    /// four fields. As four this variant reached 128 bytes on Windows — where
1573    /// `PathBuf` is wider than on the Unixes — and tripped
1574    /// `clippy::result_large_err`, which is a lint about every *success* path
1575    /// paying for the size of a failure. The message is the same either way,
1576    /// and only this error is ever built.
1577    #[error("cannot open a diagnostics file in {}: {attempts}", directory.display())]
1578    Appender {
1579        /// The diagnostics directory.
1580        directory: PathBuf,
1581        /// Both stems that were tried, and why each was refused.
1582        attempts: String,
1583    },
1584
1585    /// A global subscriber was already installed.
1586    #[error("a tracing subscriber is already installed for this process: {message}")]
1587    AlreadyInstalled {
1588        /// What `tracing_subscriber` said.
1589        message: String,
1590    },
1591}
1592
1593/// Keeps the background log-writing thread alive.
1594///
1595/// Dropping it flushes and stops that thread, so the value must be held for as
1596/// long as the program expects its diagnostics to be written. Losing it is a
1597/// silent, total loss of logs, which is why it is `#[must_use]`.
1598#[must_use = "dropping the guard stops the log writer and silently discards later diagnostics"]
1599#[derive(Debug)]
1600pub struct LoggingGuard {
1601    _worker: tracing_appender::non_blocking::WorkerGuard,
1602}
1603
1604/// Installs the redacting sink as this process's global subscriber, writing
1605/// daily-rotating files into `logs/`.
1606///
1607/// `role` decides the file, for the reason [`LogRole`] gives. `default_filter`
1608/// applies when `RUST_LOG` is unset or unparseable.
1609///
1610/// # Errors
1611///
1612/// [`LoggingError::Directory`], [`LoggingError::Appender`] and
1613/// [`LoggingError::AlreadyInstalled`]. **None of them is fatal to the caller**,
1614/// and the CLI treats all three as a warning: a `host show` that refused to
1615/// print a capacity because a log file could not be opened would be a worse
1616/// failure than the one it was reporting.
1617pub fn install(
1618    paths: &crate::paths::AppPaths,
1619    role: LogRole,
1620    default_filter: &str,
1621) -> Result<LoggingGuard, LoggingError> {
1622    use tracing_subscriber::layer::SubscriberExt as _;
1623    use tracing_subscriber::util::SubscriberInitExt as _;
1624
1625    // `AppPaths::create_all`, not `create_dir_all`. The claim `create_all`
1626    // documents — that a diagnostics file is not readable by other local
1627    // accounts — is only true if `logs/` is created at `0700`, and this is the
1628    // path a running daemon actually takes. Creating the directory here with a
1629    // bare `create_dir_all` left it at the umask default whenever `install`
1630    // won the race to create it, and `tracing_appender` then wrote 0644 files
1631    // into it: the invariant held in the test that asserts it and nowhere else.
1632    let directory = paths.logs_dir().to_path_buf();
1633    paths
1634        .create_all()
1635        .map_err(|source| LoggingError::Directory {
1636            directory: directory.clone(),
1637            source,
1638        })?;
1639
1640    let appender = open_appender(&directory, role)?;
1641    let (writer, worker) = tracing_appender::non_blocking(appender);
1642
1643    let filter = tracing_subscriber::EnvFilter::try_from_default_env()
1644        .unwrap_or_else(|_| tracing_subscriber::EnvFilter::new(default_filter));
1645
1646    tracing_subscriber::registry()
1647        .with(filter)
1648        .with(RedactingLayer::new(writer))
1649        .try_init()
1650        .map_err(|error| LoggingError::AlreadyInstalled {
1651            message: error.to_string(),
1652        })?;
1653
1654    Ok(LoggingGuard { _worker: worker })
1655}
1656
1657/// Opens today's diagnostics file, falling back to one only this account can
1658/// have created.
1659///
1660/// # Why there is a second attempt at all
1661///
1662/// [`LogRole`] keeps the daemon and the operator apart, which is enough on a
1663/// host where nothing else ever writes here. Two things still put a file the
1664/// caller cannot append to under the stem it wants:
1665///
1666/// 1. **A privileged command.** Signing in to the machine-scoped store is
1667///    `sudo runner-manager auth login`, and that run writes the *operator's*
1668///    stem as `root`. Every unprivileged command for the rest of that day then
1669///    finds a `root`-owned file under it.
1670/// 2. **A host upgraded into this change.** Files written before the roles were
1671///    split are all under [`OPERATOR_LOG_STEM`], and on a boot-mode host the
1672///    ones from the last few days belong to `root`.
1673///
1674/// Neither is worth losing diagnostics over, and neither can be repaired from
1675/// inside an unprivileged process — it may not chown the file, may not change
1676/// its mode, and must not delete an existing log. So it writes beside it, under
1677/// a stem carrying this account's identity, which no other account will choose.
1678///
1679/// The fallback is deliberately *not* the first choice. A file named after
1680/// whoever happened to open it first is a file an operator has to go looking
1681/// for; the plain stem stays the plain stem, and the qualified one appears only
1682/// on a host that has the collision.
1683fn open_appender(
1684    directory: &Path,
1685    role: LogRole,
1686) -> Result<tracing_appender::rolling::RollingFileAppender, LoggingError> {
1687    let stem = role.file_stem();
1688    let first = match build_appender(directory, stem) {
1689        Ok(appender) => return Ok(appender),
1690        Err(error) => error,
1691    };
1692
1693    let qualified = account_qualified_stem(stem);
1694    match build_appender(directory, &qualified) {
1695        Ok(appender) => Ok(appender),
1696        // Both stems are named. An operator told only about the second would go
1697        // looking for a file this process never reached for first.
1698        Err(second) => Err(LoggingError::Appender {
1699            directory: directory.to_path_buf(),
1700            attempts: format!(
1701                "neither {stem}.<date> ({first}) nor {qualified}.<date> ({second}) could be \
1702                 appended to"
1703            ),
1704        }),
1705    }
1706}
1707
1708/// One attempt, through the constructor that **returns** its failure.
1709///
1710/// `tracing_appender::rolling::daily` is the same thing with an `.expect` on
1711/// the end, and that `.expect` is a panic in the middle of an operator's `status`
1712/// command. The whole reason this function exists is that the builder hands the
1713/// error back instead.
1714fn build_appender(
1715    directory: &Path,
1716    stem: &str,
1717) -> Result<tracing_appender::rolling::RollingFileAppender, String> {
1718    tracing_appender::rolling::RollingFileAppender::builder()
1719        .rotation(tracing_appender::rolling::Rotation::DAILY)
1720        .filename_prefix(stem)
1721        .build(directory)
1722        .map_err(|error| error.to_string())
1723}
1724
1725/// `runner-manager.log` becomes `runner-manager.<account>.log`.
1726///
1727/// The stem carries its own `.log`, so the account goes before it rather than
1728/// after: the appender appends `.<date>`, and `runner-manager.log.uid-501` would
1729/// read as a rotated file rather than as another account's.
1730fn account_qualified_stem(stem: &str) -> String {
1731    let account = account_tag();
1732    match stem.rsplit_once('.') {
1733        Some((head, tail)) => format!("{head}.{account}.{tail}"),
1734        None => format!("{stem}.{account}"),
1735    }
1736}
1737
1738/// Something stable, filename-safe, and different for every local account.
1739///
1740/// The numeric effective user id rather than a name: it needs no lookup, cannot
1741/// contain a path separator, and is the identity the filesystem actually
1742/// compared when it refused the open.
1743#[cfg(unix)]
1744fn account_tag() -> String {
1745    // SAFETY: `geteuid` takes no argument, touches no memory this process owns,
1746    // and is documented never to fail.
1747    let uid = unsafe { libc::geteuid() };
1748    format!("uid-{uid}")
1749}
1750
1751/// As the Unix half, from the one identity Windows exposes without a lookup.
1752///
1753/// Sanitized rather than trusted: this becomes a file name, and `USERNAME` is
1754/// an ordinary environment variable that a caller may set to anything at all,
1755/// including something holding a path separator.
1756#[cfg(windows)]
1757fn account_tag() -> String {
1758    let raw = std::env::var("USERNAME").unwrap_or_default();
1759    let safe: String = raw
1760        .chars()
1761        .filter(|c| c.is_ascii_alphanumeric() || *c == '-' || *c == '_')
1762        .take(32)
1763        .collect();
1764    if safe.is_empty() {
1765        "other-account".to_string()
1766    } else {
1767        format!("user-{safe}")
1768    }
1769}
1770
1771#[cfg(test)]
1772mod tests {
1773    use super::*;
1774
1775    use std::sync::{Arc, Mutex};
1776
1777    use tracing_subscriber::layer::SubscriberExt as _;
1778
1779    // -----------------------------------------------------------------------
1780    // Capturing what the sink actually wrote
1781    // -----------------------------------------------------------------------
1782
1783    #[derive(Clone, Default)]
1784    struct Capture(Arc<Mutex<Vec<u8>>>);
1785
1786    impl Capture {
1787        fn text(&self) -> String {
1788            String::from_utf8_lossy(&self.0.lock().expect("not poisoned")).into_owned()
1789        }
1790    }
1791
1792    struct CaptureWriter(Arc<Mutex<Vec<u8>>>);
1793
1794    impl Write for CaptureWriter {
1795        fn write(&mut self, buf: &[u8]) -> std::io::Result<usize> {
1796            self.0.lock().expect("not poisoned").extend_from_slice(buf);
1797            Ok(buf.len())
1798        }
1799
1800        fn flush(&mut self) -> std::io::Result<()> {
1801            Ok(())
1802        }
1803    }
1804
1805    impl<'a> tracing_subscriber::fmt::MakeWriter<'a> for Capture {
1806        type Writer = CaptureWriter;
1807
1808        fn make_writer(&'a self) -> Self::Writer {
1809            CaptureWriter(Arc::clone(&self.0))
1810        }
1811    }
1812
1813    /// A sink with the same plumbing and no redaction at all.
1814    ///
1815    /// Exists so that `the_scan_catches_a_sink_that_does_not_redact` can prove
1816    /// the secret-injection scan below is capable of failing. Without it, a scan
1817    /// that found nothing would be indistinguishable from a scan that looked
1818    /// nowhere.
1819    #[derive(Debug, Clone)]
1820    struct PassthroughLayer<W>(W);
1821
1822    #[derive(Default)]
1823    struct PassthroughVisitor(Map<String, Value>);
1824
1825    impl Visit for PassthroughVisitor {
1826        fn record_str(&mut self, field: &Field, value: &str) {
1827            self.0
1828                .insert(field.name().to_string(), Value::String(value.to_string()));
1829        }
1830
1831        fn record_debug(&mut self, field: &Field, value: &dyn fmt::Debug) {
1832            self.0.insert(
1833                field.name().to_string(),
1834                Value::String(format!("{value:?}")),
1835            );
1836        }
1837    }
1838
1839    impl<S, W> Layer<S> for PassthroughLayer<W>
1840    where
1841        S: Subscriber + for<'a> LookupSpan<'a>,
1842        W: for<'a> tracing_subscriber::fmt::MakeWriter<'a> + 'static,
1843    {
1844        fn on_event(&self, event: &Event<'_>, _ctx: Context<'_, S>) {
1845            let mut visitor = PassthroughVisitor::default();
1846            event.record(&mut visitor);
1847            let mut line = serde_json::to_string(&Value::Object(visitor.0)).unwrap_or_default();
1848            line.push('\n');
1849            let _ = self.0.make_writer().write_all(line.as_bytes());
1850        }
1851    }
1852
1853    // -----------------------------------------------------------------------
1854    // The secret-injection log scan (`07-security.md`, security gate)
1855    // -----------------------------------------------------------------------
1856
1857    /// A user access token, in GitHub's user-to-server format.
1858    const USER_TOKEN: &str = "ghu_16C7e42F292c6912E7710c838347Ae178B4a";
1859    /// An installation-style token, to prove the prefix rule is not one-off.
1860    const SERVER_TOKEN: &str = "ghs_1CGGYnBAtn5ov3M0aTHhP7l3ZKuMhIB3pnPd";
1861    /// A fine-grained personal access token.
1862    const FINE_GRAINED: &str =
1863        "github_pat_11ABCDEFG0abcdefghijkl_ZYXWVUTSRQPONMLKJIHGFEDCBA9876543210zyxwv";
1864    /// Stands in for an encoded JIT configuration: base64, and long.
1865    const JIT_BLOB: &str = "eyJhZ2VudE5hbWUiOiJydW5uZXItbWFuYWdlciIsImVuY29kZWQiOiJhYmNkZWZnaGlqa2xtbm9wcXJzdHV2d3h5ejAxMjM0NTY3ODkrLz09In0=";
1866    /// A workspace path, which `07-security.md` also requires redacted.
1867    const WORKSPACE: &str = "/var/lib/runner-manager/runtime/9f2c/attempt-1";
1868    /// The same, on the platform the persona is least likely to be using but
1869    /// the CI matrix definitely is.
1870    const WINDOWS_WORKSPACE: &str = r"C:\Users\operator\AppData\Local\runner-manager\runtime\9f2c";
1871    /// A GitHub App installation assertion: a JSON Web Token.
1872    ///
1873    /// Its two `.` separators are the point. `is_opaque_char` excludes `.`, so
1874    /// the opaque-run rule sees three short runs rather than one long one, and
1875    /// every other secret here is caught by that rule when it stands alone.
1876    const JWT: &str = "eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.\
1877                       eyJpc3MiOiIxMjM0NTYiLCJpYXQiOjE3MDAwMDAwMDAsImV4cCI6MTcwMDAwMDYwMH0.\
1878                       c2lnbmF0dXJlLXRoYXQtaXMtb3BhcXVlLWFuZC1sb25nLWVub3VnaC10by1tYXR0ZXI";
1879
1880    /// An error whose `Debug` carries the HTTP body it was given.
1881    ///
1882    /// This is the shape `d2`'s secret store and `d3`'s installer will hand to
1883    /// `tracing::error!(reason = ?err)`, and it is not the same shape as any of
1884    /// the JSON above: `Debug` on a `String` escapes the quotes inside it, so
1885    /// the keys arrive spelled `\"password\"` rather than `"password"`.
1886    #[derive(Debug)]
1887    struct StoreError {
1888        body: String,
1889    }
1890
1891    fn secrets() -> Vec<&'static str> {
1892        vec![
1893            USER_TOKEN,
1894            SERVER_TOKEN,
1895            FINE_GRAINED,
1896            JIT_BLOB,
1897            JWT,
1898            WORKSPACE,
1899            WINDOWS_WORKSPACE,
1900        ]
1901    }
1902
1903    /// Routes every secret through `sink` in every shape a caller might use,
1904    /// and reports the first one that came out the other side.
1905    ///
1906    /// A helper returning `Result` rather than a test body, because the same
1907    /// injection has to be run against a deliberately non-redacting sink to
1908    /// show that it is capable of finding anything at all.
1909    fn scan_for_leaks<L>(layer: L, capture: &Capture) -> Result<String, String>
1910    where
1911        L: Layer<tracing_subscriber::Registry> + Send + Sync + 'static,
1912    {
1913        let subscriber = tracing_subscriber::registry().with(layer);
1914
1915        tracing::subscriber::with_default(subscriber, || {
1916            for secret in secrets() {
1917                // 1. In the message body, which is the one field that must be
1918                //    allowed and therefore cannot be protected by the allowlist.
1919                tracing::info!("starting runner with {secret}");
1920
1921                // 2. As a credential header in the message body, scheme and all.
1922                tracing::info!("request failed; Authorization: Bearer {secret}");
1923                tracing::warn!("retrying with x-api-key={secret}");
1924
1925                // 3. In a field nobody listed — the case that must be safe with
1926                //    no edit to this file.
1927                tracing::info!(runner_token = %secret, "registered");
1928
1929                // 4. In a field that *is* listed, which is where the allowlist
1930                //    alone would not save anything.
1931                tracing::info!(outcome = %secret, event = "started", "registered");
1932                tracing::info!(message = %secret, "ignored");
1933
1934                // 5. Through `Debug` rather than `Display`.
1935                tracing::info!(?secret, "debug shaped");
1936
1937                // 6. Embedded in a structured value rather than standing
1938                //    alone. Every shape above presents the secret as its
1939                //    own whitespace-delimited word, which is the one shape
1940                //    the opaque-run rule catches for free -- so the scan
1941                //    could pass while the sink leaked anything embedded.
1942                //    Compact JSON is what `serde_json::to_string` emits and
1943                //    what an HTTP error body arrives as, and it was emitted
1944                //    verbatim: `redact_core` recursed into the value for
1945                //    `=` only, and the JSON quote left the key spelled
1946                //    `encoded_jit_config"`, which is not on CREDENTIAL_KEYS.
1947                for key in ["encoded_jit_config", "runner_token", "pat"] {
1948                    tracing::error!("registration failed: {{\"{key}\":\"{secret}\"}}");
1949                    tracing::error!("registration failed: {{ \"{key}\": \"{secret}\" }}");
1950                    tracing::error!("registration failed: {key}:{secret}");
1951                }
1952
1953                // 7. An `Authorization` header value, in the same three
1954                //    embedded shapes.
1955                tracing::error!("response: {{\"authorization\":\"Bearer {secret}\"}}");
1956                tracing::error!("response: {{ \"authorization\": \"Bearer {secret}\" }}");
1957                tracing::error!("response: authorization:Bearer {secret}");
1958
1959                // 9. A compact object with **more than one field**, the
1960                //    credential key second. Shape 6 above differs from this by
1961                //    field order and by nothing else, and `serde_json::to_string`
1962                //    is what chooses the order -- so shape 6 could pass while an
1963                //    error body from a struct with two fields leaked. The rules
1964                //    split a word once, on the first separator they find, so
1965                //    only `runner_id` was ever examined.
1966                for key in ["encoded_jit_config", "runner_token", "access_token"] {
1967                    tracing::error!(
1968                        "registration failed: {{\"runner_id\":42,\"{key}\":\"{secret}\"}}"
1969                    );
1970                    tracing::error!(
1971                        "registration failed: {{\"status\":422,\"message\":\"bad\",\"{key}\":\"{secret}\"}}"
1972                    );
1973                    // The spaced spelling of the same thing, which was already
1974                    // safe -- by accident, because the value is its own word.
1975                    tracing::error!(
1976                        "registration failed: {{ \"runner_id\": 42, \"{key}\": \"{secret}\" }}"
1977                    );
1978                }
1979
1980                // 10. Nested, one level and two. The value recursion goes to
1981                //     `redact_value`, which is a leaf and never re-enters
1982                //     `redact_core`, so an object inside an object was emitted
1983                //     whole.
1984                tracing::error!("response: {{\"body\":{{\"runner_token\":\"{secret}\"}}}}");
1985                tracing::error!(
1986                    "response: {{\"error\":{{\"status\":422,\"body\":{{\"encoded_jit_config\":\"{secret}\"}}}}}}"
1987                );
1988
1989                // 11. A form-encoded body with the credential parameter second.
1990                //     `&` was not a separator this module knew, so the whole
1991                //     body was one word and only `scope` was ever judged.
1992                tracing::error!("token exchange failed: scope=repo&access_token={secret}");
1993                tracing::error!(
1994                    "token exchange failed: grant_type=refresh&refresh_token={secret}&scope=repo"
1995                );
1996
1997                // 12. Backslash-escaped JSON, through `Debug` rather than
1998                //     `Display`. Shape 5 is also `Debug`, but it puts the
1999                //     secret on an *unlisted* field, which is dropped wholesale
2000                //     -- so the `Debug` lens and the compact-JSON lens were both
2001                //     here and had never been crossed. `reason` is allowed, so
2002                //     this one reaches the scrubber, with its keys spelled
2003                //     `\"runner_token\"`.
2004                let failure = StoreError {
2005                    body: format!("{{\"runner_token\":\"{secret}\"}}"),
2006                };
2007                tracing::error!(reason = ?failure, "the secret store rejected the request");
2008
2009                // 13. A URL earlier in the same word. `redact_core` split on the
2010                //     first `://` and handed everything before it to
2011                //     `redact_url` as a "scheme" and everything after it as
2012                //     the URL -- and the terminal arm of `redact_url` echoes
2013                //     both verbatim. So one URL anywhere in a word put the
2014                //     whole of the rest of that word beyond every rule below.
2015                //     `documentation_url` is in essentially every GitHub REST
2016                //     error body, which makes that "any error body logged
2017                //     alongside a credential".
2018                tracing::error!(
2019                    "api failed: {{\"message\":\"Bad credentials\",\"documentation_url\":\"https://docs.github.com/rest\",\"token\":\"{secret}\"}}"
2020                );
2021                // The reverse order leaked for the mirror reason: everything
2022                // before the `://` became the scheme, and was echoed.
2023                tracing::error!(
2024                    "api failed: {{\"token\":\"{secret}\",\"documentation_url\":\"https://docs.github.com/rest\"}}"
2025                );
2026                // A nested object behind a URL, which is the shape a clone
2027                // failure arrives in.
2028                tracing::error!(
2029                    "clone failed: {{\"remote\":\"https://github.com/o/r.git\",\"body\":{{\"password\":\"{secret}\"}}}}"
2030                );
2031
2032                // 14. A value with no key of its own: an array element. The
2033                //     structural cut recursed on the *raw* fragment, and the
2034                //     terminal fallback then handed it to `redact_value` with
2035                //     its quote still attached -- so `"ghu_…` failed
2036                //     `starts_with("ghu_")`, failed `is_opaque_char` on the
2037                //     quote, failed the `eyJ` test, and failed
2038                //     `looks_like_path`. This is shape 9's defect one step
2039                //     down: "only the first pair is examined" became "a value
2040                //     with no key of its own is never examined", and it
2041                //     survived because there was no array among the twelve
2042                //     shapes above.
2043                tracing::error!("registration failed: {{\"tokens\":[\"{secret}\",\"x\"]}}");
2044                tracing::error!("registration failed: {{\"tokens\":[\"x\",\"{secret}\"]}}");
2045                tracing::error!("registration failed: [\"x\",\"{secret}\"]");
2046                tracing::error!("registration failed: [{{\"id\":1}},[\"{secret}\"]]");
2047                let listed = StoreError {
2048                    body: format!("{{\"tokens\":[\"x\",\"{secret}\"]}}"),
2049                };
2050                tracing::error!(reason = ?listed, "the secret store rejected the list");
2051
2052                // 15. `;` as the separator. It is in `WRAPPERS`, so it was
2053                //     recognised as punctuation, and it was not in
2054                //     `STRUCTURAL`, so it never cut a word. It is what
2055                //     separates the pairs of a Windows connection string, of a
2056                //     credential string, and of a cookie header written
2057                //     without a space -- `d2`'s territory exactly.
2058                tracing::error!("store rejected: Server=host;Database=x;Password={secret};");
2059                tracing::error!("store rejected: user=operator;password={secret}");
2060                tracing::error!("store rejected: theme=dark;session={secret}");
2061
2062                // 16. Wrapped in an element rather than in a quote.
2063                //     `split_wrappers` strips only the *outermost* `<` and
2064                //     `>`, so `<string>…</string>` arrived as
2065                //     `string>…</string`, which matches no rule at all. `d3`'s
2066                //     installers handle launchd plists, which is where this
2067                //     shape comes from.
2068                tracing::error!("plist rejected: <string>{secret}</string>");
2069                tracing::error!("plist rejected: <key>token</key><string>{secret}</string>");
2070
2071                // 17. A `;` or an `&` *after a URL in the same word*. This is
2072                //     shape 15 and shape 11 with one thing changed: a URL
2073                //     earlier in the word. `STRUCTURAL` has nine characters and
2074                //     `is_url_terminator` recognised seven of them -- `;` and
2075                //     `&` were missing -- and the URL branch runs *ahead* of
2076                //     the structural cut. So `split_url` swallowed everything
2077                //     to the next terminator, including the separator that
2078                //     would have cut the word, and `redact_url`'s terminal arm
2079                //     echoed it verbatim.
2080                //
2081                //     The L3 fix (bounding the URL) and the L1 fix (cutting on
2082                //     `;` and `&`) each work alone and did not compose: this
2083                //     commit added `;` to `STRUCTURAL` *for connection strings*
2084                //     while leaving the path a URL opens straight through it.
2085                //     A connection string whose `Server=` is a URL is exactly
2086                //     `d2`'s shape, and a systemd `Environment=` line is
2087                //     `d3`'s.
2088                tracing::error!(
2089                    "store rejected: Server=https://vault.local/api;Password={secret};"
2090                );
2091                tracing::error!("keychain error: url=https://kc.local;secret={secret}");
2092                tracing::error!("unit rejected: Environment=API=https://a.com/v1;TOKEN={secret}");
2093                tracing::error!("token exchange failed: cb=https://a.com/x&access_token={secret}");
2094                tracing::error!(
2095                    "dsn rejected: dsn=https://sentry.local/1;password={secret};user=x"
2096                );
2097                tracing::error!(
2098                    "callback failed: redirect=https://a.com/cb&state=1&access_token={secret}"
2099                );
2100                let routed = StoreError {
2101                    body: format!("Server=https://vault.local/api;Password={secret};"),
2102                };
2103                tracing::error!(reason = ?routed, "the secret store rejected the connection string");
2104
2105                // 18. A credential key whose value is not in its own fragment.
2106                //     The empty-value skip is right, but its justification --
2107                //     "the value is in the next fragment, where the structural
2108                //     cut reaches it" -- was false: `redact_fragment` carried
2109                //     nothing across the cut, so nothing ever reached it. An
2110                //     array, a nested object, an empty pair and a plist
2111                //     key/value pair are four spellings of the same gap.
2112                tracing::error!("store rejected: {{\"password\":[\"{secret}\"]}}");
2113                tracing::error!("store rejected: {{\"password\":{{\"v\":\"{secret}\"}}}}");
2114                tracing::error!("store rejected: Password=;{secret}");
2115                tracing::error!("plist rejected: <key>password</key><string>{secret}</string>");
2116
2117                // 19. A structural character *inside* a credential value. `,`
2118                //     and `&` could already split a secret; this commit added
2119                //     `;`, `<` and `>`, so each of those became a new character
2120                //     that can cut a punctuated password in half and leave the
2121                //     tail standing.
2122                tracing::error!("store rejected: {{\"password\":\"a<{secret}>b\"}}");
2123                tracing::error!("store rejected: {{\"password\":\"a&{secret},b\"}}");
2124
2125                // 20. A secret in a URL *path*. `redact_url` applies no shape
2126                //     rule to the path at all, so a token that reaches one is
2127                //     echoed whole -- and the token-prefix rule is documented
2128                //     as a belt that catches a credential anywhere.
2129                //
2130                //     The two workspace paths are excluded, and the exclusion
2131                //     is a real limitation rather than a convenience. A path is
2132                //     judged one segment at a time, because judging the whole
2133                //     of a URL path with `looks_like_path` would redact every
2134                //     URL with two segments in it -- which is the
2135                //     over-redaction the module documentation explicitly
2136                //     promises a full URL escapes. A filesystem path pasted
2137                //     into a URL path is therefore indistinguishable from an
2138                //     ordinary deep URL path: `/var/lib/…` and `/repos/o/r/…`
2139                //     are the same shape, segment by segment. A *credential* in
2140                //     a path is caught, because a credential has a shape of its
2141                //     own; a path in a path is not.
2142                if !looks_like_path(secret) {
2143                    tracing::error!("download failed: https://github.com/o/r/raw/{secret}/f");
2144                    tracing::error!("download failed: https://github.com/o/r/raw/{secret}");
2145                }
2146
2147                // 8. Carried on a span rather than on the event.
2148                let span = tracing::info_span!("attempt", jit_config = %secret);
2149                let _entered = span.enter();
2150                tracing::info!(event = "inside_span", "in a span");
2151            }
2152        });
2153
2154        let output = capture.text();
2155        for secret in secrets() {
2156            for needle in needles(secret) {
2157                if output.contains(&needle) {
2158                    return Err(format!(
2159                        "the sink emitted a secret verbatim: {secret}\n\
2160                         (found as: {needle})\n--- output ---\n{output}"
2161                    ));
2162                }
2163            }
2164        }
2165        Ok(output)
2166    }
2167
2168    /// Every spelling a secret can have in the sink's output.
2169    ///
2170    /// The sink writes JSON, so a secret containing a character `serde_json`
2171    /// escapes never appears in the output as it was written. `WINDOWS_WORKSPACE`
2172    /// is a raw string with single backslashes and is emitted with doubled ones,
2173    /// so scanning for the literal could not have matched it — not even against
2174    /// `PassthroughLayer`, which redacts nothing at all. The path really is
2175    /// redacted, so this was a hole in the coverage rather than a leak, but a
2176    /// needle that cannot match is a check that cannot fail.
2177    fn needles(secret: &str) -> Vec<String> {
2178        // Strip the quotes `to_string` adds; what is left is the text exactly
2179        // as it appears inside a JSON string literal.
2180        let escape = |text: &str| {
2181            let json = serde_json::to_string(text).expect("a string is serialisable");
2182            json[1..json.len() - 1].to_string()
2183        };
2184
2185        let once = escape(secret);
2186        // Twice, because a secret can be escaped twice on the way out, and
2187        // shape 12 is where that happens: `Debug` on a `String` escapes the
2188        // quotes *and* the backslashes inside it, and the sink then
2189        // JSON-encodes what `Debug` produced. A Windows path arrives in that
2190        // line with four backslashes where the secret has one, so the
2191        // once-escaped needle cannot match it there -- not even against
2192        // `PassthroughLayer`, which redacts nothing at all. That cell of the
2193        // matrix was a check incapable of failing, which is the same defect
2194        // this helper was written to fix one level down.
2195        let twice = escape(&once);
2196
2197        let mut spellings = vec![secret.to_string(), once, twice];
2198        // Consecutive-only is enough: each level of escaping is a superset of
2199        // the last, so equal spellings are always adjacent.
2200        spellings.dedup();
2201        spellings
2202    }
2203
2204    #[test]
2205    fn the_scan_looks_for_a_needle_that_can_actually_occur() {
2206        // Guards the helper above: if `needles` ever stops producing the
2207        // JSON-escaped spelling, the Windows workspace silently stops being
2208        // scanned for and every assertion about it becomes vacuous.
2209        let windows = needles(WINDOWS_WORKSPACE);
2210        assert_eq!(
2211            windows.len(),
2212            3,
2213            "a backslash path has three spellings, one per level of escaping it \
2214             can pass through on the way out: {windows:?}"
2215        );
2216        assert!(
2217            windows[1].contains(r"\\Users\\operator"),
2218            "the once-escaped spelling is what an ordinary JSON line contains: {windows:?}"
2219        );
2220        assert!(
2221            windows[2].contains(r"\\\\Users\\\\operator"),
2222            "the twice-escaped spelling is what a Debug rendering inside a JSON \
2223             line contains: {windows:?}"
2224        );
2225
2226        // The third spelling is not hypothetical: it is exactly what the sink
2227        // writes for shape 12, and without it that cell of the scan was a
2228        // check that could not fail.
2229        let shape_twelve = serde_json::to_string(&Value::String(format!(
2230            "{:?}",
2231            StoreError {
2232                body: format!("{{\"runner_token\":\"{WINDOWS_WORKSPACE}\"}}"),
2233            }
2234        )))
2235        .expect("serialisable");
2236        assert!(
2237            shape_twelve.contains(&windows[2]),
2238            "the twice-escaped needle must be findable in what shape 12 emits: {shape_twelve}"
2239        );
2240        assert!(
2241            !shape_twelve.contains(&windows[1]),
2242            "and the once-escaped one must not be, or the gap was never there: {shape_twelve}"
2243        );
2244
2245        // Confirms the premise: the raw spelling genuinely cannot occur in the
2246        // sink's output, which is why scanning only for it proved nothing.
2247        let rendered = serde_json::to_string(&Value::String(WINDOWS_WORKSPACE.to_string()))
2248            .expect("serialisable");
2249        assert!(
2250            !rendered.contains(WINDOWS_WORKSPACE),
2251            "if this ever contains the raw path, the original scan was fine after all: {rendered}"
2252        );
2253        assert!(rendered.contains(&windows[1]), "{rendered}");
2254
2255        // A secret with nothing to escape has exactly one spelling.
2256        assert_eq!(needles(USER_TOKEN), vec![USER_TOKEN.to_string()]);
2257    }
2258
2259    #[test]
2260    fn the_secret_injection_scan_finds_nothing() {
2261        let capture = Capture::default();
2262        let output = scan_for_leaks(RedactingLayer::new(capture.clone()), &capture)
2263            .unwrap_or_else(|complaint| panic!("{complaint}"));
2264
2265        // Something must have been written, or "no secrets found" is trivially
2266        // true and means nothing.
2267        assert!(!output.trim().is_empty(), "the sink wrote nothing at all");
2268        assert!(
2269            output.contains(REDACTION),
2270            "nothing was redacted, so nothing was routed through the sink:\n{output}"
2271        );
2272    }
2273
2274    #[test]
2275    fn the_scan_catches_a_sink_that_does_not_redact() {
2276        let capture = Capture::default();
2277        let complaint = scan_for_leaks(PassthroughLayer(capture.clone()), &capture)
2278            .expect_err("a sink with no redaction must be caught");
2279        assert!(
2280            complaint.contains("emitted a secret verbatim"),
2281            "the complaint must name the failure mode: {complaint}"
2282        );
2283    }
2284
2285    // -----------------------------------------------------------------------
2286    // The allowlist
2287    // -----------------------------------------------------------------------
2288
2289    fn emit(capture: &Capture, body: impl FnOnce()) -> String {
2290        let subscriber = tracing_subscriber::registry().with(RedactingLayer::new(capture.clone()));
2291        tracing::subscriber::with_default(subscriber, body);
2292        capture.text()
2293    }
2294
2295    #[test]
2296    fn a_field_nobody_listed_is_redacted_by_default() {
2297        let capture = Capture::default();
2298        let output = emit(&capture, || {
2299            tracing::info!(
2300                a_field_added_by_a_later_task = "supersecret-value",
2301                "an event"
2302            );
2303        });
2304
2305        assert!(
2306            !output.contains("supersecret-value"),
2307            "an unlisted field leaked its value:\n{output}"
2308        );
2309        assert!(output.contains(REDACTION), "{output}");
2310        // The name survives: knowing the field was there is useful, and a field
2311        // name is not a credential.
2312        assert!(
2313            output.contains("a_field_added_by_a_later_task"),
2314            "the field name should be kept:\n{output}"
2315        );
2316    }
2317
2318    #[test]
2319    fn an_unlisted_numeric_field_is_redacted_too() {
2320        // The rule has no exception for a type, because "a credential is never
2321        // an integer" is exactly the kind of reasoning that stops being true
2322        // once somebody logs an installation-scoped identifier they should not.
2323        let capture = Capture::default();
2324        let output = emit(&capture, || {
2325            tracing::info!(unlisted_number = 8_675_309_u64, "an event");
2326        });
2327        assert!(!output.contains("8675309"), "{output}");
2328        assert!(output.contains(REDACTION), "{output}");
2329    }
2330
2331    #[test]
2332    fn listed_fields_survive() {
2333        let capture = Capture::default();
2334        let output = emit(&capture, || {
2335            tracing::info!(
2336                event = "reconciled",
2337                policy_id = "9f2c1a44-0000-4000-8000-000000000001",
2338                count = 3,
2339                outcome = "started",
2340                "reconciliation finished"
2341            );
2342        });
2343
2344        for expected in [
2345            "reconciled",
2346            "9f2c1a44-0000-4000-8000-000000000001",
2347            "started",
2348            "reconciliation finished",
2349        ] {
2350            assert!(
2351                output.contains(expected),
2352                "{expected} missing from:\n{output}"
2353            );
2354        }
2355        assert!(output.contains("\"count\":3"), "{output}");
2356    }
2357
2358    #[test]
2359    fn the_allowlist_is_sorted_and_has_no_duplicates() {
2360        // `is_field_allowed` uses a binary search, so an unsorted list would not
2361        // merely be untidy — it would silently start redacting fields that are
2362        // on it.
2363        let mut sorted = ALLOWED_FIELDS.to_vec();
2364        sorted.sort_unstable();
2365        assert_eq!(
2366            ALLOWED_FIELDS,
2367            &sorted[..],
2368            "ALLOWED_FIELDS must stay sorted"
2369        );
2370
2371        let mut deduped = sorted.clone();
2372        deduped.dedup();
2373        assert_eq!(sorted.len(), deduped.len(), "ALLOWED_FIELDS has duplicates");
2374
2375        for name in ALLOWED_FIELDS {
2376            assert!(is_field_allowed(name), "{name} is listed but not allowed");
2377        }
2378        assert!(!is_field_allowed("authorization"));
2379        assert!(!is_field_allowed("runner_token"));
2380    }
2381
2382    #[test]
2383    fn every_record_is_one_parseable_json_object_per_line() {
2384        let capture = Capture::default();
2385        let output = emit(&capture, || {
2386            tracing::info!(event = "one", "first");
2387            tracing::warn!(event = "two", "second");
2388        });
2389
2390        let lines: Vec<&str> = output
2391            .lines()
2392            .filter(|line| !line.trim().is_empty())
2393            .collect();
2394        assert_eq!(lines.len(), 2, "{output}");
2395        for line in lines {
2396            let value: Value = serde_json::from_str(line)
2397                .unwrap_or_else(|error| panic!("not JSON: {error}\n{line}"));
2398            assert!(value.get("timestamp").is_some(), "{line}");
2399            assert!(value.get("level").is_some(), "{line}");
2400            assert!(value.get("logger").is_some(), "{line}");
2401            assert!(value.get("fields").is_some(), "{line}");
2402        }
2403    }
2404
2405    #[test]
2406    fn span_fields_are_redacted_and_carried() {
2407        let capture = Capture::default();
2408        let output = emit(&capture, || {
2409            let span = tracing::info_span!("attempt", attempt_id = "abc-123", jit = %JIT_BLOB);
2410            let _entered = span.enter();
2411            tracing::info!(event = "inside", "in the span");
2412        });
2413
2414        assert!(output.contains("\"name\":\"attempt\""), "{output}");
2415        assert!(
2416            output.contains("abc-123"),
2417            "the listed span field survives:\n{output}"
2418        );
2419        assert!(!output.contains(JIT_BLOB), "a span field leaked:\n{output}");
2420    }
2421
2422    /// Asserts that [`install`] created its directories the restrictive way.
2423    ///
2424    /// `install` used to call `std::fs::create_dir_all(logs_dir)` directly.
2425    /// That is invisible on Windows and *nearly* invisible on Unix — the
2426    /// directory exists either way, and `AppPaths::create_all`'s own test still
2427    /// passed, because it tests `create_all` rather than the path a running
2428    /// daemon actually takes. So this checks the two things that distinguish
2429    /// them:
2430    ///
2431    /// 1. **All four directories exist.** `create_dir_all(logs_dir)` makes
2432    ///    exactly one. This half runs on every platform, which matters because
2433    ///    Windows is the leg most likely to be run locally.
2434    /// 2. **On Unix the mode is `0700`.** This is the invariant `create_all`
2435    ///    documents — that a diagnostics file is not readable by other local
2436    ///    accounts — and `tracing_appender` writes 0644 files into whatever
2437    ///    directory it is given, so a `0755` `logs/` defeats it entirely.
2438    fn assert_install_created_restricted_directories(paths: &crate::paths::AppPaths) {
2439        for (purpose, path) in paths.all() {
2440            assert!(
2441                path.is_dir(),
2442                "install must create the {purpose} directory too: going through \
2443                 AppPaths::create_all is what applies the restriction, and creating only \
2444                 logs/ is the bug this asserts against ({})",
2445                path.display()
2446            );
2447
2448            #[cfg(unix)]
2449            {
2450                use std::os::unix::fs::PermissionsExt as _;
2451
2452                let mode = std::fs::metadata(path)
2453                    .expect("the directory exists")
2454                    .permissions()
2455                    .mode()
2456                    & 0o777;
2457                assert_eq!(
2458                    mode, 0o700,
2459                    "the {purpose} directory is mode {mode:04o}; a diagnostics file under a \
2460                     group- or world-readable directory is readable by other local accounts"
2461                );
2462            }
2463        }
2464    }
2465
2466    /// The one test that exercises [`install`] rather than assembling a
2467    /// subscriber by hand: the rolling file appender, the non-blocking writer,
2468    /// and the guard that has to be held for any of it to reach disk.
2469    ///
2470    /// It installs a *global* subscriber, which a process can only do once, so
2471    /// it is the only test here that may. Everything else uses
2472    /// `with_default`, which is thread-local and does not conflict — including
2473    /// on threads that run after this one.
2474    #[test]
2475    #[serial_test::serial(global_subscriber)]
2476    fn install_writes_redacted_json_into_the_logs_directory() {
2477        // `install` honours `RUST_LOG`, and a developer who has set it to
2478        // something restrictive would otherwise see this fail for a reason that
2479        // has nothing to do with the code. CI sets no such variable.
2480        if std::env::var_os("RUST_LOG").is_some() {
2481            return;
2482        }
2483
2484        let root = tempfile::tempdir().expect("a temporary directory");
2485        let paths = crate::paths::AppPaths::rooted_at(root.path());
2486
2487        let outcome = install(&paths, LogRole::Operator, "trace");
2488
2489        // Asserted before the early return below, because `install` creates the
2490        // directories before it touches the global subscriber: this holds
2491        // whether or not this test won the race to install one.
2492        assert_install_created_restricted_directories(&paths);
2493
2494        let Ok(guard) = outcome else {
2495            // Another test binary in the same process already installed one.
2496            // Not this test's failure, and not worth making the suite
2497            // order-dependent over.
2498            return;
2499        };
2500
2501        tracing::info!(event = "installed", runner_token = %USER_TOKEN, "hello from the sink");
2502
2503        // Dropping the guard flushes and stops the background writer, which is
2504        // the documented way to be sure the line reached disk.
2505        drop(guard);
2506
2507        let written: Vec<PathBuf> = std::fs::read_dir(paths.logs_dir())
2508            .expect("the log directory was created")
2509            .filter_map(Result::ok)
2510            .map(|entry| entry.path())
2511            .collect();
2512        assert_eq!(
2513            written.len(),
2514            1,
2515            "expected one rotating log file: {written:?}"
2516        );
2517
2518        let contents = std::fs::read_to_string(&written[0]).expect("readable");
2519        assert!(contents.contains("hello from the sink"), "{contents}");
2520        assert!(
2521            !contents.contains(USER_TOKEN),
2522            "the file sink must redact exactly like the in-memory one:\n{contents}"
2523        );
2524        assert!(contents.contains(REDACTION), "{contents}");
2525        serde_json::from_str::<Value>(contents.lines().next().expect("a line"))
2526            .expect("each line is a JSON object");
2527    }
2528
2529    // -----------------------------------------------------------------------
2530    // Two accounts, one logs/ directory
2531    // -----------------------------------------------------------------------
2532
2533    /// The daemon and the operator do not reach for the same file.
2534    ///
2535    /// Cheap, and it is the whole of the separation: everything else about
2536    /// [`LogRole`] follows from these two names being different.
2537    #[test]
2538    fn the_daemon_and_the_operator_write_different_files() {
2539        assert_ne!(
2540            LogRole::Service.file_stem(),
2541            LogRole::Operator.file_stem(),
2542            "a boot-mode daemon runs as another account and creates its file 0644; sharing a \
2543             stem hands whichever of them opened it first the day's file and locks the other out"
2544        );
2545        assert_eq!(LogRole::Operator.file_stem(), OPERATOR_LOG_STEM);
2546        assert_eq!(LogRole::Service.file_stem(), SERVICE_LOG_STEM);
2547    }
2548
2549    /// A file this account may not append to is written *beside*, not panicked
2550    /// on.
2551    ///
2552    /// This is the reported 0.1.17 crash, reproduced at the mode that caused
2553    /// it. On the host it was ownership — `root` had rolled the file over at
2554    /// midnight — and an unprivileged test cannot make a `root`-owned file, so
2555    /// it makes an unwritable one instead: the appender's `OpenOptions` refuse
2556    /// both with the same `EACCES`, and refusing was what
2557    /// `tracing_appender::rolling::daily` turned into a panic.
2558    #[cfg(unix)]
2559    #[test]
2560    fn a_log_file_this_account_cannot_append_to_is_written_beside() {
2561        use std::os::unix::fs::PermissionsExt as _;
2562
2563        // `root` is not subject to the mode bits, so it would sail through the
2564        // very open this test needs to fail.
2565        if account_tag() == "uid-0" {
2566            return;
2567        }
2568
2569        let root = tempfile::tempdir().expect("a temporary directory");
2570        let paths = crate::paths::AppPaths::rooted_at(root.path());
2571        paths
2572            .create_all()
2573            .expect("the four directories are created");
2574
2575        // Every date the appender might choose, so the test cannot flake on a
2576        // run that straddles midnight UTC.
2577        let logs = paths.logs_dir().to_path_buf();
2578        let today = chrono::Utc::now();
2579        for date in [
2580            (today - chrono::Duration::days(1))
2581                .format("%Y-%m-%d")
2582                .to_string(),
2583            today.format("%Y-%m-%d").to_string(),
2584            (today + chrono::Duration::days(1))
2585                .format("%Y-%m-%d")
2586                .to_string(),
2587        ] {
2588            let taken = logs.join(format!("{OPERATOR_LOG_STEM}.{date}"));
2589            std::fs::write(&taken, b"another account's diagnostics").expect("writable");
2590            std::fs::set_permissions(&taken, std::fs::Permissions::from_mode(0o444))
2591                .expect("the mode is applied");
2592        }
2593
2594        let mut appender = open_appender(&logs, LogRole::Operator)
2595            .expect("a diagnostics file is opened beside the one this account may not append to");
2596        appender
2597            .write_all(b"a line\n")
2598            .expect("the line is written");
2599        appender.flush().expect("the line is flushed");
2600
2601        let qualified = account_qualified_stem(OPERATOR_LOG_STEM);
2602        let written: Vec<String> = std::fs::read_dir(&logs)
2603            .expect("the directory is readable")
2604            .flatten()
2605            .map(|entry| entry.file_name().to_string_lossy().into_owned())
2606            .collect();
2607        assert!(
2608            written.iter().any(|name| name.starts_with(&qualified)),
2609            "expected a file under {qualified:?} beside the unwritable ones, and found {written:?}"
2610        );
2611        for name in &written {
2612            if name.starts_with(&qualified) {
2613                continue;
2614            }
2615            assert_eq!(
2616                std::fs::read(logs.join(name)).expect("readable"),
2617                b"another account's diagnostics",
2618                "{name} belongs to another account and must not have been touched"
2619            );
2620        }
2621    }
2622
2623    // -----------------------------------------------------------------------
2624    // The scrubber, in isolation
2625    // -----------------------------------------------------------------------
2626
2627    #[test]
2628    fn tokens_are_redacted_wherever_they_appear() {
2629        for token in [USER_TOKEN, SERVER_TOKEN, FINE_GRAINED] {
2630            for shape in [
2631                token.to_string(),
2632                format!("using {token} now"),
2633                format!("(\"{token}\")"),
2634                format!("token={token}"),
2635                format!("Authorization: Bearer {token}"),
2636                format!("authorization={token}"),
2637                // A URL's userinfo. This is the canonical token-authenticated
2638                // git remote, so it is what a clone or fetch error carries, and
2639                // it is the shape the `://` branch used to hand back verbatim.
2640                format!("https://x-access-token:{token}@github.com/owner/repo.git"),
2641                format!("fatal: could not read from https://{token}@github.com/o/r"),
2642                // Userinfo with no password half.
2643                format!("https://{token}@github.com/o/r.git"),
2644                // A fragment, which used to be stripped only when it was a
2645                // query string.
2646                format!("https://github.com/login/oauth#access_token={token}"),
2647                format!("https://github.com/x?a=1#token={token}"),
2648            ] {
2649                let redacted = redact(&shape);
2650                assert!(
2651                    !redacted.contains(token),
2652                    "{shape:?} survived redaction as {redacted:?}"
2653                );
2654            }
2655        }
2656    }
2657
2658    #[test]
2659    fn a_url_keeps_what_diagnoses_it_and_loses_what_authenticates_it() {
2660        // The host and the path are the diagnosable part and must survive, or
2661        // nobody keeps this redaction.
2662        assert_eq!(
2663            redact(
2664                "https://x-access-token:ghu_16C7e42F292c6912E7710c838347Ae178B4a@github.com/owner/repo.git"
2665            ),
2666            format!("https://{REDACTION}@github.com/owner/repo.git")
2667        );
2668        assert_eq!(
2669            redact("https://user:hunter2@api.github.com/repos/o/r?page=2"),
2670            format!("https://{REDACTION}@api.github.com/repos/o/r?{REDACTION}")
2671        );
2672        // A password containing an `@`: the host is what follows the last one.
2673        assert_eq!(
2674            redact("https://user:p@ss@github.com/o/r"),
2675            format!("https://{REDACTION}@github.com/o/r")
2676        );
2677        // Userinfo on a bare authority, with no path at all.
2678        assert_eq!(
2679            redact("https://token@github.com"),
2680            format!("https://{REDACTION}@github.com")
2681        );
2682        // An `@` after the first `/` is part of the path, not userinfo.
2683        assert_eq!(
2684            redact("https://github.com/@owner/repo"),
2685            "https://github.com/@owner/repo"
2686        );
2687    }
2688
2689    #[test]
2690    fn an_encoded_jit_configuration_is_redacted() {
2691        assert_eq!(redact(JIT_BLOB), REDACTION);
2692        let sentence = format!("handing off {JIT_BLOB} to the runner");
2693        let redacted = redact(&sentence);
2694        assert!(!redacted.contains(JIT_BLOB), "{redacted}");
2695        assert!(redacted.starts_with("handing off "), "{redacted}");
2696        assert!(redacted.ends_with(" to the runner"), "{redacted}");
2697    }
2698
2699    #[test]
2700    fn a_secret_embedded_in_a_structured_value_is_redacted() {
2701        // Compact JSON is what `serde_json::to_string` emits and what an
2702        // HTTP error body arrives as; spaced JSON is the only shape that
2703        // used to be safe, and it was safe by accident -- the value is its
2704        // own word there, so the opaque-run rule caught it without the key
2705        // ever being recognised.
2706        for shape in [
2707            format!("{{\"encoded_jit_config\":\"{JIT_BLOB}\"}}"),
2708            format!("{{ \"encoded_jit_config\": \"{JIT_BLOB}\" }}"),
2709            format!("encoded_jit_config:{JIT_BLOB}"),
2710            format!("{{\"runner_token\":\"{USER_TOKEN}\"}}"),
2711            format!("runner_token:{USER_TOKEN}"),
2712            format!("pat:{USER_TOKEN}"),
2713            format!("{{\"authorization\":\"Bearer {USER_TOKEN}\"}}"),
2714        ] {
2715            let redacted = redact(&shape);
2716            assert!(
2717                !redacted.contains(JIT_BLOB) && !redacted.contains(USER_TOKEN),
2718                "leaked from {shape}:\n{redacted}"
2719            );
2720        }
2721
2722        // A credential short enough to clear the opaque-run threshold and
2723        // without a GitHub prefix has nothing but the key to give it away,
2724        // so it leaked from the spaced shape too: `\"password\":` left the
2725        // key spelled `password\"`, which is on no list.
2726        for shape in [
2727            "{\"password\":\"hunter2\"}",
2728            "{ \"password\": \"hunter2\" }",
2729        ] {
2730            let redacted = redact(shape);
2731            assert!(
2732                !redacted.contains("hunter2"),
2733                "leaked from {shape}: {redacted}"
2734            );
2735        }
2736
2737        // The neighbouring context still survives: this is a scrubber, not
2738        // a deleter.
2739        let body = format!("registration failed: {{\"encoded_jit_config\":\"{JIT_BLOB}\"}}");
2740        let redacted = redact(&body);
2741        assert!(redacted.starts_with("registration failed: "), "{redacted}");
2742        assert!(redacted.contains("encoded_jit_config"), "{redacted}");
2743    }
2744
2745    #[test]
2746    fn a_credential_key_is_found_wherever_it_sits_in_a_compact_structure() {
2747        // Every shape the test above covers puts the credential in the
2748        // *first* key/value pair, and the rules split a word once, on the
2749        // first separator they find. So redaction was a function of field
2750        // order -- and `serde_json::to_string` is what decides field order for
2751        // any struct with more than one field, which is what an error body is.
2752        //
2753        // Nesting and a form-encoded body are the same defect wearing
2754        // different punctuation: the value recursion ends at `redact_value`,
2755        // which is a leaf, and `&` was not a separator at all.
2756        for shape in [
2757            // The credential key second, in a compact object.
2758            format!("{{\"runner_id\":42,\"encoded_jit_config\":\"{JIT_BLOB}\"}}"),
2759            format!("{{\"status\":422,\"message\":\"bad\",\"runner_token\":\"{USER_TOKEN}\"}}"),
2760            // Nested one level, and two.
2761            format!("{{\"body\":{{\"runner_token\":\"{USER_TOKEN}\"}}}}"),
2762            format!(
2763                "{{\"error\":{{\"status\":422,\"body\":{{\"encoded_jit_config\":\"{JIT_BLOB}\"}}}}}}"
2764            ),
2765            // An array of objects, which is what a list endpoint returns.
2766            format!("[{{\"id\":1}},{{\"access_token\":\"{USER_TOKEN}\"}}]"),
2767            // Form-encoded, credential second and in the middle.
2768            format!("scope=repo&access_token={USER_TOKEN}"),
2769            format!("grant_type=refresh&refresh_token={USER_TOKEN}&scope=repo"),
2770        ] {
2771            let redacted = redact(&shape);
2772            assert!(
2773                !redacted.contains(JIT_BLOB) && !redacted.contains(USER_TOKEN),
2774                "leaked from {shape}:\n{redacted}"
2775            );
2776            assert!(redacted.contains(REDACTION), "{shape} -> {redacted}");
2777        }
2778
2779        // The pairs around it stay legible. A body whose every field came back
2780        // `[redacted]` diagnoses nothing, and is how a redaction gets turned
2781        // off rather than fixed.
2782        let redacted = redact(&format!(
2783            "failed: {{\"runner_id\":42,\"encoded_jit_config\":\"{JIT_BLOB}\"}}"
2784        ));
2785        assert!(redacted.starts_with("failed: "), "{redacted}");
2786        assert!(redacted.contains("\"runner_id\":42"), "{redacted}");
2787        assert!(redacted.contains("encoded_jit_config"), "{redacted}");
2788
2789        // A credential short enough to clear the opaque-run threshold and
2790        // carrying no GitHub prefix has nothing but its key to give it away,
2791        // so it is the case the shape rules cannot rescue.
2792        for shape in [
2793            "{\"user\":\"operator\",\"password\":\"hunter2\"}",
2794            "{\"body\":{\"password\":\"hunter2\"}}",
2795            "user=operator&password=hunter2",
2796        ] {
2797            assert!(
2798                !redact(shape).contains("hunter2"),
2799                "leaked from {shape}: {}",
2800                redact(shape)
2801            );
2802        }
2803
2804        // Ordinary punctuation still survives the cut, or the structure that
2805        // makes a log line readable is gone with the secret.
2806        assert_eq!(
2807            redact("desired 3, active 1, headroom 2"),
2808            "desired 3, active 1, headroom 2"
2809        );
2810        assert_eq!(
2811            redact("labels=[linux,x64,self-hosted]"),
2812            "labels=[linux,x64,self-hosted]"
2813        );
2814    }
2815
2816    #[test]
2817    fn backslash_escaped_json_reaches_the_key_rules_and_the_value_rules() {
2818        // `tracing::error!(reason = ?err)` reaches this module through
2819        // `record_debug` and `format!("{:?}")`, and `Debug` on a `String`
2820        // escapes the quotes inside it. So an error whose `Debug` embeds an
2821        // HTTP body arrives with its keys spelled `\"password\"` -- and `\`
2822        // is not in `WRAPPERS`, so `trim_matches(WRAPPERS)` left it welded on
2823        // and no list contained the result. `reason` is on ALLOWED_FIELDS, so
2824        // this reaches the scrubber rather than being dropped.
2825        let failure = StoreError {
2826            body: "{\"password\":\"hunter2\"}".to_string(),
2827        };
2828        // The fixture carries the secret before anything has looked at it. A
2829        // premise, asserted rather than assumed: a fixture that quietly
2830        // stopped carrying one would make every assertion below vacuously
2831        // true.
2832        assert!(failure.body.contains("hunter2"), "{}", failure.body);
2833        let rendered = format!("{failure:?}");
2834        // The premise, asserted rather than assumed: `Debug` really does
2835        // produce the escaped spelling. If it ever stops, this test is
2836        // measuring something else.
2837        assert!(
2838            rendered.contains("\\\"password\\\""),
2839            "the premise is that Debug escapes the quotes: {rendered}"
2840        );
2841        assert!(
2842            !redact(&rendered).contains("hunter2"),
2843            "leaked: {}",
2844            redact(&rendered)
2845        );
2846
2847        for shape in [
2848            "{\\\"password\\\":\\\"hunter2\\\"}",
2849            "Error { body: \"{\\\"access_token\\\":\\\"hunter2\\\"}\" }",
2850        ] {
2851            assert!(
2852                !redact(shape).contains("hunter2"),
2853                "leaked from {shape}: {}",
2854                redact(shape)
2855            );
2856        }
2857
2858        // The escaped spelling of a long secret leaked too, and for the same
2859        // reason: the key was unrecognised, so nothing below it ever ran.
2860        let escaped = format!("{{\\\"encoded_jit_config\\\":\\\"{JIT_BLOB}\\\"}}");
2861        assert!(!redact(&escaped).contains(JIT_BLOB), "{}", redact(&escaped));
2862
2863        // Trimming the key is only half of it, and it is the half that is
2864        // easy to mistake for the whole. `runner_token` is deliberately *not*
2865        // on CREDENTIAL_KEYS -- a field name is not what makes a value a
2866        // secret, and this module's own documentation says so -- so that pair
2867        // is caught by the token-prefix rule reading its *value*, or it is not
2868        // caught at all. The escaped quote hid the value from that rule
2869        // exactly as it hid the key from the key rules, and a key-only fix
2870        // leaves this one leaking through the sink.
2871        assert!(
2872            !is_credential_key("runner_token"),
2873            "the premise of this case is an unlisted key; once it is listed, \
2874             this stops exercising the value side at all"
2875        );
2876        let escaped_value = format!("{{\\\"runner_token\\\":\\\"{USER_TOKEN}\\\"}}");
2877        assert!(
2878            !redact(&escaped_value).contains(USER_TOKEN),
2879            "the value side leaked: {}",
2880            redact(&escaped_value)
2881        );
2882
2883        // The trim is unconditional on a key and conditional on a value: a
2884        // backslash comes off a value only where it is escaping punctuation.
2885        // `split_wrappers` runs before `looks_like_path`, so putting `\` in
2886        // `WRAPPERS` would trim away the leading `\\` that UNC detection keys
2887        // on and turn a share path back into ordinary text.
2888        assert_eq!(redact(r"\\fileserver\share\jit"), PATH_REDACTION);
2889        assert_eq!(redact(r"\\?\C:\Users\operator\runtime"), PATH_REDACTION);
2890        // And the escaped spelling of a share path is still a share path:
2891        // `\\` escapes nothing, so it survives the trim that `\"` does not.
2892        assert_eq!(redact(r"\\\\fileserver\\share\\jit"), PATH_REDACTION);
2893        // A path ending in its own separator is still a path without it. The
2894        // separator is punctuation, so it is trimmed, judged, and put back.
2895        let trailing = redact("C:\\Users\\operator\\runtime\\");
2896        assert!(trailing.starts_with(PATH_REDACTION), "{trailing}");
2897        assert!(!trailing.contains("operator"), "{trailing}");
2898    }
2899
2900    #[test]
2901    fn a_url_does_not_make_the_rest_of_its_word_unredactable() {
2902        // `redact_core` split on the first `://` and handed *everything*
2903        // before it to `redact_url` as a scheme and everything after it as a
2904        // URL. The terminal arm of `redact_url` echoes both verbatim, so a
2905        // single URL anywhere in a word turned the whole of the rest of that
2906        // word into text no rule below could reach. The only thing that saved
2907        // the shape was a `?` or `#` inside the URL, which made `redact_url`
2908        // replace the tail.
2909        //
2910        // `documentation_url` is in essentially every GitHub REST error body,
2911        // so this was any such body logged alongside a credential.
2912        let body = format!(
2913            "{{\"message\":\"Bad credentials\",\"documentation_url\":\"https://docs.github.com/rest\",\"token\":\"{USER_TOKEN}\"}}"
2914        );
2915        let redacted = redact(&body);
2916        assert!(!redacted.contains(USER_TOKEN), "leaked: {redacted}");
2917        // The URL is the diagnosable part and must survive, or nobody keeps
2918        // this redaction.
2919        assert!(
2920            redacted.contains("https://docs.github.com/rest"),
2921            "the URL should survive: {redacted}"
2922        );
2923
2924        // The mirror order: everything before the `://` became the "scheme".
2925        let reversed = format!(
2926            "{{\"token\":\"{USER_TOKEN}\",\"documentation_url\":\"https://docs.github.com/rest\"}}"
2927        );
2928        let redacted = redact(&reversed);
2929        assert!(!redacted.contains(USER_TOKEN), "leaked: {redacted}");
2930
2931        // A nested object behind a URL was missed even when the URL's own
2932        // userinfo was caught, because the miss and the catch happened in the
2933        // same call.
2934        let clone = format!(
2935            "{{\"remote\":\"https://x-access-token:{USER_TOKEN}@github.com/o/r.git\",\"body\":{{\"password\":\"hunter2\"}}}}"
2936        );
2937        let redacted = redact(&clone);
2938        assert!(!redacted.contains(USER_TOKEN), "leaked: {redacted}");
2939        assert!(!redacted.contains("hunter2"), "leaked: {redacted}");
2940        assert!(
2941            redacted.contains("@github.com/o/r.git"),
2942            "the remote should stay diagnosable: {redacted}"
2943        );
2944
2945        // A URL still owns its own query string, which is why the URL check
2946        // sits ahead of the structural cut: `?` and `#` are not structural
2947        // characters, and an OAuth response puts the token after one of them.
2948        assert_eq!(
2949            redact(&format!(
2950                "{{\"url\":\"https://api.github.com/x?token={USER_TOKEN}\"}}"
2951            )),
2952            format!("{{\"url\":\"https://api.github.com/x?{REDACTION}\"}}")
2953        );
2954        // And a URL standing on its own is untouched.
2955        assert_eq!(
2956            redact("GET https://api.github.com/repos/o/r/actions/runners"),
2957            "GET https://api.github.com/repos/o/r/actions/runners"
2958        );
2959
2960        // A URL that is a credential key's *own* value keeps its scheme, host
2961        // and path like every other URL -- a URL is not a token, and this
2962        // module keeps exactly that much of one everywhere else. What must not
2963        // happen is the key's empty value being redacted on the way past,
2964        // which put the URL out behind a `[redacted]` that had replaced
2965        // nothing: `token=[redacted]https://evil.example/x`. Bounding the URL
2966        // is what created that shape, and pass one declining an empty value is
2967        // what closes it.
2968        assert_eq!(
2969            redact("token=https://evil.example/x"),
2970            "token=https://evil.example/x"
2971        );
2972        assert_eq!(
2973            redact("{\"token\":\"https://evil.example/x\"}"),
2974            "{\"token\":\"https://evil.example/x\"}"
2975        );
2976        // An empty credential value is nothing to redact wherever it sits, and
2977        // saying otherwise reports a secret in a place none was.
2978        assert_eq!(redact("{\"password\":\"\"}"), "{\"password\":\"\"}");
2979    }
2980
2981    #[test]
2982    fn an_array_element_is_judged_without_the_quote_that_wraps_it() {
2983        // The structural cut recursed on the *raw* fragment, and the terminal
2984        // fallback then called `redact_value` with the wrappers still on. A
2985        // fragment carrying no `:` or `=` of its own -- which is exactly what
2986        // an array element is -- therefore reached the shape rules as
2987        // `"ghu_…`: `starts_with("ghu_")` fails, `is_opaque_char` fails on the
2988        // quote, `starts_with("eyJ")` fails, and `looks_like_path` never gets
2989        // a clean look at `"C:\Users\…`.
2990        //
2991        // This is the defect the structural cut was written to close, one step
2992        // further down: "only the first key/value pair is ever examined"
2993        // became "a value with no key of its own is never examined". It
2994        // survived a round because `scan_for_leaks` had no array among its
2995        // twelve shapes.
2996        for shape in [
2997            format!("{{\"tokens\":[\"{USER_TOKEN}\",\"x\"]}}"),
2998            format!("{{\"tokens\":[\"x\",\"{USER_TOKEN}\"]}}"),
2999            format!("[\"x\",\"{USER_TOKEN}\"]"),
3000            format!("[{{\"id\":1}},[\"{USER_TOKEN}\"]]"),
3001            // The `Debug`-escaped spelling of the same thing.
3002            format!("{{\\\"tokens\\\":[\\\"x\\\",\\\"{USER_TOKEN}\\\"]}}"),
3003        ] {
3004            let redacted = redact(&shape);
3005            assert!(
3006                !redacted.contains(USER_TOKEN),
3007                "leaked from {shape}: {redacted}"
3008            );
3009            assert!(redacted.contains(REDACTION), "{shape} -> {redacted}");
3010        }
3011
3012        // Every other shape rule was reachable the same way.
3013        let jit = format!("{{\"items\":[\"{JIT_BLOB}\"]}}");
3014        assert!(!redact(&jit).contains(JIT_BLOB), "{}", redact(&jit));
3015        let assertions = format!("{{\"assertions\":[\"{JWT}\"]}}");
3016        assert!(
3017            !redact(&assertions).contains(JWT),
3018            "{}",
3019            redact(&assertions)
3020        );
3021        let opaque = "ZYXWVUTSRQPONMLKJIHGFEDCBA9876543210zyxwvut";
3022        assert!(opaque.len() > OPAQUE_RUN_THRESHOLD, "{}", opaque.len());
3023        let listed = format!("[\"x\",\"{opaque}\"]");
3024        assert!(!redact(&listed).contains(opaque), "{}", redact(&listed));
3025
3026        // A path in an array. `07-security.md` requires paths redacted, and
3027        // only `looks_like_path` can see one -- with a clean look at the value
3028        // and not before.
3029        let roots = format!("{{\"roots\":[\"x\",\"{WINDOWS_WORKSPACE}\"]}}");
3030        let redacted = redact(&roots);
3031        assert!(!redacted.contains(WINDOWS_WORKSPACE), "leaked: {redacted}");
3032        assert!(redacted.contains(PATH_REDACTION), "{redacted}");
3033
3034        // The array survives being scrubbed: this is a scrubber, not a
3035        // deleter.
3036        assert_eq!(
3037            redact("labels=[\"linux\",\"x64\"]"),
3038            "labels=[\"linux\",\"x64\"]"
3039        );
3040    }
3041
3042    #[test]
3043    fn a_semicolon_cuts_a_word_the_way_a_comma_does() {
3044        // `;` was in `WRAPPERS` -- recognised as punctuation -- and not in
3045        // `STRUCTURAL`, so it never cut a word. It is the separator for a
3046        // Windows connection string, for a credential string, and for cookie
3047        // pairs written without a space, which is `d2`'s territory exactly.
3048        // `Set-Cookie: theme=dark; session=…` was safe only because of the
3049        // space after the `;`.
3050        for shape in [
3051            "store rejected: Server=host;Database=x;Password=hunter2;",
3052            "store rejected: user=operator;password=hunter2",
3053            "Server=host;Password=hunter2;Database=x",
3054        ] {
3055            let redacted = redact(shape);
3056            assert!(
3057                !redacted.contains("hunter2"),
3058                "leaked from {shape}: {redacted}"
3059            );
3060        }
3061
3062        let cookies = format!("theme=dark;session={USER_TOKEN}");
3063        let redacted = redact(&cookies);
3064        assert!(!redacted.contains(USER_TOKEN), "leaked: {redacted}");
3065
3066        // Separators go back verbatim, so ordinary prose is unaffected.
3067        assert_eq!(
3068            redact("started; then reconciled; then idled"),
3069            "started; then reconciled; then idled"
3070        );
3071        assert_eq!(
3072            redact("desired 3;active 1;headroom 2"),
3073            "desired 3;active 1;headroom 2"
3074        );
3075    }
3076
3077    #[test]
3078    fn an_element_wrapped_value_is_redacted() {
3079        // `split_wrappers` strips only the *outermost* `<` and `>`, so
3080        // `<string>ghu_…</string>` arrived as `string>ghu_…</string`, which
3081        // matches no rule at all. `d3`'s installers handle launchd plists,
3082        // which is where this shape comes from; a systemd unit line is the
3083        // `key=value` spelling and was already covered.
3084        for shape in [
3085            format!("<string>{USER_TOKEN}</string>"),
3086            format!("<key>token</key><string>{USER_TOKEN}</string>"),
3087            format!("<dict><key>Token</key><string>{USER_TOKEN}</string></dict>"),
3088        ] {
3089            let redacted = redact(&shape);
3090            assert!(
3091                !redacted.contains(USER_TOKEN),
3092                "leaked from {shape}: {redacted}"
3093            );
3094            assert!(redacted.contains(REDACTION), "{shape} -> {redacted}");
3095        }
3096
3097        // The element names survive: they are what says the line was about a
3098        // plist at all.
3099        let plist = format!("<key>token</key><string>{JIT_BLOB}</string>");
3100        let redacted = redact(&plist);
3101        assert!(!redacted.contains(JIT_BLOB), "leaked: {redacted}");
3102        assert!(redacted.contains("<key>token</key>"), "{redacted}");
3103
3104        // An angle bracket in ordinary text is re-emitted verbatim.
3105        assert_eq!(redact("Custom<Io>"), "Custom<Io>");
3106    }
3107
3108    #[test]
3109    fn a_redacted_value_keeps_the_punctuation_that_wrapped_it() {
3110        // Pass one returned `format!("{key}{separator}{REDACTION}")` and
3111        // dropped the value's trailing wrapper, so a redacted object came out
3112        // with an unbalanced quote: `{"password":[redacted]"}`. Pass two
3113        // already put `lead` and `trail` back.
3114        //
3115        // Worth fixing because this module argues, correctly, that the
3116        // structure around a redaction is what keeps a line diagnosable -- and
3117        // a reader who cannot parse the line cannot tell a redaction from a
3118        // truncation.
3119        assert_eq!(
3120            redact("{\"password\":\"hunter2\"}"),
3121            format!("{{\"password\":\"{REDACTION}\"}}")
3122        );
3123        assert_eq!(
3124            redact(&format!("{{\"access_token\":\"{USER_TOKEN}\"}}")),
3125            format!("{{\"access_token\":\"{REDACTION}\"}}")
3126        );
3127        assert_eq!(
3128            redact(&format!(
3129                "[{{\"id\":1}},{{\"access_token\":\"{USER_TOKEN}\"}}]"
3130            )),
3131            format!("[{{\"id\":1}},{{\"access_token\":\"{REDACTION}\"}}]")
3132        );
3133        // The `Debug`-escaped spelling keeps its escaped quotes.
3134        assert_eq!(
3135            redact("{\\\"password\\\":\\\"hunter2\\\"}"),
3136            format!("{{\\\"password\\\":\\\"{REDACTION}\\\"}}")
3137        );
3138
3139        // The point of all of it: what the sink writes is still the JSON it
3140        // was handed, minus the secret.
3141        let line = redact(&format!(
3142            "{{\"runner_id\":42,\"access_token\":\"{USER_TOKEN}\"}}"
3143        ));
3144        serde_json::from_str::<Value>(&line)
3145            .unwrap_or_else(|error| panic!("a redacted body must still parse: {line} ({error})"));
3146    }
3147
3148    #[test]
3149    fn a_bare_jwt_is_redacted_despite_its_dots() {
3150        // `is_opaque_char` excludes `.`, so a JWT is three opaque runs rather
3151        // than one, each under the threshold, and a 100-character credential
3152        // printed verbatim. `Authorization: Bearer <jwt>` is caught by the
3153        // scheme rule and a credential-keyed one by the key rule, so this bit
3154        // only where a token stood alone or under a name nobody listed.
3155        assert!(
3156            JWT.len() > 100,
3157            "the premise is a long token: {}",
3158            JWT.len()
3159        );
3160        assert_eq!(redact(JWT), REDACTION);
3161        assert_eq!(
3162            redact(&format!("minted {JWT} for the installation")),
3163            format!("minted {REDACTION} for the installation")
3164        );
3165        // Sentence-final punctuation is trimmed as a wrapper first, so the
3166        // token is still recognised.
3167        assert!(!redact(&format!("minted {JWT}.")).contains(JWT));
3168        // Under a key nobody listed.
3169        assert!(!redact(&format!("assertion={JWT}")).contains(JWT));
3170        assert!(!redact(&format!("{{\"assertion\":\"{JWT}\"}}")).contains(JWT));
3171
3172        // Narrow on purpose. Adding `.` to `is_opaque_char` is the obvious fix
3173        // and the wrong one: it swallows every long dotted word there is.
3174        for ordinary in [
3175            "com.example.runner.manager.platform.process.identity.token",
3176            "runner-manager.2026.08.21.log",
3177            "api.github.com",
3178            "9f2c1a44-0000-4000-8000-000000000001.attempt.json",
3179        ] {
3180            assert_eq!(redact(ordinary), ordinary, "over-redacted {ordinary}");
3181        }
3182    }
3183
3184    #[test]
3185    fn the_sink_redacts_a_short_credential_that_only_its_key_gives_away() {
3186        // `scan_for_leaks` cannot carry this case. A credential short enough
3187        // to clear the opaque-run threshold and carrying no GitHub prefix is
3188        // indistinguishable from an ordinary word when it is logged bare, so
3189        // injecting it through shape 1 would demand a redaction no shape rule
3190        // can deliver. Here the key is always present, which is the situation
3191        // `d2`'s secret store is actually in -- and it is through the sink,
3192        // not through `redact` alone, because that distinction is why these
3193        // shapes survived a round.
3194        let capture = Capture::default();
3195        let output = emit(&capture, || {
3196            tracing::error!("store rejected: {{\"user\":\"operator\",\"password\":\"hunter2\"}}");
3197            tracing::error!("store rejected: {{\"body\":{{\"password\":\"hunter2\"}}}}");
3198            tracing::error!("store rejected: user=operator&password=hunter2");
3199            let failure = StoreError {
3200                body: "{\"password\":\"hunter2\"}".to_string(),
3201            };
3202            tracing::error!(reason = ?failure, "store rejected the request");
3203
3204            // A `;` or an `&` after a URL in the same word. The URL branch runs
3205            // ahead of the structural cut and `split_url` did not stop at
3206            // either character, so the separator that would have cut the word
3207            // was swallowed into the URL and echoed.
3208            tracing::error!("store rejected: Server=https://vault.local/api;Password=hunter2;");
3209            tracing::error!("keychain error: url=https://kc.local;password=hunter2");
3210            tracing::error!("unit rejected: Environment=API=https://a.com/v1;PASSWORD=hunter2");
3211            tracing::error!("token exchange failed: cb=https://a.com/x&password=hunter2");
3212
3213            // A credential key whose value is in a *later* fragment. The
3214            // empty-value skip is right; the claim that the structural cut
3215            // reaches such a value was not, because nothing carried the key
3216            // across the cut.
3217            tracing::error!("store rejected: {{\"password\":[\"hunter2\"]}}");
3218            tracing::error!("store rejected: {{\"password\":{{\"v\":\"hunter2\"}}}}");
3219            tracing::error!("store rejected: Password=;hunter2");
3220            tracing::error!("plist rejected: <key>password</key><string>hunter2</string>");
3221            tracing::error!(
3222                "plist rejected: <dict><key>password</key><string>hunter2</string></dict>"
3223            );
3224
3225            // A structural character inside the value. `,` and `&` could
3226            // already split a secret; this commit added `;`, `<` and `>`.
3227            //
3228            // The pieces are spelled so that none of them is a substring of
3229            // anything the line legitimately keeps -- `ss` would have "found" a
3230            // leak in the surviving key `password`, which is a check that fails
3231            // for the wrong reason and is no better than one that cannot fail.
3232            tracing::error!("store rejected: {{\"password\":\"qq<vv>xx\"}}");
3233            tracing::error!("store rejected: {{\"password\":\"j1&k2,m3\"}}");
3234            tracing::error!("store rejected: {{\"password\":\"n4;p5<q6>r7,t8\"}}");
3235        });
3236
3237        assert!(
3238            !output.contains("hunter2"),
3239            "a short credential leaked through the sink:\n{output}"
3240        );
3241        // The punctuated passwords: every piece a structural character can cut
3242        // one into has to go, not just the piece that kept the key company.
3243        for piece in [
3244            "qq", "vv", "xx", "j1", "k2", "m3", "n4", "p5", "q6", "r7", "t8",
3245        ] {
3246            assert!(
3247                !output.contains(piece),
3248                "a structural character split a secret and the tail survived \
3249                 ({piece}):\n{output}"
3250            );
3251        }
3252        assert!(output.contains(REDACTION), "{output}");
3253        // Sixteen lines, or the sink was never reached and "no secret found" is
3254        // true of an empty string.
3255        assert_eq!(
3256            output
3257                .lines()
3258                .filter(|line| !line.trim().is_empty())
3259                .count(),
3260            16,
3261            "{output}"
3262        );
3263    }
3264
3265    /// The multi-word plist spelling, which is worse than the one-word one:
3266    /// the credential key and the value are in the same word, but the value
3267    /// itself is two words, so closing it needs the fragment carry to reach the
3268    /// word-level follow-on rule.
3269    #[test]
3270    fn a_credential_value_that_runs_past_its_own_word_is_still_redacted() {
3271        let capture = Capture::default();
3272        let output = emit(&capture, || {
3273            tracing::error!("plist rejected: <key>password</key><string>correct horse</string>");
3274            tracing::error!("plist rejected: <key>password</key> <string>correct horse</string>");
3275        });
3276        for piece in ["correct", "horse"] {
3277            assert!(
3278                !output.contains(piece),
3279                "a multi-word credential value leaked ({piece}):\n{output}"
3280            );
3281        }
3282    }
3283
3284    /// The other half of the fragment carry: it must not swallow the ordinary,
3285    /// *closed* pairs sitting next to a credential.
3286    ///
3287    /// Every shape here is one the carry has an opportunity to over-redact —
3288    /// a closed empty value, a neighbouring key, the pairs of a connection
3289    /// string, and the element names of a plist. Redaction that eats the
3290    /// diagnostics is redaction nobody keeps, and a carry with no terminator
3291    /// is exactly how that happens.
3292    #[test]
3293    fn the_fragment_carry_stops_where_the_value_does() {
3294        for (input, expected) in [
3295            (
3296                "{\"password\":\"abc\",\"user\":\"bob\"}",
3297                format!("{{\"password\":\"{REDACTION}\",\"user\":\"bob\"}}"),
3298            ),
3299            (
3300                "Server=host;Password=hunter2;User=bob",
3301                format!("Server=host;Password={REDACTION};User=bob"),
3302            ),
3303            (
3304                "{\"password\":\"\",\"user\":\"bob\"}",
3305                "{\"password\":\"\",\"user\":\"bob\"}".to_string(),
3306            ),
3307            (
3308                "<dict><key>password</key><string>hunter2</string></dict>",
3309                format!("<dict><key>password</key><string>{REDACTION}</string></dict>"),
3310            ),
3311            (
3312                "{\"password\":\"qq<vv>xx\"}",
3313                format!("{{\"password\":\"{REDACTION}<{REDACTION}>{REDACTION}\"}}"),
3314            ),
3315        ] {
3316            assert_eq!(redact(input), expected, "from {input}");
3317        }
3318
3319        // A closed value ends the claim, so the word after it is ordinary text.
3320        assert_eq!(
3321            redact("{\"password\":\"x\"} ok"),
3322            format!("{{\"password\":\"{REDACTION}\"}} ok")
3323        );
3324        assert_eq!(
3325            redact("Server=host;Password=hunter2; ok"),
3326            format!("Server=host;Password={REDACTION}; ok")
3327        );
3328
3329        // `{"password":""} ok` is *not* on that list, and the reason is worth
3330        // recording rather than asserting away. `split_wrappers` strips the
3331        // empty value with the rest of the trailing punctuation, so the word
3332        // reaches `redact_word` as the core `password":` — a bare credential
3333        // key, which the stem rule has always read as "the value is the next
3334        // word". That rule predates the carry, is untouched by it, and fires
3335        // first; the empty-value skip below it never gets a look. So the `ok`
3336        // is redacted, exactly as it was before any of this.
3337        assert_eq!(
3338            redact("{\"password\":\"\"} ok"),
3339            format!("{{\"password\":\"\"}} {REDACTION}")
3340        );
3341        // What the empty-value skip does still guarantee is the thing it was
3342        // written for: no `[redacted]` is invented *inside* the pair itself.
3343        assert_eq!(redact("{\"password\":\"\"}"), "{\"password\":\"\"}");
3344
3345        // Ordinary prose is untouched: the carry is only ever armed by a
3346        // credential key.
3347        assert_eq!(
3348            redact("started; then reconciled; then idled"),
3349            "started; then reconciled; then idled"
3350        );
3351        assert_eq!(redact("Custom<Io>"), "Custom<Io>");
3352    }
3353
3354    #[test]
3355    fn paths_are_redacted_on_both_families() {
3356        assert_eq!(redact(WORKSPACE), PATH_REDACTION);
3357        assert_eq!(redact(WINDOWS_WORKSPACE), PATH_REDACTION);
3358        assert_eq!(redact("\\\\fileserver\\share\\jit"), PATH_REDACTION);
3359        assert_eq!(
3360            redact("~/Library/Application Support/x"),
3361            format!("{PATH_REDACTION} Support/x")
3362        );
3363        assert_eq!(
3364            redact("runtime=/var/lib/runner-manager/runtime"),
3365            format!("runtime={PATH_REDACTION}")
3366        );
3367    }
3368
3369    #[test]
3370    fn a_url_survives_but_its_query_string_and_fragment_do_not() {
3371        assert_eq!(
3372            redact("GET https://api.github.com/repos/owner/repo/actions/runners"),
3373            "GET https://api.github.com/repos/owner/repo/actions/runners"
3374        );
3375        assert_eq!(
3376            redact("https://api.github.com/x?access_token=ghu_secret"),
3377            format!("https://api.github.com/x?{REDACTION}")
3378        );
3379        // A fragment carries a token in an OAuth implicit-flow response, and
3380        // is never needed to diagnose an HTTP call.
3381        assert_eq!(
3382            redact("https://api.github.com/x#access_token=ghu_secret"),
3383            format!("https://api.github.com/x#{REDACTION}")
3384        );
3385        // Whichever comes first ends the diagnosable part.
3386        assert_eq!(
3387            redact("https://api.github.com/x?page=2#access_token=ghu_secret"),
3388            format!("https://api.github.com/x?{REDACTION}")
3389        );
3390    }
3391
3392    /// `;` and `&` end a URL, but only ahead of its query string.
3393    ///
3394    /// `STRUCTURAL` has nine characters and `is_url_terminator` recognised
3395    /// seven of them. Because the URL branch runs *ahead* of the structural
3396    /// cut, a URL earlier in a word made `split_url` swallow everything to the
3397    /// next terminator — including the `;` or `&` that would have cut the word
3398    /// — and `redact_url`'s terminal arm echoes what it is given. So the L3 fix
3399    /// (bounding a URL) and the L1 fix (cutting on `;` and `&`) did not
3400    /// compose, and this commit added `;` to `STRUCTURAL` for connection
3401    /// strings while leaving the path a URL opens straight through it.
3402    #[test]
3403    fn a_separator_after_a_url_still_cuts_the_word() {
3404        for shape in [
3405            format!("store rejected: Server=https://vault.local/api;Password={USER_TOKEN};"),
3406            format!("keychain error: url=https://kc.local;secret={USER_TOKEN}"),
3407            format!("unit rejected: Environment=API=https://a.com/v1;TOKEN={USER_TOKEN}"),
3408            format!("token exchange failed: cb=https://a.com/x&access_token={USER_TOKEN}"),
3409            format!("dsn rejected: dsn=https://sentry.local/1;password={USER_TOKEN};user=x"),
3410        ] {
3411            let redacted = redact(&shape);
3412            assert!(
3413                !redacted.contains(USER_TOKEN),
3414                "leaked from {shape}:\n{redacted}"
3415            );
3416        }
3417
3418        // The URL itself still survives, or nobody keeps this redaction.
3419        assert_eq!(
3420            redact("store rejected: Server=https://vault.local/api;Password=hunter2;"),
3421            format!("store rejected: Server=https://vault.local/api;Password={REDACTION};")
3422        );
3423
3424        // The obvious fix is the wrong one, and this is what it costs. Making
3425        // `;` and `&` terminators everywhere ends a URL *inside* its own query
3426        // string, and `redact_url` replaces a query wholesale precisely because
3427        // a token can be in any parameter and this module does not guess which:
3428        // `?[redacted]` would become `?[redacted]&state=hunter2`. So the cut is
3429        // restricted to the part of the URL ahead of the first `?` or `#`.
3430        assert_eq!(
3431            redact("https://a.com/cb?code=1&state=hunter2"),
3432            format!("https://a.com/cb?{REDACTION}")
3433        );
3434        assert_eq!(
3435            redact("https://a.com/cb#code=1&state=hunter2"),
3436            format!("https://a.com/cb#{REDACTION}")
3437        );
3438        assert_eq!(
3439            redact(&format!(
3440                "https://a.com/cb?a=1;b=2&access_token={USER_TOKEN}"
3441            )),
3442            format!("https://a.com/cb?{REDACTION}")
3443        );
3444
3445        // And leaving `\` out of the terminators is still right: a Windows
3446        // path used as a URL password puts the `@` that identifies the
3447        // userinfo behind the first backslash, and `redact_url` echoes what it
3448        // is given.
3449        assert_eq!(
3450            redact(r"https://x-access-token:C:\Users\op\p@github.com/o/r.git"),
3451            format!("https://{REDACTION}@github.com/o/r.git")
3452        );
3453    }
3454
3455    /// A secret in a URL *path* is echoed verbatim.
3456    ///
3457    /// `redact_url` applied no shape rule to the path at all. The design
3458    /// intends a path to be diagnosable — and it still is — but the
3459    /// token-prefix rule is documented as a belt that catches a credential
3460    /// *anywhere*, and a path was the one place it never ran.
3461    #[test]
3462    fn a_secret_in_a_url_path_is_redacted_and_the_rest_of_the_path_is_not() {
3463        for shape in [
3464            format!("https://github.com/o/r/raw/{USER_TOKEN}/f"),
3465            format!("https://api.github.com/{JIT_BLOB}"),
3466            format!("https://a.com/x/{JWT}?page=2"),
3467        ] {
3468            let redacted = redact(&shape);
3469            assert!(
3470                !redacted.contains(USER_TOKEN)
3471                    && !redacted.contains(JIT_BLOB)
3472                    && !redacted.contains(JWT),
3473                "leaked from {shape}:\n{redacted}"
3474            );
3475        }
3476
3477        assert_eq!(
3478            redact(&format!("https://github.com/o/r/raw/{USER_TOKEN}/f")),
3479            format!("https://github.com/o/r/raw/{REDACTION}/f")
3480        );
3481
3482        // The rest of the path is exactly as diagnosable as it was: a segment
3483        // is judged on its own, and an ordinary one is not a secret.
3484        for url in [
3485            "https://api.github.com/repos/owner/repo/actions/runners",
3486            "https://github.com/actions/runner/releases/download/v2.330.0/actions-runner-linux-x64-2.330.0.tar.gz",
3487            "https://github.com/o/r.git",
3488            "https://github.com/@owner/repo",
3489            "https://api.github.com/repos/o/r/actions/runners/42",
3490        ] {
3491            assert_eq!(redact(url), url, "over-redacted a diagnosable path: {url}");
3492        }
3493    }
3494
3495    /// A large message must not take the process down.
3496    ///
3497    /// The URL branch recursed into `redact_core` on both sides of the cut. The
3498    /// termination argument was sound — every call is on a strictly shorter
3499    /// slice — and it was silent on *depth*, which was linear in the length of
3500    /// the input. A message of ~2000 URL-carrying items, about 86 KB, exited
3501    /// `0xc00000fd STATUS_STACK_OVERFLOW`.
3502    ///
3503    /// **A stack overflow is not catchable — it aborts the process.** A large
3504    /// HTTP error body is attacker-influenceable content, so this was a way to
3505    /// kill the agent from inside its own log sink, which is worse than any
3506    /// leak: a sink that is not running redacts nothing.
3507    ///
3508    /// Note what this test can and cannot do. An overflow aborts the test
3509    /// binary rather than failing an assertion, so the value of this test is
3510    /// that **the suite completes at all**; the assertions below only confirm
3511    /// it did the work rather than skipping it.
3512    #[test]
3513    fn a_large_message_does_not_overflow_the_stack() {
3514        // 20000 items is ~860 KB and an order of magnitude past the ~2000 that
3515        // was measured to abort. It is also what the commit *before* the URL
3516        // branch survived, because that branch returned without recursing --
3517        // so this is a regression guard with the margin the regression had.
3518        const ITEMS: usize = 20_000;
3519        let body = r#"{"url":"https://api.github.com/repos/o/r"},"#.repeat(ITEMS);
3520        assert!(body.len() > 800_000, "the premise is a large body");
3521
3522        let redacted = redact(&body);
3523        assert!(
3524            redacted.contains("https://api.github.com/repos/o/r"),
3525            "the URLs are the diagnosable part and must survive"
3526        );
3527
3528        // The same depth, reached through the structural cut and through a
3529        // credential value rather than through a bare URL.
3530        let nested = format!("{{\"error\":{{\"body\":{{\"list\":[{}]}}}}}}", body);
3531        assert!(!redact(&nested).is_empty());
3532
3533        // A long run with no URL in it at all, so the structural loop is
3534        // exercised on its own.
3535        let flat = "{\"a\":1},".repeat(ITEMS);
3536        assert!(!redact(&flat).is_empty());
3537
3538        // And one long unbroken URL, where the *path* loop is what iterates.
3539        let long_path = format!("https://a.com/{}", "seg/".repeat(ITEMS));
3540        assert!(!redact(&long_path).is_empty());
3541    }
3542
3543    #[test]
3544    fn a_credential_header_loses_its_scheme_and_its_value() {
3545        assert_eq!(
3546            redact("Authorization: Bearer abc123"),
3547            format!("Authorization: {REDACTION} {REDACTION}")
3548        );
3549        assert_eq!(
3550            redact("Authorization: Bearer abc123, Accept: application/json"),
3551            // The comma ends the header value, so the next header's name is not
3552            // swallowed.
3553            format!("Authorization: {REDACTION} {REDACTION} Accept: application/json")
3554        );
3555        assert_eq!(
3556            redact("x-api-key=hunter2"),
3557            format!("x-api-key={REDACTION}")
3558        );
3559        assert_eq!(
3560            redact("Cookie: session=abc"),
3561            format!("Cookie: {REDACTION}")
3562        );
3563
3564        // A webhook signature is not a credential the way a token is -- it is
3565        // derived, and it verifies rather than authenticates -- but it has a
3566        // SHA-256's exact shape, so the digest carve-out rendered it as a
3567        // labelled 12-character prefix rather than redacting it. Twelve hex
3568        // characters of an HMAC are of no use to anybody, which is why the
3569        // carve-out is safe in general; the header is on the list anyway,
3570        // because there is no diagnostic worth having in a signature and one
3571        // entry is cheaper than the argument.
3572        let hmac = "9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08";
3573        assert_eq!(
3574            redact(&format!("X-Hub-Signature-256: sha256={hmac}")),
3575            format!("X-Hub-Signature-256: {REDACTION}")
3576        );
3577        // Pass one puts the value's wrapping quote back, so what comes out is
3578        // still the object that went in. It used to emit
3579        // `{"x-hub-signature-256":[redacted]"}` -- an unbalanced quote in a
3580        // line this module argues has to stay diagnosable.
3581        //
3582        // This case is also `redact_core`'s two-pass witness, and the comment
3583        // there names it: the `=` comes first and names nothing, but what
3584        // follows it is a whole HMAC -- non-empty -- so a merged pass inspects
3585        // that value and returns before ever reaching the `:` that names
3586        // `x-hub-signature-256`.
3587        assert_eq!(
3588            redact(&format!("{{\"x-hub-signature-256\":\"sha256={hmac}\"}}")),
3589            format!("{{\"x-hub-signature-256\":\"{REDACTION}\"}}")
3590        );
3591        // The SHA-1 spelling GitHub still sends alongside it.
3592        assert_eq!(
3593            redact(&format!("X-Hub-Signature: sha1={hmac}")),
3594            format!("X-Hub-Signature: {REDACTION}")
3595        );
3596        // The carve-out itself is untouched: a bare digest still renders as a
3597        // prefix, because a checksum gate that cannot say which digest it got
3598        // is a gate nobody can act on.
3599        assert_eq!(redact(hmac), "sha256:9f86d081884c…");
3600    }
3601
3602    #[test]
3603    fn ordinary_text_survives_intact() {
3604        // Redaction that eats the diagnostics is redaction nobody keeps. These
3605        // are the shapes this product's own log lines actually take.
3606        for text in [
3607            "reconciliation finished",
3608            "runner exited idle without work",
3609            "desired 3, active 1, headroom 2",
3610            "attempt 9f2c1a44-0000-4000-8000-000000000001 is busy",
3611            "http 403 after 2 retries",
3612            "windows/x64 is a documented pair",
3613        ] {
3614            assert_eq!(redact(text), text, "redaction damaged an ordinary message");
3615        }
3616    }
3617
3618    /// A SHA-256 is 64 opaque characters and would otherwise be swallowed by
3619    /// the opaque-run rule. `07-security.md` makes checksum verification a
3620    /// security gate, and a gate that cannot say *which* digest it got is a
3621    /// gate nobody can act on, so a digest renders as a labelled prefix.
3622    #[test]
3623    fn a_digest_renders_as_a_prefix_rather_than_disappearing() {
3624        let digest = "9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08";
3625
3626        assert_eq!(redact(digest), "sha256:9f86d081884c…");
3627        // The point of the change: expected-versus-actual stays legible.
3628        let other = "60303ae22b998861bce3b28f33eec1be758a213c86c93c076dbe9f558c11c752";
3629        assert_ne!(
3630            redact(digest),
3631            redact(other),
3632            "two digests must stay distinguishable"
3633        );
3634
3635        // In the sentence shape `e2` will actually write.
3636        assert_eq!(
3637            redact(&format!("checksum mismatch: expected {digest} got {other}")),
3638            "checksum mismatch: expected sha256:9f86d081884c… got sha256:60303ae22b99…"
3639        );
3640
3641        // A prefix a caller logged itself is below the threshold and untouched.
3642        assert_eq!(redact(&digest[..12]), digest[..12].to_string());
3643
3644        // An already-labelled digest is labelled once, not twice, and is
3645        // truncated like a bare one.
3646        //
3647        // Before `redact_core` recursed into a `:` value this arrived at
3648        // `redact_value` whole, matched nothing there -- a `:` is not an
3649        // opaque character -- and was printed in full. Recursing hands
3650        // `as_sha256_digest` the digest on its own, which re-attaches its own
3651        // label, so the redundant one is dropped rather than doubled.
3652        //
3653        // The truncation is the part worth having: a 64-character lowercase
3654        // hex run is only *usually* a digest, an HMAC-SHA256 signature has
3655        // the same shape, and `sha256:<hmac>` used to be emitted whole.
3656        assert_eq!(redact(&format!("sha256:{digest}")), "sha256:9f86d081884c…");
3657        // The `=` spelling had the same doubling and is fixed with it; the
3658        // caller's own separator survives either way.
3659        assert_eq!(redact(&format!("sha256={digest}")), "sha256=9f86d081884c…");
3660    }
3661
3662    #[test]
3663    fn the_digest_exception_is_exactly_sixty_four_lowercase_hex_and_nothing_else() {
3664        // The narrower this carve-out is, the less there is to reason about.
3665        // Anything that is not precisely a lowercase SHA-256 stays redacted by
3666        // the opaque-run rule.
3667        let digest = "9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08";
3668
3669        assert_eq!(
3670            redact(&digest.to_ascii_uppercase()),
3671            REDACTION,
3672            "uppercase hex is not the shape this exception recognises"
3673        );
3674        assert_eq!(
3675            redact(&format!("{digest}0")),
3676            REDACTION,
3677            "65 characters is not a SHA-256"
3678        );
3679        // 64 characters of base64, which is what an encoded secret of that
3680        // length looks like: not hex, so not exempt.
3681        let base64ish = "ZYXWVUTSRQPONMLKJIHGFEDCBA9876543210zyxwvutsrqponmlkjihgfedcba98";
3682        assert_eq!(base64ish.len(), 64);
3683        assert_eq!(redact(base64ish), REDACTION);
3684
3685        // And the JIT blob, which is the thing the opaque-run rule exists for,
3686        // must not have been weakened by any of this.
3687        assert_eq!(redact(JIT_BLOB), REDACTION);
3688    }
3689
3690    #[test]
3691    fn a_uuid_survives() {
3692        // 36 characters, below the opaque-run threshold on purpose: every
3693        // identifier in this product's domain model is a UUID, and redacting
3694        // them would make the logs useless.
3695        let id = "9f2c1a44-0000-4000-8000-000000000001";
3696        assert_eq!(redact(id), id);
3697        assert!(id.len() < OPAQUE_RUN_THRESHOLD);
3698    }
3699
3700    #[test]
3701    fn redaction_preserves_whitespace_and_line_structure() {
3702        let input = "line one\nAuthorization: Bearer x\nline three";
3703        let redacted = redact(input);
3704        assert_eq!(redacted.lines().count(), 3, "{redacted}");
3705        assert!(redacted.starts_with("line one\n"), "{redacted}");
3706        assert!(redacted.ends_with("\nline three"), "{redacted}");
3707    }
3708}