Skip to main content

Module logging

Module logging 

Source
Expand description

The structured log sink, and the redaction that is not optional.

07-security.md’s threat table, on “Logs disclose repository secrets”: “Structured allowlist logging with unconditional redaction of tokens, headers, JIT blobs, and paths.” Its security gate is a secret-injection log scan, and its release gate is that “the user access token and the encoded JIT configuration are absent from logs, databases, snapshots, crash reports, and CLI output”.

§Allowlist, not denylist — and why that is the whole design

A denylist redacts the fields somebody remembered to name. It is correct on the day it is written and wrong on the day a later task adds a field, which is the day it matters. So this sink inverts the default: a field whose name is not in ALLOWED_FIELDS has its value replaced outright. A task that adds runner_token to a log call gets [redacted] with no review, no ceremony, and no leak. Adding a field to the allowlist is a deliberate edit to this file that shows up in a diff.

The field name itself is kept. Knowing that an event carried a runner_token field is useful, and the name is not the secret.

§Two layers, because one is not enough

The allowlist protects fields. It cannot protect the message body, which has to be allowed or the logs say nothing — and a message body is exactly where a secret ends up when somebody writes info!("failed with {authorization}"). So every string that survives the allowlist is then scrubbed by value shape (redact): GitHub token prefixes, credential header names, long opaque runs such as an encoded JIT configuration, and filesystem paths.

Scrubbing by shape over-redacts, deliberately. Some of that is worth knowing about before it surprises somebody:

  • A slash-rooted word is treated as a filesystem path, and a URL path on its own looks exactly like one. Log a full URL — https://api.github.com/… — and it survives intact; log a bare /repos/owner/repo and it becomes [path].
  • Any unbroken run of 40 or more base64, base64url, or hex characters is treated as an opaque secret. There is exactly one carve-out: a value that is precisely 64 lowercase hex characters is a SHA-256 digest, and renders as a labelled 12-character prefix rather than disappearing. A digest is not a secret, and 07-security.md makes checksum verification a security gate whose most useful diagnostic is expected-versus-actual.
  • A JSON Web Token is redacted whole. Its . separators split it into runs the opaque-run rule is too short-sighted to catch, so it is recognised by shape instead: two or three base64url segments whose header begins eyJ.
  • A URL keeps its scheme, host and path and loses everything that authenticates it: the user:password@ userinfo, the query string, and the fragment. The userinfo matters more than it looks — a token-authenticated git remote is https://x-access-token:ghu_…@github.com/owner/repo.git, so it is the shape a clone or fetch failure arrives in. The path stays diagnosable but is not exempt: each segment goes through the same shape rules on its own, because a token in …/raw/ghu_…/f is a token, and the alternative was the one place the belt never ran. A 40-character git object name is opaque enough to go with it.
  • A word ending in : or = whose stem is a credential header name causes the next two words to be redacted, so Authorization: Bearer ghu_… loses both the scheme and the token.

A word is cut on structural punctuation — ,, ;, {, }, [, ], <, > and & — before any of that runs, and each fragment is then judged the way a whole word is: unwrapped, judged on its core, and re-emitted with its punctuation put back. Without that cut only the first key/value pair in a compact structure is ever examined, and redaction becomes a function of field order: {"encoded_jit_config":"…"} was caught and {"runner_id":42,"encoded_jit_config":"…"} was not, while serde_json::to_string is what decides which of the two an error body is. Nesting, a form-encoded body, a ;-separated connection string and a plist element are all that same defect in different punctuation. So is an array element, one step further down — a fragment judged with its quote still attached matches no shape rule at all, and an array element is the one fragment that has no key of its own to give it away.

Because the cut is flat, a credential’s value does not have to be in the same fragment as the key that names it — {"password":["hunter2"]}, {"password":{"v":"hunter2"}} and a plist’s <key>password</key><string>hunter2</string> all put it one or more fragments away — so a carry is threaded along the fragments to say that a key is still waiting for its value, or that a quoted value was cut before its closing quote. It steps over element names, because markup is not the value a key named, and it stops at the punctuation that visibly closes the value, because a redaction reported where no secret was is a false signal in the one log a reader consults to find out whether anything leaked.

A URL is cut out of the text around it rather than being allowed to own the rest of the word. Its scheme is the run of scheme characters immediately before the ://, and it ends at the first character that cannot appear in a URL — plus, ahead of its query string only, at a ; or an &, which are structural characters everywhere else and are query syntax after the ?. Everything on either side goes back through the rules. So {"documentation_url":"https://…","token":"ghu_…"} — which is what a GitHub REST error body looks like — keeps the URL and redacts the token, rather than the URL swallowing the token, and Server=https://vault.local/api;Password=… keeps the URL and redacts the password rather than the URL swallowing the ; that separates them.

The text after a URL is iterated over, not recursed into. That is a memory-safety property, not a style: recursing there made stack depth linear in the length of the message, and ~86 KB of URL-carrying JSON exited STATUS_STACK_OVERFLOW. A stack overflow is not catchable and takes the process with it, so an attacker-influenceable error body could kill the agent from inside its own log sink — and a sink that is not running redacts nothing at all. For the same reason every search in split_url is bounded by the URL’s own end: a pass that never returns is as effective a denial as one that overflows.

A key is also trimmed of backslashes, which a value never is: Debug on a String escapes the quotes inside it, so a body reached through error!(reason = ?err) spells its keys \"password\". The trim_key function documents why the same trim must not be applied to a value; it is named rather than linked because it is private and this module’s documentation is not.

Every one of those is a case where being wrong costs a slightly less readable log line, against a case where being wrong the other way costs a disclosed credential.

§What this does not do

It does not stop a caller printing to standard output, and it does not redact a Debug derive somewhere else in the program. Those are held by different controls: secrecy::SecretString for the two values that matter (07-security.md’s credential inventory), and crate::process::SpawnSpec::spawn_with_handoff for the command line.

Structs§

LoggingGuard
Keeps the background log-writing thread alive.
RedactingLayer
A tracing layer that writes one redacted JSON object per event.

Enums§

LogRole
Whose diagnostics these are, and therefore which file they go in.
LoggingError
The sink could not be installed.

Constants§

ALLOWED_FIELDS
The field names this sink emits verbatim. Everything else is replaced with REDACTION.
OPERATOR_LOG_STEM
The file stem LogRole::Operator writes.
PATH_REDACTION
What replaces something that was recognisably a filesystem path. Distinct from REDACTION so a reader can tell “a path was here” from “a secret was here” without either being disclosed.
REDACTION
What replaces a value this sink will not emit.
SERVICE_LOG_STEM
The file stem LogRole::Service writes, and what service status reports as the daemon’s log.

Functions§

install
Installs the redacting sink as this process’s global subscriber, writing daily-rotating files into logs/.
is_field_allowed
Whether this sink will emit a field’s value rather than replacing it.
redact
Scrubs a string of anything that looks like a credential, an encoded JIT configuration, or a filesystem path.