Expand description
Credential scrubbing and safe truncation (plan §3, roadmap M5).
Every transcript line passes through scrub before it is written to
disk or broadcast as a worker.message event, replacing common credential
shapes with [REDACTED]. This is defense in depth, not a guarantee — the
permission layer (§4.7) is the primary control.
The rule set is a fixed, OnceLock-compiled list of regexes plus one
entropy-gated pass. There are deliberately no external dependencies
(no secret-scanning crate): everything is regex + a hand-rolled Shannon
entropy helper. The design goal is high recall on real credential shapes
while keeping false positives low enough that ordinary prose, git SHAs,
UUIDs, and placeholder tokens survive untouched.
§Rule ordering
Rules run in list order and the output of each feeds the next, so the most specific patterns must run first:
- PEM private-key blocks (multi-line) — removed whole before anything inside them can match a narrower rule.
- GCP service-account
private_keyJSON — the escaped PEM body that lives on a single JSON line. - Vendor-specific fixed-prefix tokens (Anthropic, OpenAI incl.
sk-proj-, GoogleAIza, Stripe, npm, GitHub, AWS, Slack, JWT). These have unmistakable shapes, so they run before any generic rule. - Credential headers —
Authorization: Bearer/Basic. Scheme word kept, credential redacted. - Connection-string passwords —
scheme://user:PASSWORD@host. Only the password segment is redacted; user and host stay for diagnosis. - Generic assignment catch-all —
key/secret/token/password = value. Runs late so a vendor rule gets first crack at the value. Implemented as an allowlist-aware closure pass (not a static replacement) so placeholder tokens, UUIDs, and git SHAs assigned to secret-ish names survive. - Entropy-gated bare token — a high-entropy base64/hex blob that sits
next to a broader secret-ish key name (
access_token,client_secret,auth, …) not covered by rule 6. This is the only rule that reasons about the content of the value, and it is gated behind both a key-name context match and the allowlist plus a 4.0 bits/char entropy floor, so random-looking prose, git SHAs, and UUIDs are never touched.
Rules 1–5 are static OnceLock regex replacements; rules 6–7 are
OnceLock-compiled regexes applied through closures so they can consult the
allowlist and (for rule 7) entropy. Every pass is deterministic.
truncate_chars cuts long content on a char boundary so multibyte
text can never panic the engine or produce invalid UTF-8.
Structs§
- Secret
Finding - A secret detector hit. Never carries the secret value itself.
- Secret
Scan - Result of scanning and redacting a text payload.
Constants§
- SECRET_
ALLOWLIST_ PATH - Tracked repository file containing one waived secret fingerprint per line.
Functions§
- filter_
allowed - format_
findings - read_
allowlist_ text - scan_
paths - Scan file contents about to be committed by the engine.
- scan_
text - Find secrets in
text. - scan_
text_ at - Find secrets in
text, usinglocationonly for diagnostics. - scan_
unified_ diff - Scan only added lines in a unified git diff.
- scrub
- scrub_
and_ truncate scrubthentruncate_chars— scrubbing happens first so truncation can never split a secret into an unrecognizable (and unredacted) prefix.- scrub_
json_ value - Redact every string leaf in a JSON value. Findings carry JSON-pointer-ish
locations rooted at
location. - scrub_
with_ findings - Scan and redact anything that looks like a credential.
- truncate_
chars - Truncate to at most
maxcharacters (not bytes), appending… [truncated]when anything was cut. Always cuts on acharboundary, so multibyte input can never split.