Skip to main content

Module redact

Module redact 

Source
Expand description

12.2 step 4: redacting known secret shapes from text before it’s stored (docs/design/12.2-onboard.md, section 5). The hooks’ transcript ingest always runs it; mnemo thread ingest --redact turns it on for a direct call.

It’s a best effort: it removes shapes it knows, each starting and ending at a word boundary and with a minimum length taken from the vendor’s real format, so prose isn’t redacted. Each match becomes [redacted: <kind>]. It works on any text: every position it cuts at is next to an ASCII byte, so it never splits a character. The shapes:

  • PEM private-key blocks, from the BEGIN line to the END line. A marker with no partner (a block split across texts) takes the text up to the end, or from the start, only when key material is next to it (a Proc-Type header, or 40 or more base64 characters on the lines beside it); a marker only mentioned in prose loses the marker alone;
  • AWS access key ids: AKIA or ASIA and exactly 16 of [A-Z0-9];
  • GitHub tokens: ghp_, gho_, ghu_, ghs_ or ghr_ and at least 36 of [A-Za-z0-9], or github_pat_ and at least 22 of [A-Za-z0-9_];
  • Slack tokens: xoxa-, xoxb-, xoxp-, xoxr- or xoxs- and at least 10 of [A-Za-z0-9-];
  • API keys: sk- (including sk-ant- and sk-proj-) and at least 20 of [A-Za-z0-9_-]; sk_live_ or rk_live_ and at least 20 of [A-Za-z0-9]; AIza and at least 35 of [A-Za-z0-9_-]; glpat- and at least 20 of [A-Za-z0-9_-]; npm_ and at least 36 of [A-Za-z0-9]; hf_ and at least 30 of [A-Za-z0-9];
  • JWTs: three base64url parts joined by dots, the first starting eyJ, each at least 10 characters;
  • MNemo recovery codes (MN1-…), checked with the recovery-code parser, with hyphens, spaces or line breaks between the groups;
  • the value in NAME=value, NAME: value or "name": "value":
    • when the name ends in PASSWORD or PASSPHRASE (any case): any value of 8 or more characters, and an unquoted one runs to the end of the line;
    • when it ends in TOKEN, SECRET, _KEY, -KEY or APIKEY: a value of 8 or more characters that isn’t a plain word (it has a digit or a character outside [A-Za-z_.-], or it mixes upper and lower case and is 20 or more letters). Names whose last part before _KEY or -KEY is cache, sort, partition, primary, foreign, unique, idempotency, dedup or row are database words, and stay;
    • a value starting with $, <, {{ or %{ is a reference or a placeholder, and stays;
  • the credential in Authorization: <scheme> <credential> (8 or more characters), and after Bearer (20 or more);
  • the word after --passphrase or --password (8 or more characters), and after --token or --key when it’s 16 or more of [A-Za-z0-9+/=_-] and isn’t a plain word (so a memory key like design.v2 or a file path stays);
  • the password in a URL’s scheme://user:password@.

Structs§

Redacted
What secrets found and removed.

Constants§

KINDS
The kinds, in the order counts are reported.

Functions§

add_counts
Sum the counts of several redactions, in KINDS order.
secrets
Replace every known secret shape in text with [redacted: <kind>].