Expand description
12.2 step 4: redacting known secret shapes from text before it’s stored
(docs/design/12.2-onboard.md, section 5). The hooks’ transcript ingest
always runs it; mnemo thread ingest --redact turns it on for a direct
call.
It’s a best effort: it removes shapes it knows, each starting and ending
at a word boundary and with a minimum length taken from the vendor’s real
format, so prose isn’t redacted. Each match becomes [redacted: <kind>].
It works on any text: every position it cuts at is next to an ASCII
byte, so it never splits a character. The shapes:
- PEM private-key blocks, from the
BEGINline to theENDline. A marker with no partner (a block split across texts) takes the text up to the end, or from the start, only when key material is next to it (aProc-Typeheader, or 40 or more base64 characters on the lines beside it); a marker only mentioned in prose loses the marker alone; - AWS access key ids:
AKIAorASIAand exactly 16 of[A-Z0-9]; - GitHub tokens:
ghp_,gho_,ghu_,ghs_orghr_and at least 36 of[A-Za-z0-9], orgithub_pat_and at least 22 of[A-Za-z0-9_]; - Slack tokens:
xoxa-,xoxb-,xoxp-,xoxr-orxoxs-and at least 10 of[A-Za-z0-9-]; - API keys:
sk-(includingsk-ant-andsk-proj-) and at least 20 of[A-Za-z0-9_-];sk_live_orrk_live_and at least 20 of[A-Za-z0-9];AIzaand at least 35 of[A-Za-z0-9_-];glpat-and at least 20 of[A-Za-z0-9_-];npm_and at least 36 of[A-Za-z0-9];hf_and at least 30 of[A-Za-z0-9]; - JWTs: three base64url parts joined by dots, the first starting
eyJ, each at least 10 characters; - MNemo recovery codes (
MN1-…), checked with the recovery-code parser, with hyphens, spaces or line breaks between the groups; - the value in
NAME=value,NAME: valueor"name": "value":- when the name ends in
PASSWORDorPASSPHRASE(any case): any value of 8 or more characters, and an unquoted one runs to the end of the line; - when it ends in
TOKEN,SECRET,_KEY,-KEYorAPIKEY: a value of 8 or more characters that isn’t a plain word (it has a digit or a character outside[A-Za-z_.-], or it mixes upper and lower case and is 20 or more letters). Names whose last part before_KEYor-KEYiscache,sort,partition,primary,foreign,unique,idempotency,deduporroware database words, and stay; - a value starting with
$,<,{{or%{is a reference or a placeholder, and stays;
- when the name ends in
- the credential in
Authorization: <scheme> <credential>(8 or more characters), and afterBearer(20 or more); - the word after
--passphraseor--password(8 or more characters), and after--tokenor--keywhen it’s 16 or more of[A-Za-z0-9+/=_-]and isn’t a plain word (so a memory key likedesign.v2or a file path stays); - the password in a URL’s
scheme://user:password@.
Structs§
Constants§
- KINDS
- The kinds, in the order counts are reported.
Functions§
- add_
counts - Sum the counts of several redactions, in
KINDSorder. - secrets
- Replace every known secret shape in
textwith[redacted: <kind>].