Skip to main content

Module secret_gate

khive_runtime

Module secret_gate 

Source
Expand description

Write-time secret detection gate (issue #76).

Scans caller-supplied content strings before any storage write. A match causes a hard RuntimeError::SecretDetected that names the detector and carries a masked excerpt — it never echoes the full candidate back.

Scope: credentials only — API keys, tokens, private keys, passwords, and connection strings with embedded credentials. General PII such as email addresses, phone numbers, and company names is intentionally NOT blocked; those are normal knowledge-graph content.

Detection is layered, cheap-first:

  1. Known-prefix / known-shape patterns — AWS AKIA/ASIA, GitHub tokens, OpenAI sk-proj-, Anthropic sk-ant-, Stripe live keys, Fly.io tokens, Vercel secrets, Slack xox*, JWT triples, PEM private-key headers, Age secret keys, URL userinfo (scheme://user:pass@). Bare sk- is also checked but only when NOT followed by a known safe word boundary (e.g. sk-learn, sk-image).
  2. High-entropy token heuristic — base64/hex/base64url runs ≥ 24 chars near a trigger word (key, secret, password, credential, bearer, auth, apikey, api_key, access_key, private_key). The word token alone is NOT a trigger to avoid blocking tokenizer_*, token_count, etc.

Allowlist (false-positive suppression):

  • Pure hex strings (sha256, git SHA) — passed unconditionally.
  • UUID canonical form (xxxxxxxx-xxxx-…) — passed.
  • Base64/base64url content hashes with an explicit sha<N>- prefix (SRI hashes, npm lockfile integrity) — passed when not preceded by a known-vendor prefix. Bare base64 tokens without the sha<N>- prefix are NOT passed.
  • Strings that are entirely ASCII punctuation/whitespace (e.g. code) — not subject to the entropy heuristic, only the literal-prefix checks apply.
  • Non-ASCII characters (CJK prose, accented text, emoji) act as token delimiters for the entropy heuristic: only maximal ASCII runs are entropy-checked. Real base64/hex/base64url credentials are ASCII, and shannon_entropy runs over UTF-8 bytes — multibyte codepoints inflate the byte-wise entropy and false-positive on natural-language non-Latin content. Treating non-ASCII as a delimiter (rather than skipping any whitespace token that merely contains it) keeps CJK prose unflagged while still catching an ASCII credential glued to CJK text/punctuation/fullwidth whitespace. The literal-prefix checks (Layer 1) treat any non-ASCII-alphanumeric char (CJK, accented text, emoji) as a token boundary, so a known-prefix secret is caught whether the adjacent non-ASCII sits before the prefix (数据AKIA…) or after it (AKIA…数据).

Structs§

SecretMatch
Returned when a write would store credential-looking content.

Functions§

check
Hard-block content from being written.
check_json
Recursively scan a JSON value for credential-shaped strings.
check_tags
Scan a string-tagged slice (entity/note tags).
mask_secrets
Redact every detected secret span in text, replacing each with ***MASKED***.