---
title: Redaction rules
description: Every pattern rtb-redact applies, in the order it applies them, with thresholds, examples, and what each rule does not match.
---
# Redaction rules
`string` and `string_into` apply the same seven rules, in a fixed order, with no
configuration. This page lists each one: the pattern as it appears in
`src/lib.rs`, what it replaces, and (the part that usually matters more) what
it leaves alone.
Every example below is the literal output of `rtb_redact::string` at version
0.6.3.
## The fast path, and what it skips
Before any rule runs, `string` scans the input for an **anchor**. If none is
present it returns the input borrowed, without allocating and without running a
single pattern. An input is anchored if it contains any of:
- one of the characters `@`, `=`, `?`, `-`, `_`, `.`
- the literal `-----BEGIN `
- a case-insensitive `bearer `, `basic ` or `token `, **with a trailing
space**, not a tab or newline
- a run of 40 or more characters from `A-Z a-z 0-9 + / = _ -`
Almost every real log line clears that bar, which is the point: the check exists
to make clean strings cheap, not to be exact.
It does have teeth, though. A string whose only secret is made purely of letters
and digits, with no anchor character anywhere else in the line, is returned
untouched even though a rule below would have matched it:
```text
in : AKIAABCDEFGHIJKLMNOP out: AKIAABCDEFGHIJKLMNOP (unchanged)
in : key AKIAABCDEFGHIJKLMNOP here
out: key AKIAABCDEFGHIJKLMNOP here (unchanged)
in : aws_key=AKIAABCDEFGHIJKLMNOP
out: aws_key=[redacted] (the `_` and `=` anchor it)
```
The two credential formats this reaches are AWS access key IDs (`AKIA…`,
`ASIA…`) and Google API keys (`AIza…`), both of which are pure alphanumerics
shorter than 40 characters. Provider tokens that carry `-`, `_` or `.` in their
own shape (`sk-…`, `ghp_…`, `glpat-…`, `xox…-…`, `SG.…`) anchor themselves and
are unaffected.
Tab- and newline-separated auth schemes hit the same edge:
```text
in : Bearer<TAB>abcdef1234567890 out: unchanged
in : Bearer abcdef1234567890 out: Bearer [redacted]
```
## The order the rules run in
Each rule runs over the output of the one before it:
1. [URL userinfo](#url-userinfo)
2. [Authorization-style schemes](#authorization-style-schemes)
3. [Sensitive query parameters](#sensitive-query-parameters)
4. [PEM private-key blocks](#pem-private-key-blocks)
5. [Well-known provider prefixes](#well-known-provider-prefixes)
6. [JSON Web Tokens](#json-web-tokens)
7. [Long opaque runs](#long-opaque-runs)
The order is deliberate: high-confidence, narrowly-shaped rules run first so
that the broad length-based rule at the end only ever sees what is left. The
PEM rule runs ahead of the token rules for the same reason: a key body is a
long opaque run, and collapsing the whole block first keeps the output readable
instead of leaving a half-masked key.
The numbered comments in `src/lib.rs` (`// 1.`, `// 7.`, `// 4.` …) are the
original design document's numbering, not the execution order. The list above is
the execution order.
The output is **idempotent**: redacting an already-redacted string returns it
unchanged, because `[redacted]` contains no character any rule matches on.
## URL userinfo
```text
([a-zA-Z][a-zA-Z0-9+.-]*)://[^:\s/?#]+:[^@\s]+@
```
Replaces `user:password@` with `[redacted]@`, keeping the scheme, host and path.
```text
in : connect to postgres://app:hunter2@db.internal/mydb
out: connect to postgres://[redacted]@db.internal/mydb
```
**Any scheme matches:** `postgres://`, `redis://`, `amqp://`, not just HTTP.
The scheme's case is preserved as written.
Not matched:
- **Userinfo with no password.** `https://token@github.com/x/y.git` is left as
it is; the pattern requires the `user:password` pair.
- **An empty password.** `https://user:@host/` does not match either.
- **A password containing `@`.** The value runs only up to the first `@`, so
`https://u:p@ss@host/` becomes `https://[redacted]@ss@host/` and the tail of
the password survives.
## Authorization-style schemes
```text
(?i)\b(Bearer|Basic|Token)\s+[A-Za-z0-9_\-.+/=]+
```
Keeps the scheme word, replaces the credential:
```text
in : Authorization: Bearer ghp_abcdefghijklmnopqrstuvwxyz123456
out: Authorization: Bearer [redacted]
```
The scheme is re-emitted exactly as it was written (`bearer` stays lowercase),
and whatever whitespace separated it from the credential is normalised to a
single space.
Only these three scheme words are recognised. `Digest`, `ApiKey`, `Negotiate`
and `Signature` are not.
**This rule over-redacts English prose.** It cannot tell a scheme from the same
word used as a noun, so the word after one is destroyed:
```text
in : we offer a Basic tier out: we offer a Basic [redacted]
in : the Token bucket algorithm out: the Token [redacted] algorithm
```
If your log lines routinely contain those words as prose, expect this. Losing a
word is the deliberate trade against leaking a token. See
[Why it over-redacts](../explanation/why-shape-matching.md#why-it-would-rather-over-redact).
## Sensitive query parameters
```text
words: `apikey`, `api_key`, `api-key`, `access_token`, `refresh_token`, `token`,
`password`, `passwd`, `secret`, `signature`, `sig`, `auth`, `x-api-key`.
Two consequences of how loosely the name is anchored:
- **The `?` or `&` is optional**, so bare assignments are caught too:
`password=hunter2` and `PASSWORD=hunter2` in an environment dump both redact.
- **There is no left-hand word boundary**, so any name *ending* in a recognised
one matches: `csrf_token=`, `client_secret=` and `x-api-key=` all redact.
Names that are **not** in the list, and are therefore left in the clear:
| `key=` | `?key=AIza…` only redacts because the Google prefix rule catches the value, not because `key` is recognised |
| `authorization=` | `auth` must be followed immediately by `=` |
| `pwd=`, `credential=`, `session=`, `sessionid=` | no rule covers these |
**The value runs further than you may expect.** It ends at `&`, whitespace or
`#`, and at nothing else. A `;` or `,` separator does not stop it, so
neighbouring non-secret data is swallowed:
```text
in : ?password=hunter2;other=1 out: ?password=[redacted]
in : ?password=hunter2#frag out: ?password=[redacted]#frag
```
## PEM private-key blocks
```text
(?s)-----BEGIN [A-Z ]*PRIVATE KEY-----.*?-----END [A-Z ]*PRIVATE KEY-----
```
The whole block, header to footer, is replaced with a canonical three-line form:
```text
in : -----BEGIN RSA PRIVATE KEY-----
MIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQC7VJTUt9Us8cKj
-----END RSA PRIVATE KEY-----
out: -----BEGIN PRIVATE KEY-----
[redacted]
-----END PRIVATE KEY-----
```
**The key type is normalised away.** `RSA`, `EC` and `OPENSSH` all come out as
plain `PRIVATE KEY`, so the output does not tell you which kind of key was
present. Multiple blocks in one string are each replaced.
Not matched:
- **Anything that is not a private key.** `-----BEGIN CERTIFICATE-----` and
`-----BEGIN PUBLIC KEY-----` are not private-key blocks and this rule ignores
them. Their bodies may still be caught by the long-opaque rule if the base64
lines are 40 characters or more, but the `BEGIN`/`END` lines are re-emitted
verbatim, so a certificate comes back as a `CERTIFICATE` header wrapped around a
`[redacted]` body, not as a normalised `PRIVATE KEY` block.
- **Lowercase markers.** The pattern is case-sensitive; `-----begin rsa private
key-----` passes straight through.
- **A block with no `END` line.** A truncated key (the common case when a log
line is cut short) does not match this rule. Its body is usually still caught
by the long-opaque rule, but the `BEGIN` header stays.
## Well-known provider prefixes
```text
xox[baprs]-[A-Za-z0-9\-]+ SG\.[A-Za-z0-9_\-]{22,}\.[A-Za-z0-9_\-]{43,}
```
A match is replaced **whole**, prefix included, with `[redacted]`, and the output
does not tell you which provider the token belonged to.
**A match shorter than 20 characters in total is left alone.** That is what
keeps `sk-abc` in a sentence from being mangled:
```text
in : short sk-abc tail out: short sk-abc tail (6 chars)
in : sk-abcdefghijklmnopq out: [redacted] (20 chars)
```
There is no word boundary at the end and no whitespace requirement, so a token
inside a URL path or a quoted JSON value is still caught.
Formats **not** carried here: GitHub fine-grained PATs (`github_pat_…`), GitLab
runner and deploy tokens (`glrt-`, `gldt-`), Stripe (`sk_live_…`), Twilio
(`SK…`), and every internal or bespoke credential format. Some are long enough
to be caught by the long-opaque rule instead (which depends on the surrounding
punctuation, not on the token) and some are not caught at all.
## JSON Web Tokens
```text
eyJ[A-Za-z0-9_\-]+\.[A-Za-z0-9_\-]+\.[A-Za-z0-9_\-]+
```
Three base64url segments beginning `eyJ`, replaced whole with `[redacted]`,
**but only when the match is 100 characters or longer**. A shorter JWT-shaped
string is left exactly as it is:
```text
in : auth: eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwi…
out: auth: [redacted] (155 chars)
in : eyJhbGciOi.eyJzdWIi.Sflabc (26 chars)
out: eyJhbGciOi.eyJzdWIi.Sflabc (unchanged)
```
A real token with a signed payload clears 100 characters comfortably; an
unsigned or minimal one may not, and the dots in it stop the long-opaque rule
from catching it either.
Only the `eyJ` opening is recognised. That is base64 for `{"`, so it covers
tokens whose header is a JSON object, which is every conventional JWT. A JWE or
a token with a non-JSON header does not match.
## Long opaque runs
```text
- **Punctuation defeats it.** A 40-character token in parentheses, quotes, or
followed by a `;` or `,` is not whitespace-bounded and is not matched by this
rule. `Set-Cookie: sid=<40 chars>; Path=/` keeps its value in full.
- **Two long runs separated by a single space, only the first is redacted.** The
match consumes its trailing space, leaving the second run with no leading
boundary:
```text
in : <40 chars> <40 chars>
out: [redacted] <40 chars>
```
A two-character separator such as `\r\n` does not have this problem.
### Why 40, and what it costs
40 is low enough to catch most opaque session and API tokens and high enough to
clear a UUID. What sits either side of the line:
| MD5 digest | 32 | No |
| UUID with hyphens | 36 | No |
| **git SHA-1 / commit hash** | **40** | **Yes** |
| SHA-256 digest | 64 | Yes |
A bare commit hash in a log line is replaced with `[redacted]`. That is the
known cost of the threshold, and the most common false positive in practice, since
`git log` output and CI job logs are full of them. The reasoning behind
accepting it is in
[Why the threshold is 40](../explanation/why-shape-matching.md#why-the-threshold-is-40-characters-here-and-41-in-the-go-module).
Long file paths pay the same price when they contain no `.`:
`GET /api/v1/resources/subresources/items/list HTTP` becomes
`GET [redacted] HTTP`, because `/` is in the alphabet.
## Quick lookup: is this redacted?
| `postgres://app:hunter2@db/mydb` | `postgres://[redacted]@db/mydb` |
| `https://token@github.com/x/y` | unchanged |
| `Authorization: Bearer <token>` | `Authorization: Bearer [redacted]` |
| `?api_key=<value>` | `?api_key=[redacted]` |
| `?key=<value>` | unchanged unless the value itself matches a rule |
| `sk-ant-api03-…` (20+ chars) | `[redacted]` |
| `AKIA…` alone on a line | unchanged (no anchor) |
| JWT of 100+ chars | `[redacted]` |
| JWT under 100 chars | unchanged |
| git SHA-1, whitespace-bounded | `[redacted]` |
| UUID | unchanged |
| `"password": "hunter2"` (JSON) | unchanged (JSON uses `:`, not `=`) |
| `Cookie: session=abc123` | unchanged (`session` is not a recognised name) |
## Related
- [API reference](api.md): the functions that apply these rules.
- [What rtb-redact does not do](../explanation/what-rtb-redact-does-not-do.md):
the limits stated as limits, with the reasoning.
- [Cover a secret the rules miss](../how-to/cover-a-secret-the-rules-miss.md):
what to do when your credential is in the "not matched" column.