# forbidden-strings
Linear-time deny-list scanner for Git repos,
built on the in-house `forbidden-regex` engine
(`package/rust-module/forbidden-regex`).
A native Rust binary with a sub-commit-budget
startup,
it scans working-tree files line by line against a deny list of literals and
restricted-dialect regexes and reports each match as an opaque,
redacted finding.
Rules split into a baseline embedded in the binary
(`data/builtin-rules.txt`,
activated by `--builtin-rules`),
a committed shared appendix
(`forbidden-strings.append.txt`),
a per-repo gitignored appendix
(`forbidden-strings.append.local.txt`),
or a CI-only secret (`FORBIDDEN_STRINGS_LIST`).
The matched substring,
the surrounding line,
and the rule pattern are never printed in
failure output,
so a rule body that would itself leak if committed (a customer name,
an
unreleased project codename,
a pre-disclosure partner ID) can live as an appendix or CI
secret without exposure on public CI logs.
## What's different
- **Native binary startup.
** Rust with `lto = true`,
`codegen-units = 1`,
`opt-level = 3`,
`panic = "unwind"`,
`overflow-checks = true`,
`strip = true`.
No Node startup and no WASM init.
Explicit regex rules compile once per distinct content;
later invocations reload compact exact-literal groups plus regex-only engine bytes and rebuild one Aho-Corasick
matcher,
which meets the measured sub-100 ms pre-commit budget.
- **Linear-time matching.
** The engine is derivative and product based with no
backtracking,
so no rule combination can exhibit catastrophic-backtracking behaviour.
A set-level SIMD prefilter lets clean lines skip per-rule work.
- **Set-algebra rules.
** Intersection `A & B` and complement `~(A)` are first-class in
the dialect,
so "match X but not Y" needs no lookaround.
PCRE-family engines
(gitleaks,
trufflehog,
secretlint,
plain RE2) cannot do this;
their workaround is
per-rule allowlists,
which scale badly.
- **Sensitive rules can live out-of-band.
** The committed baseline holds non-sensitive
rules;
the gitignored appendix and the CI-only `FORBIDDEN_STRINGS_LIST` secret hold
sensitive rules.
Failure output never prints the matched substring,
the surrounding
line,
or the rule pattern,
so a rule body itself can be a secret.
## When to pick something else
`forbidden-strings` deliberately omits features other scanners ship as core capabilities:
- **CEL-based post-match filtering** (entropy thresholds,
BPE token efficiency,
git-author
predicates,
file-path globs,
string allowlists).
No equivalent here.
- **Async HTTP validation**.
No way to call a provider API to confirm a detected secret
is live;
staleness review is on you.
- **Git history scanning**.
The walker enumerates working-tree files only.
No equivalent
of `gitleaks git` that scans every diff in every commit.
- **SARIF / JSON / CSV output**.
Hits go to stderr as plain text;
no machine-readable
format for GitHub code-scanning upload or CI dashboards.
- **Per-rule path scoping**.
Every rule runs against every non-skipped file;
the scanner
cannot apply rule X only to YAML files.
- **Per-rule allowlists**.
No way to say "rule X but skip when it matches in path Y".
- **No streaming or stdin input.
** Files only;
the walker enumerates from disk.
If you need any of those,
betterleaks or gitleaks is the right tool.
## Prerequisites
- **Rust toolchain**.
Install via mise:
`mise install rust`.
- **mise** itself,
since build commands are `mise run` tasks.
## Build
```sh
mise run //package/cli/forbidden-strings:build
```
The release binary lands at `package/cli/forbidden-strings/target/release/forbidden-strings`.
Root `cli-git.config.ts` gives that path to the bundled `security/forbidden-strings` policy;
nothing needs to be on `$PATH`.
## Setup
The scanner needs exactly one rules file at scan time.
How you produce it is up to you.
### Without file-enforcer (most consumers)
Put rules in a file named `forbidden-strings.local.txt` at the repo root
(named tail-format sections,
or legacy one-rule-per-line;
see "Rule file format"),
or
pass `--rules <PATH>` / set `FORBIDDEN_STRINGS_RULES=<PATH>` to point at any other path.
That is the whole setup.
For a zero-file start,
pass `--builtin-rules` to scan with the
embedded betterleaks-ported baseline (see "Built-in baseline" below).
Add the file to
`.gitignore` if the rules themselves are sensitive;
otherwise commit it.
The "Rule file
format" section describes the line syntax.
In CI,
materialise the file from a secret (see
"GitHub Actions" below) so the rule bodies never enter version control.
### With file-enforcer (this monorepo's workflow)
Inside the Monochromatic monorepo,
no rules file exists at the repository root
(see `doc/decision/gitignore-negations.md`).
The pieces:
- The betterleaks baseline ships inside the scanner binary
(`data/builtin-rules.txt`,
regenerated by the two-stage chain behind
`mise run //package/cli/forbidden-strings:generate:rules`:
the TS porter `src/mise.port-betterleaks.ts`,
then the `dialectport` bin in
`package/rust-module/forbidden-regex.bench/src/bin/`);
repo invocations activate
it with `--builtin-rules` (the cli-git policy sets `builtinRules: true`).
- `forbidden-strings.append.txt` is the committed shared appendix of non-sensitive
repo-wide rules.
- `forbidden-strings.append.local.txt` is the per-repo additions.
Gitignored,
free-form,
edited by hand.
Place sensitive literals (codenames,
customer names,
partner IDs) here.
- `.cache/forbidden-strings.rules.txt` is the authoritative runtime text consumed by the scanner.
File-enforcer concatenates the two appendixes into the gitignored `.cache/` scratch dir.
The generated root `mise.toml` `[env]` points `FORBIDDEN_STRINGS_RULES` at it
(absolute via `{{config_root}}`).
Do not edit directly.
- A compiled runtime artifact lives under the current user's native cache root,
partitioned by scanner version,
platform,
and SHA-256 of the exact runtime text.
It stores exact-literal groups directly and precompiled engine bytes only for explicit regex rules.
File-enforcer invokes `forbidden-strings compile-rules` after generating the text when the
release scanner exists.
A later scan repairs a missing or rejected artifact automatically.
Run `mise run file-enforcer` after editing either appendix to regenerate the runtime text
and eagerly compile its cache artifact.
The generator is `generateForbiddenStringsRules` in `file-enforcer.config.ts`.
During fresh setup before the release scanner exists,
file-enforcer reports that eager compilation was skipped;
the first later scan compiles and publishes the artifact.
A stale pre-0.4.0 release binary exists but does not understand `compile-rules`,
so file-enforcer fails instead of silently claiming eager compilation;
rebuild the scanner before rerunning file-enforcer.
If you
fork this scanner into a project that does not use file-enforcer,
drop the appendix split
and follow the single-file workflow above.
## Usage
```sh
# scan a specific file list (uses ./forbidden-strings.local.txt by default)
forbidden-strings path/to/file other/file
# scan every working-tree file (.gitignore respected; .git/.jj skipped)
forbidden-strings --all
```
The rules path is resolved in this order:
`--rules <PATH>` flag (highest),
then `FORBIDDEN_STRINGS_RULES` environment variable,
then `./forbidden-strings.local.txt` in the current working directory.
Every runtime-rules scan uses read-write caching by default with no opt-out.
`FORBIDDEN_STRINGS_CACHE_DIR` may set an absolute cache root.
Without the override,
the scanner uses `$XDG_CACHE_HOME` or `$HOME/.cache` on XDG-oriented Unix,
`$HOME/Library/Caches` on macOS,
and `%LOCALAPPDATA%` on Windows.
```sh
# explicit path
forbidden-strings --rules ./other-rules.txt --all
# via env var (CI-friendly: materialize from a secret, then run)
FORBIDDEN_STRINGS_RULES=./materialized.txt forbidden-strings --all
# eagerly compile one rules file without scanning
forbidden-strings compile-rules --rules ./other-rules.txt
# print version and exit
forbidden-strings --version # or -V
```
`compile-rules` requires an explicit text path,
derives the user-cache destination,
and prints nothing on success.
It reuses a valid artifact and otherwise compiles,
rechecks the source snapshot,
and atomically publishes owner-only cache data.
Cache misses and rejected artifacts fall back to authoritative text and emit one compact
JSON warning on stderr before findings.
A failed cache write emits another warning but does not invalidate the already compiled scan.
Warnings contain only closed reason and recovery tokens,
never source paths,
cache paths,
digests,
rule text,
or operating-system error text.
### Built-in baseline (`--builtin-rules`)
The binary embeds the betterleaks-ported baseline ruleset.
The text form
(`data/builtin-rules.txt`) is exported as the library constant
`forbidden_strings::BUILTIN_RULES`;
the scan path loads the baseline from a serialized
`RegexSet` precompiled at build time (compiling the full baseline at each startup is not
viable,
so the cost is paid once during the build).
It is pure opt-in:
without the flag the
scanner never reads it,
so existing invocations behave exactly as before the flag existed.
With `--builtin-rules`:
- The baseline is appended after the resolved rules file.
Baseline findings render
as `rule=<name>` (the upstream betterleaks id),
so they identify themselves
regardless of position.
Unnamed legacy rules fall back to numeric ids that do not
shift:
your rules keep ids `0..user_len` and the baseline takes `user_len..`.
A
runtime rule whose section name collides with a baseline name fails the load
closed (the finding token must stay unambiguous).
- When no rules file resolves at all
(no `--rules`,
no env var,
and no `./forbidden-strings.local.txt` in cwd),
the baseline alone is the ruleset;
passing the flag is itself the configuration.
- An explicitly named missing file
(`--rules <path>` or the env var pointing at a path that does not exist)
still exits 2:
silently scanning without your rules would be a false-clean result.
```sh
# zero-file quick start: scan the tree with the embedded baseline only
forbidden-strings --builtin-rules --all
# your rules plus the baseline
forbidden-strings --builtin-rules --rules ./rules.txt --all
```
`--all` and positional files are mutually exclusive in practice:
if both are passed,
the
walker output silently overwrites the positional list.
Use one or the other.
## Rule file format
Two formats,
autodetected per file on the first significant line;
one file never mixes
them.
### Tail format (named sections)
A rule file is what `tail --verbose -n +1` over per-rule files would produce:
a
`==> name <==` header line opens a section whose body is one rule.
The name (first
character `[a-z0-9]`,
then any of `[a-z0-9.-]`,
unique within the loaded file) is the
rule's identity:
findings render it as `rule=<name>`.
- A body with exactly one significant line (non-blank,
not `#`-leading) is classified
by the two line shapes below,
exactly as in the legacy format.
- A body with more than one significant line is one verbatim regex pattern handed to
the always-verbose engine as-is (blank lines and first-column `#` comments inside it
are the engine's own to consume).
- Any line whose trimmed form starts with `==>` without being exactly a strict header
fails the load closed with its line number (case typos,
indentation,
missing-space
arrows,
and genuine arrow content alike);
write genuine `==> ` content in a regex
body with the `[=]=> ` reshape.
- Empty sections,
duplicate names,
and significant content before the first header
are fail-closed load errors,
each reported by line number only.
### Legacy format (one rule per line)
A file whose first significant line is not a strict header parses line-by-line,
byte-for-byte as before;
such rules carry no name and findings fall back to
`rule=N` numeric ids.
Two line shapes (shared with single-line tail sections):
- A bare line is a **case-sensitive literal**.
It is escaped into the engine's verbose
dialect and matched as a substring.
A literal under eight bytes is word-boundary
gated at each end whose neighbouring byte is an ASCII word byte,
so a short code
like `ACR` matches the whole token `ACR` but no longer fires glued inside
`ACRYLIC`;
eight bytes and longer match anywhere,
and over-matching in that
direction is a ratified preference.
If a short literal must match glued substrings
too,
write it as a regex (`/ACR/`).
- A line of the shape `/PATTERN/FLAGS` is a **regex** in the `forbidden-regex` dialect.
The first `/` and the last `/` delimit the pattern.
`FLAGS` is a trailing run of
ASCII-lowercase letters;
if the trailing run is not all-lowercase,
the whole line is
treated as a literal instead (so `/foo/I` is a literal scan for the seven bytes `/foo/I`,
not a case-insensitive regex).
Empty and whitespace-only lines are ignored.
A line whose first non-whitespace byte is `#`
is a comment.
One leading UTF-8 BOM is stripped from the source.
An empty source (no
non-blank,
non-comment line) is a rule-file error.
### Flags policy
The engine is always in multiline and verbose mode,
so the only accepted flags are the
ones those two modes already imply:
- `m` (multiline) and `x` (verbose) are accepted as no-ops and dropped.
- **Any other flag letter is a hard,
fail-closed load error.
** Silently dropping an `i` or
an `s` would change match semantics (case folding,
dot-matches-newline),
so the loader
rejects the whole ruleset rather than weaken a rule.
Need one of those locally?
Restructure
the pattern (for case-insensitivity,
spell the alternatives:
`[Aa][Bb][Cc]`).
### Supported constructs
The dialect is a deliberately restricted subset (see
`package/rust-module/forbidden-regex/README.md` for the full engine spec):
- Literals and the escapes `\t`,
`\b` (word boundary),
backslash-escaped metacharacters,
and backslash-escaped whitespace.
- Character classes:
`[abc]`,
`[a-z]`,
`[a-zA-Z]`,
negated `[^...]`,
and the shorthands
`\d \w \s \D \W \S` (usable inside classes too).
- `.` matches any byte except a newline.
- Grouping and alternation:
`(?:a|b)`.
Groups are **non-capturing only**.
- Bounded repetition:
`a?`,
`a{3}`,
`a{3,6}`.
- Anchors:
`^`,
`$`,
`\b`.
The word set is ASCII `[A-Za-z0-9_]`;
`^` and `$` anchor at
line boundaries.
- Set algebra:
intersection `&`,
complement `~(...)`.
Matching is an unanchored search over a single line's raw bytes:
a pattern matches if it
matches any substring.
Because verbose mode is always on,
unescaped whitespace outside
character classes is ignored,
so a rule may be written across several physical characters
for readability;
to match a literal space use `\ `,
`\t`,
or `[ ]`.
### Set-algebra operators
Two top-level set operators that pure-PCRE engines lack:
- `A & B` (intersection):
matches strings matched by both `A` and `B`.
- `~(A)` (complement):
matches strings that do NOT match `A`.
Operators `&` and `|` take single-atom operands:
a literal,
a class,
`.`,
an anchor,
a
`(?:...)` group,
or a `~(...)`.
A concatenation or a quantified atom must be wrapped in
`(?:...)` to be an operand,
so there is no operator precedence to remember.
`~(...)` is
always parenthesized.
A pattern that can match the empty string is rejected (unanchored,
it
would match everything),
so `~(Y)` alone is rejected while `(?:X) & ~(Y)` with a concrete
`X` compiles.
Example:
ban any five-digit key except the all-zeros placeholder:
```text
/(?:key_[0-9]{5}) & ~(key_00000)/
```
### Rejected at compile time (fail-closed)
Anything outside the supported set is a hard compile error naming the offending rule's
opaque index,
and the whole load fails closed (a bad ruleset never degrades to a partial
scan):
`*`,
`+`,
unbounded `{n,}`,
`\xNN` byte escapes,
capturing `(`,
lookaround and
inline-flag groups (`(?` not followed by `:`),
backreferences,
unknown escapes,
unbalanced
brackets,
stacked quantifiers,
`{n,m}` with `n` greater than `m`,
repetition whose
expansion exceeds the engine's cap,
and any pattern that can match the empty string.
Load errors are redacted:
a compile error carries only the opaque rule index (0-based position in the
compiled set) and the engine's own static reason;
a tail-format structural error (near-header,
duplicate name,
empty section,
pre-header content) carries only a source line number;
rule text appears in neither.
The redacted error type is `LoadError` in `src/rule/frx/error.rs`.
## Output
For each violation:
```text
PATH:LINE rule=<token>
```
- `LINE` is the 1-based line number.
- The rule token is the rule's **section name** for tail-format rules (and the
betterleaks id for baseline rules),
or the 0-based numeric engine id for unnamed
legacy rules.
Unnamed runtime rules take ids `0..user_len`;
under
`--builtin-rules` the baseline's unnamed ids are offset past them.
The finding is
columnless:
the engine reports per-line rule indices,
not spans,
so no
`COL_START..COL_END` segment appears.
- One finding is emitted per `(line, rule)` pair.
- **The matched substring,
the line content,
and the rule pattern are never printed.
**
Only the path,
line number,
and rule identity token appear,
so a failing CI log never
becomes a leak surface.
Name sensitive rules with deliberately opaque section names
(the name is the only rule-derived text that surfaces);
a numeric legacy index is
looked up against the local rule file.
Two synthetic findings keep the scan fail-closed:
- **Read errors.
** A file that cannot be opened (broken symlink,
permission denied,
deleted
during scan) produces `PATH: read error: <reason>` on stderr and counts toward the
exit-1 total.
A secret-scanning gate must not pass silently on a file it could not inspect.
- **Engine errors.
** If the matcher panics on a file,
the `catch_unwind` boundary in
`scan_one_set` (`src/frx_scan.rs`) catches it and emits `PATH: engine error`,
again
counting toward exit 1 rather than aborting or exiting clean.
Hits go to **stderr**,
not stdout;
redirecting `2>/dev/null` silently loses the report.
Within a file,
findings are emitted in set order (runtime rules before the baseline),
then
by line;
across files,
ordering is rayon-scheduler-determined,
stable on a given input but
not alphabetic.
Callers that need deterministic cross-file reports should pipe the output
into `sort`.
Exit codes:
- `0`:
no violations.
- `1`:
one or more violations (real hits,
read errors,
or engine errors).
- `2`:
usage error or rule-file error.
## Security model
The redaction guarantee is what lets a rule body itself be a secret.
Two boundaries carry
it:
- **Load path.
** Rule compilation reports only `LoadError` (`src/rule/frx/error.rs`),
whose
every variant is an opaque index plus the engine's static reason;
no pattern bytes reach
a diagnostic.
The compiler builds through `RegexSet::new` / `RegexSet::from_bytes`,
neither of which logs the pattern.
- **Scan path.
** Findings are formatted as `PATH:LINE rule=N` in `src/frx_scan.rs`;
the
matched bytes and the line content are never included.
The fail-closed `catch_unwind`
boundary emits only `PATH: engine error`.
Keep sensitive rule bodies out of tracked files:
use the gitignored `forbidden-strings.append.local.txt` or the CI-only
`FORBIDDEN_STRINGS_LIST` secret,
never the committed baseline or appendix.
Treat compiled artifacts as sensitive as their source:
the runtime envelope retains exact bare-literal bytes and is confidential policy data,
not a redaction.
On Unix,
file-enforcer enforces mode `0600` on the local appendix and generated runtime text;
the scanner enforces private application directories and mode `0600` on artifacts.
In CI,
pipe secrets through `printenv` rather than interpolating them into a workflow command;
shell expansion can leak values even when log masking is enabled.
Hosted GitHub runners use their native account cache for the derived artifact.
A self-hosted or shared runner should set `FORBIDDEN_STRINGS_CACHE_DIR` to job-scoped storage
when its account cache survives jobs.
## Integration
### Local cli-git policy
Root `cli-git.config.ts` enables `security/forbidden-strings` at error severity.
The
PATH-shadowed cli-git wrapper evaluates selected would-be-committed bytes before commit,
landed commit bytes before automatic push,
and Git-native outgoing ranges before manual
push.
Native `--no-verify` skips Git hooks but does not skip this policy.
Run an explicit read-only check through the built shim with:
```sh
git cli-git check --policy security/forbidden-strings --all
```
The policy invokes the repository-built scanner directly.
Scanner infrastructure failures
remain distinct exit-2 engine failures;
findings exit `1`.
### GitHub Actions
`.github/workflows/forbidden-strings.yml` remains independent of cli-git trust and local
wrapper state.
It downloads the release matching the scanner crate version,
verifies the
archive's GitHub build-provenance attestation,
materializes the committed baseline plus
shared appendix and optional repository secret,
then invokes the scanner binary directly.
Pull-request and merge-queue jobs scan changed files relative to `origin/main`;
pushes to
`main` scan files changed across the push event's before and after commits.
The same precedence applies locally and in CI:
`--rules` > `FORBIDDEN_STRINGS_RULES` > `./forbidden-strings.local.txt`.
## Walker behaviour
- **`--all` semantics.
** Walks the working tree via `ignore::WalkBuilder` in `src/walk.rs`:
`.hidden(false)` (dotfiles like `.github/`,
`.npmrc` ARE scanned),
`.ignore(false)` (the
`.ignore` file is NOT consulted;
`.gitignore` stays enabled).
Files force-added past
`.gitignore` (`git add -f`) are recovered via an in-process `gix-index` read of
`.git/index`;
no git subprocess.
- **`.git/` and `.jj/` skipped.
** Internal VCS state is never scanned.
- **Symlinks NOT followed.
** `WalkBuilder`'s default `follow_links` is false;
symlinked
directories are not descended,
symlinked files surface as a read-error synthetic hit on a
broken target.
- **Non-UTF-8 paths silently dropped.
** Index entries that are not valid UTF-8 are excluded
from the walk.
- **Binary-file 8 KiB tail cap.
** Files whose first 8 KiB contains a NUL byte are scanned
only in the first 8 KiB.
The leading window always runs,
so secrets there fire;
the tail
past 8 KiB is skipped.
Constant `BIN_PROBE_SIZE` and `read_with_binary_check` in
`src/lib.rs`.
- **Self-skip set.
** During `--all`,
canonical paths are auto-skipped so rule bodies do not
self-match:
the materialised rules file (whatever `--rules` / env var / default resolves
to),
plus three generated-source paths:
- `package/cli/forbidden-strings/data/betterleaks-default-config.toml`
- `package/cli/forbidden-strings/data/builtin-rules.txt`
- `package/cli/forbidden-strings/src/port-betterleaks-relaxations.ts`
Skip is path-anchored via `std::fs::canonicalize`,
not basename-anchored,
so an
unrelated file named `forbidden-strings.local.txt` in a subdirectory is still scanned.
Paths that fail to canonicalize from the current cwd are silently dropped from the set.
Explicit positional arguments bypass the `--all` skip;
the scanner's own
`forbidden-strings.*.txt` config files at cwd are skipped in both modes
(`is_config_file_at_cwd` in `src/lib.rs`).
## Performance
The scanner is a native Rust binary tuned for a sub-commit-budget startup and linear-time
matching;
the release profile (`Cargo.toml`) sets `lto`,
`codegen-units = 1`,
`opt-level = 3`,
`panic = "unwind"`,
`overflow-checks = true`,
and `strip`.
`panic = "unwind"` and
`overflow-checks = true` are load-bearing for the fail-closed `catch_unwind` boundary,
not
speed:
the forbidden-regex engine documents that it expects the caller's unwind boundary,
and a wrapped overflow would otherwise let a corrupt rule fail open.
Full bench methodology and per-version regression history are in `PERF.md`.
Runtime cache verification measures first compilation separately from repeated artifact loads.
A valid artifact must preserve finding identity and matching behavior before its timing is accepted.
The cache directory has no automatic retention policy;
deleting its `forbidden-strings/` child is always safe because authoritative text regenerates it.
## Fuzzing
Coverage-guided fuzzing lives in its own package,
`package/fuzz/forbidden-strings`,
so a
scoped nightly toolchain does not force this published crate onto nightly.
The scanner
exposes a curated internal surface (`fuzz_api`,
behind the `fuzzing` Cargo feature) for the
targets to drive.
The teardown that removed the old engine also retired the gate,
shard,
and
routing targets that fuzzed it;
the surviving targets are retargeted onto the
`forbidden-regex` load and scan path.
See that package's README for prerequisites,
mise
commands,
the bounded-container wrapper,
and crash-reproduction guidance.
## Architecture
- **Two-form loader.
** `src/rule/frx` owns the rule-file format and preserves whether each rule was written as a bare literal or an
explicit regex.
Bare literals still escape into the verbose dialect for the public engine compiler;
runtime scanning instead retains their exact bytes.
`/PATTERN/FLAGS` lines and multiline tail sections remain restricted regex rules,
validated under their original global rule ids.
- **Hybrid runtime matcher.
** `src/runtime_matcher.rs` de-duplicates exact literals into one overlapping Aho-Corasick matcher and compiles only
explicit regex rules into `RegexSet`.
Both subset-local outputs map back to original ids,
sort,
and de-duplicate before finding attribution.
- **Content-addressed runtime cache.
** `src/runtime_cache/` hashes exact authoritative text,
selects a scanner-version and platform partition,
validates scanner-owned envelope metadata,
rebuilds the literal matcher from compact groups,
and decodes optional regex-only engine bytes.
Scan-time repair and `compile-rules` share the same atomic publisher.
- **Line-based batch scan.
** `src/frx_scan.rs` splits each file's bytes into lines once and hands the buffer plus line-start offsets through a
common matcher interface.
Runtime sets merge Aho-Corasick and regex-subset ids;
the built-in baseline retains `RegexSet::line_matches`.
Each set runs under a `catch_unwind` boundary so a matcher fault fails closed as a synthetic finding.
- **Build-time baseline precompilation.
** `build.rs` compiles `data/builtin-rules.txt` through the engine once at build time and serializes it
(`to_bytes`);
`lib.rs` embeds the blob with `include_bytes!` and the loader rebuilds it via the validating
`from_bytes`,
never recompiling.
This trusted embedded path remains separate from the mutable runtime cache envelope.
Cargo compiles build scripts' dependencies unoptimized by default,
so `Cargo.toml` sets `build-override` `opt-level = 3` for the release and dev profiles,
and the engine builds the baseline's rules on one worker thread per core
(see [the Cargo build-override write-up](../../../doc/troubleshooting/cargo-build-override-opt-level.md)).
- **Concurrent load and walk.
** Rule loading and `--all` file walking run concurrently via
`rayon::join` (they share no state);
files then fan out across the rayon thread pool for
the parallel scan.
- **`ignore` crate walker + in-process gix-index union.
** `--all` uses `ignore::WalkBuilder`
(honouring `.gitignore`,
`.git/info/exclude`,
and global excludes) and unions the result
with an in-process `gix_index::File` read of `.git/index` so `git add -f` files are still
discovered.
See `src/walk.rs`.
- **Bundled `data/betterleaks-default-config.toml`.
** Upstream-vendored provenance for the
betterleaks port;
the embedded baseline is derived from it,
and
`port-betterleaks-relaxations.ts` records the lossy translations applied during the port.