Silens Scan
Privacy-first Rust core for detecting, querying and safely transforming secrets and sensitive data.
Silens Scan is designed as a reusable engine rather than an application shell. Callers provide UTF-8 text and retain control of I/O, storage and presentation. The core performs detection, validation, deterministic normalization, reporting and share-safe transformations without storing matched secret values inside public findings.
The serial core is suitable for native applications and WASM/PWA integrations. Native callers can optionally enable Rayon-backed parallel scanning across independent inputs.
Capabilities
- Deterministic UTF-8 scanning
- Literal, prefix, suffix and regex rules
- Capture-aware regex projection
- Deterministic and contextual validators
- Selectable built-in detector catalog
- Immutable per-source
ScanReport - Ordered batch
ScanResults<K> - Lazy query/filter API with explicit sorting
- Aggregated
ScanSummary - Optional remediation guidance per finding
- Safe redaction
- Semantic template generation
- Deterministic keyed pseudonymization
- Deterministic keyed synthetic replacements
- Share-bundle construction
- Optional Serde support
- Optional Rayon parallel batch scanning
- Unicode-aware source locations
- Stable serial/parallel ordering
- Reusable cached built-in scanner
- No matched secret value stored in public findings
Install
[]
= "0.1"
Optional features are deliberately independent:
= { = "0.1", = ["serde", "parallel"] }
Feature Default Purpose
serde no Serialize and deserialize public data contracts
parallel no Scan independent inputs concurrently with Rayon
The default build keeps the core serial and does not pull Rayon or Serde.
Quick start
Scanning always uses a batch-oriented API. One source is simply a one-element batch.
use Scanner;
let scanner = default;
let results = scanner.scan;
println!;
for in results.findings
Caller-provided keys are preserved in the result. Each ScanEntry<K>
also retains the original source byte length, while Finding contains
metadata and coordinates rather than a copy of the matched secret.
Detection model
The built-in scanner combines multiple matching families behind one compiled execution pipeline:
UTF-8 source
│
▼
CompiledRuleSet
│
├── Aho-Corasick ── literal / prefix
├── regex engine ── pattern / capture
└── specialized ─── suffix
│
▼
candidate validation
│
├── deterministic validators
└── contextual validators
│
▼
deterministic normalization
│
▼
line / Unicode-column finalization
│
▼
ScanReport → ScanResults<K>
A single source is scanned serially. With parallel, independent
sources are distributed through Rayon while the same immutable scanner
is reused.
Built-in detection
The current built-in catalog covers detector families for:
- GitHub
- Stripe
- Cloudflare
- Slack
- Telegram
- JWT
- AWS
- Azure
- GCP
- password and passphrase fields
- sensitive hashes
- generic API keys, tokens and secrets
The canonical selectable built-in pack is exposed as
builtins::CURRENT.
Custom rules can be composed with the builder:
use ;
let scanner = builder
.rule
.rule
.build?;
# Ok::
Reports, queries and summaries
ScanResults<K> is the batch result. It exposes direct iteration as
well as a lazy query surface.
use ;
let scanner = default;
let results = scanner.scan;
let high_priority = results
.query
.minimum_severity
.sort;
for in high_priority.iter
let summary = results.summary;
println!;
Queries can filter by exact or minimum severity, exact or minimum confidence, exact rule identifier, and convenience predicates such as critical/high-priority/high-confidence. Sorting is explicit and happens after lazy filtering.
ScanSummary reports:
- sources and bytes scanned
- reports with and without findings
- total findings
- counts per severity
- clean/critical status helpers
Remediation
A finding may carry an optional Remediation. Remediation is
presentation-safe guidance attached to the rule, not an automatic
mutation of the source.
for in results.findings
This allows applications to present actionable guidance such as rotating a credential, replacing a private key, changing a password or removing a sensitive value without embedding application-specific copy into the scanner pipeline.
Transformations
The transform module turns a ScanReport plus the original source
into share-safe output.
Redaction
Redaction is the conservative transform. It replaces detected spans and safely merges overlaps.
use ;
let scanner = builder
.rule
.build?;
let source = "TOKEN=SECRET";
let results = scanner.scan;
let report = results.single_report.expect;
assert_eq!;
# Ok::
Use redact_with when a custom replacement marker is required.
Semantic templates
Templates preserve the semantic rule identity while discarding the matched value:
TOKEN=<SILENS:credential>
TemplateOptions can configure the namespace and deterministic per-rule
numbering. Template generation rejects ambiguous overlapping findings
instead of guessing which semantic placeholder should win.
Pseudonymization
pseudonymize replaces findings with deterministic keyed pseudonyms.
The same input value under the same key produces the same pseudonym,
allowing correlation without retaining the original value.
use PseudonymizationOptions;
let options = new;
The key is supplied by the caller and is not part of a public finding.
Synthetic replacements
synthesize creates deterministic keyed synthetic values. For supported
detector families it preserves useful structural characteristics while
intentionally producing validator-invalid output, so generated material
can be used safely in examples and shareable artifacts.
use SynthesisOptions;
let options = new;
Share bundles
ShareBundle applies one explicit transformation mode to a batch while
preserving source keys and a manifest containing the transformation
mode, summary and generation time.
Available modes are represented by ShareMode; the manifest exposes the
non-secret ShareModeKind.
Use share bundles when an application needs to package transformed sources and scan metadata together for export, support or collaboration workflows.
Canonical examples and golden corpus
The repository contains a canonical fixture corpus under:
examples/fixtures/
├── inputs/
└── outputs/
├── reports/
├── redacted/
├── templates/
├── pseudonymized/
└── synthesized/
These are not hand-written expected outputs.
examples/generate_fixtures.rs runs the real scanner and transformation
APIs to regenerate them:
cargo run --example generate_fixtures --features serde
Golden tests independently generate the same artifacts in memory and compare them with the committed outputs. This keeps documentation examples tied to actual crate behavior.
The shared corpus scanner intentionally exercises custom prefix, regex and literal rules plus remediation metadata.
Parallel scanning
Enable parallel to scan independent sources concurrently:
= { = "0.1", = ["parallel"] }
let results = scanner.parallel_scan;
Serial and parallel execution use the same per-source pipeline and preserve input order. Silens Scan does not split one source into chunks and does not create a private thread pool; Rayon uses the current pool.
Parallelism is therefore an application choice, not a semantic difference in scanning.
Serde
Enable serde when results need to cross an application boundary or be
persisted/serialized:
= { = "0.1", = ["serde"] }
let json = to_string_pretty?;
The canonical fixture generator uses this feature for JSON report artifacts.
Coordinate contract
Locations have an explicit mixed coordinate contract:
start: zero-based UTF-8 byte offset, inclusiveend: zero-based UTF-8 byte offset, exclusiveline: one-basedcolumn: one-based Unicode scalar position
Byte offsets are suitable for exact source slicing. Line/column values are presentation-oriented.
Silens Scan does not copy the matched source value into a public
Finding.
Privacy and trust boundary
Silens Scan is a core library. It does not decide where source data comes from or where transformed output is sent.
The intended boundary is:
application owns I/O
│
▼
&str input
│
▼
Silens Scan
│
├── metadata-only findings
└── explicit transformations
This makes the serial core suitable for local-first WASM/PWA and desktop use: applications can keep scanning and transformation local to the process/browser. Network access, remote repositories, uploads, authentication and persistence belong to the integrating application rather than this crate.
Performance
Criterion benchmarks below were measured with Rust 1.97.1. They are workload- and machine-specific engineering baselines, not portable performance guarantees.
Core scanning
Workload Median time Throughput
Built-ins, 1 KiB 20.25 µs 66.81 MiB/s Built-ins, 64 KiB 253.90 µs 247.64 MiB/s Built-ins, 1 MiB 3.759 ms 266.13 MiB/s 64 custom literal rules 48.09 µs 1.27 GiB/s 512 custom literal rules 48.93 µs 1.25 GiB/s 64 custom mixed rules 151.32 µs 413.04 MiB/s Full built-in pipeline 253.94 µs 246.12 MiB/s
Match density
For a 64 KiB input:
Density Findings Median time Throughput
None 0 211.97 µs 294.85 MiB/s Sparse 7 ~255 µs ~246--248 MiB/s Dense 1496 ~2.85 ms ~22 MiB/s
The dense fixture intentionally produces roughly one final finding every 43 bytes, making result production and validation a substantial part of the workload.
Parallel batch scanning
The benchmark scans 32 independent sources.
Source size Serial Parallel Approx. speedup
4 KiB × 32 1.027 ms 244.49 µs 4.20× 64 KiB × 32 8.161 ms 1.307 ms 6.24× 1 MiB × 32 121.16 ms 18.24 ms 6.64×
Parallel throughput reaches approximately:
- 563 MiB/s for 4 KiB sources
- 1.50 GiB/s for 64 KiB sources
- 1.71 GiB/s for 1 MiB sources
Build cost
Ruleset Median build time
Built-in pack, cold compile 49.69 ms 4 custom literal rules 15.58 µs 64 custom literal rules 60.95 µs 512 custom literal rules 299.24 µs
The default built-in scanner uses a shared compiled cache, so the cold built-in compile cost is paid once per process and reused afterward.
Performance policy
The 0.1.0 engine is performance-frozen unless a correctness issue is
found. Manual SIMD has deliberately not been introduced: current
workloads do not justify the extra architecture-specific complexity. It
can be reconsidered if real-world profiling exposes a concrete
bottleneck.
Testing and hardening
The release test surface includes:
- unit tests across rules, validators, reporting and transformations
- doctests for public APIs
- canonical end-to-end fixtures
- golden output verification
- serial/parallel equivalence
- deterministic transformation checks
- UTF-8 scalar-column checks
- CRLF preservation
- empty-input behavior
- dense repeated-finding stress
- large single-line input
- Serde round-trip coverage
The canonical corpus is intended to remain part of the public documentation as the crate evolves.
Scope
Silens Scan intentionally stops at the reusable scanning core.
Application-level concerns such as file selection, remote repository loading, authenticated workflows, uploads, UI, persistence and subscription features belong to consumers such as the Silens Scan web application and Silens Studio.
A standalone CLI is not required by the core architecture and is not
part of the 0.1.0 scope.
Release status
0.1.0 targets the first public crates.io release.
The core capability set is frozen. Remaining release work is documentation, example polish, package metadata and publication validation.
License
MIT