velociredactor 0.1.0

Redact secrets and PII from text and structured files, with stable numbered redactions
Documentation

velociredactor

velociredactor is a Rust library for redacting secrets and personal data from text and structured files.

Each redacted value becomes a token named REDACTION-N. N numbers distinct secrets in order of first appearance, and equal values share a number. Exact values and regular expressions can allow confirmed false positives.

Structured format support preserves keys and formatting while replacing values. Comment scanning is configurable. Raw mode treats the complete input as plain text.

The CLI README covers the command-line program, installation, and command reference.

Quick start

Add the crate to a project:

[dependencies]
velociredactor = "0.1"

The default builder includes the built-in configuration:

use velociredactor::{Allow, FormatHint, Redactor};

let redactor = Redactor::builder().build();
let input = br#"{"db_password": "hunter2", "note": "hello"}"#;

let redaction = redactor.redact(input, FormatHint::Name("json")).unwrap();
assert_eq!(redaction.findings()[0].id, 1);
assert_eq!(redaction.findings()[0].token(), "REDACTION-1");

let output = redaction.render(&Allow::none()).unwrap();
assert_eq!(
    output,
    br#"{"db_password": "REDACTION-1", "note": "hello"}"#
);

// Render the same findings again while allowing that value through.
assert_eq!(redaction.render(&Allow::values(["hunter2"])).unwrap(), input);

[Redactor::redact] separates detection from rendering. A [Redaction] can render the same findings repeatedly with different [Allow] lists. [Redactor::redact_str] provides a compact plain-text API that replaces every finding.

Configuration

Configuration defines the values to scan, documentation placeholders, active detectors, recognized formats, and allowed findings.

[config::Config] represents the complete configuration. Config::builtin returns the copy compiled into the crate. Config::builtin_source returns its commented YAML source. Config::apply applies a configuration to a [RedactorBuilder].

A configuration loaded from a file replaces the built-in configuration completely. Start a custom configuration from Config::builtin_source.

[RedactorBuilder::new] creates an empty builder. [Redactor::builder] creates a builder with the built-in detectors, formats, and policy.

Formats

Default features provide structured handling for:

  • JSON and JSON Lines
  • YAML
  • TOML
  • XML
  • HCL
  • INI and dotenv files
  • Java properties
  • CSV, TSV, and pipe-separated values
  • binary property lists
  • plain text

[FormatHint] chooses a format by name, path, content, or raw text. An inferred structured format that fails to parse falls back to plain text and records a warning. An explicitly named format returns its parse error.

Cargo features

The default feature set enables parallel detection and every bundled structured format.

  • parallel uses Rayon for detector execution.
  • json, yaml, toml, xml, hcl, ini, dotenv, properties, csv, and plist enable their corresponding formats.
  • privacy-filter enables the OpenAI Privacy Filter model detector.
  • privacy-filter-cuda enables NVIDIA CUDA execution.
  • privacy-filter-accelerate enables Apple Accelerate.
  • privacy-filter-openblas enables a system OpenBLAS installation.

Choose one BLAS backend feature for a build. The privacy-filter model is downloaded separately.

Extending

Implement [detect::Detector] to add detection logic and register it with [RedactorBuilder::detector]. Bundled detectors such as [detect::BETTERLEAKS_RULESET] and [detect::EmailDetector] use the same interface. A detector that needs every document value together uses [detect::Detector::document_scope].

Implement [format::Format] to add a file format and register it with [RedactorBuilder::format]. The [format::LeafVisitor] interface communicates values and replacements while the format owns parsing and serialization.

Implement [policy::LeafPolicy] to control which structured values are scanned and which objects provide credential context.

The repository includes a complete custom detector and format in examples/custom.rs.

License

velociredactor is available under the MIT License.