Skip to main content

Crate velociredactor

Crate velociredactor 

Source
Expand description

velociredactor logo

§velociredactor

velociredactor is a Rust library for redacting secrets and personal data from text and structured files.

Each redacted value becomes a token named REDACTION-N. N numbers distinct secrets in order of first appearance, and equal values share a number. Exact values and regular expressions can allow confirmed false positives.

Structured format support preserves keys and formatting while replacing values. Comment scanning is configurable. Raw mode treats the complete input as plain text.

The CLI README covers the command-line program, installation, and command reference.

§Quick start

Add the crate to a project:

[dependencies]
velociredactor = "0.1"

The default builder includes the built-in configuration:

use velociredactor::{Allow, FormatHint, Redactor};

let redactor = Redactor::builder().build();
let input = br#"{"db_password": "hunter2", "note": "hello"}"#;

let redaction = redactor.redact(input, FormatHint::Name("json")).unwrap();
assert_eq!(redaction.findings()[0].id, 1);
assert_eq!(redaction.findings()[0].token(), "REDACTION-1");

let output = redaction.render(&Allow::none()).unwrap();
assert_eq!(
    output,
    br#"{"db_password": "REDACTION-1", "note": "hello"}"#
);

// Render the same findings again while allowing that value through.
assert_eq!(redaction.render(&Allow::values(["hunter2"])).unwrap(), input);

Redactor::redact separates detection from rendering. A Redaction can render the same findings repeatedly with different Allow lists. Redactor::redact_str provides a compact plain-text API that replaces every finding.

§Configuration

Configuration defines the values to scan, documentation placeholders, active detectors, recognized formats, and allowed findings.

config::Config represents the complete configuration. Config::builtin returns the copy compiled into the crate. Config::builtin_source returns its commented YAML source. Config::apply applies a configuration to a RedactorBuilder.

A configuration loaded from a file replaces the built-in configuration completely. Start a custom configuration from Config::builtin_source.

RedactorBuilder::new creates an empty builder. Redactor::builder creates a builder with the built-in detectors, formats, and policy.

§Formats

Default features provide structured handling for:

  • JSON and JSON Lines
  • YAML
  • TOML
  • XML
  • HCL
  • INI and dotenv files
  • Java properties
  • CSV, TSV, and pipe-separated values
  • binary property lists
  • plain text

FormatHint chooses a format by name, path, content, or raw text. An inferred structured format that fails to parse falls back to plain text and records a warning. An explicitly named format returns its parse error.

§Cargo features

The default feature set enables parallel detection and every bundled structured format.

  • parallel uses Rayon for detector execution.
  • json, yaml, toml, xml, hcl, ini, dotenv, properties, csv, and plist enable their corresponding formats.
  • privacy-filter enables the OpenAI Privacy Filter model detector.
  • privacy-filter-cuda enables NVIDIA CUDA execution.
  • privacy-filter-accelerate enables Apple Accelerate.
  • privacy-filter-openblas enables a system OpenBLAS installation.

Choose one BLAS backend feature for a build. The privacy-filter model is downloaded separately.

§Extending

Implement detect::Detector to add detection logic and register it with RedactorBuilder::detector. Bundled detectors such as detect::BETTERLEAKS_RULESET and detect::EmailDetector use the same interface. A detector that needs every document value together uses detect::Detector::document_scope.

Implement format::Format to add a file format and register it with RedactorBuilder::format. The format::LeafVisitor interface communicates values and replacements while the format owns parsing and serialization.

Implement policy::LeafPolicy to control which structured values are scanned and which objects provide credential context.

The repository includes a complete custom detector and format in examples/custom.rs.

§License

velociredactor is available under the MIT License.

Modules§

agent
Files an AI coding agent must read through velociredactor.
config
The configuration: everything velociredactor knows, as data.
detect
Secret and PII detectors.
format
Input formats.
policy
Rules for which values in structured data are scanned at all.

Structs§

Allow
Secrets to leave in place, identified by exact value or by a pattern the whole value matches.
Finding
One redacted secret. Every occurrence of the same text shares a finding.
FormatError
A format failed to parse or re-serialize its input.
Glob
A glob pattern.
Redaction
The result of scanning an input: identified secrets, ready to render.
Redactor
Finds and redacts secrets. Build one with Redactor::builder.
RedactorBuilder
Configures a Redactor.

Enums§

Error
Errors returned while building a Redactor or redacting input.
FormatHint
How to choose the format of an input.

Constants§

TOKEN_PREFIX
Every replacement token starts with this.

Functions§

find_tokens
Byte ranges of the replacement tokens in s.
is_redaction_token
Whether s is exactly a replacement token.
token
The replacement text for a secret: REDACTION-<n>, where n is the 1-based id of the distinct value in this document.