guardrail-rs

A zero-Python LLM security proxy written in Rust.

guardrail-rs is a reverse-proxy that sits between your application and an LLM provider (OpenAI, Anthropic, Azure OpenAI, or any OpenAI-compatible endpoint). It inspects every chat-completion request before it leaves your infrastructure, blocking prompt injection attempts, redacting PII, and enforcing custom policy rules. The default regex + PII pipeline took 47 µs on an 8 KB request in our measurement (see Latency).

Installation

cargo install guardrail-cli

This builds and installs the guardrail binary from crates.io (Rust 1.88+ required). Every crate in this workspace — guardrail-core, guardrail-classifiers, guardrail-config, guardrail-proxy, and guardrail-cli — is published independently, so you can also depend on just guardrail-core to embed the pipeline in your own Rust application with no HTTP server at all.

Alternatives:

# Docker (build locally from the Dockerfile in this repo)
docker build -t guardrail-rs .
docker run -p 8080:8080 -v $(pwd)/guardrail.toml:/etc/guardrail/guardrail.toml guardrail-rs

# ...or the full local stack (guardrail + Ollama + Prometheus + Grafana):
docker compose up -d

# Build from source
git clone https://github.com/Mattral/guardrail-rs.git && cd guardrail-rs
cargo install just  # one-time
just build

Want to try it in a couple of minutes with zero local setup? See Try it in Colab below — it installs Rust, runs cargo install guardrail-cli against the real crates.io release, and walks through blocking a prompt injection and redacting PII live.

At a glance

flowchart LR
    A["Your app<br/>(OpenAI/Anthropic SDK)"] -->|"POST /v1/chat/completions"| B["guardrail-rs<br/>(drop-in proxy)"]
    B --> C{{"Pipeline<br/>(stages)"}}
    C -->|"allow / redact"| D["Upstream LLM<br/>OpenAI · Anthropic · Azure · …"]
    C -->|"block"| E["403 + JSON error<br/>(never leaves your network)"]

Why guardrail-rs?

• No Python, no PyTorch, no GPU required. A single static binary (~15 MB), built from the Rust source in this repository.
• Fast without ONNX. In the default configuration (regex + PII), the pipeline took 4.7 µs for a 512 B request and 47 µs for 8 KB in our measurement. Cost grows with input size, mostly from PII redaction. The optional ONNX classifiers cost tens to hundreds of milliseconds per request; see Latency.
• Drop-in. Point your existing OpenAI/Anthropic SDK's base_url at guardrail-rs — no application code changes required.
• Fails open by default. A stage that errors on a request doesn't take down your production traffic; switch to fail-closed with pipeline.on_error = "block" or per ONNX stage with on_error.
• Hot-reloadable configuration. Update rules and policies without dropping connections.
• Observable. Structured audit logs (never logging raw PII) and Prometheus metrics out of the box.

Quick start

Once installed (see Installation above):

# 1. Copy and edit the example configuration
cp guardrail.example.toml guardrail.toml
# edit guardrail.toml: set [upstream].url

# 2. Validate the configuration
guardrail validate --config guardrail.toml
# (from source: just validate / cargo run -p guardrail-cli -- validate --config guardrail.toml)

# 3. Run the proxy
guardrail run --config guardrail.toml
# (from source: just run / cargo run -p guardrail-cli -- run --config guardrail.toml)

Then point your application at http://localhost:8080 instead of https://api.openai.com:

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8080/v1",
    api_key="sk-...",  # forwarded to the real upstream unchanged
)

Environment variable overrides

Configuration can be overridden without editing guardrail.toml:

Hot reload (Unix)

Send SIGHUP to reload configuration without dropping connections:

pkill -HUP guardrail
# or: just reload

What gets checked

The onnx_injection and toxicity stages are experimental. They require building with --features onnx and providing ONNX model files (see docs/onnx-models.md). CI runs them against tiny fixture models, which checks the code path but not detection quality. The real models are only smoke-tested, manually. Everything else works with zero external dependencies.

CLI

guardrail run --config guardrail.toml       # start the proxy
guardrail validate --config guardrail.toml  # check config without starting
guardrail check "some text" --config guardrail.toml
                                             # run text through the pipeline
                                             # and print the decision as JSON

Configuration

See guardrail.example.toml for a fully-annotated reference configuration, and docs/ for detailed guides:

• docs/architecture.md — pipeline design and stage contract
• docs/configuration.md — full TOML schema reference
• docs/policy-rules.md — writing custom policy rules
• docs/deployment.md — Docker, Kubernetes, and bare-metal deployment
• docs/onnx-models.md — enabling semantic classifiers
• docs/threat-model.md — what guardrail-rs protects against (and what it doesn't)
• docs/stage-api.md — implementing custom pipeline stages
• docs/benchmarks.md — performance targets and how to run benchmarks

Examples

See examples/README.md for client examples (curl, Python, Node.js, Anthropic SDK) against a running proxy. To embed guardrail-rs as a library with no HTTP server at all:

cargo run --example minimal -p guardrail-cli

Try it in Colab

No local setup required. The notebook installs Rust, runs cargo install guardrail-cli against the real published crate on crates.io, starts the proxy against a small local mock upstream (no API key needed), and then sends three requests to show:

1. A clean request passing through untouched.
2. A prompt-injection attempt getting blocked with a 403 and the prompt_injection error code.
3. A message containing an email address getting redacted to [EMAIL] before it ever reaches the (mock) upstream.

First run takes a few minutes while Rust compiles the crate — that's the cost of testing the real crates.io artifact rather than a pre-built binary.

Related: GuardRail Studio (optional)

A separate control-plane app — dashboard, policy editor, audit log, metrics — that drives this proxy. It does not reimplement detection.

• Repo: https://github.com/Mattral/GuardRail-Studio
• Talks to a running guardrail process (/v1/chat/completions, /healthz, /metrics on the same port, guardrail.toml + SIGHUP)
• Not required to use guardrail-cli

Project layout

crates/
  guardrail-core         # Pipeline trait, request/decision types, policy engine
  guardrail-classifiers  # Regex injection scanner, PII redactor, ONNX classifiers
  guardrail-config       # TOML config schema, validation, hot-reload
  guardrail-proxy        # HTTP server, request forwarding, metrics, audit log
  guardrail-cli          # `guardrail` binary
  guardrail-test-suite   # End-to-end integration tests

Development

# Install just (task runner)
cargo install just

# Build
just build

# Test (requires cargo-nextest: cargo install cargo-nextest)
just test

# Lint
just lint

# Format
just fmt

# Full CI check locally
just ci

# Generate coverage report (requires cargo-tarpaulin)
just coverage

See justfile for all available recipes (just --list).

License

Licensed under either of Apache License, Version 2.0 or MIT license at your option.

Contributing

Contributions are welcome! Please see CONTRIBUTING.md for guidelines on submitting issues and pull requests, including how to add new prompt-injection rules or PII entity types.