Assay

Policy-as-Code for AI Agents.

Runs offline. No telemetry. No vendor lock-in.

Assay validates AI agent behavior against policies. Record traces, generate policies, run deterministic CI gates, produce evidence bundles for audit. Works with any MCP-compatible agent.

Open Core: Engine + baseline packs are MIT/Apache-2.0. Compliance packs (EU AI Act, SOC2) are commercial. See ADR-016.

Try It Now

No install required. Opens a browser-based terminal with Assay pre-compiled and demo fixtures ready.

# In the Codespace terminal:
make demo        # See the full break & fix flow
make test        # Run a safe trace (PASS)
make fail        # Run an unsafe trace (FAIL)

Or install locally:

Install

cargo install assay-cli

Quickstart

From scratch

# Generate policy + config from project defaults
assay init --ci

# Run smoke tests (uses bundled traces, no API calls)
assay ci --config ci-eval.yaml --trace-file traces/ci.jsonl

From an existing trace

# Generate policy from recorded agent behavior
assay init --from-trace trace.jsonl

# Validate
assay validate --config eval.yaml --trace-file trace.jsonl

From an MCP Inspector session

# Import trace
assay import --format inspector session.json --out-trace traces/session.jsonl

# Run tests
assay run --config eval.yaml --trace-file traces/session.jsonl

Commands

Testing & Validation

Policy & Config

Evidence & Compliance

Runtime

Misc

CI Integration

GitHub Actions

- uses: Rul1an/assay/assay-action@v2

The action installs assay, runs your gate, uploads SARIF to the Security tab, and posts a PR comment with results.

# .github/workflows/assay.yml
name: Assay Gate
on: [push, pull_request]

permissions:
  contents: read
  pull-requests: write
  security-events: write

jobs:
  assay:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: Rul1an/assay/assay-action@v2

Or generate a workflow:

assay init --ci github   # writes .github/workflows/assay.yml
assay init --ci gitlab   # writes .gitlab-ci.yml

Manual CI

assay ci \
  --config eval.yaml \
  --trace-file traces/golden.jsonl \
  --sarif reports/sarif.json \
  --junit reports/junit.xml \
  --pr-comment reports/pr-comment.md \
  --replay-strict

Exit codes: 0 pass, 1 test failure, 2 config error, 3 infra error.

Configuration

Two files: a test config (eval.yaml) and a policy (policy.yaml).

eval.yaml — defines what to test:

version: 1
suite: "my_agent"
model: "trace"
tests:
  - id: "deploy_args"
    input:
      prompt: "deploy_staging"
    expected:
      type: args_valid
      schema:
        deploy_service:
          type: object
          required: [env]
          properties:
            env: { type: string, enum: [staging, prod] }

policy.yaml — defines what's allowed:

version: "1.0"
name: "my-policy"
allow: ["*"]
deny:
  - "exec"
  - "shell"
  - "bash"
constraints:
  - tool: "read_file"
    params:
      path:
        matches: "^/app/.*|^/data/.*"

Policy packs: assay init --pack default|hardened|dev

Evidence Bundles

Tamper-evident .tar.gz bundles containing manifest.json (SHA-256 hashes, Merkle root) and events.ndjson (CloudEvents format, content-addressed IDs).

assay evidence export --profile profile.yaml --out bundle.tar.gz
assay evidence verify bundle.tar.gz
assay evidence lint --pack eu-ai-act-baseline bundle.tar.gz
assay evidence diff baseline.tar.gz current.tar.gz

Python SDK

pip install assay

from assay import AssayClient

client = AssayClient("traces.jsonl")
client.record_trace(tool_call)

Pytest plugin:

@pytest.mark.assay(trace_file="test_traces.jsonl")
def test_agent():
    pass

Project Structure

crates/
  assay-cli/        CLI binary
  assay-core/       Eval engine, store, trace replay, report formatters
  assay-metrics/    Built-in metrics (args_valid, sequence_valid, regex_match, etc.)
  assay-evidence/   Evidence bundles, lint engine, diff, sanitize
  assay-mcp-server/ MCP proxy for runtime enforcement
  assay-sim/        Attack simulation harness
  assay-monitor/    eBPF/LSM runtime (Linux)
  assay-policy/     Policy compilation (kernel + userspace tiers)
  assay-registry/   Pack registry client (DSSE, OIDC, lockfile)
  assay-common/     Shared types
  assay-ebpf/       Kernel eBPF programs
assay-python-sdk/   Python SDK (PyO3 + pytest plugin)

Contributing

cargo test --workspace
cargo clippy --workspace --all-targets -- -D warnings

See CONTRIBUTING.md.

License

MIT