assay-core 2.6.0

High-performance evaluation framework for LLM agents (Core)
Documentation

Assay

Crates.io CI License

Runtime security & linting for MCP servers. Finds vulnerabilities in your Model Context Protocol configuration and fixes them automatically.

Install

Script (Recommended)

curl -fsSL https://getassay.dev/install.sh | sh

Cargo

cargo install assay-cli

Quick Verify

Validate the release integrity in a secure Docker environment:

curl -o verify.sh https://raw.githubusercontent.com/Rul1an/assay/main/scripts/verify_lsm_docker.sh
chmod +x verify.sh
./verify.sh --release-tag v2.2.2

Learning Mode

Automatically generate least-privilege policies from runtime activity.

Single-Run Analysis

Generate a policy from a single trace file:

assay generate -i trace.jsonl --heuristics

Multi-Run Stability (Profile)

Accumulate observations over multiple runs (CI pipelines, staging) to distinguish stable behavior from noise:

# Initialize a new profile
assay profile init --output profile.yaml --name my-app

# Update profile with run data
assay profile update --profile profile.yaml -i trace.jsonl --run-id ci-run-123

# Generate stable policy (gates out flaky behavior)
assay generate --profile profile.yaml --min-stability 0.8

Configuration

Assay uses standard JSON Schema for policies. Generated by assay init.

assay.yaml:

version: "2.0"
name: "mcp-default-gate"

# Global defaults
allow: ["*"]

# Explicit blocks
deny:
  - "exec*"
  - "shell*"

# Parametric constraints
constraints:
  - tool: "read_file"
    params:
      path:
        matches: "^/app/.*|^/data/.*"

See Migration Guide if upgrading from v1.x.

Documentation

Full documentation available at getassay.dev.

Contributing

Pull requests are welcome. For major changes, please open an issue first to discuss what you would like to change.

cargo test --workspace

CI: Build & Test (GitHub Actions)

Copy-paste this into .github/workflows/ci.yml to build the workspace (including binaries) and run the crate test suites on Linux/macOS/Windows:

# (see .github/workflows/ci.yml)
name: CI

on:
  push:
    branches: [ main ]
  pull_request:

jobs:
  test:
    name: Build + Test (${{ matrix.os }})
    runs-on: ${{ matrix.os }}
    strategy:
      fail-fast: false
      matrix:
        os: [ubuntu-latest, macos-latest, windows-latest]

    steps:
      - uses: actions/checkout@v4

      - name: Install Rust (stable)
      - uses: dtolnay/rust-toolchain@stable
        with:
          components: rustfmt, clippy

      - name: Rust cache
        uses: Swatinem/rust-cache@v2
        with:
          # cache all workspace crates
          workspaces: |
            . -> target

      # IMPORTANT: build binaries first so assert_cmd E2E tests can find them
      - name: Build workspace (binaries)
        run: cargo build --workspace

      - name: Test assay-core
        run: cargo test -p assay-core

      - name: Test assay-cli
        run: cargo test -p assay-cli

      - name: Test assay-mcp-server
        run: cargo test -p assay-mcp-server

License

MIT