assay-core 2.2.0

High-performance evaluation framework for LLM agents (Core)
Documentation

Assay

Crates.io CI License

Runtime security & linting for MCP servers. Finds vulnerabilities in your Model Context Protocol configuration and fixes them automatically.

Install

Script (Recommended)

curl -fsSL https://getassay.dev/install.sh | sh

Cargo

cargo install assay-cli

Quick Verify

Validate the release integrity in a secure Docker environment:

curl -o verify.sh https://raw.githubusercontent.com/Rul1an/assay/main/scripts/verify_lsm_docker.sh
chmod +x verify.sh
./verify.sh --release-tag v2.1.1

Quick Start

# 1. Scaffolding
assay init --pack default

# 2. Check for issues
assay validate

# 3. Autofix problems
assay fix --yes

Core Features

  • SOTA Inode Enforcement: Kernel-level TOCTOU prevention using open(O_PATH) + eBPF LSM.
  • Path Containment: strictly limits file access to allowed directories.
  • RCE Prevention: Blocks exec, shell, spawn via minimal overhead hooks.
  • Atomic Autofix: Safely repairs config/code with zero corruption risk.
  • Zero-Latency: Built in Rust. Microsecond-scale overhead.

Compatibility

Requirement Min Version Note
Linux Kernel 5.8+ CONFIG_BPF_LSM=y
BTF Support required CONFIG_DEBUG_INFO_BTF=y
Filesystem bpf, tracefs Automatically mounted

CI: Validate your MCP policy on every PR (GitHub Actions)

Copy this workflow to .github/workflows/assay-security.yml.

It does two things:

  1. Validates your policy file (syntax + JSON Schema compilation)
  2. Runs full assay validate and uploads SARIF to GitHub Code Scanning

Tip: pin a specific Assay version for reproducible CI results.

name: Assay MCP Security

on:
  push:
    paths:
      - "assay.yaml"
      - "policy.yaml"
      - "**/*.mcp.json"
      - ".github/workflows/assay-security.yml"
  pull_request:
    paths:
      - "assay.yaml"
      - "policy.yaml"
      - "**/*.mcp.json"
      - ".github/workflows/assay-security.yml"

jobs:
  assay:
    runs-on: ubuntu-latest
    permissions:
      contents: read
      security-events: write

    steps:
      - uses: actions/checkout@v4

      - name: Install Assay
        run: |
          curl -fsSL https://getassay.dev/install.sh | sh
          echo "$HOME/.local/bin" >> $GITHUB_PATH
          # Optional: pin version if supported by installer

      - name: Validate policy file (syntax + schema compile)
        run: |
          # Adjust to your policy path:
          assay policy validate --input policy.yaml --deny-deprecations

      - name: Assay validate (SARIF)
        run: |
          assay validate --format sarif --output results.sarif
        continue-on-error: true

      - name: Upload SARIF to GitHub Security
        uses: github/codeql-action/upload-sarif@v3
        if: always()
        with:
          sarif_file: results.sarif

      - name: Fail on issues (PR gating)
        run: |
          assay validate --format text

Recommended: pin Assay to a specific release in CI to keep behavior stable across time. When upgrading, run the workflow on a branch and review any new warnings/errors before merging.

Output Formats

  • Human (Default): ANSI-colored terminal output.
  • JSON (--format json): Strict schema for scripts/agents.
  • SARIF (--format sarif): Native GitHub Security tab integration.

Configuration

Generated by assay init. Edit assay.yaml to customize:

version: "1.0"
name: "mcp-default-gate"

# Global defaults
allow: ["*"]

# Explicit blocks
deny:
  - "exec*"
  - "shell*"

# Parametric constraints
constraints:
  - tool: "read_file"
    params:
      path:
        matches: "^/app/.*|^/data/.*"

Configuration (v2.0)

Assay uses standard JSON Schema for policies. To migrate from v1.x:

assay policy migrate --input v1-policy.yaml

See Migration Guide.

Documentation

Full documentation available at getassay.dev.

Contributing

Pull requests are welcome. For major changes, please open an issue first to discuss what you would like to change.

cargo test --workspace

CI: Build & Test (GitHub Actions)

Copy-paste this into .github/workflows/ci.yml to build the workspace (including binaries) and run the crate test suites on Linux/macOS/Windows:

# (see .github/workflows/ci.yml)
name: CI

on:
  push:
    branches: [ main ]
  pull_request:

jobs:
  test:
    name: Build + Test (${{ matrix.os }})
    runs-on: ${{ matrix.os }}
    strategy:
      fail-fast: false
      matrix:
        os: [ubuntu-latest, macos-latest, windows-latest]

    steps:
      - uses: actions/checkout@v4

      - name: Install Rust (stable)
      - uses: dtolnay/rust-toolchain@stable
        with:
          components: rustfmt, clippy

      - name: Rust cache
        uses: Swatinem/rust-cache@v2
        with:
          # cache all workspace crates
          workspaces: |
            . -> target

      # IMPORTANT: build binaries first so assert_cmd E2E tests can find them
      - name: Build workspace (binaries)
        run: cargo build --workspace

      - name: Test assay-core
        run: cargo test -p assay-core

      - name: Test assay-cli
        run: cargo test -p assay-cli

      - name: Test assay-mcp-server
        run: cargo test -p assay-mcp-server

License

MIT