assay-core 2.8.0

High-performance evaluation framework for LLM agents (Core)
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155

# Assay

[![Crates.io](https://img.shields.io/crates/v/assay-cli.svg)](https://crates.io/crates/assay-cli)
[![CI](https://github.com/Rul1an/assay/actions/workflows/ci.yml/badge.svg)](https://github.com/Rul1an/assay/actions/workflows/ci.yml)
[![License](https://img.shields.io/crates/l/assay-core.svg)](https://github.com/Rul1an/assay/blob/main/LICENSE)

**Runtime security & linting for MCP servers.**
Finds vulnerabilities in your Model Context Protocol configuration and fixes them automatically.

## Install

### Script (Recommended)
```bash
curl -fsSL https://getassay.dev/install.sh | sh
```

### Cargo
```bash
cargo install assay-cli
```

## Quick Verify

Validate the release integrity in a secure Docker environment:

```bash
curl -o verify.sh https://raw.githubusercontent.com/Rul1an/assay/main/scripts/verify_lsm_docker.sh
chmod +x verify.sh
./verify.sh --release-tag v2.2.2
```

## Learning Mode

Automatically generate least-privilege policies from runtime activity.

### Single-Run Analysis
Generate a policy from a single trace file:
```bash
assay generate -i trace.jsonl --heuristics
```

### Multi-Run Stability (Profile)
Accumulate observations over multiple runs (CI pipelines, staging) to distinguish stable behavior from noise:

```bash
# Initialize a new profile
assay profile init --output profile.yaml --name my-app

# Update profile with run data
assay profile update --profile profile.yaml -i trace.jsonl --run-id ci-run-123

# Generate stable policy (gates out flaky behavior)
assay generate --profile profile.yaml --min-stability 0.8
```

## Configuration

Assay uses standard JSON Schema for policies. Generated by `assay init`.

`assay.yaml`:
```yaml
version: "2.0"
name: "mcp-default-gate"

# Global defaults
allow: ["*"]

# Explicit blocks
deny:
  - "exec*"
  - "shell*"

# Parametric constraints
constraints:
  - tool: "read_file"
    params:
      path:
        matches: "^/app/.*|^/data/.*"
```

See [Migration Guide](docs/guides/migration.md) if upgrading from v1.x.

## Documentation

Full documentation available at [getassay.dev](https://getassay.dev).

-   [Policy Syntax (v2.0)](docs/reference/policies.md)
-   [Runtime Enforcement](docs/architecture/runtime.md)
-   [Policy Packs](https://getassay.dev/docs/packs)
-   [CI Recipes](https://getassay.dev/docs/ci)
-   [Configuration Reference](https://getassay.dev/docs/config)

## Contributing

Pull requests are welcome. For major changes, please open an issue first to discuss what you would like to change.

```bash
cargo test --workspace
```

## CI: Build & Test (GitHub Actions)

Copy-paste this into `.github/workflows/ci.yml` to build the workspace (including binaries)
and run the crate test suites on Linux/macOS/Windows:

```yaml
# (see .github/workflows/ci.yml)
name: CI

on:
  push:
    branches: [ main ]
  pull_request:

jobs:
  test:
    name: Build + Test (${{ matrix.os }})
    runs-on: ${{ matrix.os }}
    strategy:
      fail-fast: false
      matrix:
        os: [ubuntu-latest, macos-latest, windows-latest]

    steps:
      - uses: actions/checkout@v4

      - name: Install Rust (stable)
      - uses: dtolnay/rust-toolchain@stable
        with:
          components: rustfmt, clippy

      - name: Rust cache
        uses: Swatinem/rust-cache@v2
        with:
          # cache all workspace crates
          workspaces: |
            . -> target

      # IMPORTANT: build binaries first so assert_cmd E2E tests can find them
      - name: Build workspace (binaries)
        run: cargo build --workspace

      - name: Test assay-core
        run: cargo test -p assay-core

      - name: Test assay-cli
        run: cargo test -p assay-cli

      - name: Test assay-mcp-server
        run: cargo test -p assay-mcp-server
```

## License

[MIT](LICENSE)