assay-core 1.3.0

High-performance evaluation framework for LLM agents (Core)
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109

<h1 align="center">
  <br>
  <img src="docs/assets/logo.svg" alt="Assay Logo" width="200">
  <br>
  Assay
  <br>
</h1>

<h4 align="center">The CI/CD Standard for Agentic Systems</h4>

<p align="center">
  <a href="https://github.com/Rul1an/assay/actions/workflows/ci.yml">
    <img src="https://github.com/Rul1an/assay/actions/workflows/ci.yml/badge.svg" alt="CI Status">
  </a>
  <a href="https://crates.io/crates/assay">
    <img src="https://img.shields.io/crates/v/assay.svg" alt="Crates.io">
  </a>
  <a href="https://docs.assay.dev">
    <img src="https://img.shields.io/badge/docs-assay.dev-blue" alt="Documentation">
  </a>
</p>

---

**Assay** is the missing link between your AI Agent and Production. It enforces **Policy-as-Code** on Model Context Protocol (MCP) tool usage, ensuring your agents behave safely and deterministically.

**For Vibecoders**: Connect your agent output, run `assay validate`, and see if it breaks the rules.
**For Engineers**: High-performance Rust binary, rigid schema validation, and CI/CD integration.

## 🚀 Features

-   **Policy Engine**: Enforce schema strictness and sequence ordering (`search_before_escalate`).
-   **The Doctor**: Smart diagnostics (`assay doctor`) that fix typo'd tool names and config issues automatically.
-   **CI-Native**: Generates GitHub/GitLab workflows instantly (`assay init-ci`).
-   **Stateless Python SDK**: Validate traces directly in `pytest` without a database.

## ⚡ Quick Start

### 1. Instant Verification
Don't guess. Verify.

```bash
# Install (macOS/Linux)
curl -sSL https://assay.dev/install.sh | sh

# Generate a demo environment (policy + traces) and run checks
assay demo
```

### 2. CI/CD in 10 Seconds
Stop writing YAML manually.

```bash
# Generate a ready-to-merge GitHub Actions workflow
assay init-ci --provider github
```

### 3. The Clinic (Debugging)
Something wrong? Let the Doctor diagnose it.

```bash
# Analyzes your config, policies, and traces for common issues
assay doctor
```

## 🐍 Python SDK

Integrate directly into your `pytest` suite. No server required.

```bash
pip install assay-it
```

```python
from assay import validate

def test_agent_compliance(traces):
    """
    Validate agent traces against your defined policy.
    Raises strict errors on violations.
    """
    report = validate(
        policy_path="assay.yaml",
        traces=traces
    )

    assert report["passed"], f"Policy Violation: {report['violations']}"
```

> **Note**: Full docstrings and type hints included. Your IDE will love you.

## 🛠️ CLI Reference

| Command | Purpose |
| :--- | :--- |
| `assay validate` | Run stateless validation on trace files. |
| `assay doctor` | Diagnose config issues and fuzzy-match known errors. |
| `assay init-ci` | Generate CI workflow templates (GitHub/GitLab). |
| `assay run` | Execute and capture traces from an agent (Advanced). |

## 📚 Documentation

-   [**Getting Started**](https://docs.assay.dev/getting-started/)
-   [**Python Quickstart**](https://docs.assay.dev/python-quickstart/)
-   [**Configuration Schema**](https://docs.assay.dev/config/)

## License

MIT.