MCPSEC: MCP Security Benchmark Framework

Version 1.0.0 | Apache-2.0 License

MCPSEC is an open, vendor-neutral security benchmark for evaluating MCP (Model Context Protocol) gateway security. It defines 10 formal security properties and 64 reproducible attack test cases across 12 attack classes, derived from real-world penetration testing of MCP deployments.

Why MCPSEC?

AI agents with tool access are a new attack surface. OWASP defined the ASVS for web apps — MCPSEC does the same for MCP gateways.

Most MCP gateways offer tool-level allowlists and nothing more. MCPSEC tests for what actually matters: injection evasion, encoded exfiltration, schema mutation, confused deputy attacks, audit tampering, and more.

Quick Start

# Build the benchmark harness
cargo build -p mcpsec

# Run against a gateway
cargo run -p mcpsec -- --target http://localhost:3000 --output results/my-gateway.json

# Run with markdown report
cargo run -p mcpsec -- --target http://localhost:3000 --format markdown

What It Tests

10 Security Properties (P1-P10)

See PROPERTIES.md for formal definitions.

12 Attack Classes (A1-A12)

Total: 64 test cases. See ATTACKS.md for full catalog.

Scoring

See SCORING.md for weights and methodology.

Gateway Interface

MCPSEC tests any gateway that exposes an HTTP evaluation endpoint. The harness sends crafted payloads and checks whether the gateway correctly identifies or blocks each attack.

POST /api/evaluate
Content-Type: application/json

{"tool":"bash","function":"exec","parameters":{"command":"..."}}

The gateway should return a JSON response with a verdict field indicating Allow, Deny, or equivalent.

Project Structure

mcpsec/
├── README.md              # This file
├── PROPERTIES.md          # 10 formal security properties
├── ATTACKS.md             # 12 attack classes, 64 test cases
├── METHODOLOGY.md         # How to run, how to score
├── SCORING.md             # Scoring rubric and tiers
├── Cargo.toml             # Standalone Rust crate
├── src/
│   ├── lib.rs             # Public API
│   ├── runner.rs          # HTTP client for gateway testing
│   ├── report.rs          # JSON/Markdown report generation
│   ├── scoring.rs         # Score calculation
│   └── attacks/           # 12 attack modules (a01-a12)
├── tests/
│   └── self_test.rs       # Validate harness logic
└── results/               # Reference benchmark results

Philosophy

1. Open and reproducible. Every test case is documented with exact payloads and pass/fail criteria. No black-box scoring.
2. Vendor-neutral. Any MCP gateway can be benchmarked. The harness tests observable behavior, not implementation details.
3. Derived from real attacks. Every test case corresponds to a real attack vector discovered through penetration testing of MCP deployments.
4. No security theater. We test what competitors miss: Unicode homoglyphs, multi-layer encoding, schema mutation, audit chain integrity — not just "does the allowlist work."

Contributing

MCPSEC is open to contributions. To propose a new attack class or security property:

1. Open an issue describing the attack vector
2. Include a proof-of-concept payload
3. Define clear pass/fail criteria
4. Reference relevant OWASP or CVE identifiers

License

Apache-2.0 — use it, fork it, benchmark your competitors.

The benchmark framework itself is Apache-2.0 to encourage adoption. The Vellaveto gateway is licensed under a three-tier model (MPL-2.0 / Apache-2.0 / BUSL-1.1) — see LICENSING.md.