# Results
Armorer Guard is built for the hot path of AI-agent runtimes: scan locally,
return structured JSON, and decide before untrusted text becomes context or
model output becomes action.
## Headline Snapshot
| Native classifier latency | 0.0247 ms avg |
| Classifier macro F1 | 0.9833 |
| Classifier micro F1 | 0.9819 |
| Classifier micro recall | 1.0000 |
| Classifier exact match | 0.9724 |
| Classifier validation rows | 1,411 |
These numbers describe the exported Rust-native semantic classifier embedded in
the runtime. End-to-end scanner latency also includes redaction, policy checks,
normalization, and JSON IO.
## Promptfoo Red-Team Snapshot
Latest local Promptfoo-derived red-team hard split:
| Cases | 146 |
| Passed | 93 |
| Failed | 53 |
| Accuracy | 0.9178 |
| Precision | 0.9429 |
| Recall | 0.7674 |
| F1 | 0.8462 |
| Avg end-to-end duration | 14.86 ms |
| p50 end-to-end duration | 12.25 ms |
| p95 end-to-end duration | 23.85 ms |
This suite is intentionally broader than prompt injection only. It includes
agentic, coding-agent, data-exfiltration, dangerous-tool-call, system-prompt,
and application-safety categories. Some categories are outside Armorer Guard's
current enforcement target, so this is a visibility benchmark, not a polished
marketing-only score.
## Hard Agent-Boundary Snapshot
Latest local hard integration split:
| Cases | 5,926 |
| Passed | 4,050 |
| Failed | 1,876 |
| Accuracy | 0.6912 |
| Precision | 0.9326 |
| Recall | 0.5201 |
| F1 | 0.6678 |
| Avg end-to-end duration | 9.25 ms |
| p95 end-to-end duration | 10.74 ms |
The hard split mixes third-party prompt-injection examples, benign near misses,
tool-call contexts, and policy surfaces. It is useful precisely because it still
shows misses: Armorer Guard should improve against hard attacks without turning
benign developer or support workflows into false positives.
## What It Catches
Representative lanes exposed by the runtime:
| `credential_lane` | API keys, bot tokens, passwords, generic secret assignments |
| `semantic_lane` | prompt injection, exfiltration, system prompt extraction, safety bypass |
| `policy_lane` | dangerous tool calls, outbound credential disclosure, action-stage escalation |
| `similarity_lane` | trainable development exemplars owned by the project |
Example reasons returned by the scanner:
```text
detected:credential
semantic:prompt_injection
semantic:data_exfiltration
semantic:system_prompt_extraction
semantic:sensitive_data_request
semantic:destructive_command
policy:dangerous_tool_call
policy:credential_disclosure
```
## Why Local Rust
Armorer Guard does not call a hosted model in the scanning path. The classifier
coefficients are exported into `src/semantic_classifier_native.tsv` and loaded
by the Rust runtime. That gives agent builders a predictable pre-tool-call gate:
no network hop, no prompt leakage to a judge model, and machine-readable reasons
that can be enforced by a policy engine.
The Python package is intentionally thin. It shells out to the same Rust binary
so Python users get the same verdicts as Rust and CLI users.
## Reproduce
Build and test the scanner:
```bash
cargo build --release
cargo test
python3 -m pytest -q
```
Smoke-bench the local CLI path:
```bash
hyperfine \
--warmup 10 \
Use `inspect-json` for policy and tool-call context:
```bash
printf '%s' '{
"text": "{\"command\":\"rm -rf /\"}",
"context": {
"eval_surface": "tool_call_args",
"trace_stage": "action",
"tool_name": "Bash"
}