Crate traitclaw_eval 

Evaluation framework for the TraitClaw AI agent framework.

Provides EvalSuite, TestCase, and Metric abstractions for measuring agent quality. Includes built-in metrics for relevancy and keyword matching.

Quick Start

use traitclaw_eval::{EvalSuite, TestCase, KeywordMetric, Metric};

let suite = EvalSuite::new("quality_tests")
    .add_case(TestCase::new("greeting", "Say hello")
        .expect_contains("hello"));

assert_eq!(suite.name(), "quality_tests");
assert_eq!(suite.cases().len(), 1);

let metric = KeywordMetric;
let score = metric.score("Say hello", "Hello! How can I help?", &["hello"]);
assert!(score > 0.0);

Re-exports

pub use export::EvalReportExport;

pub use metrics::JudgeProvider;

pub use metrics::LlmJudgeMetric;

pub use metrics::SchemaValidationMetric;

pub use metrics::ToolUsageMetric;

pub use runner::AsyncMetric;

pub use runner::EvalAgent;

pub use runner::EvalRunner;

pub use runner::SyncMetricAdapter;

Modules

export

Report export — JSON and CSV serialization for EvalReport.

metrics

Specialized metrics for LLM output evaluation.

runner

Async Metric trait and EvalRunner execution engine.

Structs

EvalReport

An evaluation report.

EvalSuite

A suite of evaluation test cases.

KeywordMetric

Built-in keyword matching metric.

LengthRelevancyMetric

Built-in length-based relevancy metric.

TestCase

A single evaluation test case.

TestResult

A single test case result.

Traits

Metric

Trait for evaluation metrics.