Crate klieo_eval 

Recall-quality metrics for klieo memory pipelines.

Pure scoring — no I/O, no async, no dependency on the memory traits. Callers run the recall against whatever pipeline they want (Qdrant, sqlite-vec, hybrid GraphRAG, …), collect the returned ids into a RecallSample, and hand the slice to score_recall.

Metrics

• hit_rate — fraction of queries with at least one expected id in the top-K result list.
• mean reciprocal rank (MRR) — mean of 1 / rank-of-first-hit across queries; non-hit contributes 0.
• mean precision@K — |expected ∩ top-K| / K, averaged across queries.
• mean recall@K — |expected ∩ top-K| / |expected|, averaged across queries with at least one expected id. Queries with empty expected do not contribute to the mean.
• mean NDCG@K — normalised discounted cumulative gain over the top-K results, binary relevance (1 if expected, 0 otherwise), averaged across queries with at least one expected id.

Example

use klieo_eval::{score_recall, RecallSample};

let samples = vec![RecallSample::new(
    "ICT risk management",
    vec!["dora-art-5".into(), "dora-art-6".into()],
    vec![
        "dora-art-5".into(),
        "ai-act-art-6".into(),
        "dora-art-6".into(),
    ],
)];
let report = score_recall(&samples, 5);
assert!((report.hit_rate() - 1.0).abs() < 1e-9);
assert!(report.mean_reciprocal_rank() > 0.99);

Re-exports

pub use agent_eval::eval_capture;Deprecated

pub use agent_eval::check_regression;

pub use agent_eval::eval_capture_determinism;

pub use agent_eval::EvalMetrics;

pub use agent_eval::Regression;

pub use live_eval::eval_capture_live;

pub use live_eval::LiveEvalMetrics;

pub use classifier_eval::extract_label;

pub use classifier_eval::score_classification;

pub use classifier_eval::ClassificationCase;

pub use classifier_eval::ClassificationReport;

pub use classifier_eval::LabelSpec;

pub use classifier_eval::LabelStat;

pub use classifier_eval::Miss;

Modules

agent_eval

Replay-first agent regression eval (ADR-047).

classifier_eval

Deterministic, structured label-match eval for classifier agents. Pure scoring — no I/O, no provider, no agent driver. A classifier’s output is a closed label set, so exact-label-match gates the actual decision while tolerating the prose a real LLM wraps it in.

live_eval

Live agent-loop re-drive eval (ADR-048).

Structs

PerQueryScore

One row of the report — every metric for one query, plus the rank of the first hit (1-based) or None when none of the expected ids appeared in the top-K window. expected_count is preserved so recall- and NDCG-shaped means can correctly skip queries with empty expected.

RecallReport

Per-query trace plus aggregate metrics computed across the scored sample slice.

RecallSample

One scored query — the question, the ground-truth ids that should land in the top-K, and the actual ranked id list returned by the recall pipeline (top result first).

Functions

score_recall

Score a batch of samples against a top-K window.