eval-magic 0.5.0

One-stop CLI for running skill evals — measure whether an agent skill actually shifts behavior.
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

//! Shared error type for the pipeline stages.
//!
//! The stages orchestrate filesystem IO, JSON (de)serialization, and schema
//! validation, so a stage failure can originate in any of those. `PipelineError`
//! unifies them behind one `thiserror` enum (the library-side convention; the CLI
//! boundary maps it to `anyhow`).

/// A recoverable failure while running a pipeline stage.
#[derive(Debug, thiserror::Error)]
pub enum PipelineError {
    /// A stage-specific failure with a ready-to-display message.
    #[error("{0}")]
    Message(String),
    /// Filesystem IO failure.
    #[error(transparent)]
    Io(#[from] std::io::Error),
    /// JSON parse/serialize failure.
    #[error(transparent)]
    Json(#[from] serde_json::Error),
    /// A produced artifact failed schema validation before write.
    #[error(transparent)]
    Validation(#[from] crate::validation::ValidationError),
}