mira-eval

The core of Mira — a Rust-first, code-first evaluation framework for agents and tools. The library is imported as mira.

Eval = Dataset(Sample…) + Subject + [Scorer…]  ×  model matrix

use mira::scorer::{contains, succeeded};
use mira::subject::subject_fn;
use mira::{Eval, Transcript, register_eval};

fn greet() -> Eval {
    Eval::new("greet")
        .sample("hi", "Say hi and tell me the answer to life.")
        .subject(subject_fn(|_s, _cx| async move {
            Transcript::response("Hi! The answer is 42.")
        }))
        .scorer(succeeded())
        .scorer(contains("42"))
        .build()
}
register_eval!(greet);

#[tokio::main]
async fn main() -> std::io::Result<()> {
    mira::Study::registered().serve().await
}

Run it with the mira-cli host:

mira --bin greet run

See the docs for the full guide and the protocol reference.

Licensed under MIT.