Struct BenchmarkSet 

pub struct BenchmarkSet {
    pub cases: Vec<BenchmarkCase>,
}

A set of benchmark cases loaded from a TOML file.

TOML format

[[cases]]
prompt = "What is the capital of France?"
reference = "Paris"
tags = ["geography", "factual"]

[[cases]]
prompt = "Explain async/await in Rust."
context = "You are a Rust expert."

Fields

cases: Vec<BenchmarkCase>

Implementations

impl BenchmarkSet

pub fn from_file(path: &Path) -> Result<Self, EvalError>

Load a benchmark set from a TOML file.

Performs size guard (10 MiB limit) and canonicalisation before reading. Symlinks that escape the file’s parent directory are rejected.

Errors

Returns EvalError::BenchmarkLoad if the file cannot be read, EvalError::BenchmarkParse if the TOML is invalid, EvalError::BenchmarkTooLarge if the file exceeds the size limit, or EvalError::PathTraversal if canonicalization reveals a symlink escape.

pub fn validate(&self) -> Result<(), EvalError>

Validate that the benchmark set is non-empty.

Errors

Returns EvalError::EmptyBenchmarkSet if cases is empty.

Trait Implementations

impl Clone for BenchmarkSet

fn clone(&self) -> BenchmarkSet

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for BenchmarkSet

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<'de> Deserialize<'de> for BenchmarkSet

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.