Struct BenchmarkCase 

pub struct BenchmarkCase {
    pub prompt: String,
    pub context: Option<String>,
    pub reference: Option<String>,
    pub tags: Option<Vec<String>>,
}

A single benchmark case.

The prompt is sent to the subject model. If context is present it is injected as a system message before the user turn. If reference is present the judge model uses it to calibrate its score.

Examples

use zeph_experiments::BenchmarkCase;

let case = BenchmarkCase {
    prompt: "Name the first element.".into(),
    context: Some("You are a chemistry expert.".into()),
    reference: Some("Hydrogen".into()),
    tags: Some(vec!["chemistry".into()]),
};
assert!(case.reference.as_deref() == Some("Hydrogen"));

Fields

prompt: String

The prompt sent to the subject model.

context: Option<String>

Optional system context injected before the user turn.

reference: Option<String>

Optional reference answer for the judge to calibrate scoring.

tags: Option<Vec<String>>

Optional tags for filtering or grouping results.

Trait Implementations

impl Clone for BenchmarkCase

fn clone(&self) -> BenchmarkCase

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for BenchmarkCase

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<'de> Deserialize<'de> for BenchmarkCase

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.