Struct NamedScorer 

pub struct NamedScorer {
    pub name: String,
    pub command: Vec<String>,
    pub version_command: Option<Vec<String>>,
}

A named, executable scorer — declared HERE and not in a corpus.

Why this lives in the references file

references.toml has always been executable input: every entry names an argv this crate spawns, and it is read and reviewed as such. A corpora/*.toml was pure data.

When VLM scoring landed, the corpus grew a [scorer] block carrying an argv, and that moved the trust boundary: a file shaped like data could suddenly run code. The risk was never that execution is possible — it is that execution became invisible, buried on line 20 of a thousand lines of hashes and prompts that a reviewer will skim.

So the argv moved back here, and the corpus now merely selects a scorer by name. Data selects from a set; code defines the set. The corpus keeps metric and scale, which are facts about the benchmark’s numbers rather than instructions to execute anything.

Fields

name: String

The name a corpus refers to.

command: Vec<String>

Argv; {predictions} is replaced with the predictions JSONL path.

version_command: Option<Vec<String>>

Optional argv printing a version string, recorded in the ledger.

Implementations

impl NamedScorer

pub fn command_line(&self) -> String

The argv as a single line, for the ledger and for printing before the spawn — an executed command should be visible, not merely permitted.

Trait Implementations

impl Clone for NamedScorer

fn clone(&self) -> NamedScorer

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for NamedScorer

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<'de> Deserialize<'de> for NamedScorer

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.