Enum Metric 

pub enum Metric {
    EndedOnFailedCall,
    ToolErrorRate,
    CutShort,
    Compactions,
    Turns,
    MalformedArgs,
}

What a candidate claims it will do.

Variants

EndedOnFailedCall

Runs that finished with their last tool call failed.

ToolErrorRate

Share of attempted tool calls the environment refused.

CutShort

Runs the harness cut short rather than the model finishing.

Compactions

Summaries taken. Fewer is better only when the work is unchanged, which is what the work guardrail is for.

Turns

Turns spent.

MalformedArgs

Arguments the model produced that did not parse.

Implementations

impl Metric

pub fn headroom(&self, recorded: &RunStats) -> f64

How much this episode can say about the metric, higher being more.

The priority for a prioritised replay draw, and it needs no new concept: it is the metric’s own value on the recorded run. Every metric here is a cost, so an episode already at zero has no room to improve — whatever the change does, that pair can only tie or worsen, and it costs a real model run per arm to learn that. An episode with a high recorded cost is the one that can discriminate.

This is prioritised experience replay’s shape with the sensor that exists today. PER samples by |TD error| because a surprising transition carries the most information; here the same argument is made with headroom, because the appraisal record that would supply a goal error is not built yet. When it is, |goal error| joins this rather than replacing it — a run can be uninformative about a metric and still be the most instructive thing that happened all week.

It is only ever a priority, never a score. Drawing the selection slice this way is safe precisely because selection only picks; the holdout, drawn uniformly, is what confirms. See judge_drawn.

pub fn of(&self, s: &RunStats) -> f64

The metric’s value for one run. Lower is better for every metric here, which is a deliberate constraint rather than a coincidence: a mixed polarity is the kind of thing that inverts a comparison silently, so anything worth predicting gets phrased as a cost.

Trait Implementations

impl Clone for Metric

fn clone(&self) -> Metric

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Copy for Metric

impl Debug for Metric

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<'de> Deserialize<'de> for Metric

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl Eq for Metric

impl PartialEq for Metric

fn eq(&self, other: &Metric) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl Serialize for Metric

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.

impl StructuralPartialEq for Metric