Skip to main content

OutcomeEvalset

Struct OutcomeEvalset 

Source
pub struct OutcomeEvalset {
    pub hash: String,
    pub field: String,
    pub higher_is_better: bool,
    pub horizons_ms: Vec<i64>,
    pub checkpoints: Vec<Checkpoint>,
    pub baseline: BaselineKind,
    pub min_effect: Option<MinEffect>,
    pub cost: Option<CostBound>,
}
Expand description

The evalset every LLM-authored, applicable proposal is measured against after apply (docs/loop.md, “Evalset-backed outcomes”). An authored lesson carries no built-in recurrence metric — nothing errors when a lesson is merely useless — so without this the Verify gate has nothing to re-measure for exactly the proposals a human was least able to judge. The host names the evalset and the field; the engine takes the baseline from the newest run journaled BEFORE the proposal and reads the current value from runs journaled AFTER the apply. No baseline run → no metric, never a fabricated one.

Fields§

§hash: String

The evalset hash (the subject is evalset:<hash> in agent:harness).

§field: String

The summary field to read: passed, failed, total, error_rate, or any numeric field the host’s harness writes into the summary.

§higher_is_better: bool

Which direction is an improvement — passed and an accuracy are higher-is-better, failed and error_rate are not. Stated by the host because getting it wrong would revert an improvement.

§horizons_ms: Vec<i64>

Checkpoints after apply, in ms. Default 1d / 7d / 30d. The older spelling; checkpoints wins when both are given.

§checkpoints: Vec<Checkpoint>

The schedule in the deployment’s own unit — {"after_ms": n}, {"after_runs": n} or {"after_grains": n} (a bare integer is ms). A benchmark or CI harness wants [{"after_runs": 1}]: measure at the next graded run after the apply, however soon that is. Empty (the default) means horizons_ms.

§baseline: BaselineKind

Which run before the apply the verdict compares against (default newest_before_apply; see BaselineKind for why high_water is opt-in).

§min_effect: Option<MinEffect>

The minimum effect size a verdict needs to call a regression (default none: any drop past floating-point slack is a regression).

§cost: Option<CostBound>

A cost bound read beside the quality field (default none).

Implementations§

Source§

impl OutcomeEvalset

Source

pub fn schedule(&self) -> Vec<Checkpoint>

The effective schedule: checkpoints when set, else horizons_ms as time checkpoints.

Trait Implementations§

Source§

impl Clone for OutcomeEvalset

Source§

fn clone(&self) -> Self

Returns a duplicate of the value. Read more
1.0.0 (const: unstable) · Source§

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source. Read more
Source§

impl Debug for OutcomeEvalset

Source§

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter. Read more
Source§

impl<'de> Deserialize<'de> for OutcomeEvalset

Source§

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>
where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer. Read more
Source§

impl PartialEq for OutcomeEvalset

Source§

fn eq(&self, other: &Self) -> bool

Equality operator ==. Read more
1.0.0 (const: unstable) · Source§

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=. Read more
Source§

impl Serialize for OutcomeEvalset

Source§

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>
where __S: Serializer,

Serialize this value into the given Serde serializer. Read more
Source§

impl StructuralPartialEq for OutcomeEvalset

Auto Trait Implementations§

Blanket Implementations§

Source§

impl<T> Any for T
where T: 'static + ?Sized,

Source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
Source§

impl<T> Borrow<T> for T
where T: ?Sized,

Source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
Source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

Source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
Source§

impl<T> CloneToUninit for T
where T: Clone,

Source§

unsafe fn clone_to_uninit(&self, dest: *mut u8)

🔬This is a nightly-only experimental API. (clone_to_uninit)
Performs copy-assignment from self to dest. Read more
Source§

impl<T> DeserializeOwned for T
where T: for<'de> Deserialize<'de>,

Source§

impl<T> From<T> for T

Source§

fn from(t: T) -> T

Returns the argument unchanged.

Source§

impl<T, U> Into<U> for T
where U: From<T>,

Source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

Source§

impl<T> ToOwned for T
where T: Clone,

Source§

type Owned = T

The resulting type after obtaining ownership.
Source§

fn to_owned(&self) -> T

Creates owned data from borrowed data, usually by cloning. Read more
Source§

fn clone_into(&self, target: &mut T)

Uses borrowed data to replace owned data, usually by cloning. Read more
Source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

Source§

type Error = !

The type returned in the event of a conversion error.
Source§

fn try_from(value: U) -> Result<T, !>

Performs the conversion.
Source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

Source§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
Source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.