pub struct HarnessMeasureRequest {
pub model: String,
pub split: String,
pub held_in_fraction: f64,
pub split_seed: u64,
pub max_turns: u32,
pub tasks_dir: Option<PathBuf>,
}Expand description
What to replay. Serde-derived: it is the harness_measure param of
evolution.run verbatim.
The defaults mirror car_bench::harness_bench::HarnessBenchConfig::default()
exactly — held-out, a 0.5 held-in fraction, seed 0, 20 turns — so an
in-daemon measurement and a car-bench-harness CLI run are the same
measurement over the same task split. A default that drifted from the
CLI’s would silently make an operator’s baseline file incomparable with a
daemon-measured candidate.
Fields§
§model: StringModel id under test. REQUIRED — metrics not attributable to a model are not metrics, and a router substitution would attribute a token count to the wrong model.
split: StringWhich half of the deterministic split. Default "held-out" — the
regression-gate half, the tasks a mutation was not tuned on.
held_in_fraction: f64Share of tasks assigned to held-in.
split_seed: u64Seed for the deterministic split shuffle. Fix it for a lineage: two runs at different seeds are over different task sets.
max_turns: u32Assistant-loop turn cap per task.
tasks_dir: Option<PathBuf>Override the task suite directory. None = the suite built into the
binary.
Trait Implementations§
Source§impl Clone for HarnessMeasureRequest
impl Clone for HarnessMeasureRequest
Source§impl Debug for HarnessMeasureRequest
impl Debug for HarnessMeasureRequest
Source§impl<'de> Deserialize<'de> for HarnessMeasureRequest
impl<'de> Deserialize<'de> for HarnessMeasureRequest
Source§fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>where
__D: Deserializer<'de>,
fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>where
__D: Deserializer<'de>,
Source§impl PartialEq for HarnessMeasureRequest
impl PartialEq for HarnessMeasureRequest
Source§impl Serialize for HarnessMeasureRequest
impl Serialize for HarnessMeasureRequest
impl StructuralPartialEq for HarnessMeasureRequest
Auto Trait Implementations§
impl Freeze for HarnessMeasureRequest
impl RefUnwindSafe for HarnessMeasureRequest
impl Send for HarnessMeasureRequest
impl Sync for HarnessMeasureRequest
impl Unpin for HarnessMeasureRequest
impl UnsafeUnpin for HarnessMeasureRequest
impl UnwindSafe for HarnessMeasureRequest
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
impl<ST, DT> CastableFrom<ST, Initialized, Initialized> for DT
impl<ST, DT> CastableFrom<ST, Uninit, Uninit> for DT
Source§impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> DeserializeOwned for Twhere
T: for<'de> Deserialize<'de>,
impl<S, T> Duplex<S> for Twhere
T: FromSample<S> + ToSample<S>,
impl<T> ErasedDestructor for Twhere
T: 'static,
Source§impl<S> FromSample<S> for S
impl<S> FromSample<S> for S
fn from_sample_(s: S) -> S
Source§impl<T> Instrument for T
impl<T> Instrument for T
Source§fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
Source§fn in_current_span(self) -> Instrumented<Self> ⓘ
fn in_current_span(self) -> Instrumented<Self> ⓘ
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more