Skip to main content

HarnessMeasurer

Trait HarnessMeasurer 

Source
pub trait HarnessMeasurer: Send + Sync {
    // Required method
    fn measure<'life0, 'life1, 'life2, 'life3, 'async_trait>(
        &'life0 self,
        request: &'life1 HarnessMeasureRequest,
        harness_config: Option<&'life2 HarnessConfig>,
        memgine_config: Option<&'life3 MemgineConfig>,
    ) -> Pin<Box<dyn Future<Output = Result<HarnessMetrics, String>> + Send + 'async_trait>>
       where Self: 'async_trait,
             'life0: 'async_trait,
             'life1: 'async_trait,
             'life2: 'async_trait,
             'life3: 'async_trait;
}
Expand description

What the daemon needs in order to grade a harness candidate ITSELF.

Implemented above this crate (car-bench owns the task suite and the assistant-loop replay) and installed on ServerState by the daemon binary, because car-bench depends on car-server-core and the dependency cannot run the other way.

Injected rather than called directly for the same reason the rest of this module injects execution: the evolution.run orchestration — mutual exclusion, the dry-run rule, which mutations are measured at all, and the gate wiring — stays unit-testable against a stub that spends no model calls.

Required Methods§

Source

fn measure<'life0, 'life1, 'life2, 'life3, 'async_trait>( &'life0 self, request: &'life1 HarnessMeasureRequest, harness_config: Option<&'life2 HarnessConfig>, memgine_config: Option<&'life3 MemgineConfig>, ) -> Pin<Box<dyn Future<Output = Result<HarnessMetrics, String>> + Send + 'async_trait>>
where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait, 'life3: 'async_trait,

Replay the requested split in process under harness_config and memgine_config, and fold the runs’ own event logs into one HarnessMetrics.

harness_config is the operating config the replay must run UNDER — None means the runtime default. Measuring a candidate without installing its config produces a run byte-identical to the baseline, so an implementation that ignores this argument reports a comparison of a config with itself.

memgine_config is the context-assembly config the replay’s memory fixtures are seeded under — None means the memgine default. It is the second pillar’s twin of the argument above, and the identical warning applies: a “candidate” context measurement taken without installing the candidate config is a second measurement of the default, and the gate would be comparing a config with itself. It matters because a bench task that declares a memory: fixture is replayed with a real memgine attached and the shipped recall tool advertised, so the assembled context — and therefore the answer the task is graded on — genuinely moves with conversation_keep_recent.

ONE trait, not one per pillar: the replay is the same replay over the same split with the same seed, and only which config is varied differs. Two traits would let the two arms drift into measuring different task sets, at which point a context grade and a harness grade stop being comparable to each other or to a car-bench-harness CLI run.

Every call must be a REAL measurement or an Err. Returning a default/zero document is the one thing this trait must never do: an all-zero HarnessMetrics is structurally indistinguishable from a measurement of a harness that spends nothing, and the regression gate would read it as one.

Dyn Compatibility§

This trait is dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementors§