pub enum BaselineKind {
NewestBeforeApply,
HighWater,
}Expand description
Which run journaled before the apply an evalset verdict compares
against (docs/loop.md, “Evalset-backed outcomes”).
newest_before_apply is the marginal question — did THIS apply make the
agent worse than it was the moment before — and it is the default because
it never proposes reverting a rule for a drop an earlier rule caused.
high_water asks the question an operator actually has — is the agent
worse than the best it has been — and it is a policy choice, not the
default, because on a noisy evalset (or a deployment that does not
journal a run between applies) it attributes the whole fall from the
peak to whichever rule was applied last. That confounding is the trade;
measured need: on the ad-buy corpus (crates/areev-bench/ADBUY.md, seed
3) an agent that reached 238 of 280 and then fell to 128 measured held
against the day-one run of 35, because day one was the only run before
the apply.
Variants§
NewestBeforeApply
The newest run journaled strictly before the apply.
HighWater
The best run journaled strictly before the apply — max for a higher-is-better field, min otherwise.