pub enum Assertion {
ToolCallOrder {
expected: Vec<ExpectedToolCall>,
},
ToolCallCount {
name: String,
expected: usize,
},
StatePredicate {
pointer: String,
expected: Value,
},
NoToolCall {
names: Vec<String>,
},
MaxCost {
usd: f64,
},
MaxLatency {
ms: u64,
},
}Expand description
One deterministic check over a run’s evidence.
Serialized with an assertion tag so assertions can live in config
files and reports: {"assertion":"tool_call_count","name":"calculator","expected":2}.
Variants§
ToolCallOrder
The expected tool calls appear in the observed trajectory as an ordered subsequence (extra calls in between are allowed), each satisfying its argument matchers.
Fields
expected: Vec<ExpectedToolCall>The expected subsequence, in order.
ToolCallCount
The tool name was called exactly expected times.
StatePredicate
The run’s final state has expected at pointer (JSON pointer,
RFC 6901 — e.g. /messages/3/content).
Fields
NoToolCall
None of the blacklisted tools were called.
MaxCost
The run’s total journaled cost is at most usd.
MaxLatency
The run’s wall latency is at most ms.
Implementations§
Source§impl Assertion
impl Assertion
Sourcepub fn name(&self) -> String
pub fn name(&self) -> String
The stable report key for this assertion. Aggregation groups results
by it, so it includes the parameters that distinguish two assertions
of the same kind (state[/answer] vs state[/score]).
Sourcepub fn evaluate(&self, evidence: &RunEvidence) -> AssertionResult
pub fn evaluate(&self, evidence: &RunEvidence) -> AssertionResult
Evaluate against one run’s evidence. Pure: same evidence, same verdict.