Expand description
One evaluation of the sequence-rule language, and a record of what it evaluated.
Two things live here, and the second is the reason the first moved.
One implementation, most of the way. The rule language had two evaluators. assay-metrics’
sequence_valid handled Require, Before and Blocklist and resolved no aliases;
assay-mcp-server’s check_sequence handled all eight variants and did resolve them.
The same suite YAML therefore got two answers, and the silent one was the pass: a
never_after rule, which is the shape a credential-read-then-egress policy is written
in, fell through the metric’s _ arm and reported a clean run. assay-metrics cannot
call assay-mcp-server (the dependency runs the other way), so the shared home is here.
assay-mcp-server has not called through yet: its JSON violation shape is a published tool
contract and porting it means preserving message text field by field. Until it does the two
are guarded by assay-mcp-server/tests/sequence_eval_parity.rs rather than by a shared call,
which is the fallback CLAUDE.md sanctions and the weaker of the two options. What that test
guards is not hypothetical: a differential over every trace of length <= 5 on a three-symbol
alphabet found 213 after disagreements between the copies at this module’s first commit.
Those were closed by the after rewrite, not by TraceExtent; the extent parameter creates
divergences of its own, by design, which is why the parity test pins the proxy’s reading.
A record, not a verdict. Each rule yields a RuleEvaluation naming the rule, the
call indices it read, and what it found. A consumer recomputes the conclusion from the
carried span rather than accepting a severity, which is what ADR-042 requires of a claim:
bounded, and checkable by someone who does not trust the producer. Nothing here
aggregates: there is no score, no whole-run verdict, and a caller that wants one has to
write the reduction itself and own it.
RuleOutcome::NotExercised is the member that makes the record worth carrying. A
before rule whose then tool never appears passes without its antecedent ever firing,
and a rule kind this build does not implement passes for a different reason entirely.
Both used to be indistinguishable from a rule that ran and held. They are separate values
now, for the same reason crate::metrics_api::Exercised exists one layer down.
Structs§
- Rule
Evaluation - One rule’s evaluation against one call sequence.
- Sequence
Call - One call as the rule language sees it.
Enums§
- Rule
Outcome - What one rule found. Deliberately three values: a rule that did not run is not a rule that passed, and folding them loses the distinction this module exists to keep.
- Trace
Extent - Whether more calls may still arrive.
Functions§
- evaluate_
rules - Evaluate every rule against the ordered tool-call names, returning one record per rule.