Skip to main content

SafetyScorer

heartbit_core::eval

Struct SafetyScorer

pub struct SafetyScorer { /* private fields */ }

Expand description

Scores agent execution for guardrail safety.

Checks for GuardrailDenied events in the event collector. Score is 0.0 if any denial occurred, 1.0 otherwise. Warnings pass (only denials fail).

Important: The collector accumulates events across cases. When using EvalRunner::run with multiple cases, attach the same collector via EvalRunner::with_event_collector so per-case events are isolated. When calling EvalRunner::score_result manually, call clear_events yourself between cases.

Implementations§

impl SafetyScorer

pub fn new(collector: EventCollector) -> Self

Create a safety scorer that reads from the given event collector.

Trait Implementations§

impl EvalScorer for SafetyScorer

fn name(&self) -> &str

Scorer name for reporting.

fn score( &self, _case: &EvalCase, _output: &str, _tool_calls: &[String], ) -> (f64, Vec<String>)

Score the agent’s execution against the eval case. Read more

fn pass_threshold(&self) -> f64

Minimum score to pass (default: 1.0 for binary pass/fail).

Auto Trait Implementations§

impl Freeze for SafetyScorer

impl RefUnwindSafe for SafetyScorer

impl Send for SafetyScorer

impl Sync for SafetyScorer

impl Unpin for SafetyScorer

impl UnsafeUnpin for SafetyScorer

impl UnwindSafe for SafetyScorer

Blanket Implementations§

impl<T> Any for T
where T: 'static + ?Sized,

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more

impl<T> Borrow<T> for T
where T: ?Sized,

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more

impl<T> BorrowMut<T> for T
where T: ?Sized,

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more

impl<T> From<T> for T

fn from(t: T) -> T

Returns the argument unchanged.

impl<T> Instrument for T

fn instrument(self, span: Span) -> Instrumented<Self>

Instruments this type with the provided Span, returning an Instrumented wrapper. Read more

fn in_current_span(self) -> Instrumented<Self>

Instruments this type with the current Span, returning an Instrumented wrapper. Read more

impl<T, U> Into<U> for T
where U: From<T>,

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

impl<T> PolicyExt for T
where T: ?Sized,

fn and<P, B, E>(self, other: P) -> And<T, P>
where T: Policy<B, E>, P: Policy<B, E>,

Create a new Policy that returns Action::Follow only if self and other return Action::Follow. Read more

fn or<P, B, E>(self, other: P) -> Or<T, P>
where T: Policy<B, E>, P: Policy<B, E>,

Create a new Policy that returns Action::Follow if either self or other returns Action::Follow. Read more

impl<T> Same for T

type Output = T

Should always be Self

impl<T, U> TryFrom<U> for T
where U: Into<T>,

type Error = Infallible

The type returned in the event of a conversion error.

fn try_from(value: U) -> Result<T, <T as TryFrom<U>>::Error>

Performs the conversion.

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.

impl<T> WithSubscriber for T

fn with_subscriber<S>(self, subscriber: S) -> WithDispatch<Self>
where S: Into<Dispatch>,

Attaches the provided Subscriber to this type, returning a WithDispatch wrapper. Read more

fn with_current_subscriber(self) -> WithDispatch<Self>

Attaches the current default Subscriber to this type, returning a WithDispatch wrapper. Read more