pub struct EvaluationReport {Show 17 fields
pub schema_version: u32,
pub fixture_schema_version: u32,
pub response_schema_version: u32,
pub dataset_id: String,
pub provider: String,
pub models: Vec<String>,
pub question_schema_hash: String,
pub fixture_content_hash: String,
pub completeness: Completeness,
pub metrics: Vec<QuestionMetrics>,
pub total_input_tokens: u64,
pub total_output_tokens: u64,
pub total_input_cost_usd: Option<f64>,
pub total_output_cost_usd: Option<f64>,
pub total_cost_usd: Option<f64>,
pub latency_p50_ms: Option<f64>,
pub latency_p95_ms: Option<u64>,
}Expand description
Deterministic offline evaluation report.
Fields§
§schema_version: u32Report schema version.
fixture_schema_version: u32Fixture schema version.
response_schema_version: u32Saved response schema version.
dataset_id: StringDataset identifier.
provider: StringProvider identifier from saved responses.
models: Vec<String>Actual model versions observed, sorted and deduplicated.
question_schema_hash: StringStable FNV-1a hash of case IDs and typed question schemas.
fixture_content_hash: StringStable FNV-1a hash of all labeled fixture content.
completeness: CompletenessValid, failed, and missing case counts.
metrics: Vec<QuestionMetrics>Per-question and grouped statistics.
total_input_tokens: u64Total input tokens from valid responses.
total_output_tokens: u64Total output tokens from valid responses.
total_input_cost_usd: Option<f64>Estimated input-token charge, if its price is known.
total_output_cost_usd: Option<f64>Estimated output-token charge, if its price is known.
total_cost_usd: Option<f64>Estimated combined charge, only when both prices are known.
latency_p50_ms: Option<f64>Median end-to-end latency of valid responses.
latency_p95_ms: Option<u64>Nearest-rank 95th percentile latency of valid responses.
Implementations§
Source§impl EvaluationReport
impl EvaluationReport
Sourcepub fn metric_for(
&self,
question: &str,
language: Option<&str>,
risk: Option<&str>,
) -> Option<&QuestionMetrics>
pub fn metric_for( &self, question: &str, language: Option<&str>, risk: Option<&str>, ) -> Option<&QuestionMetrics>
Find metrics for a question and a specific language/risk slice.
Sourcepub fn validate(&self) -> Result<(), EvaluationError>
pub fn validate(&self) -> Result<(), EvaluationError>
Validate a report loaded from a file before calibration or comparison.
§Errors
Returns EvaluationError::Invalid for malformed metadata or statistics.
Trait Implementations§
Source§impl Clone for EvaluationReport
impl Clone for EvaluationReport
Source§impl Debug for EvaluationReport
impl Debug for EvaluationReport
Source§impl<'de> Deserialize<'de> for EvaluationReport
impl<'de> Deserialize<'de> for EvaluationReport
Source§fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>where
__D: Deserializer<'de>,
fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>where
__D: Deserializer<'de>,
Auto Trait Implementations§
impl Forget for EvaluationReport
impl Freeze for EvaluationReport
impl Move for EvaluationReport
impl RefUnwindSafe for EvaluationReport
impl Send for EvaluationReport
impl Sync for EvaluationReport
impl Unpin for EvaluationReport
impl UnsafeUnpin for EvaluationReport
impl UnwindSafe for EvaluationReport
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Source§impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> DeserializeOwned for Twhere
T: for<'de> Deserialize<'de>,
Source§impl<T> Instrument for T
impl<T> Instrument for T
Source§fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
Source§fn in_current_span(self) -> Instrumented<Self> ⓘ
fn in_current_span(self) -> Instrumented<Self> ⓘ
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more