Struct EvaluationDatasetMetricConfig 

#[non_exhaustive]
pub struct EvaluationDatasetMetricConfig {
    pub task_type: EvaluationTaskType,
    pub dataset: Option<EvaluationDataset>,
    pub metric_names: Vec<String>,
}

Defines the prompt datasets, built-in metric names and custom metric names, and the task type.

Fields (Non-exhaustive)

Non-exhaustive structs could have additional fields added in future. Therefore, non-exhaustive structs cannot be constructed in external crates using the traditional Struct { .. } syntax; cannot be matched against without a wildcard ..; and struct update syntax will not work.

task_type: EvaluationTaskType

The the type of task you want to evaluate for your evaluation job. This applies only to model evaluation jobs and is ignored for knowledge base evaluation jobs.

dataset: Option<EvaluationDataset>

Specifies the prompt dataset.

metric_names: Vec<String>

The names of the metrics you want to use for your evaluation job.

For knowledge base evaluation jobs that evaluate retrieval only, valid values are "Builtin.ContextRelevance", "Builtin.ContextCoverage".

For knowledge base evaluation jobs that evaluate retrieval with response generation, valid values are "Builtin.Correctness", "Builtin.Completeness", "Builtin.Helpfulness", "Builtin.LogicalCoherence", "Builtin.Faithfulness", "Builtin.Harmfulness", "Builtin.Stereotyping", "Builtin.Refusal".

For automated model evaluation jobs, valid values are "Builtin.Accuracy", "Builtin.Robustness", and "Builtin.Toxicity". In model evaluation jobs that use a LLM as judge you can specify "Builtin.Correctness", "Builtin.Completeness", "Builtin.Faithfulness", "Builtin.Helpfulness", "Builtin.Coherence", "Builtin.Relevance", "Builtin.FollowingInstructions", "Builtin.ProfessionalStyleAndTone", You can also specify the following responsible AI related metrics only for model evaluation job that use a LLM as judge "Builtin.Harmfulness", "Builtin.Stereotyping", and "Builtin.Refusal".

For human-based model evaluation jobs, the list of strings must match the name parameter specified in HumanEvaluationCustomMetric.

Implementations

impl EvaluationDatasetMetricConfig

pub fn task_type(&self) -> &EvaluationTaskType

The the type of task you want to evaluate for your evaluation job. This applies only to model evaluation jobs and is ignored for knowledge base evaluation jobs.

pub fn dataset(&self) -> Option<&EvaluationDataset>

Specifies the prompt dataset.

pub fn metric_names(&self) -> &[String]

The names of the metrics you want to use for your evaluation job.

For knowledge base evaluation jobs that evaluate retrieval only, valid values are "Builtin.ContextRelevance", "Builtin.ContextCoverage".

For knowledge base evaluation jobs that evaluate retrieval with response generation, valid values are "Builtin.Correctness", "Builtin.Completeness", "Builtin.Helpfulness", "Builtin.LogicalCoherence", "Builtin.Faithfulness", "Builtin.Harmfulness", "Builtin.Stereotyping", "Builtin.Refusal".

For automated model evaluation jobs, valid values are "Builtin.Accuracy", "Builtin.Robustness", and "Builtin.Toxicity". In model evaluation jobs that use a LLM as judge you can specify "Builtin.Correctness", "Builtin.Completeness", "Builtin.Faithfulness", "Builtin.Helpfulness", "Builtin.Coherence", "Builtin.Relevance", "Builtin.FollowingInstructions", "Builtin.ProfessionalStyleAndTone", You can also specify the following responsible AI related metrics only for model evaluation job that use a LLM as judge "Builtin.Harmfulness", "Builtin.Stereotyping", and "Builtin.Refusal".

For human-based model evaluation jobs, the list of strings must match the name parameter specified in HumanEvaluationCustomMetric.

impl EvaluationDatasetMetricConfig

pub fn builder() -> EvaluationDatasetMetricConfigBuilder

Creates a new builder-style object to manufacture EvaluationDatasetMetricConfig.

Trait Implementations

impl Clone for EvaluationDatasetMetricConfig

fn clone(&self) -> EvaluationDatasetMetricConfig

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for EvaluationDatasetMetricConfig

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl PartialEq for EvaluationDatasetMetricConfig

fn eq(&self, other: &EvaluationDatasetMetricConfig) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl StructuralPartialEq for EvaluationDatasetMetricConfig