Struct HumanEvaluationConfig 

#[non_exhaustive]
pub struct HumanEvaluationConfig {
    pub human_workflow_config: Option<HumanWorkflowConfig>,
    pub custom_metrics: Option<Vec<HumanEvaluationCustomMetric>>,
    pub dataset_metric_configs: Vec<EvaluationDatasetMetricConfig>,
}

Specifies the custom metrics, how tasks will be rated, the flow definition ARN, and your custom prompt datasets. Model evaluation jobs use human workers only support the use of custom prompt datasets. To learn more about custom prompt datasets and the required format, see Custom prompt datasets.

When you create custom metrics in HumanEvaluationCustomMetric you must specify the metric's name. The list of names specified in the HumanEvaluationCustomMetric array, must match the metricNames array of strings specified in EvaluationDatasetMetricConfig. For example, if in the HumanEvaluationCustomMetric array your specified the names "accuracy", "toxicity", "readability" as custom metrics then the metricNames array would need to look like the following \["accuracy", "toxicity", "readability"\] in EvaluationDatasetMetricConfig.

Fields (Non-exhaustive)

Non-exhaustive structs could have additional fields added in future. Therefore, non-exhaustive structs cannot be constructed in external crates using the traditional Struct { .. } syntax; cannot be matched against without a wildcard ..; and struct update syntax will not work.

human_workflow_config: Option<HumanWorkflowConfig>

The parameters of the human workflow.

custom_metrics: Option<Vec<HumanEvaluationCustomMetric>>

A HumanEvaluationCustomMetric object. It contains the names the metrics, how the metrics are to be evaluated, an optional description.

dataset_metric_configs: Vec<EvaluationDatasetMetricConfig>

Use to specify the metrics, task, and prompt dataset to be used in your model evaluation job.

Implementations

impl HumanEvaluationConfig

pub fn human_workflow_config(&self) -> Option<&HumanWorkflowConfig>

The parameters of the human workflow.

pub fn custom_metrics(&self) -> &[HumanEvaluationCustomMetric]

A HumanEvaluationCustomMetric object. It contains the names the metrics, how the metrics are to be evaluated, an optional description.

If no value was sent for this field, a default will be set. If you want to determine if no value was sent, use .custom_metrics.is_none().

pub fn dataset_metric_configs(&self) -> &[EvaluationDatasetMetricConfig]

Use to specify the metrics, task, and prompt dataset to be used in your model evaluation job.

impl HumanEvaluationConfig

pub fn builder() -> HumanEvaluationConfigBuilder

Creates a new builder-style object to manufacture HumanEvaluationConfig.

Trait Implementations

impl Clone for HumanEvaluationConfig

fn clone(&self) -> HumanEvaluationConfig

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for HumanEvaluationConfig

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl PartialEq for HumanEvaluationConfig

fn eq(&self, other: &HumanEvaluationConfig) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl StructuralPartialEq for HumanEvaluationConfig