Struct SparkStandaloneAutoscalingConfig

pub struct SparkStandaloneAutoscalingConfig {
    pub graceful_decommission_timeout: Option<Duration>,
    pub remove_only_idle_workers: Option<bool>,
    pub scale_down_factor: Option<f64>,
    pub scale_down_min_worker_fraction: Option<f64>,
    pub scale_up_factor: Option<f64>,
    pub scale_up_min_worker_fraction: Option<f64>,
}

Basic autoscaling configurations for Spark Standalone.

This type is not used in any activity, and only used as part of another schema.

Fields

graceful_decommission_timeout: Option<Duration>

Required. Timeout for Spark graceful decommissioning of spark workers. Specifies the duration to wait for spark worker to complete spark decommissioning tasks before forcefully removing workers. Only applicable to downscaling operations.Bounds: 0s, 1d.

remove_only_idle_workers: Option<bool>

Optional. Remove only idle workers when scaling down cluster

scale_down_factor: Option<f64>

Required. Fraction of required executors to remove from Spark Serverless clusters. A scale-down factor of 1.0 will result in scaling down so that there are no more executors for the Spark Job.(more aggressive scaling). A scale-down factor closer to 0 will result in a smaller magnitude of scaling donw (less aggressive scaling).Bounds: 0.0, 1.0.

scale_down_min_worker_fraction: Option<f64>

Optional. Minimum scale-down threshold as a fraction of total cluster size before scaling occurs. For example, in a 20-worker cluster, a threshold of 0.1 means the autoscaler must recommend at least a 2 worker scale-down for the cluster to scale. A threshold of 0 means the autoscaler will scale down on any recommended change.Bounds: 0.0, 1.0. Default: 0.0.

scale_up_factor: Option<f64>

Required. Fraction of required workers to add to Spark Standalone clusters. A scale-up factor of 1.0 will result in scaling up so that there are no more required workers for the Spark Job (more aggressive scaling). A scale-up factor closer to 0 will result in a smaller magnitude of scaling up (less aggressive scaling).Bounds: 0.0, 1.0.

scale_up_min_worker_fraction: Option<f64>

Optional. Minimum scale-up threshold as a fraction of total cluster size before scaling occurs. For example, in a 20-worker cluster, a threshold of 0.1 means the autoscaler must recommend at least a 2-worker scale-up for the cluster to scale. A threshold of 0 means the autoscaler will scale up on any recommended change.Bounds: 0.0, 1.0. Default: 0.0.

Trait Implementations

impl Clone for SparkStandaloneAutoscalingConfig

fn clone(&self) -> SparkStandaloneAutoscalingConfig

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for SparkStandaloneAutoscalingConfig

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for SparkStandaloneAutoscalingConfig

fn default() -> SparkStandaloneAutoscalingConfig

Returns the “default value” for a type.

impl<'de> Deserialize<'de> for SparkStandaloneAutoscalingConfig

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl Serialize for SparkStandaloneAutoscalingConfig

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.

impl Part for SparkStandaloneAutoscalingConfig