Struct ElasticConfig 

pub struct ElasticConfig {
    pub base_learning_rate: f64,
    pub reference_world_size: usize,
    pub dataset_shards: usize,
    pub min_world_size: usize,
    pub max_world_size: usize,
    pub warmup_steps: usize,
}

Configuration of an elastic training run.

Fields

base_learning_rate: f64

Base (reference) learning rate, valid at reference_world_size workers.

reference_world_size: usize

World size the base_learning_rate was tuned for (the linear-scaling reference). Must be at least one.

dataset_shards: usize

Number of dataset shards D to distribute across the workers.

min_world_size: usize

Minimum allowed world size: a leave that would drop below this is rejected.

max_world_size: usize

Maximum allowed world size: a join that would rise above this is rejected.

warmup_steps: usize

Number of gradual-warmup steps applied to the learning rate after a resize (0 disables warmup).

Implementations

impl ElasticConfig

pub fn new( base_learning_rate: f64, reference_world_size: usize, dataset_shards: usize, min_world_size: usize, max_world_size: usize, warmup_steps: usize, ) -> Result<Self>

Create and validate an elastic configuration.

Errors

Returns an error when base_learning_rate is non-finite or non-positive, reference_world_size, dataset_shards or min_world_size are zero, or max_world_size < min_world_size.

pub fn validate(&self) -> Result<()>

Validate the configuration invariants.

Errors

See ElasticConfig::new.

Trait Implementations

impl Clone for ElasticConfig

fn clone(&self) -> ElasticConfig

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for ElasticConfig

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for ElasticConfig

fn default() -> Self

Returns the “default value” for a type.

impl PartialEq for ElasticConfig

fn eq(&self, other: &ElasticConfig) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl StructuralPartialEq for ElasticConfig