Enum ArchitectureStrategy

pub enum ArchitectureStrategy {
    Transformer {
        component_specific_lr: bool,
        layer_wise_decay: bool,
        attention_warmup: usize,
    },
    ConvolutionalNet {
        layer_type_lr: bool,
        depth_scaling: bool,
        bn_special_handling: bool,
    },
    RecurrentNet {
        rnn_gradient_clip: Option<f64>,
        weight_type_lr: bool,
    },
    Custom {
        rules: HashMap<String, LayerConfig>,
    },
}

Architecture-specific optimization strategy

Variants

Transformer

Transformer-specific optimizations

Fields

component_specific_lr: bool

Use different learning rates for different components

layer_wise_decay: bool

Apply layer-wise learning rate decay

attention_warmup: usize

Warmup steps for attention parameters

ConvolutionalNet

CNN-specific optimizations

Fields

layer_type_lr: bool

Use different learning rates for conv vs fc layers

depth_scaling: bool

Apply depth-wise learning rate scaling

bn_special_handling: bool

Batch norm parameter handling

RecurrentNet

RNN-specific optimizations

Fields

rnn_gradient_clip: Option<f64>

Gradient clipping specifically for RNNs

weight_type_lr: bool

Different learning rates for recurrent vs linear weights

Custom

Custom architecture strategy

Fields

rules: HashMap<String, LayerConfig>

Custom optimization rules

Trait Implementations

impl Clone for ArchitectureStrategy

fn clone(&self) -> ArchitectureStrategy

Returns a duplicate of the value.

const fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for ArchitectureStrategy

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.