Struct FcdHyper 

pub struct FcdHyper {
    pub steps: usize,
    pub lr: f64,
    pub kl_w: f64,
    pub eval_every: usize,
    pub bs: usize,
    pub seq: usize,
    pub seed: u64,
    pub polish_only: bool,
}

Training hyper-parameters (defaults = the certified recipe).

Fields

steps: usize

lr: f64

kl_w: f64

eval_every: usize

bs: usize

seq: usize

seed: u64

polish_only: bool

Polish WITHOUT converting attention: the --o1 spec then only names which layers are trainable, and both teacher and student keep exact attention. This is the shape the notebook’s FCD correction has — restore accuracy a compression already cost by training the last layers on real text — and it is what a narrowed FFN needs, where the damage is in the FFN, not in the attention kernel.

Trait Implementations

impl Clone for FcdHyper

fn clone(&self) -> FcdHyper

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for FcdHyper

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for FcdHyper

fn default() -> Self

Returns the “default value” for a type.