Skip to main content

post

Function post 

Source
pub fn post(
    e: &Engine,
    topology: &HyperTopology,
    f: &CudaSlice<f32>,
    residual: &CudaSlice<f32>,
    mix: &HcMix,
    t: usize,
    hidden: usize,
) -> Result<CudaSlice<f32>, Box<dyn Error>>
Expand description

One site’s post-branch half (hc_post): out[t, k, :] = post[t, k]·f[t, :] + Σ_j comb[t, j, k]·residual[t, j, :]. residual is the site’s INPUT stream state, not the layer’s — the MLP site’s residual is the attention site’s output.