Skip to main content

pre_exact

Function pre_exact 

Source
pub fn pre_exact(
    e: &Engine,
    topology: &HyperTopology,
    site: &HyperSite,
    x: &CudaSlice<f32>,
    t: usize,
    hidden: usize,
) -> Result<(CudaSlice<f32>, HcMix), Box<dyn Error>>
Expand description

pre with the DECODE-EXACT mixing GEMM: each token’s mix coefficients come from the SAME m=1 cuBLASLt program the serial T=1 decode step runs (linear_t1_into is linear at m == 1 on a row view — same config, same weight pointer, same input bytes), instead of one m=t call whose n-dependent reduction split changes every output bit (the lt_ndep probe documented on Engine::linear_decode_exact). Everything after the GEMM is the per-token kernel set pre already runs — block-per-token programs whose per-token bytes do not depend on t. This is the entry the BATCHED hyper decode walk uses so that row b of a B-row tick is bit-identical to that session’s solo decode_step_hyper step.