Skip to main content

forward_batch_graph

Function forward_batch_graph 

Source
pub fn forward_batch_graph(
    _model: &Arc<CmfModel>,
    _kv_id: u64,
    _layers: &[GraphLayer<'_>],
    _invf: &[f32],
    _h: &mut [f32],
    _nh: usize,
    _nkv: usize,
    _hd: usize,
    _rd: usize,
    _hidden: usize,
    _inter: usize,
    _positions: &[usize],
    _cap: usize,
    _gemma: bool,
    _eps: f32,
    _k: usize,
) -> bool
Expand description

Batched prefill: k contiguous positions through the whole graph in one submit (projections/FFN as GEMMs, attention/GDN looped over scratch). h is [k·hidden] in/out; positions len k. wgpu only.