Skip to main content

forward_token_graph

Function forward_token_graph 

Source
pub fn forward_token_graph(
    _model: &Arc<CmfModel>,
    _kv_id: u64,
    _layers: &[GraphLayer<'_>],
    _invf: &[f32],
    _h: &mut [f32],
    _nh: usize,
    _nkv: usize,
    _hd: usize,
    _rd: usize,
    _hidden: usize,
    _inter: usize,
    _position: usize,
    _cap: usize,
    _gemma: bool,
    _eps: f32,
    lm_head: Option<(&GraphW<'_>, usize)>,
    final_norm: &[f32],
    logits: &mut Vec<f32>,
) -> bool
Expand description

Whole-token decode graph on wgpu: the entire layer stack in ONE submit, hidden resident, one readback. Updates h in place. false = refusal.