Struct Gemma4Engine 

pub struct Gemma4Engine {
    pub weights: Gemma4Weights,
    pub hp: Gemma4Hparams,
}

Fields

weights: Gemma4Weights

hp: Gemma4Hparams

Implementations

impl Gemma4Engine

pub fn probe_kernels(&self)

See crate::decoder::Decoder::probe_kernels. Called at the end of loading, before the registry is sealed.

This engine implements Engine::forward_token and nothing else: there is no batched path, so a 512-token prefill is 512 sequential single-token passes and every projection is a batch = 1 matvec. On Metal that is one command buffer, one commit and one wait per projection per token, which is why Gemma-4-E2B measures slower on Metal than on CPU while producing correct output. The registry records that as a missing engine capability rather than leaving it to be rediscovered from a benchmark.

pub fn new_state(&self) -> Gemma4DecodeState

Trait Implementations

impl Engine for Gemma4Engine

type State = Gemma4DecodeState

fn new_state(&self) -> Gemma4DecodeState

Builds fresh (empty) per-layer state for a new request.

fn vocab_size(&self) -> usize

fn forward_token( &self, token_id: usize, pos: usize, state: &mut Self::State, ) -> Vec<f32>