Struct EmbedHost 

pub struct EmbedHost {
    pub raw: Vec<u8>,
    pub ggml_type: GgmlType,
    pub n_embd: usize,
}

Host-resident embedding table for row gather (dequant only the needed token rows).

Fields

raw: Vec<u8>

ggml_type: GgmlType

n_embd: usize

Implementations

impl EmbedHost

pub fn from_gguf(g: &GgufFile, name: &str) -> Self

pub fn from_source(src: &dyn TensorSource, name: &str) -> Self

pub fn qt_and_row_bytes(&self, n_embd: usize) -> (i32, usize)

QT int + row_bytes for this embed table’s dtype (for the device embed-gather kernel). CUDA-GRAPH-PLAN Phase 1. Mirrors the GpuTensor qtype mapping.

pub fn rows(&self, n_embd: usize) -> usize

Rows this embedding table actually holds — the ONLY bound that matters for a gather, and not the same number as cfg.vocab_size on a padded table.

pub fn try_gather( &self, n_embd: usize, tokens: &[u32], ) -> Result<Vec<f32>, String>

Gather rows for tokens -> [T, n_embd] f32, dequant per row from raw bytes, REFUSING an out-of-range id by name.

WHY THIS IS A RESULT AND NOT A SLICE. 2026-09-02, 2x B200, boot D (MEMRA_PRIME_CHUNK=4096 MEMRA_MOE_F16G=1): the mode-1 expert GEMM corrupted the trunk, the logits went bad, the sampler handed back an id near i32::MAX, and this function’s self.raw[off..off + row_bytes] panicked with range start index 17592186036224 out of range. That panic took the GPU WORKER down, the respawn hit it again, and every request after it was connection refused — one numeric bug in one door became a fleet outage (research/glm5-b200-20260902/box/prefill/, and LAW: engine panics are fleet-fatal). The numeric bug is fixed at its own site; this guard is the reason the NEXT one costs a request instead of a fleet. The message names the id, the bound and the likely upstream, because a bounds error with no attribution sends the next reader to the wrong file.

pub fn gather(&self, n_embd: usize, tokens: &[u32]) -> Vec<f32>

Self::try_gather for the call sites that cannot return an error (probes, gates). Serving paths take try_gather — see its header for why the difference is load-bearing.