Struct Weights 

pub struct Weights {
    pub cfg: Lfm2Config,
    pub embed: Buffer,
    pub embed_q4: Q4,
    pub embedding_norm: Buffer,
    pub layers: Vec<Layer>,
    pub mtp: Option<MtpHead>,
    pub embed_f16: Option<Buffer>,
    pub embed_split: Option<(Buffer, u32)>,
    pub ple: Option<PleWeights>,
}

Fields

cfg: Lfm2Config

embed: Buffer

[vocab, hidden] f32 — the token-gather row. Gemma-3: pre-scaled by sqrt(hidden) at load (the architecture’s embedding normalizer), so the gather kernel stays scale-free.

embed_q4: Q4

[vocab, hidden] Q4_0 — the tied lm_head (UNscaled).

embedding_norm: Buffer

layers: Vec<Layer>

mtp: Option<MtpHead>

MTP draft head when the checkpoint ships one (qwen3.5-MoE multimodal checkpoints do).

embed_f16: Option<Buffer>

Q1/GGUF (Bonsai-class) models only: an f16 copy of the gather table. The f32 embed stays the source for the buffer-COPY staging paths (no size limit on copies), but a storage BINDING caps at 2 GiB-4 on wgpu-Vulkan — the gather kernels bind THIS table as two row-aligned sub-ranges instead (Metal-only builds could bind the f32 whole; the split keeps one code path for all backends).

embed_split: Option<(Buffer, u32)>

Q1/GGUF models only: the UPPER rows of the f32 staging table + the first row it holds. Vulkan’s max_buffer_size (~4 GiB) cannot hold the 5 GB f32 table in one buffer, and the staging paths COPY rows out by (buffer, offset) — so the table ships as two halves and Self::embed_row_src picks per token. None = embed holds every row.

ple: Option<PleWeights>

Gemma-4 edge: model-level per-layer-embedding weights; None everywhere else.

Implementations

impl Weights

pub fn embed_row_src(&self, tok: u32) -> (&Buffer, u64)

The (buffer, byte offset) holding token tok’s f32 embedding row — the staging-copy sources. Split models (Q1/GGUF, table > max_buffer_size) resolve the half here.

pub fn mtp_engine_weights(&self) -> Option<Weights>

Assemble the MTP draft head as a standalone ONE-LAYER MODEL sharing this model’s embedding and lm_head (wgpu buffers are ref-counted — clones share VRAM): stage_first = false because its residual stream is fed by the fc merge instead of a token gather, stage_last = true for mtp.norm + the shared head. The draft engine therefore reuses every architecture arm of the generic plan builder — a future Gemma-MoE (or dense) draft head takes exactly the same path, no draft-specific kernels or plans.

pub fn load_shard( ctx: &GpuCtx, dir: impl AsRef<Path>, start: usize, end: usize, ) -> Result<Self>

Load only layers start..end for pipeline-parallel sharding (Qwen3 for now): stage 0 (start == 0) owns the f32 embedding gather; the last stage (end == n_layers) owns the final norm + tied Q4 head; middle stages hold neither (1-element dummy buffers — their plan never binds them). Layer indices stay ABSOLUTE in tensor names but local in layers.

pub fn load_shard_bytes( ctx: &GpuCtx, config_json: &[u8], blobs: Vec<Vec<u8>>, start: usize, end: usize, ) -> Result<Self>

Load a shard from in-memory config.json bytes + safetensors blob(s) — the wasm/browser path, where a worker’s weights arrive by shipping (inferencelayer::shard OP_SHIP), never from a filesystem. blobs are whole safetensors files in order (single- or HF-sharded).

pub fn load(ctx: &GpuCtx, dir: impl AsRef<Path>) -> Result<Self>

Load config.json + the checkpoint from dir, honouring the weight-precision policy in OSFKB_DECODE_PRECISION (default all-Q4 — see Precision).

pub fn load_with_precision( ctx: &GpuCtx, dir: impl AsRef<Path>, precision: Precision, ) -> Result<Self>

Load with an explicit weight-precision policy. Quantization is a speed/fidelity trade the CALLER owns: 4-bit weights are the dominant decode-bandwidth lever, but they cost real accuracy (a 270M gemma-3 loses ~2× the per-step agreement a 1B does, measured against an f32 reference), so a caller serving prose or classification may want f16 where a caller serving grammar-constrained output does not.

A precision this build cannot SERVE is REFUSED rather than silently served as Q4 — shipping a model the caller did not ask for is the one outcome a precision knob must never produce.

Servable today: uniform q4 (the default, byte-for-byte historical) and uniform f16 (the reference arm the quantization ruler measures against — see crate::kld). MIXED policies and any Q8 group still refuse: the mixed case needs a per-SITE bind-group choice in the plan rather than one family switch, because the f16 kernels take four bindings where the Q4 kernels take five, so a mixed plan cannot share one bind-group shape.