Module model 

Dense transformer model: loads GGUF weights to GPU (Stage-1: dequant→f32), runs the shared full-attention + SwiGLU forward graph. Arch-agnostic via ModelConfig; this path is exactly the dense-transformer graph (qwen3) and the full-attention layers of hybrids.

Structs

EmbedHost

Host-resident embedding table for row gather (dequant only the needed token rows).

ExpertLayout

One layer’s stacked 256-expert tensor, raw GGUF quant bytes held HOST-RESIDENT.

Fp8BlockScales

Device-resident block-128 weight-scale grid for an e4m3 operand (B1b, lane fp8st 2026-08-03).

Fp8Weight

FP8-native prefill operand: raw checkpoint e4m3 codes [out_f, in_f] row-major (EXACT — the weight side of the FP8 GEMM does no re-quantization) + its weight scale(s). Per-tensor class: scale is the dequant scalar folded into the GEMM’s scale pointer together with the per-batch activation scale, blk == None. Block-128 class (Qwen official FP8): blk == Some and scale == 1.0 — see Fp8BlockScales for the resident layout contract.

HostExps

Layer

Model

Enums

GpuTensor

A weight tensor resident on GPU. Quantized weights stay in GGUF block bytes (Quant); small non-quant tensors (norms, sometimes embed/lm_head) are kept dequantized as f32 (Float). This keeps VRAM ~= on-disk quant size (fixes the f32-on-load OOM).

HostBuf

One layer’s stacked 256-expert tensor, raw GGUF quant bytes held HOST-RESIDENT.

Functions

full_prec_enabled

FULL-PRECISION LOADER MODE (MEMRA_FULL_PREC=1, default OFF — MTP-heal research platform). Bypasses the standing loader law (large BF16/F8 -> Q8_0/NVFP4 re-encode, the “Float-poison” tripwire). Under this flag every weight loads as Float and compute rides the Stage-A f32 oracle path end to end — SLOW IS FINE, this mode exists for exactness (the MTP acceptance CEILING at full precision), not speed. Large 2D matmul weights stay bf16-resident (GpuTensor::FloatBf16) with dequant-on-use so the 9B (~18GB bf16) + f32 activations fit 24GB instead of blowing to ~38GB as an all-f32 materialization. The Float-poison tripwire warnings are CORRECT behavior here and are suppressed. See docs/FLAGS.md and HANDOVER “MEMRA DUAL-SHAPE”.

repack_nvfp4_split

Host-side split-plane repack of NVFP4 GGUF block bytes (A6). Input: out_f rows of in_f/64 36-byte blocks ([4B UE4M3 scales][32B packed e2m1]). Output (same length): quant plane (out_f x nsb64 x 32B) followed by scale plane (out_f x nsb64 x 4B). Pure byte permutation.

residency_census_report

Human-readable residency census: one line per qtype that took at least one 2D weight, plus a total. Callers print it right after load — see run-gen’s MEMRA_RESIDENCY_CENSUS=1.

rp_enabled

A6 repack seam: default ON, MEMRA_RP=0 restores the GGUF block layout everywhere (rollback/A-B).

unpack_nvfp4_split

Inverse of repack_nvfp4_split (the roundtrip gate).

Type Aliases

TensorMap