Expand description
Exact compressed-tensors W4 adapter for the Marlin physical ABI.
The supported checkpoint subset is intentionally narrow: pack-quantized INT4 weights, fixed group size, optional format-typed asymmetric zero points, and no activation quantization. Repacking is cold-path CPU work performed while static plan resources are initialized.
Structs§
- Compressed
Tensors Marlin Safetensors Source - Mmap-backed safetensors archive with an exact compressed-tensors-to-Marlin adapter. Dense components keep the archive’s zero-copy/transcode behavior.