Crate haagenti_neural 

Neural Compression

10x compression using learned codebooks for model weights.

Architecture

┌─────────────────────────────────────────────────────────────────┐
│                    Neural Compression                            │
├─────────────────────────────────────────────────────────────────┤
│                                                                  │
│  Traditional: weights.safetensors (16GB)                        │
│               ↓ LZ4                                              │
│               weights.hct (8GB) — 2:1 compression               │
│                                                                  │
│  Neural:      weights.safetensors (16GB)                        │
│               ↓ Learned Encoder                                  │
│               weights.nct (1.6GB) — 10:1 compression            │
│               ↓ Learned Decoder (GPU)                            │
│               weights (reconstructed)                            │
│                                                                  │
│  ┌──────────────────────────────────────────────────────────┐   │
│  │              Compression Pipeline                         │   │
│  │                                                           │   │
│  │  Encoder (offline, CPU):                                 │   │
│  │  ┌─────────┐    ┌─────────┐    ┌─────────┐              │   │
│  │  │ Weight  │ -> │ Vector  │ -> │ Codebook│ -> indices   │   │
│  │  │ Tensor  │    │ Quantize│    │ Lookup  │              │   │
│  │  └─────────┘    └─────────┘    └─────────┘              │   │
│  │                                                           │   │
│  │  Decoder (runtime, GPU):                                 │   │
│  │  ┌─────────┐    ┌─────────┐    ┌─────────┐              │   │
│  │  │ Indices │ -> │ Codebook│ -> │ Residual│ -> weights   │   │
│  │  │         │    │ Lookup  │    │ Refine  │              │   │
│  │  └─────────┘    └─────────┘    └─────────┘              │   │
│  └──────────────────────────────────────────────────────────┘   │
└─────────────────────────────────────────────────────────────────┘

Layer-Specific Codebooks

Different layer types use different codebook sizes:

• Attention Q/K: 4096 centroids, 12-bit indices
• Attention V/O: 8192 centroids, 13-bit indices
• FFN: 2048 centroids, 11-bit indices
• Normalization: 256 centroids, 8-bit (lossless)

Modules

prelude

Prelude for common imports

Structs

Codebook

A codebook containing centroids for vector quantization

CodebookConfig

Configuration for a codebook

CodebookStats

Statistics from codebook training

CodebookTrainer

Codebook trainer using K-means clustering

DecoderConfig

Configuration for the decoder

EncodedTensor

Encoded tensor data

EncoderConfig

Configuration for the encoder

GpuDecoder

GPU-accelerated decoder (placeholder for CUDA implementation)

LayerCodebook

Collection of codebooks for different layer types

NctFile

NCT file container

NctHeader

NCT (Neural Compressed Tensor) file header.

NctMetadata

NCT file metadata

NeuralDecoder

Neural decoder for model decompression

NeuralEncoder

Neural encoder for model compression

RefinerConfig

Configuration for residual refiner

ResidualRefiner

Residual refiner for improving reconstruction quality

TrainingConfig

Configuration for codebook training

Enums

LayerType

Layer type for codebook selection

NeuralError

Errors that can occur during neural compression

Constants

DEFAULT_COMPRESSION_RATIO

Default compression ratio target

NCT_EXTENSION

File extension for neural compressed tensors

NCT_MAGIC

Magic bytes for NCT format

Type Aliases

Result

Result type for neural compression operations