Crate haagenti_hct 

Haagenti Compressed Tensor (HCT) Format

High-performance compressed tensor storage for neural network weights, with HoloTensor holographic compression support for progressive loading.

Overview

HCT provides two complementary storage modes:

• Standard HCT: Block-compressed tensor storage with random access
• HoloTensor: Holographic compression enabling progressive reconstruction

Standard HCT Format

Block-based compression with LZ4 or Zstd for fast random access:

use haagenti_hct::{HctWriter, HctReader, CompressionAlgorithm, DType};
use std::fs::File;

// Write compressed tensor
let mut writer = HctWriter::new(
    File::create("weights.hct")?,
    CompressionAlgorithm::Zstd,
    DType::F16,
    &[4096, 4096],
)?;
writer.write_data(&weight_data)?;
writer.finish()?;

// Read tensor
let mut reader = HctReader::open("weights.hct")?;
let data = reader.read_all()?;

HoloTensor Format

Holographic compression enables progressive reconstruction from partial data:

use haagenti_hct::{
    HoloTensorEncoder, HoloTensorDecoder,
    HolographicEncoding, DType,
};

// Encode with spectral holography (8 fragments)
let encoder = HoloTensorEncoder::new(HolographicEncoding::Spectral)
    .with_fragments(8);
let (header, fragments) = encoder.encode_1d(&weights)?;

// Reconstruct from partial fragments (any 4 of 8 for ~90% quality)
let mut decoder = HoloTensorDecoder::new(header);
decoder.add_fragment(fragments[0].clone())?;
decoder.add_fragment(fragments[3].clone())?;
decoder.add_fragment(fragments[5].clone())?;
decoder.add_fragment(fragments[7].clone())?;

let approx_data = decoder.reconstruct()?;

Encoding Schemes

Scheme	Best For	Min Quality	Progressive
Spectral (DCT)	Dense MLP weights	60%	Smooth curve
Random Projection	High-dimensional	10%	Linear curve
Low-Rank Distributed	Attention layers	30%	Sharp knee

Feature Flags

• lz4 - LZ4 compression for base blocks
• zstd - Zstd compression for better ratios
• full - All features (default)

Re-exports

pub use tensor::DEFAULT_BLOCK_SIZE;

pub use tensor::FLAG_BLOCK_CHECKSUMS;

pub use tensor::FLAG_HEADER_CHECKSUM;

pub use tensor::FLAG_HOLOGRAPHIC;

pub use tensor::FLAG_QUANTIZATION;

pub use tensor::FLAG_TENSOR_NAME;

pub use tensor::HCT_MAGIC;

pub use tensor::HCT_VERSION;

pub use tensor::HCT_VERSION_V2;

pub use tensor::BlockIndex;

pub use tensor::CompressionAlgorithm;

pub use tensor::DType;

pub use tensor::HctHeader;

pub use tensor::BlockIndexV2;

pub use tensor::QuantizationMetadata;

pub use tensor::QuantizationScheme;

pub use tensor::HctReader;

pub use tensor::HctReaderV2;

pub use tensor::HctWriter;

pub use tensor::HctWriterV2;

pub use tensor::compress_file;

pub use tensor::ChecksumError;

pub use tensor::CompressionStats as HctCompressionStats;

pub use holotensor::HOLO_FLAG_ESSENTIAL_FIRST;

pub use holotensor::HOLO_FLAG_FRAGMENT_CHECKSUMS;

pub use holotensor::HOLO_FLAG_HEADER_CHECKSUM;

pub use holotensor::HOLO_FLAG_INTERLEAVED;

pub use holotensor::HOLO_FLAG_QUALITY_CURVE;

pub use holotensor::HOLO_FLAG_QUANTIZATION;

pub use holotensor::HOLO_MAGIC;

pub use holotensor::HOLO_VERSION;

pub use holotensor::FragmentIndexEntry;

pub use holotensor::HoloFragment;

pub use holotensor::HolographicEncoding;

pub use holotensor::QualityCurve;

pub use holotensor::HoloTensorHeader;

pub use holotensor::SeededRng;

pub use holotensor::SpectralDecoder;

pub use holotensor::SpectralEncoder;

pub use holotensor::RphDecoder;

pub use holotensor::RphEncoder;

pub use holotensor::LrdfDecoder;

pub use holotensor::LrdfEncoder;

pub use holotensor::HoloTensorDecoder;

pub use holotensor::HoloTensorEncoder;

pub use holotensor::HoloTensorReader;

pub use holotensor::HoloTensorWriter;

pub use holotensor::decode_from_file;

pub use holotensor::decode_from_file_progressive;

pub use holotensor::encode_to_file;

pub use holotensor::open_holotensor;

pub use holotensor::read_holotensor;

pub use holotensor::write_holotensor;

Modules

holotensor

HoloTensor: Holographic Compression for Neural Network Weights

prelude

Prelude module for common imports.

tensor

Compressed Tensor Format (.hct) for LLM weight storage.

Enums

Error

Compression error types.

Functions

dct_1d

1D Discrete Cosine Transform Type-II using FFT.

dct_2d

2D DCT via separable 1D transforms (row then column).

idct_1d

1D Inverse Discrete Cosine Transform Type-II (aka DCT-III).

idct_2d

2D IDCT via separable 1D transforms.

Type Aliases

Result

Result type alias for compression operations.