Module embedding_compression_codec 

Embedding Compression Codec

Production-quality codec for compressing dense embedding vectors using multiple pure-Rust algorithms: scalar quantization, product quantization, delta coding, run-length encoding, and hybrid PQ+RLE.

Supported Methods

• ScalarQuantization — uniform min-max quantization to N bits
• ProductQuantization — split into subvectors, quantize each independently
• DeltaCoding — delta-encode sorted values, then scalar quantize
• RunLengthEncoding — RLE on quantized values
• HybridPQ — product quantization + RLE on residuals

Example

use ipfrs_semantic::{EmbeddingCompressionCodec, EccMethod, EccCodecConfig};

let mut codec = EmbeddingCompressionCodec::new();
let id = codec.register_codec("my-sq8", EccMethod::ScalarQuantization, 8, 64);
let embedding = vec![0.1f64; 128];
let compressed = codec.compress(id, &embedding).unwrap();
let decompressed = codec.decompress(&compressed).unwrap();
let mse = codec.reconstruction_error(&embedding, &decompressed);
assert!(mse < 1e-3);

Structs

EccCodecConfig

Configuration for creating a new codec.

EccCodecSpec

Registered codec specification stored in the codec registry.

EccCodecStats

Aggregate statistics returned by EmbeddingCompressionCodec::codec_stats.

EccCompressed

Output of a compression operation.

EccCompressionRecord

One entry in the codec’s compression audit log.

EmbeddingCompressionCodec

Codec for compressing dense embedding vectors.

Enums

EccError

Errors produced by EmbeddingCompressionCodec.

EccMethod

Compression algorithm used by a codec.

Type Aliases

EccCodecId

Numeric identifier for a registered codec.

EccEmbeddingCompressionCodec

Type alias for the codec itself.