Struct VectorCompressor 

pub struct VectorCompressor { /* private fields */ }

High-performance vector compressor for embedding data.

This struct provides methods to compress and decompress vectors using various algorithms to reduce storage space and improve I/O performance. It maintains statistics about compression ratios and processing performance.

Examples

use turboprop::compression::{VectorCompressor, CompressionConfig, CompressionAlgorithm};

// Create a compressor with scalar quantization
let config = CompressionConfig {
    algorithm: CompressionAlgorithm::ScalarQuantization,
    quantization_bits: 8,
    enable_delta_compression: false,
    clustering_threshold: 0.95,
    kmeans_iterations: 10,
    subvector_size: 8,
    codebook_size: 256,
};
let mut compressor = VectorCompressor::new(config);

// Compress a batch of vectors
let vectors = vec![
    vec![1.0, 2.0, 3.0],
    vec![4.0, 5.0, 6.0],
];
let compressed = compressor.compress_batch(&vectors).unwrap();

// Decompress back to original form
let decompressed = compressor.decompress_batch(&compressed).unwrap();

println!("Compression ratio: {:.2}%", compressor.stats().compression_percentage());

Implementations

impl VectorCompressor

pub fn new(config: CompressionConfig) -> Self

Create a new vector compressor with the specified configuration.

Arguments

• config - Configuration specifying the compression algorithm and parameters

Examples

use turboprop::compression::{VectorCompressor, CompressionConfig, CompressionAlgorithm};

// Create compressor for scalar quantization
let config = CompressionConfig {
    algorithm: CompressionAlgorithm::ScalarQuantization,
    quantization_bits: 8,
    enable_delta_compression: false,
    clustering_threshold: 0.95,
    kmeans_iterations: 10,
    subvector_size: 8,
    codebook_size: 256,
};
let compressor = VectorCompressor::new(config);

pub fn compress_batch( &mut self, vectors: &[Vec<f32>], ) -> Result<Vec<CompressedVector>>

Compress a batch of vectors using the configured algorithm.

This method processes multiple vectors simultaneously for better performance. The compression algorithm and parameters are determined by the configuration provided during construction.

Arguments

• vectors - Slice of vectors to compress (each vector should have the same dimension)

Returns

• Result<Vec<CompressedVector>> - Vector of compressed representations

Errors

Returns an error if:

• Vectors have inconsistent dimensions
• Compression algorithm fails due to invalid parameters
• Insufficient memory for processing

Examples

use turboprop::compression::{VectorCompressor, CompressionConfig, CompressionAlgorithm};

let config = CompressionConfig::default();
let mut compressor = VectorCompressor::new(config);

let vectors = vec![
    vec![1.0, 2.0, 3.0, 4.0],
    vec![5.0, 6.0, 7.0, 8.0],
];

match compressor.compress_batch(&vectors) {
    Ok(compressed) => {
        println!("Compressed {} vectors", compressed.len());
    },
    Err(e) => eprintln!("Compression failed: {}", e),
}

pub fn decompress_batch( &self, compressed: &[CompressedVector], ) -> Result<Vec<Vec<f32>>>

Decompress a batch of vectors

pub fn stats(&self) -> &CompressionStats

Get compression statistics