Module memory_pool 

Memory pool management

Memory Pool Management for Quantization

This module provides advanced memory pooling capabilities to reduce allocation overhead during quantization operations, particularly beneficial for batch processing and inference scenarios.

Features

• Pre-allocated Pools: Reusable memory pools for common tensor sizes
• Dynamic Sizing: Automatic pool expansion based on usage patterns
• Memory Analytics: Tracking allocation patterns and optimization opportunities
• Thread Safety: Concurrent access for multi-threaded quantization operations

Usage

use torsh_quantization::memory_pool::{MemoryPool, PoolConfig};
use torsh_tensor::Tensor;

// Create a memory pool with configuration
let config = PoolConfig::default();
let mut pool = MemoryPool::new(config);

// Allocate a tensor from the pool
let tensor = pool.allocate_tensor(&[1024, 1024], torsh_core::DType::F32)?;

// Use the tensor for quantization operations
// ... quantization work ...

// Return tensor to pool for reuse
pool.release_tensor(tensor);

Structs

MemoryAnalytics

Memory usage analytics with advanced metrics

MemoryPool

Thread-safe memory pool for tensor allocation

PoolConfig

Configuration for memory pool behavior

PoolUtilizationReport

Detailed pool utilization report