Module memory 

Advanced Memory Optimization for Geospatial Data

This module provides comprehensive memory management features optimized for geospatial workloads, including:

• Custom allocators (slab, buddy) for efficient tile allocation
• Memory-mapped I/O with read-ahead and write-behind
• Zero-copy data transfers between operations
• Arena allocators for batch processing
• Memory pools for buffer reuse
• NUMA-aware allocation for multi-socket systems
• Huge pages support for large datasets

Performance Targets

• 50% reduction in memory usage for large datasets
• 2-3x faster allocation for small objects
• Zero-copy transfers where possible
• NUMA local access > 90%

Example

use oxigdal_core::memory::{Pool, PoolConfig};
use oxigdal_core::error::Result;

// Create a memory pool for 4KB buffers
let config = PoolConfig::new()
    .with_size_class(4096, 100);
let pool = Pool::with_config(config)?;

// Allocate a buffer from the pool
let buffer = pool.allocate(4096)?;

Re-exports

pub use allocator::Allocator;

pub use allocator::AllocatorStats;

pub use allocator::BuddyAllocator;

pub use allocator::SlabAllocator;

pub use allocator::ThreadLocalAllocator;

pub use arena::Arena;

pub use arena::ArenaPool;

pub use arena::ArenaStats;

pub use hugepages::HugePageAllocator;

pub use hugepages::HugePageConfig;

pub use hugepages::HugePageSize;

pub use hugepages::HugePageStats;

pub use mmap::MemoryMap;

pub use mmap::MemoryMapConfig;

pub use mmap::MemoryMapMode;

pub use numa::NumaAllocator;

pub use numa::NumaConfig;

pub use numa::NumaNode;

pub use numa::NumaStats;

pub use pool::Pool;

pub use pool::PoolConfig;

pub use pool::PoolStats;

pub use zero_copy::SharedBuffer;

pub use zero_copy::ZeroCopyBuffer;

pub use zero_copy::ZeroCopyConfig;

Modules

allocator

Custom Memory Allocators for Geospatial Data

arena

Arena Allocators for Batch Operations

hugepages

Huge Pages Support

mmap

Optimized Memory-Mapped I/O

numa

NUMA-Aware Memory Allocation

pool

Memory Pool Management

zero_copy

Zero-Copy Data Transfers

Constants

GPU_ALIGNMENT

Memory alignment for GPU transfers (256 bytes)

HUGE_PAGE_SIZE

Huge page size (2MB on most systems)

MAX_POOL_SIZE

Maximum pool size before compaction (1GB)

PAGE_SIZE

Default page size (4KB)

SIMD_ALIGNMENT

Memory alignment for SIMD operations (64 bytes for AVX-512)