Expand description
Memory management interfaces for device memory operations
This module provides device memory management abstractions, separate from KV cache management. It handles raw memory allocation, transfers, and memory pool management across different devices.
Structs§
- Allocation
Requirements - Requirements for memory allocation
- Best
FitStrategy - Best-fit allocation strategy
- Defragmentation
Stats - Defragmentation statistics
- Event
Handle - Event handle for stream synchronization
- Memory
Access Stats - Memory access pattern statistics
- Memory
Handle - Memory handle representing allocated memory
- Memory
Handle Info - Information about a memory handle
- Memory
Info - Memory information for a device
- Memory
Manager Config - Memory manager configuration
- Memory
Pool Config - Memory pool configuration
- Memory
Transfer - Memory transfer specification
- Stream
Handle - Stream handle for asynchronous operations
Enums§
- Access
Pattern Type - Detected access patterns
- Memory
Access - Memory access modes
- Memory
Pressure - Memory pressure levels
- Memory
Type - Memory types for different use cases
- Memory
Usage Hint - Memory usage hints for optimization
Traits§
- Advanced
Memory Manager - Advanced memory operations
- Allocation
Strategy - Memory allocation strategy
- Device
Memory Manager - Device memory manager for raw memory operations
- Global
Memory Monitor - Global memory monitor for system-wide memory tracking
- Memory
Manager Factory - Memory manager factory
- Stream
Manager - Stream manager for asynchronous operations