Expand description
Inference Memory Tracker
Tracks memory allocations made during inference sessions, detecting leaks, peaks, and per-rule memory usage.
Structs§
- Inference
Memory Tracker - Tracks memory allocations made during a single inference session.
- Memory
Snapshot - A point-in-time view of tracked memory usage.
- Region
Info - Metadata about a single allocated memory region.
- Tracker
Stats - Aggregate counters for an
InferenceMemoryTrackersession.
Enums§
- Alloc
Event - An event that can be recorded by
InferenceMemoryTracker.