Struct MemoryPlanOptions 

pub struct MemoryPlanOptions {
    pub allocate_params: bool,
    pub allocate_inputs: bool,
    pub allocate_constants: bool,
    pub arena_no_reuse: bool,
    pub pin_output_ancestors: bool,
}

Assign buffers using a greedy best-fit algorithm.

Sorts buffers by size (largest first), then for each buffer finds the smallest free gap in the arena during its live interval. This is a simplified version of XLA’s GlobalDecreasingSizeBestFitHeap. Controls which graph boundaries receive arena slots during planning.

Inference graphs use Self::inference (all boundaries allocated). Backward graphs in a training pair use Self::backward_activations_only: parameters borrow offsets from the forward plan via SharedWeightLayout so weights are not stored twice in the activation arena.

Fields

allocate_params: bool

allocate_inputs: bool

allocate_constants: bool

arena_no_reuse: bool

When true (or env RLX_ARENA_NO_REUSE=1), every tensor gets a unique arena slot.

pin_output_ancestors: bool

When true (default), pin the entire transitive ancestor DAG of the graph outputs to the final step. That’s a conservative guard for out-of-order execution, but it destroys slot reuse on deep feed-forward graphs (the HiFi-GAN decoder hit a 5 GB arena). In-order executors (CPU, wgpu) can set this false: only the output nodes are pinned (read-back protection), which is sufficient and keeps the arena small.

Implementations

impl MemoryPlanOptions

pub fn inference() -> MemoryPlanOptions

pub fn backward_activations_only() -> MemoryPlanOptions

Activations + inputs/constants only; params bound via SharedWeightLayout.

Trait Implementations

impl Clone for MemoryPlanOptions

fn clone(&self) -> MemoryPlanOptions

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Copy for MemoryPlanOptions

impl Debug for MemoryPlanOptions

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter.

impl Default for MemoryPlanOptions

fn default() -> MemoryPlanOptions

Returns the “default value” for a type.

impl Eq for MemoryPlanOptions

impl PartialEq for MemoryPlanOptions

fn eq(&self, other: &MemoryPlanOptions) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl StructuralPartialEq for MemoryPlanOptions