Struct ResourceBudget 

pub struct ResourceBudget {
    pub max_ram_bytes: Option<usize>,
    pub max_resident_experts: Option<usize>,
}

How much a model may hold resident. None fields mean “unbounded / derive”.

Fields

max_ram_bytes: Option<usize>

Max resident weight RAM (bytes). None → the physical-RAM-derived soft budget (soft_memory_budget_bytes).

max_resident_experts: Option<usize>

Max MoE experts resident PER LAYER. None → derive from max_ram_bytes (or all experts if RAM is unbounded).

Implementations

impl ResourceBudget

pub const UNBOUNDED: Self

Unbounded: full RAM, all experts resident.

pub fn from_env() -> Self

Read from the environment. RLX_MAX_RAM_BYTES and RLX_MAX_RESIDENT_EXPERTS override; absent fields fall back to derivation at query time.

pub fn effective_ram_bytes(&self) -> Option<usize>

The effective RAM ceiling: explicit max_ram_bytes, else the soft physical-RAM budget (None only when RAM is unknown and unset).

pub fn resident_experts( &self, num_experts: usize, bytes_per_expert: usize, backbone_bytes: usize, ) -> usize

How many experts to keep resident PER LAYER: honor an explicit max_resident_experts, else derive from the RAM ceiling by reserving backbone_bytes (attention/norms/embed) and dividing the remainder by bytes_per_expert. Always in 1..=num_experts.

pub fn expert_pool_config( &self, num_experts: usize, bytes_per_expert: usize, backbone_bytes: usize, refresh: ExpertRefreshPolicy, ) -> ExpertPoolConfig

Build an ExpertPoolConfig honoring this budget (offload kicks in when the resident count is below num_experts).

pub fn needs_offload(&self, model_bytes: usize, num_experts: usize) -> bool

Whether weight offload/streaming should engage: the model doesn’t fit the RAM ceiling, or an explicit expert cap is below the expert count.

Trait Implementations

impl Clone for ResourceBudget

fn clone(&self) -> ResourceBudget

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Copy for ResourceBudget

impl Debug for ResourceBudget

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for ResourceBudget

fn default() -> ResourceBudget

Returns the “default value” for a type.

impl Eq for ResourceBudget

impl PartialEq for ResourceBudget

fn eq(&self, other: &ResourceBudget) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl StructuralPartialEq for ResourceBudget