Struct LoadBalancer 

pub struct LoadBalancer { /* private fields */ }

Load balancer for tensor distribution

Implementations

impl LoadBalancer

pub fn new(cluster: Arc<ClusterState>, config: LoadBalanceConfig) -> Self

Create new load balancer

pub fn with_runtime_profile( cluster: Arc<ClusterState>, profile: &RuntimeProfile, ) -> Self

Create a load balancer with configuration derived from the local runtime profile.

pub fn config(&self) -> LoadBalanceConfig

Access the current configuration.

pub fn distribute_layers( &self, layers: &[LayerSpec], ) -> Result<LoadDistributionPlan, String>

Distribute tensor layers across nodes based on VRAM capacity.

Optimized using cursor-based index traversal instead of costly vector draining and shifting elements, improving complexity from O(N^2) to O(N). Further optimized by sorting references to NodeResources instead of cloning them, and dynamically avoiding layer cloning/sorting if they are already in sequential order.

pub fn distribute_layers_with_runtime_profile( &self, layers: &[LayerSpec], profile: &RuntimeProfile, ) -> Result<LoadDistributionPlan, String>

Distribute layers and directly chunk large node allocations to match worker parallelism.

OPTIMIZATION: Delegates directly to distribute_layers_internal with Some(max_layers_per_slice) to generate chunked tensor slices in a single pass during greedy assignment. This completely eliminates intermediate single-slice vector allocations (vec![slice]), avoids intermediate LoadDistributionPlan construction, and bypasses multi-pass post-processing traversals.

pub fn rebalance(&self) -> bool

Rebalance load based on current node metrics

pub fn shed_load(&self) -> Vec<(String, String)>

Shed load from overloaded nodes to underloaded ones.

OPTIMIZATION: Collects overloaded and underloaded node candidates in a single-pass loop over cluster metrics under lock, completely eliminating intermediate active_nodes vector allocations and inlining best target lookup.

pub fn reconfigure_from_system_profile(&self, profile: &SystemProfile)

Re-tune load-balancing thresholds from a system profile detected at runtime.

pub fn subscribe_to_watcher( self: &Arc<Self>, watcher: &SystemProfileWatcher, ) -> JoinHandle<()>

Subscribe to a SystemProfileWatcher’s broadcast channel and automatically re-tune whenever hardware changes are detected.

Returns a JoinHandle that can be cancelled by dropping it.

pub fn distribute_with_deadlock_prevention( &self, layers: &[LayerSpec], ) -> Result<LoadDistributionPlan, String>

Distribute layers with deadlock prevention

Trait Implementations

impl Clone for LoadBalancer

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for LoadBalancer

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.