Struct Cluster 

pub struct Cluster {
    pub cfg: ClusterConfig,
    pub caps: Vec<NodeCaps>,
    pub plan: Vec<Assignment>,
    pub chosen_precision: Option<String>,
    /* private fields */
}

A configured, probed, planned cluster.

Fields

cfg: ClusterConfig

caps: Vec<NodeCaps>

Probed hardware, one per node (same order as cfg.nodes). Empty until Cluster::probe.

plan: Vec<Assignment>

Layer→node plan. Empty until Cluster::plan. Standby stages follow the active ones and mirror their layer ranges.

chosen_precision: Option<String>

Set when the precision ladder had to step down from the stored width.

Implementations

impl Cluster

pub fn from_config(cfg: ClusterConfig) -> Self

pub fn from_path(p: impl AsRef<Path>) -> Result<Self>

pub fn probe(&mut self, remote_bin: &str) -> Result<&[NodeCaps]>

Probe every node’s hardware. remote_bin is the worker binary path on the remote hosts (its --probe mode self-reports); nodes without ssh are probed locally.

pub fn discover( &mut self, remote_bin: &str, ) -> Result<(usize, Vec<(String, String)>)>

Find machines from the [discover] section and append them to cfg.nodes. Returns (added, failures); a host that will not probe is reported rather than fatal, so one dead machine does not stop the rest.

Idempotent: nodes already present (by addr) are left alone, so hand-pinned entries survive and re-running adds nothing.

pub fn autoplan( &mut self, index: &[TensorEntry], opts: &CostOptions, remote_bin: &str, ) -> Result<&[Assignment]>

Discover → probe → cost → plan, in one call.

The DX this module exists for: point it at a checkpoint’s tensor index and get an assignment back, without hand-writing node blocks or working out per-layer bytes. remote_bin is the worker binary on the remote hosts (its --probe mode self-reports).

Discovery is skipped when [discover] is absent, so an all-manual config still works and this is just probe + plan with the cost derived for you.

pub fn retire(&mut self, addr: &str, model: ModelCost) -> Result<&[Assignment]>

Retire a node, moving its work off without a restart.

Marks it Draining and, if a standby is mirroring it, promotes that standby in place — it already holds the same layers, so this is a pointer swap. With no standby the remaining active nodes are re-planned, which works but reloads weights, so a cluster you intend to drain should carry spares.

Re-plans from the caps already probed; call Cluster::probe first if the hardware may have changed.

pub fn active_nodes(&self) -> Vec<&NodeConfig>

Active nodes remaining after any draining/standby ones are excluded.

pub fn kv_profile(&self) -> KvProfile

The KV profile the last plan used, after the objective’s override and unknown-handling are applied.

pub fn plan_table(&self) -> String

The plan as a table — what each machine ends up holding.

Shows the disk column and the predicted per-stage time, because for an MoE those are what the assignment actually turned on; a plan that looks RAM-balanced can still be badly skewed by one node’s slow disk.

pub fn plan(&mut self, model: ModelCost) -> Result<&[Assignment]>

Plan placement from the model cost model + probed caps (probe first).

pub fn worker_argv(&self, i: usize) -> Vec<String>

The worker command-line for node i from its plan + config — so the model crate’s worker binary honours device / precision / rng / kv / layers.

pub fn launch(&self, remote_bin: &str) -> Result<Vec<Child>>

Spawn a worker process per node (ssh for remote, local otherwise). The caller drives them via Cluster::drive, then reaps the children.

pub fn drive(&self, inputs: Vec<NamedTensor>) -> Result<ClusterRun>

Drive one forward through the planned stages, timing each, and collect the output. inputs seeds the first stage (e.g. input_ids). Boundary tensor name is hidden/hidden_in per the model’s stage I/O convention.