Crate rlx_distributed 

Multi-node distributed inference for RLX models.

Layered on top of rlx-driver’s transports (TcpTransport, ThunderboltTransport, and rlx-mlx’s MlxTransport) and ProcessGroup:

• config — hosts.json parsing + process-group construction (DistConfig::connect).
• partition — pipeline-parallel layer assignment (pipeline_layer_range, block_role).
• pipeline — the model-agnostic relay (PipelineCoordinator) driving per-rank BlockRunners.

A model family provides a BlockRunner for its layer block (e.g. rlx_qwen3’s Qwen3PipelineStage); everything else here is reusable.

use rlx_distributed::{DistConfig, ParallelMode, PipelineCoordinator};

let cfg = DistConfig::load("hosts.json", Some(rank), ParallelMode::Pipeline)?;
let group = cfg.connect()?;                 // blocks until the mesh forms
let coord = PipelineCoordinator::new(group);

let mut tokens = vec![/* prompt ids */];
for _ in 0..32 {
    let tok = coord.forward_step(&mut runner, &tokens, |logits| argmax(logits))?;
    tokens.push(tok);
}
coord.barrier()?;                           // before dropping the group

Re-exports

pub use config::DistConfig;

pub use config::Hostfile;

pub use config::ParallelMode;

pub use config::TransportBackend;

pub use launch::LocalCluster;

pub use launch::WorkerArgs;

pub use launch::free_loopback_ports;

pub use launch::worker_args;

pub use partition::BlockRole;

pub use partition::block_role;

pub use partition::pipeline_layer_range;

pub use pipeline::BlockInput;

pub use pipeline::BlockOutput;

pub use pipeline::BlockRunner;

pub use pipeline::PipelineCoordinator;

Modules

config

Launch configuration: hostfile parsing + process-group construction.

launch

Local multi-process launcher — run an N-rank cluster on one host over loopback TCP, with no hand-written hosts.json and no N terminals.

partition

Pipeline-parallel layer partitioning.

pipeline

Pipeline-parallel forward orchestration.

Structs

NetTransport

Full-mesh TCP transport implementing both Transport (two-sided send/recv) and SymmetricTransport (one-sided put/get/barrier). Construct via TcpTransport::bind or ThunderboltTransport::bind.

ProcessGroup

A handle to the collective-communication world: a rank, a size, and the transport that connects them.

TcpTransport

Portable TCP transport. Works over any reachable IP — Ethernet or the macOS Thunderbolt Bridge link. This is the baseline that pipeline- and tensor-parallel inference run on before any fast path.

ThunderboltTransport

Thunderbolt transport. Same wire protocol as TcpTransport today (TCP over the Thunderbolt Bridge IP link), but a distinct type so the symmetric one-sided heap path is the intended interface and so a future zero-copy Thunderbolt DMA backend can replace the engine behind the unchanged Transport + SymmetricTransport traits.

Enums

ReduceKind

Element-wise reduction operator for collectives.

Traits

Transport

Two-sided, tagged, byte-oriented point-to-point transport between world_size ranks.