mpi-rs — a pure-Rust Message Passing Interface

A from-scratch, pure-Rust implementation of MPI whose public API mirrors rsmpi (the mpi crate) so that existing rsmpi programs compile and run against it with little or no change.

The crucial difference from rsmpi: there is no C library. rsmpi is a set of FFI bindings that link against Open MPI or MPICH. mpi-rs implements the MPI runtime itself — process bootstrap, the network transport, and every collective algorithm — in safe Rust, on top of the standard library only (zero external dependencies in the default build).

rsmpi:  your code ── mpi (Rust bindings) ──FFI──▶ libmpi.so (C: Open MPI / MPICH)
mpi-rs: your code ── mpi (pure Rust: transport + collectives + launcher)

The crate publishes as mpi-rs but its library is named mpi, so code keeps using use mpi::traits::*;:

[dependencies]
mpi-rs = "0.1"

Quick start

use mpi::traits::*;

fn main() {
    let universe = mpi::initialize().unwrap();
    let world = universe.world();
    let size = world.size();
    let rank = world.rank();

    if rank == 0 {
        let msg = [1.0f64, 2.0, 3.0];
        world.process_at_rank(1).send(&msg[..]);
    } else if rank == 1 {
        let (msg, status) = world.process_at_rank(0).receive_vec::<f64>();
        println!("rank 1 received {msg:?} from {}", status.source_rank());
    }
}

Build and run across 4 processes with the bundled launcher:

$ cargo build --examples
$ ./target/debug/mpiexec -n 4 ./target/debug/examples/hello
Hello from rank 0 of 4 on localhost
Hello from rank 1 of 4 on localhost
Hello from rank 2 of 4 on localhost
Hello from rank 3 of 4 on localhost
All 4 processes checked in.

A program run without the launcher behaves like a C MPI singleton init: world size 1, rank 0.

How it works

Process launch & bootstrap (mpiexec / mpirun)

The launcher spawns N copies of your program and runs a tiny PMI-style rendezvous:

1. mpiexec binds a TCP rendezvous socket and spawns N children, passing each its rank, the world size and the rendezvous address in environment variables (MPI_PMI_RANK, MPI_PMI_SIZE, MPI_PMI_ROOT).
2. In mpi::initialize(), each child binds its own data socket, connects to the rendezvous and reports its address.
3. The launcher gathers all addresses and broadcasts the full table back, so every rank learns how to reach every peer. initialize() then returns.

Transport

Each rank runs a background acceptor thread; every peer connection gets its own reader thread that drains framed messages into a shared, matched inbox. A receive scans the inbox for the first message matching (communicator, source, tag) — honouring ANY_SOURCE / ANY_TAG — and blocks on a condition variable until one arrives. FIFO order per (source, tag) gives MPI's non-overtaking guarantee. Messages to self short-circuit into the local inbox.

Communicators & collectives

Every communicator carries a context id; point-to-point traffic and collective traffic run on separate contexts so they never alias, and each collective phase uses a distinct tag. Collectives use scalable algorithms — binomial-tree broadcast/reduce, dissemination barrier, ring allgather, and tree all-reduce (O(log n) steps) — with an ordered fallback for non-commutative reductions. split/dup agree on fresh contexts via a deterministic derivation from an all-gathered (color, key) table.

Robustness

MPI_Finalize (dropping the Universe) is a collective barrier, so no rank exits with messages still undelivered. MPI_Abort — and any panic (e.g. a failed assertion) — is propagated to every peer, so one rank's failure tears the whole job down promptly instead of leaving the others blocked in a receive. MPI_Wtime uses a monotonic clock. Messages above 64 KiB use a rendezvous protocol so a fast sender can't exhaust a slow receiver's memory. Mixed-endian jobs are detected and rejected at startup.

Shared memory (optional)

Build with --features shm (pulls in libc) to route same-host ranks through a POSIX shared-memory ring instead of loopback TCP — measured ~3× lower latency (19 µs → 6.5 µs) for small messages. Cross-host ranks still use TCP. The default build remains zero-dependency.

Examples

$ ./target/debug/mpiexec -n 4 ./target/debug/examples/selftest
SELFTEST PASS: all checks passed on 4 ranks.

Testing

$ cargo test          # unit tests + multi-process integration tests (n = 1,2,4,7)

The integration tests launch examples/selftest under mpiexec at several process counts and assert it passes.

Multi-host clusters

mpiexec can launch ranks on remote hosts over SSH. Ranks are assigned to the --host list round-robin; each rank auto-detects its own routable IP, so no per-node configuration is needed:

$ mpiexec -n 4 --host localhost,node2 --remote-bin /path/on/node2/prog ./prog

--remote-bin is the program path on the remote hosts (they need the binary and passwordless SSH). A host of localhost runs locally; anything else is launched with ssh <host> ….

This has been verified on a heterogeneous two-node LAN — rank 0 on macOS (arm64) and ranks on Linux (x86_64) — running the full selftest (every collective, communicator split, graph/inter-comm, pack/unpack) and the one-sided rma example across the network. The wire format is fixed little-endian, so little-endian 64-bit hosts of different OS/arch interoperate. (Cross-compile the Linux binary from macOS with e.g. zig as the linker for x86_64-unknown-linux-gnu.)

Parity with rsmpi

See PARITY.md for the full feature-by-feature status matrix. Implemented and tested (pure Rust, multi-process tests):

• environment — init, Universe, threading, timing, processor/library info
• point-to-point — all send modes, blocking + immediate_* + probes + matched probes, send_receive*
• collectives — barrier, broadcast, gather/scatter (+ varcount), allgather (+ varcount), all-to-all (+ varcount), reduce/allreduce, inclusive/exclusive scan, reduce-scatter, and their non-blocking immediate_* forms
• reductions — all built-in SystemOperations plus UserOperation closures
• communicators — split/dup/group, groups & set algebra, Cartesian and graph topologies (+ neighbourhood collectives), inter-communicators, naming, pack/unpack
• datatypes — Equivalence, Buffer/BufferMut, Partition, UserDatatype (contiguous/vector/indexed/struct), View/MutView, and #[derive(Equivalence)] (feature derive)
• one-sided RMA — Window put / get / accumulate / fence / lock
• parallel I/O — File explicit-offset & collective read / write
• attribute caching — keyvals and comm attributes
• error handling — MPI_ERR_* classes, error_string, error handlers
• dynamic processes — Root::spawn / Communicator::parent (MPI_Comm_spawn)
• requests — Request, scopes, WaitGuard/CancelGuard, wait_any/wait_all, generalized requests
• transport — TCP; single-host loopback or multi-host over SSH with auto-detected node IPs (verified heterogeneous macOS/Linux)

This covers every functional chapter of MPI, going well beyond rsmpi's shipped surface (which has no window/io/spawn). What is not claimed: every one of the standard's 400+ entry points and corner-case flags, big-endian interoperability, and the MPI_Dist_graph_create_adjacent variant — see PARITY.md.

License

MIT OR Apache-2.0.