jsmpi 0.1.0

A browser-oriented MPI compatibility layer for Rust/WASM using Web Workers
Documentation

jsmpi

A browser-oriented Rust MPI compatibility layer: run rsmpi-style programs in the browser by compiling them to WebAssembly and executing ranks through Web Workers.

Project Goals

jsmpi aims to preserve the rsmpi developer experience as much as possible when moving Rust MPI programs to the browser:

  • Keep the use mpi::traits::* style.
  • Keep the main call flow such as mpi::initialize(), world.rank(), send(), receive(), and barrier().
  • Minimize code changes outside imports and entry-point wrapping.

Current Status

The repository currently includes the following:

Implemented

  • Base jsmpi crate skeleton.
  • initialize(), Universe::world(), rank(), and size().
  • Minimal API surface for send, receive, broadcast_into, and barrier.
  • Browser runtime chain with Coordinator + Worker + WASM.
  • Blocking synchronization bridge prototype based on SharedArrayBuffer + Atomics.
  • Demo page with switchable examples.
  • Build scripts and local static serving workflow.

MPI API Completion Status

jsmpi is intentionally incremental. The list below tracks practical compatibility with common rsmpi usage.

Core Environment / Communicator

  • initialize()
  • Universe::world()
  • world.rank() / world.size()
  • world.barrier()

Point-to-Point (Blocking)

  • process_at_rank(rank).send() / send_with_tag()
  • process_at_rank(rank).receive() / receive_with_tag()
  • receive_into() / receive_into_with_tag()
  • Slice/vector helpers: send_slice*, receive_vec*, receive_slice_into*
  • Raw bytes path: send_bytes*, receive_bytes*
  • Chunked bytes path (length-prefix framing): send_bytes_chunked*, receive_bytes_chunked*
  • [~] Wildcard semantics: any_process().receive*() is available; full MPI wildcard/tag matching parity is still evolving

Point-to-Point (Non-Blocking, First Version)

  • Immediate send: immediate_send() / immediate_send_with_tag() with request test()/wait()
  • Immediate receive: immediate_receive() / immediate_receive_with_tag() with request test()/wait()/wait_into()
  • Multi-request helpers: ImmediateReceiveRequest::test_any() / wait_any() / test_all() / wait_all() / test_some() / wait_some()
  • [~] Lifecycle semantics (first version): requests expose state() (Pending/Completed/Canceled/Freed), cancel() transitions pending requests to Canceled, and free() marks requests as Freed
  • [~] Current semantics are lightweight wrappers over existing runtime behavior; full MPI request lifecycle parity (MPI_Request edge cases) is still evolving

Collectives (Common)

  • Broadcast: broadcast_into and communicator convenience broadcast_into_from
  • Gather: gather_into_root (root process form) and communicator convenience gather_into_root
  • Scatter: scatter_into_root (root process form) and communicator convenience scatter_into_root
  • Reduce sum: reduce_sum_into_root / reduce_into_root(SystemOperation::sum())
  • All-reduce sum: all_reduce_sum_into / all_reduce_into(SystemOperation::sum())
  • All-gather: all_gather_into
  • Broadcast dissemination now uses a tree-style fan-out (virtual-rank doubling), reducing root hot-spot pressure on larger communicator sizes
  • all_reduce_sum_into and all_gather_into now reuse broadcast dissemination for result fan-out
  • Broadcast timeout/retry: broadcast_into_from_with_timeout(...) / broadcast_into_from_with_retry_timeout(...)
  • Gather timeout/retry: gather_into_root_with_timeout(...) / gather_into_root_with_retry_timeout(...)
  • Scatter timeout/retry: scatter_into_root_with_timeout(...) / scatter_into_root_with_retry_timeout(...)
  • Timeout path: all_gather_into_with_timeout(...) -> Result<()>
  • Retry+timeout path: all_reduce_sum_into_from_with_retry_timeout(...) -> Result<()>

Not Yet Implemented (Planned)

  • [~] Non-blocking request family: first-version isend/irecv-style flow is available (immediate_send*, immediate_receive*, test*/wait* orchestration); full MPI request parity remains in progress
  • Full datatype and custom operation parity with rsmpi
  • Advanced communicator/topology APIs (split, cartesian topology, etc.)
  • Full algorithmic/performance parity for large-scale collectives

Current Examples

  • examples/hello_ranks.rs
  • examples/ping_pong.rs
  • examples/immediate_requests.rs
  • examples/broadcast_seed.rs

Current Documentation

  • docs/jsmpi-technical-spec.md
  • docs/jsmpi-development-guide.md
  • docs/protocol-versioning-plan.md
  • docs/production-release-checklist.md
  • docs/browser-compatibility-matrix.md
  • examples/README.md
  • demo/index.html

Production Engineering Assets

  • .github/workflows/ci.yml
  • .github/ISSUE_TEMPLATE/protocol-versioning-task.md
  • .github/ISSUE_TEMPLATE/release-readiness-task.md

Repository Layout

jsmpi/
├─ src/          # Core Rust crate
├─ src/launcher/ # Browser launcher / worker bridge
├─ examples/     # Shared native/wasm examples
├─ demo/         # Yew browser demo app
├─ scripts/      # Build and run scripts
└─ docs/         # Technical spec and development guide

Quick Start

1. Build browser demo artifacts

npm run build:demo

This runs trunk build inside demo/.

2. Start local static server

npm run serve:demo

3. Open the app

http://127.0.0.1:8080/

In the UI, you can:

  • Switch between examples.
  • Set the rank count.
  • Inspect runtime logs, communication events, and error output.

The current demo is built and served through Trunk.

4. Deploy to GitHub Pages

The repository includes a Pages workflow:

  • .github/workflows/pages.yml

The workflow automatically:

  1. Compiles examples/*.rs to wasm32-unknown-unknown.
  2. Runs wasm-bindgen to generate demo/pkg/*.
  3. Runs trunk build --release.
  4. Publishes demo/dist to GitHub Pages.

For project pages repositories, it uses /<repo>/ as public-url automatically. For <user>.github.io repositories, it uses the root path /.

Example Migration Pattern

Recommended pattern: keep cross-target differences limited to imports and entry points.

#[cfg(target_arch = "wasm32")]
use jsmpi as mpi;

use mpi::traits::*;

#[cfg(target_arch = "wasm32")]
use wasm_bindgen::prelude::*;

fn run() {
    let universe = mpi::initialize().unwrap();
    let world = universe.world();
    // Keep the rest of the MPI logic unchanged whenever possible.
}

fn main() {
    run();
}

#[cfg(target_arch = "wasm32")]
#[wasm_bindgen(start)]
pub fn start() {
    run();
}

Current Limitations

This project is still in an early prototype stage. The following areas are not complete yet:

  • Full rsmpi API/trait coverage.
  • Broader tag/wildcard semantics and non-blocking request family (isend/irecv/wait*).
  • More complete collective algorithm family and large-scale optimization strategy.
  • More robust error propagation and job termination semantics.
  • Higher-performance data paths and typed-array optimizations.

Development and Validation

Validated key commands:

cargo check --lib
cargo check --examples --target wasm32-unknown-unknown
npm run build:demo