DScale

A fast, deterministic simulation framework for testing and benchmarking distributed systems. It simulates network latency, bandwidth constraints, and process execution in an event-driven environment with support for both single-threaded and parallel execution modes.

Usage

1. Install

In your project

cargo add dscale

2. Define Messages

Messages must implement the Message trait, which allows defining a virtual_size for bandwidth simulation.

use dscale::Message;

struct MyMessage {
    data: u32,
}

impl Message for MyMessage {
    fn virtual_size(&self) -> usize {
        // Size in bytes used for bandwidth simulation.
        // Can be much bigger than real memory size to simulate heavy payloads.
        1000
    }
}

// Or (if there is no need in bandwidth)
impl Message for MyMessage {}

3. Implement Process Logic

Implement ProcessHandle to define how your process reacts to initialization, messages, and timers.

use dscale::{ProcessHandle, Rank, MessagePtr, TimerId, Jiffies};
use dscale::{broadcast, send_to, schedule_timer_after, rank, dscale_debug};
use dscale::global::configuration;

#[derive(Default)]
struct MyProcess;

impl ProcessHandle for MyProcess {
    fn on_start(&mut self) {
        dscale_debug!("Starting process {} of {}", rank(), configuration::process_number());
        schedule_timer_after(Jiffies(100));
    }

    fn on_message(&mut self, from: Rank, message: MessagePtr) {
        if let Some(msg) = message.try_as_type::<MyMessage>() {
            dscale_debug!("Received message from {}: {}", from, msg.data);
        }
    }

    fn on_timer(&mut self, _id: TimerId) {
        broadcast(MyMessage { data: 42 });
    }
}

4. Run the Simulation

Use SimulationBuilder to configure the topology, network constraints, and start the simulation.

use dscale::{SimulationBuilder, Jiffies, BandwidthConfig, Distr};

fn main() {
    let mut runner = SimulationBuilder::default()
        .add_pool::<MyClient>("Client", 1)
        .add_pool::<MyServer>("Server", 3)
        .within_pool_latency("Client", Distr::Uniform(Jiffies(1), Jiffies(5)))
        .within_pool_latency("Server", Distr::Uniform(Jiffies(1), Jiffies(5)))
        .between_pool_latency("Client", "Server", Distr::Normal {
            mean: Jiffies(10),
            std_dev: Jiffies(2),
            low: Jiffies(5),
            high: Jiffies(20),
        })
        .vnic_bandwidth(BandwidthConfig::Bounded(1000)) // 1000 bytes per Jiffy
        .time_budget(Jiffies(1_000_000))
        .simple()
        .build();

    runner.run_full_budget();
}

Parallel Execution

For large simulations, enable parallel execution to distribute process steps across multiple threads:

let mut runner = SimulationBuilder::default()
    .add_pool::<MyProcess>("Nodes", 1000)
    .within_pool_latency("Nodes", Distr::Uniform(Jiffies(1), Jiffies(10)))
    .time_budget(Jiffies(1_000_000))
    .parallel(Threads::Specific(8)) // use 8 worker threads
    .build();

runner.run_full_budget();

When is parallel mode efficient?

1. A lot of simulated processes (at least 200-300)
2. on_message execution takes most of simulation time
3. Independent work inside on_message (not so much synchronization)

[!CAUTION] You can safely run only one simulation in parallel (simple or parallel), because of thread local state. Simulation lifetime is bounded to runner instance. Once it drops you can run another simulation.

Fault Injection

DScale supports injecting network faults into simulations. Faults are scheduled as timed events — you specify when a fault starts and when it ends.

[!IMPORTANT] Fault injection requires building with the FAULT environment variable set to on:

FAULT=on cargo run --bin my_example

Without FAULT=on, all fault-related events will not do anything.

[!CAUTION] Fault injection is only supported in single-threaded (simple) mode. Using faults with parallel mode will panic at build time.

let mut runner = SimulationBuilder::default()
    .add_pool::<MyProcess>("Nodes", 5)
    .within_pool_latency("Nodes", Distr::Uniform(Jiffies(1), Jiffies(5)))
    .time_budget(Jiffies(1_000_000))
    // Break the link between rank 0 and rank 1 from time 100 to 500
    .break_link(Jiffies(100), Jiffies(500), 0, 1)
    // Isolate rank 2 (all links broken) from time 200 to 800
    .isolate(Jiffies(200), Jiffies(800), 2)
    .simple()
    .build();

runner.run_full_budget();

Public API

Simulation Control

SimulationBuilder — Configures the simulation environment.

SimulationRunner

Network Topology

Constants

Distributions

Process Interaction (Context-Aware)

These functions are available globally but must be called within the context of a running process step.

Configuration (dscale::global::configuration)

Key-Value Store (dscale::global::kv)

Thread-safe store for passing shared state, metrics, or configuration between processes or back to the host.

Macros

All logging macros prefix output with the current simulation time and process rank ([Now: ... | P...]). Controlled by the RUST_LOG environment variable.

Helpers (dscale::helpers)

Message Downcasting (MessagePtr)

Logging Configuration (RUST_LOG)

DScale output is controlled via the RUST_LOG environment variable.

• RUST_LOG=info: Shows high-level simulation status and a progress bar.
• RUST_LOG=debug: Enables all dscale_debug! macro output and all internal simulation events.
• RUST_LOG=full::path::to::your::file::or::crate=debug,another::path=debug: Filter events only for your specific file or crate.

[!WARNING]
RUST_LOG=debug and path-level debug filters only work without the --release flag.

Thanks to

• https://gitlab.com/whirl-framework
• https://github.com/jepsen-io/maelstrom
• https://github.com/systems-group/anysystem
• https://www.open-mpi.org
• https://www.nsnam.org