Module simulated 

Simulate networking between peers with configurable link behavior (i.e. drops, latency, corruption, etc.).

Both peer and link modification can be performed dynamically over the lifetime of the simulated network. This can be used to mimic transient network partitions, offline nodes (that later connect), and/or degrading link quality. Messages on a link are delivered in order, and optional per-peer bandwidth limits account for transmission delay and queueing.

Determinism

commonware-p2p::simulated can be run deterministically when paired with commonware-runtime::deterministic. This makes it possible to reproduce an arbitrary order of delivered/dropped messages with a given seed.

Bandwidth Simulation

The simulator provides a realistic model of bandwidth contention where network capacity is a shared, finite resource. This is achieved without the overhead of simulating byte-by-byte transfers.

Core Model

The bandwidth model is built on an event-based timeline. Instead of simulating continuous data flow, the scheduler calculates the key points in time where bandwidth availability changes for a peer. These changes occur whenever a transfer starts or finishes.

Each peer has a schedule for its egress (upload) and ingress (download) bandwidth. When a new message is sent, the scheduler performs the following steps:

1. Find Available Capacity: It looks at the sender’s egress schedule and the receiver’s ingress schedule to find the available bandwidth over time. The effective transfer rate is always limited by the minimum of the two.

2. Fill Time Slots: The algorithm iterates through time, finding “slots” of available bandwidth. It calculates how much of the message can be sent in each slot until the entire message is accounted for. For example, if two 10KB messages are sent over a 10KB/s link, the second message will be scheduled to start only after the first one completes.

3. Reserve Bandwidth: Once the completion time is calculated, the new transfer is added to the schedules of both the sender and receiver as a bandwidth reservation, consuming capacity for its calculated duration.

Latency vs. Transmission Delay

The simulation correctly distinguishes between two key components of message delivery:

• Transmission Delay: The time it takes to send all bytes of a message over the link. This is determined by the message size and the available bandwidth (e.g., a 10KB message on a 10KB/s link has a 1-second transmission delay).
• Network Latency: The time it takes for a byte to travel from the sender to the receiver, independent of bandwidth. This is configured via the Link properties.

The final delivery time of a message is the sum of when its transmission completes plus the simulated network latency. This model ensures that large messages correctly occupy the network link for longer periods, affecting other concurrent transfers, while still accounting for the physical travel time of the data.

Example

use commonware_p2p::simulated::{Config, Link, Network};
use commonware_cryptography::{ed25519, PrivateKey, Signer as _, PublicKey as _, PrivateKeyExt as _};
use commonware_runtime::{deterministic, Spawner, Runner, Metrics};
use std::time::Duration;

// Generate peers
let peers = vec![
    ed25519::PrivateKey::from_seed(0).public_key(),
    ed25519::PrivateKey::from_seed(1).public_key(),
    ed25519::PrivateKey::from_seed(2).public_key(),
    ed25519::PrivateKey::from_seed(3).public_key(),
];

// Configure network
let p2p_cfg = Config {
    max_size: 1024 * 1024, // 1MB
};

// Start context
let executor = deterministic::Runner::seeded(0);
executor.start(|context| async move {
    // Initialize network
    let (network, mut oracle) = Network::new(context.with_label("network"), p2p_cfg);

    // Start network
    let network_handler = network.start();

    // Register peers
    let (sender1, receiver1) = oracle.register(peers[0].clone(), 0).await.unwrap();
    let (sender2, receiver2) = oracle.register(peers[1].clone(), 0).await.unwrap();

    // Set bandwidth limits
    // peer[0]: 10KB/s egress, unlimited ingress
    // peer[1]: unlimited egress, 5KB/s ingress
    oracle.set_bandwidth(peers[0].clone(), 10_000, usize::MAX).await.unwrap();
    oracle.set_bandwidth(peers[1].clone(), usize::MAX, 5_000).await.unwrap();

    // Link 2 peers
    oracle.add_link(
        peers[0].clone(),
        peers[1].clone(),
        Link {
            latency: Duration::from_millis(5),
            jitter: Duration::from_millis(2),
            success_rate: 0.75,
        },
    ).await.unwrap();

    // ... Use sender and receiver ...

    // Update link
    oracle.remove_link(
        peers[0].clone(),
        peers[1].clone(),
    ).await.unwrap();
    oracle.add_link(
        peers[0].clone(),
        peers[1].clone(),
        Link {
            latency: Duration::from_millis(100),
            jitter: Duration::from_millis(25),
            success_rate: 0.8,
        },
    ).await.unwrap();

    // ... Use sender and receiver ...

    // Shutdown network
    network_handler.abort();
});

Structs

Config

Configuration for the simulated network.

Link

Describes a connection between two peers.

Network

Implementation of a simulated network.

Oracle

Interface for modifying the simulated network.

Receiver

Implementation of a crate::Receiver for the simulated network.

Sender

Implementation of a crate::Sender for the simulated network.

Enums

Error

Errors that can occur when interacting with the network.