Crate turmoil

source ·
Expand description

Turmoil is a framework for testing distributed systems. It provides deterministic execution by running multiple concurrent hosts within a single thread. It introduces “hardship” into the system via changes in the simulated network. The network can be controlled manually or with a seeded rng.

Hosts and Software

A turmoil simulation is comprised of one or more hosts. Hosts run software, which is represented as a Future.

Test code is also executed on a special host, called a client. This allows for an entry point into the simulated system. Client hosts have all the same capabilities as normal hosts, such as networking support.

use turmoil;

let mut sim = turmoil::Builder::new().build();

// register a host
sim.host("host", || async {

    // host software goes here

    Ok(())
});

// define test code
sim.client("test", async {

    // we can interact with other hosts from here

    Ok(())
});

// run the simulation and handle the result
_ = sim.run();

Networking

Simulated networking types that mirror tokio::net are included in the turmoil::net module.

Turmoil is not yet oppinionated on how to structure your application code to swap in simulated types under test. More on this coming soon…

Network Manipulation

The simulation has the following network manipulation capabilities:

  • partition, which introduces a network partition between hosts
  • repair, which repairs a network partition between hosts
  • hold, which holds all “in flight” messages between hosts. Messages are available for introspection using Sim’s links method.
  • release, which releases all “in flight” messages between hosts

Tracing

The tracing crate is used to emit important events during the lifetime of a simulation.

This can be configured using RUST_LOG=turmoil=info.

Feature flags

  • regex: Enables regex host resolution through ToIpAddrs

tokio_unstable

Turmoil uses unhandled_panic to forward host panics as test failures. See unstable features to opt in.

Modules

  • net
    This module contains the simulated TCP/UDP networking types.

Structs

  • Builder
    A builder that can be used to configure the simulation.
  • Datagram
    UDP datagram.
  • LinkIter
    An iterator for the link, providing access to sent messages that have not yet been delivered.
  • LinksIter
    An iterator for the network topology, providing access to all active links in the simulated network.
  • SentRef
    Provides a reference to a message that is currently inflight on the network from one host to another.
  • Sim
    A handle for interacting with the simulation.

Enums

  • IpVersion
    The kinds of networks that can be simulated in turmoil
  • Protocol
    Supported network protocols.
  • Segment
    This is a simplification of real TCP.

Traits

Functions

  • elapsed
    Returns how long the currently executing host has been executing for in virtual time.
  • hold
    Hold messages between two hosts, or sets of hosts, until release is called.
  • lookup
    Lookup an IP address by host name.
  • lookup_many
    Lookup an IP address by host name. Use regex to match a number of hosts.
  • partition
    Partition two hosts, or sets of hosts, resulting in all messages sent between them to be dropped.
  • release
    The opposite of hold. All held messages are immediately delivered.
  • repair
    Repair the connection between two hosts, or sets of hosts, resulting in messages to be delivered.
  • reverse_lookup
    Perform a reverse DNS lookup, returning the hostname if the entry exists.

Type Aliases