Crate moonpool

Expand description

§Moonpool

Deterministic simulation testing for distributed systems in Rust.

Moonpool enables you to write distributed system logic once, test it with simulated networking for reproducible debugging, then deploy with real networking—all using identical application code.

Inspired by FoundationDB’s simulation testing.

§Crate Architecture

┌─────────────────────────────────────────────────────────────┐
│              moonpool (this crate)                          │
│   Re-exports all functionality from sub-crates             │
├──────────────────────────┬──────────────────────────────────┤
│  moonpool-transport      │       moonpool-sim               │
│  • Peer connections      │       • SimWorld runtime         │
│  • Wire format           │       • Chaos testing            │
│  • NetTransport + RPC    │       • Buggify macros           │
│  • #[service] macro      │       • 14 assertion macros      │
│    (via transport-derive)│       • Multiverse exploration   │
│                          │         (via moonpool-explorer)  │
├──────────────────────────┴──────────────────────────────────┤
│                     moonpool-core                           │
│  Provider traits: Time, Task, Network, Random, Storage      │
│  Core types: UID, Endpoint, NetworkAddress                  │
└─────────────────────────────────────────────────────────────┘

§Quick Start

use moonpool::{SimulationBuilder, WorkloadTopology};

SimulationBuilder::new()
    .topology(WorkloadTopology::ClientServer { clients: 2, servers: 1 })
    .run(|ctx| async move {
        // Your distributed system workload
    });

§Which Crate to Use

Use case	Crate
Full framework (recommended)	`moonpool`
Provider traits only	`moonpool-core`
Simulation without transport	`moonpool-sim`
Transport without simulation	`moonpool-transport`
Fork-based exploration internals	`moonpool-explorer`
Proc-macro internals	`moonpool-transport-derive`

§Documentation

moonpool_core - Provider traits and core types
moonpool_sim - Simulation runtime and chaos testing
moonpool_transport - Network transport layer

Modules§

__tokio: A runtime for writing reliable network applications without compromising speed.
chaos: Chaos testing infrastructure for deterministic fault injection. Chaos testing infrastructure for deterministic fault injection.
error: Error types for transport operations. Error types for the moonpool messaging layer.
executor: The deterministic single-threaded executor (seeded-random scheduling). The moonpool deterministic executor: a single-threaded, seeded-random async task scheduler purpose-built for simulation.
flags: Address flags.
network: Network simulation and configuration. Network simulation and configuration.
observability: Production-friendly observability layer (replaces legacy Timeline + Invariant). Trace-based observability for moonpool simulations.
peer: Resilient peer connection management. Resilient peer connection management.
prelude: Common imports for application code.
providers: Provider implementations for simulation. Provider implementations for simulation.
rpc: RPC layer with typed request/response patterns. FDB-style static messaging with fixed endpoints.
runner: Simulation runner and orchestration framework. Simulation runner and orchestration framework.
select_support: Runtime support for the deterministic select! macro.
sim: Core simulation engine for deterministic testing. Core simulation engine for deterministic testing.
simulations: Simulation workloads and binary targets. Simulation workloads and binary targets.
storage: Storage simulation and configuration. Storage simulation and configuration.
wire: Wire format with CRC32C checksums. Wire format for packet serialization.

Macros§

assert_always: Assert that a condition is always true.
assert_always_greater_than: Assert that val > threshold always holds.
assert_always_greater_than_or_equal_to: Assert that val >= threshold always holds.
assert_always_less_than: Assert that val < threshold always holds.
assert_always_less_than_or_equal_to: Assert that val <= threshold always holds.
assert_always_or_unreachable: Assert that a condition is always true when reached, but the code path need not be reached. Does not panic if never evaluated.
assert_reachable: Assert that a code path is reachable (should be reached at least once).
assert_sometimes: Assert a condition that should sometimes be true, tracking stats and triggering exploration.
assert_sometimes_all: Compound boolean assertion: all named bools should sometimes be true simultaneously.
assert_sometimes_each: Per-value bucketed sometimes assertion with optional quality watermarks.
assert_sometimes_greater_than: Assert that val > threshold sometimes holds. Forks on watermark improvement.
assert_sometimes_greater_than_or_equal_to: Assert that val >= threshold sometimes holds. Forks on watermark improvement.
assert_sometimes_less_than: Assert that val < threshold sometimes holds. Forks on watermark improvement.
assert_sometimes_less_than_or_equal_to: Assert that val <= threshold sometimes holds. Forks on watermark improvement.
assert_unreachable: Assert that a code path should never be reached.
buggify: Buggify with 25% probability
buggify_knob: Buggify a config knob value within bounds.
buggify_with_prob: Buggify with custom probability
select: Waits on multiple concurrent branches, returning when the first completes, with a deterministic, seed-controlled polling order.

Structs§

AdaptiveConfig: Configuration for adaptive batch-based timeline splitting.
AssertionStats: Statistics for a tracked assertion.
Attrition: Built-in attrition configuration for automatic process reboots.
BugRecipe: A captured bug recipe with its root seed for deterministic replay.
ChaosConfiguration: Configuration for chaos injection in simulations.
Endpoint: Endpoint = Address + Token.
EndpointMap: Maps endpoint tokens to message receivers.
EventQueue: A priority queue for scheduling events in chronological order.
ExplorationConfig: Configuration for exploration.
ExplorationReport: Report from fork-based exploration.
FailureMonitor: Reactive failure monitor for address and endpoint tracking.
FaultContext: Context for fault injectors — gives access to SimWorld fault injection methods.
InMemoryStorage: In-memory storage with deterministic fault injection.
InstallGuard: Drop-guard returned by SimulationLayer::install. Restores the previous subscriber when dropped.
InterfaceMethod: Unified method handle for both local (server) and remote (client) modes.
JsonCodec: JSON codec using serde_json.
LocalityConfig: Configuration for laying processes out across a failure-domain topology.
LocalityInfo: Resolved failure-domain locality for a single process instance.
MachineRegistry: Registry mapping process IPs to their resolved locality.
MonitorConfig: Configuration for connection health monitoring.
NetNotifiedQueue: Type-safe message queue with async notification.
NetTransport: Central transport coordinator (FDB NetTransport equivalent).
NetTransportBuilder: Builder for NetTransport that eliminates common footguns.
NetworkAddress: Network address (IPv4/IPv6 + port + flags).
NetworkConfiguration: Configuration for network simulation parameters
OpenOptions: Options for opening a file.
PacketHeader: Packet header for wire format.
Peer: A resilient peer that manages connections to a remote address.
PeerConfig: Configuration for peer behavior and reconnection parameters.
PeerMetrics: Metrics and state information for a peer connection.
ProcessTags: Resolved tags for a specific process instance.
ReplyFuture: Future that resolves when a reply is received from the server.
ReplyPromise: Promise for sending a reply to a request.
RequestEnvelope: Envelope wrapping a request with its reply endpoint.
RequestStream: Stream for receiving typed requests with reply promises.
ScheduledEvent: An event scheduled for execution at a specific simulation time.
SectorBitSet: A simple bitset for tracking sector states.
ServerHandle: Handle for a running RPC server.
ServiceEndpoint: Client-side typed endpoint for making RPC calls.
SimContext: Simulation context provided to workloads.
SimFaultRecord: A fault recorded by the simulation engine, stamped with the sim time at which it occurred.
SimNetworkProvider: Simulated networking implementation
SimProviders: Simulation providers bundle for deterministic testing.
SimRandomProvider: Random provider for simulation that uses the thread-local deterministic RNG.
SimStorageProvider: Simulated storage provider for deterministic testing.
SimTaskProvider: Task provider that spawns onto the deterministic executor driving the current simulation iteration.
SimTime: FormatTime implementation that writes simulation time reported by a Clock instead of wall-clock time.
SimTimeProvider: Simulation time provider that integrates with SimWorld.
SimWorld: The central simulation coordinator that manages time and event processing.
SimulationBuilder: Builder pattern for configuring and running simulation experiments.
SimulationLayer: A tracing::Layer that captures plain trace events emitted inside process/workload spans.
SimulationLayerHandle: Cheap-to-clone handle to a SimulationLayer’s captured state.
SimulationMetrics: Core metrics collected during a simulation run.
SimulationReport: Comprehensive report of a simulation run with statistical analysis.
SleepFuture: Future that completes after a specified simulation time duration.
StateHandle: Shared state handle for cross-workload publish/get communication.
StorageConfiguration: Configuration for storage simulation parameters.
TagRegistry: Registry mapping process IPs to their resolved tags.
TokioJoinHandle: JoinHandle produced by TokioTaskProvider.
TokioNetworkProvider: Real Tokio networking implementation.
TokioProviders: Production providers using Tokio runtime.
TokioRandomProvider: Production random provider using thread-local RNG.
TokioStorageFile: Wrapper for Tokio File to implement our trait.
TokioStorageProvider: Real Tokio storage implementation.
TokioTaskProvider: Tokio-based task provider.
TokioTcpListener: Wrapper for Tokio TcpListener to implement our trait.
TokioTimeProvider: Real time provider using Tokio’s time facilities.
TraceEvent: One captured trace event — the same shape as a production log line.
UID: 128-bit unique identifier.
WeakSimWorld: A weak reference to a simulation world.
WorkloadTopology: Topology information provided to workloads and processes.

Enums§

AssertCmp: Comparison operator for numeric assertions.
AssertKind: The kind of assertion being tracked.
AttritionScope: The failure domain a reboot targets.
Chaos: A chaos surface to enable, and how to sample it per seed.
ChaosMode: How an enabled chaos surface is sampled each seed.
ClientId: Strategy for assigning client IDs to workload instances.
CodecError: Error type for codec operations.
ConnectFailureMode: Connection establishment failure mode for fault injection.
ConnectionStateChange: Connection state changes
DomainLevel: The level of a failure domain in the locality hierarchy.
Event: Events that can be scheduled in the simulation.
FailureStatus: Status of a network address or endpoint.
FieldValue: A single structured field value captured from a tracing event.
IterationControl: Configuration for how many iterations a simulation should run.
JoinError: Error returned by TaskProvider::JoinHandle when a task did not complete normally.
LatencyDistribution: A pluggable latency distribution for per-operation latency sampling.
MessagingError: Errors that can occur in the messaging layer.
NetworkAddressParseError: Error parsing a network address from string.
NetworkOperation: Network data operations
Parallelism: Controls how many children can run in parallel during splitting.
PeerError: Errors that can occur during peer operations.
RebootKind: The type of reboot to perform on a process.
ReplyError: Errors that can occur during request-response operations.
RpcError: Unified error type for RPC operations.
SimFaultEvent: Fault events automatically recorded by the simulator.
SimulationError: Errors that can occur during simulation operations.
StorageError: Errors that can occur during simulated storage operations.
StorageOperation: Storage operations that can be scheduled in the simulation.
TimeError: Errors that can occur during time operations.
WellKnownToken: Well-known endpoint tokens.
WireError: Wire format error types.
WorkloadCount: How many instances of a workload to spawn per iteration.

Constants§

HEADER_SIZE: Header size: 4 (length) + 4 (checksum) + 16 (token) = 24 bytes.
MAX_PAYLOAD_SIZE: Maximum payload size (1MB).
SECTOR_SIZE: Size of a disk sector in bytes.
SIM_FAULT_EVENT_NAME: Well-known event name for simulator-emitted fault events.
WELL_KNOWN_RESERVED_COUNT: Number of reserved well-known token slots.

Traits§

Clock: Narrow Send + Sync source of simulation time, suitable for plugging into the SimTime formatter.
FaultInjector: A fault injector that introduces failures during the chaos phase.
Invariant: An invariant validated after every simulation step.
MessageCodec: Pluggable message serialization format.
MessageReceiver: Trait for receiving deserialized messages from the transport layer.
NetworkProvider: Provider trait for creating network connections and listeners.
Process: A process that participates in simulation as part of the system under test.
Providers: Bundle of all provider types for a runtime environment.
RandomProvider: Provider trait for random number generation.
StorageFile: Trait for file handles that support async read/write/seek operations.
StorageProvider: Provider trait for file storage operations.
TaskProvider: Provider for spawning tasks.
TcpListenerTrait: Trait for TCP listeners that can accept connections.
TimeProvider: Provider trait for time operations.
TraceQuery: Read-only view of captured trace events provided to invariants.
TransportHandle: Object-safe transport abstraction that erases provider generics.
Workload: A workload that participates in simulation testing.

Functions§

assertion_results: Get current assertion statistics for all tracked assertions.
buggify_init: Initialize buggify for simulation run
buggify_reset: Reset/disable buggify
clear_rng_breakpoints: Clear all RNG breakpoints.
current_sim_seed: Get the current simulation seed.
deserialize_packet: Deserialize a packet, validating checksum.
format_timeline: Format a recipe as a human-readable timeline string.
get_reply: At-least-once delivery: send reliably, retransmit on reconnect.
get_reply_unless_failed_for: At-least-once delivery with sustained failure timeout.
has_always_violations: Check whether any always-type assertion was violated during this iteration.
init_sim_tracing: Install a basic stdout tracing_subscriber::fmt subscriber at max_level.
invariant_fn: Wrap a closure as an Invariant.
make_decode_fn: Create a DecodeFn by capturing a concrete codec.
make_encode_fn: Create an EncodeFn by capturing a concrete codec.
parse_timeline: Parse a timeline string back into a recipe.
reset_always_violations: Reset the violation counter. Must be called at the start of each iteration.
reset_assertion_results: Reset all assertion statistics.
reset_rng_call_count: Reset the RNG call count to zero.
reset_sim_rng: Reset the thread-local simulation RNG to a fresh state.
rng_call_count: Get the current RNG call count.
sample_duration: Sample a random duration from a range
sample_latency: Sample a latency from a LatencyDistribution using the simulation RNG.
send: Fire-and-forget delivery: send request unreliably with no reply.
send_request: Send a typed request and return a future for the response.
serialize_packet: Serialize a packet with token and payload.
set_rng_breakpoints: Set RNG breakpoints for deterministic replay.
set_sim_seed: Set the seed for the thread-local simulation RNG.
set_swarm_op_seed: Set the per-seed base for workload operation-alphabet swarm masks.
sim_random: Generate a random value using the thread-local simulation RNG.
sim_random_range: Generate a random value within a specified range using the thread-local simulation RNG.
sim_random_range_or_default: Generate a random value within the given range, returning the start value if the range is empty.
swarm_op_enabled: Report whether operation op_id is enabled for the current iteration’s operation-alphabet swarm subset.
try_deserialize_packet: Try to deserialize from a buffer that may contain partial data.
try_get_reply: At-most-once delivery: send unreliably, race reply against disconnect.
validate_assertion_contracts: Validate all assertion contracts based on their kind.

Type Aliases§

DecodeFn: Type-erased decode function captured from a concrete MessageCodec.
EncodeFn: Type-erased encode function captured from a concrete MessageCodec.
PeerReceiver: Type alias for the peer receiver channel. Used when taking ownership via take_receiver().
SimulationResult: A type alias for Result<T, SimulationError>.
TokioTransport: A NetTransport backed by the production TokioProviders bundle (and the default JsonCodec) — the typical production transport type.

Attribute Macros§

service: Attribute macro for defining RPC service interfaces.

Crate moonpool

Crate moonpool Copy item path

§Moonpool

§Crate Architecture

§Quick Start

§Which Crate to Use

§Documentation

Modules§

Macros§

Structs§

Enums§

Constants§

Traits§

Functions§

Type Aliases§

Attribute Macros§

Crate moonpool