snare 3.1.1

Ergonomic in-process network testing on top of snare-interpose: write the code under test against std, describe the peer with a tester builder
Documentation

snare

Deterministic, in-process simulation of the operating system for Rust tests — without changing the code under test.

Your code keeps calling std::net::TcpStream, std::fs::File, std::env::var, libc::sched_setscheduler, clock_gettime, raw AF_PACKET/io_uring — exactly as it does in production. snare interposes at the OS-call boundary underneath it and serves those calls from an in-memory model: a virtual network, a virtual filesystem, a simulated host, a virtual clock. The test drives the other side of the wire.

use snare::prelude::*;
use std::io::{BufRead, BufReader, Write};
use std::net::TcpStream;

let sim = Sim::new();
sim.run(|| {
    // The "server" the code under test talks to — scripted by the test.
    let server = connect_tester::<Line>("127.0.0.2:9000")
        .then_action(|msg, _from| TesterAction::Send(Line(format!("echo:{}", msg.0))));

    // The code under test: a real TcpStream. Its socket calls are serviced from memory.
    let client = std::thread::spawn(|| {
        let mut s = TcpStream::connect("127.0.0.2:9000").unwrap();
        s.write_all(b"hello\n").unwrap();
        let mut line = String::new();
        BufReader::new(s).read_line(&mut line).unwrap();
        line
    });

    run_testers!(server);
    assert_eq!(client.join().unwrap(), "echo:hello\n");
});

No shim types, no dependency injection, no trait objects threaded through the code under test. It is a plain TcpStream.

The paradigm

snare 1.x required the code under test to import snare's shim types (snare::net::TcpStream instead of std::net::TcpStream, and so on). That only tests code you could edit to use the shims, and it never covers what a dependency does three crates down.

snare 2 removes that constraint. It patches the import tables of the loaded program (ELF GOT / Mach-O __got / PE IAT) so that the libc and Win32 symbols a managed thread calls — socket, open, getenv, clock_gettime, ioctl, sched_setscheduler, … — land in snare's hooks first. A hook offers the call to the test's in-memory model; if the model declines, the call falls through to the real OS. Nothing the code under test imports changes. This reaches straight through std and every dependency.

A function's address stored in data is redirected too: C dispatch tables such as SQLite's aSyscall[] ({ "fstat", fstat }, …) and Rust statics like static F: unsafe extern "C" fn(..) = libc::fstat are filled by the loader from data relocations (ELF R_X86_64_64/R_AARCH64_ABS64, Mach-O binds in __const/__data), and snare rewrites those words alongside the import slots. A pointer taken in code, one read from data and one returned by dlsym (or GetProcAddress) therefore all name the hook, and compare equal. On Windows such a pointer is the address of the linker's import thunk, which jumps through the patched IAT. A copy the program makes at run time before the first Sim is built (into the heap or a lazily initialized static) keeps the real address.

Every resource or object used by the code under test must be created under its active simulation. Run initialization inside Sim::run, including constructors, dependency initialization, worker pools and lazy initialization. Objects created before entering the simulation, or created with interposition disabled and then handed to the code under test, violate this usage invariant.

Passthrough (snare::real) is reserved for other-side emulation and the test harness. The code under test, including its dependencies and initialization, must remain under interposition. The harness can configure the simulated world before the run and use independent real resources for its own bookkeeping, fixtures and native comparisons.

Two needs, one library:

  • Deterministic unit tests — write them in-crate or as integration tests, guarded by #[cfg(snare)], and run them with cargo snare test (below).
  • One process, one API — the test body, its scripted peers (testers) and any higher-level harness run in the same process as the code under test and drive the simulation through the same in-process API (Sim, testers, set_udp_policy, the time controls). There is no controller process and no cross-process protocol; when the harness or other-side emulator needs the real machine it says so with snare::real.

How it works

        code under test (plain std / libc / raw syscalls)
                         │  its OS symbols are interposed
                         ▼
  snare-interpose:  import-table patch ──► per-thread hooks ──► Domain
                                                                 ├─ Layer stack   (clock, entropy, sleep)
                                                                 ├─ Net backend   (sockets, netlink, raw L2)
                                                                 ├─ Fs  backend   (files, /sys, /proc, /dev)
                                                                 ├─ Host backend  (scheduling, NIC tuning)
                                                                 └─ Env backend   (getenv/setenv)
                                                                 │ declined?
                                                                 ▼
                                                            the real OS
  • snare-interpose is the engine: the import-table patchers, the per-thread hook functions, the Domain (a thread's simulated world), the Layer/Net/Fs/Host/Env backend traits, and the real() escape hatch. It has no opinion about what is simulated — that is a backend.
  • snare is the ergonomic layer built on it: Sim/SimBuilder, SimHost (one backend that models a whole host), HostProfile/EasyBuilder to configure it, the Fabric (TCP/UDP/raw-L2 network), VirtualFs, and the connect_tester/run_testers! scripted-peer API.

A thread becomes managed while it is inside Sim::run, and every thread it spawns inherits the simulation automatically. Only managed threads are interposed; the test's own bookkeeping is not.

What is simulated

Surface Covered
Time clock_gettime (CLOCK_MONOTONIC/REALTIME/TAI), gettimeofday, time (the UCRT's _time64 on Windows), their raw Linux syscall forms, QueryPerformanceCounter, GetTickCount64, mach_absolute_time — a discrete-event virtual clock (the default, with or without a SimHost; wall_clock() opts out) that continues where the process's earlier sims left off, or starts at a fixed epoch under deterministic() or fixed_epoch(). Time jumps to the next pending sleep or timeout once every thread is blocked, and each call that returns without blocking costs a microsecond, so busy-polls still let time move. It can also run scaled to real time (time_rate), be set forward (set_time_value), and be paused, which holds every sleep and timed wait until time is moved from outside — see Controlling time.
Entropy getrandom/getentropy/arc4random/ProcessPrng, and Linux syscall(SYS_getrandom) (getrandom 0.2, so rand 0.8's thread_rng) — seeded, so HashMap order etc. is repeatable.
Local time A sim with TZ in an isolated environment, or an /etc/localtime of its own, has its own zone: localtime_r, localtime, mktime, timelocal, ctime_r, ctime and tzset, and on macOS CoreFoundation's CFTimeZoneCopySystem/CopyDefault, resolve it from the sim's environment and files. See Local time.
TCP std::net::TcpStream/TcpListener against scripted connect_tester peers, or against each other: the code under test can bind/listen/accept (accept4 flags on Linux) and connect to its own listeners, with shutdown(2) (SHUT_RD/SHUT_WR/SHUT_RDWR) and bind conflicts (EADDRINUSE, SO_REUSEADDR) as the host OS has them. Vectored readv/writev (std's and tokio's write_vectored) gather and scatter as on the host, a datagram socket's as one datagram.
UDP std::net::UdpSocket serviced from memory on every platform: bind/connect/sendto/recvfrom/sendmsg/recvmsg (and Linux sendmmsg/recvmmsg), blocking cross-thread receive, broadcast (SO_BROADCAST), multicast (IP_ADD_MEMBERSHIP, Linux IP_MULTICAST_ALL), Linux IP_RECVERR ICMP reports on MSG_ERRQUEUE, and several addresses sharing a port. Under a SimHost it adds SCM_TXTIME launch deadlines, and its sockets work with poll and epoll. TCP and UDP run together in one Sim.
Packet timestamps Kernel receive and transmit stamps from the sim's clock on the plain Sim and under a SimHost: Linux SO_TIMESTAMP/SO_TIMESTAMPNS/SO_TIMESTAMPING (SCM_TIMESTAMP*, UDP and TCP) and MSG_ERRQUEUE transmit stamps (OPT_ID, OPT_TSONLY, TX_SCHED/TX_ACK); macOS SO_TIMESTAMP/SO_TIMESTAMP_MONOTONIC/SO_TIMESTAMP_CONTINUOUS; Windows SIO_TIMESTAMPING on UDP (WSARecvMsg's SO_TIMESTAMP, SIO_GET_TX_TIMESTAMP by SO_TIMESTAMP_ID). A packet is stamped when it reaches the socket, after its link delay. See Packet timestamps.
Raw L2 Linux AF_PACKET/SOCK_RAW + bind(sockaddr_ll); macOS /dev/bpf* (BIOCSETIF/bpf_hdr). What EtherCAT masters (ethercrab) use. Interfaces come from the sim's topology (add_nic), and frames respect their link state and MTU.
Interfaces and routing One topology per sim (NicSpec, Route, add_nic, set_link, schedule_link, route_lookup) that every backend routes real traffic through: longest-prefix routes, source selection, carrier and admin flaps that stall TCP and drop datagrams, per-interface latency/jitter/loss (NicPolicy) and counters, and bind-to-device (SO_BINDTODEVICE, IP_BOUND_IF, IP_UNICAST_IF, or set_socket_device from the test) with the host OS's errors. See Interfaces and routing.
Interface enumeration if_nametoindex/if_indextoname/if_nameindex, getifaddrs, SIOCGIF*, SimHost sysfs, rtnetlink RTM_GETLINK/GETADDR/GETROUTE, macOS NET_RT_IFLIST2, and Windows IP Helper (GetIfEntry2, GetAdaptersAddresses, ConvertInterface*, GetBestRoute2) all read the topology. See Enumerating interfaces.
Readiness epoll + eventfd (Linux) and kqueue/kevent (macOS, incl. EVFILT_USER wakers), so mio-based code works natively on both. Edge-triggered registrations (EPOLLET, EV_CLEAR) are reported once per event as the kernels do — data arriving, an eventfd write or a NOTE_TRIGGER is a new edge even on an fd already ready — so a fired mio::Waker or an idle writable socket does not wake every poll; EPOLLONESHOT, EV_ONESHOT, EV_DISPATCH and EV_DISABLE/EV_ENABLE are honoured. Terminal errors follow edge-triggered delivery; newly visible errors or timestamp reports produce new edges. Level-triggered waits repeat while ready. poll(2), WSAPoll and Winsock select block and honour their timeouts.
Socket faults and back-pressure Raised socket errors (raise_socket_error), ICMP port unreachables (injected, from a tester, and sent back automatically after the round trip for a datagram no socket takes), one-way link stalls that hold traffic in flight (quiesce), TCP send and receive buffers (SO_SNDBUF/SO_RCVBUF) and a receive window (TcpPolicy::recv_window) that block sends with SO_SNDTIMEO and host-OS writable thresholds, and SO_LINGER aborts and waiting closes — each with the host OS's codes and call order. See Socket faults and back-pressure.
Connect faults A listening address refuses (ListenerBehavior::Refusing) or drops SYNs until an instant (DelayingUntil), and an address nobody answers at makes a connect wait out the host OS's SYN retransmission plan — Linux 131 s (tcp_syn_retries, tcp_syn_linear_timeouts, TCP_SYNCNT), macOS 75 s (TCP_CONNECTIONTIMEOUT), Windows 21 s (TCP_MAXRT) — in virtual time. Nonblocking connects report EINPROGRESS/WSAEWOULDBLOCK, then poll/epoll/kqueue/select and SO_ERROR, so connect_timeout and mio behave as on the real OS. See Connect faults.
Protocol counters The host's UDP and TCP counters, driven by the sim's traffic: Linux /proc/net/snmp and /proc/net/snmp6, macOS sysctl net.inet.udp.stats, Windows GetUdpStatistics(Ex/Ex2) and GetTcpStatistics(Ex/Ex2), and proto_counters() for the test. See Protocol counters.
Netlink AF_NETLINK rtnetlink RTM_GETLINK/IFLA_STATS64 (a dump, or one link by index or name), genetlink CTRL_CMD_GETFAMILY, RTM_GETQDISC/ETF.
Files VirtualFs: an in-memory tree behind open, sequential/positional IO (vectored readv/writev/preadv/pwritev too), truncate, sync, stat and directory-stream calls, with glob passthrough to real paths. File times, virtual and real, follow the sim's clock (see File timestamps). SimHost also renders /sys, /proc, /dev/ptp*, /dev/cpu_dma_latency (the request held open reads back through SimHost::cpu_dma_latency). Raw directory syscalls and path mutation are incomplete.
Host tuning Scheduling (sched_setscheduler/affinity/priority, mlock/mlockall), resource limits (RLIMIT_RTPRIO/NICE/MEMLOCK), NIC config (ethtool rings/coalescing/channels/pause/EEE/flags/flow rules/stats, SIOC[GS]HWTSTAMP, qdiscs and ETF offload, threaded NAPI, queues/IRQs), PTP, capability gating — all modeled, never touching the real scheduler or NIC.
Kernel identity Under a SimHost, uname (and Linux SYS_uname) reports the profile's sysname/nodename/kernel_release/kernel_version/machine: a fixed kernel by default, whose Linux version string carries PREEMPT_RT when the profile does, or the build machine's own with real_uname(). gethostname still reads the real name.
Tester stages A tester's message chain: then_test / then_stateful_test drop or rewrite a message before later stages, then_edit_state updates state, then_action / then_stateful_action answer — run in the order added.
Thread classes Every thread in a Sim is a participant unless marked (snare::sched::mark_background, mark_helper, mark_driver_thread, spawn_as): only participants hold up quiescence and the deterministic schedule. busy() / setup_scope() leases hold time still; thread names from pthread_setname_np / SetThreadDescription label each thread. SimBuilder::stuck_after fails a run a spinning participant has frozen. See Stuck runs.
Cooperative primitives snare::sched::park / current_unparker, block_on / block_on_until / block_on_timeout, the Sleep future (sleep_until), WakerSet and is_driven: in a sim they wait on its clock and schedule — counted toward quiescence, their deadlines virtual timers, their wakes held at an executive's gate — and off one on real time. See Cooperative primitives.
Executive An outside simulation owns a sim's clock in-process (Sim::executive, snare::sched::attach): it lets time flow in grants up to a horizon, jumps it timer group by timer group once the sim is quiescent, and acts at timestamps of its own whose effects reach the participants only when the timestamp ends — with a quiescence/participant/timer listing and an audit of what it cannot see. See Executive.
Socket table One record per open socket of the code under test on every backend (socket_table(), socket_entry(id), socket_id(&sock), closed_sockets(), sockets_bound(addr)): kind, local and peer address, the listener an accepted stream came from, multicast memberships, queued/delivered/sent counts, the pending SO_ERROR and when it opened and closed. See Socket table.
Socket limits and privileges SO_RCVBUF/SO_SNDBUF rounded as the host OS does, each datagram admitted against the receive buffer at its arrival (Linux truesize, macOS sb_cc/sb_mbcnt, Windows payload), FIONREAD/SIOCINQ/SIOCOUTQ, SO_NREAD, SO_MEMINFO, SO_RXQ_OVFL, SO_COOKIE; one Privileges per sim gating reserved-port binds, SO_*BUFFORCE, SO_PRIORITY/SO_MARK, interface rebinds, AF_PACKET, /dev/bpf* and a SimHost's capabilities. See Socket limits and privileges.
Names getaddrinfo/freeaddrinfo, getnameinfo, gethostbyname (and Winsock's GetAddrInfoW/GetNameInfoW) answered from a per-sim host table (add_host) plus a built-in localhost — unknown names fail with the host's own code (EAI_NONAME, WSAHOST_NOT_FOUND), never real DNS; numeric hosts and hints stay the real libc's. Per-name latency and seeded failures (set_dns_policy). See Names.
Signals sigaction/signal/raise/kill/pthread_kill for SIGINT/SIGTERM/SIGHUP (Windows: SetConsoleCtrlHandler, GenerateConsoleCtrlEvent, the CRT's signal/raise) against per-sim dispositions; Sim::raise_signal delivers as the host OS would, so ctrlc and signal-hook run unchanged. The semaphores and socketpairs their threads block on (sem_*, socketpair(AF_UNIX), Windows semaphores) are simulated too. See Signals.
Recorded events A sim-wide log (recorded_events(), Sim::recorded_events) of what crossed each tester's boundary, what link policies did to datagrams, policy changes, injected faults and delivered signals, stamped on the sim's clock.
Packet capture SimBuilder::pcapng(path) (or SNARE_PCAPNG_DIR for every sim, SNARE_PCAPNG_TESTS for chosen tests) writes what crosses the sim's network to a pcapng file Wireshark opens: fabricated Ethernet/IP headers around each TCP connection (handshake, segments, ACKs at arrival, FINs, resets, SYN retransmissions), each datagram once at its sender, ICMP port unreachables and raw L2 frames verbatim, stamped on the sim's clock per interface, optionally commented with real wall time. See Packet capture.
Environment getenv/setenv/unsetenv lookups and mutations are isolated (opt-in). macOS enumeration uses the simulated snapshot; Linux vars/vars_os still read the real process environment.
Windows The thread-scheduling plane (SetThreadPriority/affinity/priority-class/timeBeginPeriod, MMCSS AvSetMmThreadCharacteristicsW/AvSetMmThreadPriority/AvRevertMmThreadCharacteristics, power throttling through Set/GetProcessInformation and Set/GetThreadInformation, CPU sets) runs against a WinHost, read back with Sim::time_periods, mmcss_tasks, process_power_throttling, thread_power_throttling, process_default_cpu_sets and thread_selected_cpu_sets, and std::net::TcpStream/TcpListener/UdpSocket are serviced from memory by a Winsock fabric (socket/bind/listen/accept/connect/send/recv/sendto/recvfrom with MSG_PEEK/MSG_WAITALL and WSAEMSGSIZE truncation, the synchronous WSASend/WSARecv/WSASendTo/WSARecvFrom/WSASendMsg and the WSARecvMsg extension, blocking recv/accept, UDP broadcast/multicast/multi-IP, don't-fragment, WSAIoctl including SIO_CPU_AFFINITY, read back with Sim::socket_cpu_affinity). WSAPoll and select readiness and WSADuplicateSocket (try_clone) are modelled, and I/O completion ports cover Mio's single-socket \Device\Afd poll profile (see OPEN_BUGS.md for its bounds); a sim socket never reaches Winsock — overlapped calls and event/window-message notification on one fail with WSAEOPNOTSUPP. A pcap/npcap interposer is in place structurally.

Registered hooks record unsupported calls in Domain::unmodelled(). An empty report is a useful check, but does not prove full simulation: unhooked functions, direct process-global reads and CPU instructions are outside that accounting.

Configuring the host: SimHost, HostProfile, EasyBuilder

For code that tunes the host (real-time schedulers, NIC configurators like fast-talker), attach a SimHost describing the machine. The OS personality is fixed by cfg!(target_os) at compile time; what you configure here are host facts.

use snare::{HostProfile, Nic, LinkStats, Sim, CAP_SYS_NICE};
use std::net::Ipv4Addr;

let host = HostProfile::new()
    .cpus(8)
    .isolated(2..8)                       // /sys/devices/system/cpu/isolated
    .governor("performance")
    .preempt_rt(true)                     // /sys/kernel/realtime
    .cap(CAP_SYS_NICE)                    // grant a capability; setters EPERM without it
    .nic(Nic::new("eth0", 2)
        .mtu(9000)
        .driver("igb", "5.6.0")
        .hardware_timestamping(true)
        .address(Ipv4Addr::new(10, 0, 0, 10))
        .queues(4, 4)
        .link_stats(LinkStats { rx_packets: 1000, ..Default::default() }))
    .env("RUNTIME_ENV", "sim")            // isolates + sets the environment
    .kernel_release("6.6.30-rt30")        // uname -r; also sysname, nodename, kernel_version, machine
    .build();

Sim::builder().host(host).build().run(|| {
    // sched_setscheduler(SCHED_FIFO), ethtool, /sys reads, SO_TIMESTAMPING, getenv — all served here.
});

The profile's CPUs are what the code under test counts: sysconf(_SC_NPROCESSORS_CONF) is cpus, _SC_NPROCESSORS_ONLN the online ones, Linux sched_getaffinity the online set, and macOS's hw.ncpu, hw.activecpu, hw.logicalcpu, hw.physicalcpu (and their _max forms) and, on arm64, one performance level hw.perflevel0.* follow it. So num_cpus and std::thread::available_parallelism report it on both.

uname reports a host named snare running a fixed kernel (Linux 6.12.0, #1 SMP PREEMPT_DYNAMIC, or #1 SMP PREEMPT_RT with preempt_rt(true); macOS Darwin 25.0.0) on the build target's machine type. HostProfile::real_uname() copies all five fields from the machine running the test instead, and works inside a sim.

EasyBuilder gives batteries-included presets when you just want a working machine:

use snare::{EasyBuilder, Sim};

let host = EasyBuilder::realtime().build();   // 8-CPU PREEMPT_RT, all caps, a hw-timestamping NIC
// also: EasyBuilder::server() / laptop() / unprivileged() / minimal(), each tweakable:
let host = EasyBuilder::laptop().cpus(2).privileged(false).build();

Sim::builder().host(host).build().run(|| { /* ... */ });

Files: VirtualFs

use snare::{FsBuilder, Sim};

let fs = FsBuilder::new()
    .file("/etc/app.conf", "mode = fast\n")
    .dir("/var/run")
    .passthrough("/usr/share/**")   // globs that reach the real filesystem
    .deny("/secret/**")             // globs that always fail
    .build();

Sim::builder().fs(fs).build().run(|| {
    assert_eq!(std::fs::read_to_string("/etc/app.conf").unwrap(), "mode = fast\n");
});

Separate opens of a file share live contents; duplicated descriptors also share offsets and status flags. Writes and truncation are visible immediately. Absolute mkdir, unlink and rename operate on the virtual tree, preserve inode identity across moves, and keep unlinked files usable through open descriptors. Creation requires a declared directory parent, and pathname traversal checks intermediate components before resolving . and ... Relative path mutations, rmdir, getdents64, fdopendir, permissions and sparse storage need additional modelling.

An open file belongs to the process, as a descriptor does in a real one: a file opened in one sim and kept past its run (a log file behind a global tracing subscriber) is the same open file when another sim's thread, or a thread outside every sim, writes to it, and the bytes land in the file system that opened it. Paths stay per sim. Once that sim is gone, calls through the descriptor fail with EBADF, and its number stays reserved until the holder closes it, so it never reaches a file the OS opened in the meantime. With a SimHost as well, the host serves the paths it models (/sys, /proc, /dev) first, the VirtualFs the rest.

Virtual files take advisory locks: fcntl record locks (F_GETLK, F_SETLK, F_SETLKW and the F_OFD_* forms on Linux and macOS) and flock. Every thread is in the one process, so process record locks never conflict with each other and go when any descriptor of the file closes, as POSIX has it; open-file-description and flock locks conflict between separate opens and go with the description's last descriptor. A blocked F_SETLKW, F_OFD_SETLKW or flock waits like any other blocked call in the sim, so it lets virtual time move and keeps a deterministic schedule. SQLite's default VFS locks a database on a VirtualFs as it would on disk. See OPEN_BUGS.md for the file-plane boundaries.

File timestamps

File times follow the sim's CLOCK_REALTIME, so code that compares a file's age with SystemTime::now() (age-based log rotation, config reloads, pruning by creation time) sees a consistent world:

  • VirtualFs: every node keeps access, modification, change and birth times in nanoseconds. Declared nodes start at the instant the sim is built (FsBuilder::times with NodeTimes overrides any of them). Creation stamps all four and the parent's modification and change times; write, pwrite, ftruncate and O_TRUNC stamp modification and change; link, unlink and rename stamp the inode's change time and the parents'; reads and directory listings stamp the access time under Linux's relatime rule (only when it is not after the modification or change time, or is a day old). utimensat, futimens, utimes, futimes and macOS setattrlist/fsetattrlist (std's File::set_times, including macOS set_created) set what they name. stat, fstat, fstatat and macOS st_birthtime, and Linux statx (STATX_ATIME/MTIME/CTIME/BTIME), report them. Where Linux and macOS differ, the model takes the host's side: a read at end of file marks the access only on Linux; setting only the access time marks the change time only on Linux; on macOS a modification time set before the birth time moves the birth time back (fs_times compares each step with the host).
  • Real files in a sim on a virtual clock: what the sim creates or changes on the real file system is recorded per (device, inode) at the sim's time, by the same rules (writes through descriptors the sim opened for writing, O_CREAT/O_TRUNC, mkdir, symlink, link, unlink, rename, the utimensat family), and stat reports it; the real file keeps real times. A change the sim cannot see (another process's write, a write through a descriptor opened before the sim) shows at the sim's time of the first stat that notices it. A real file the sim never touched shows its host times shifted by one constant, the host clock minus the sim's when the sim was built: it keeps its age relative to the sim's start and its order among real files, instead of sitting years after a 2023 sim clock. A sim on the real clock (wall_clock) reports host times unchanged. A real file a sim opened for writing goes back to the OS when the sim drops, so a log file kept past the run stays writable.
  • SimHost files under /proc, /sys and /dev carry the sim's boot instant; like procfs and sysfs, the /proc and /sys ones report no birth time.

Local time

The C library resolves the local time zone inside itself, where no hook sees it: glibc reads TZ and opens the zone file through its own internal calls, and macOS's libc and CoreFoundation run from the shared cache. A sim that decides its zone — TZ set in an isolated environment (HostProfile::env, isolate_env), or an /etc/localtime its VirtualFs serves — has snare resolve it instead, following tzset(3), and read the files through the sim's file plane, so declined paths reach the real ones:

  • TZ unset: /etc/localtime (a symlink into the real /usr/share/zoneinfo is followed), or UTC when there is none.
  • TZ empty: UTC.
  • TZ (after an optional :) names an absolute TZif file or one under TZDIR (/usr/share/zoneinfo by default), or is a POSIX TZ string such as EST5EDT,M3.2.0,M11.1.0.
use snare::{FsBuilder, HostProfile, Sim};

let host = HostProfile::new().env("TZ", "Asia/Tokyo").build();
Sim::builder().host(host).build().run(|| {
    let mut tm: libc::tm = unsafe { std::mem::zeroed() };
    unsafe { libc::localtime_r(&1_700_000_000, &mut tm) };
    assert_eq!(tm.tm_gmtoff, 9 * 3600);
});

let fs = FsBuilder::new().symlink("/etc/localtime", "/usr/share/zoneinfo/Europe/Oslo");
let host = HostProfile::new().isolate_env().build();
Sim::builder().fs(fs.build()).host(host).build().run(|| {
    // localtime_r reports CET; on macOS iana-time-zone reports "Europe/Oslo".
});

localtime_r, localtime, mktime, timelocal, ctime_r and ctime use that zone, tzset makes every thread resolve it again, and on macOS CFTimeZoneCopySystem and CFTimeZoneCopyDefault return a CoreFoundation zone of the same name (or built from the sim's TZif bytes), so time's UtcOffset::current_local_offset, iana-time-zone and anything else reading local time through them see the sim's zone; chrono and jiff read TZ and /etc/localtime themselves. Rule times, mktime's handling of gaps and a tm_isdst the time does not have follow glibc on Linux and tzcode on macOS, checked against them by local_time_os_truth. A sim that decides no zone — no isolated environment and no /etc/localtime of its own, as a plain Sim — leaves local time to the C library and the machine's zone. See OPEN_BUGS.md for what stays process-wide.

Determinism and quiescence

Time is virtual, so a test that "waits a second" finishes instantly and every run sees the same sim times (and the same absolute timestamps under deterministic() or fixed_epoch()); randomness is seeded (SimBuilder::seed), per thread, so HashMap order, random ids and link faults replay. When every thread is blocked with no timer left to fire, the run is deadlocked and the blocked simulated wait gives up (EAGAIN) rather than hanging the test. A thread blocked on a word in static data (Linux futex, Windows WaitOnAddress; std's stdout and stderr locks, a static mutex) is the exception: a thread outside the sim, such as another test's, may hold it, so the run waits in real time for it to be let go instead of reading as deadlocked. Timers due at one instant all fire there: a time skip lands just past the earliest deadline (1 ns; on Windows 100 ns, or one QueryPerformanceCounter tick where that is longer, so Instant reads past it), and the next skip waits until every timed wait it reached has returned, so a park, condvar or channel timeout — which notices its deadline only between real-time slices of at most 2 ms — wakes at its deadline even when a sleeper on another thread is due at the same instant and would otherwise skip straight on. Likewise a thread woken from a native wait — a futex, condition variable or semaphore wake, or the exit of the thread it joins — holds off every time skip until it has run. A thread blocked on a lock that no thread of its sim holds — one held by another test running in parallel, an unmanaged thread, or std's own process-wide locks such as the one every thread start and exit takes — waits on the world outside, not on the sim, so it holds off time skips too until it gets the lock. The pthread mutex hooks record which mutexes each of a sim's threads holds, so a contended pthread mutex (std's Mutex on macOS) is judged exactly; a futex word names no holder, so on Linux a lock wait on a word in static data (std's own statics) first waits up to 1 ms of real time uncounted, which an outside holder in a short critical section lets go within, and a lock in heap memory held outside the sim is still counted as a wait in it.

Thread interleaving is deterministic too with Sim::builder().deterministic(): one thread runs at a time, and control passes only where a thread waits — a socket, sleep, mutex, condition variable, channel, park, join or yield — to the next thread in a fixed order. Those waits are emulated rather than handed to the kernel, so the waker always moves its waiters to the run queue itself and nothing is woken behind the scheduler's back; locks shared with threads outside the simulation (another test running in parallel) are waited out without letting their timing reorder this one, for up to 50 ms of real time each, after which the waiter passes the baton on and waits in the schedule for the holder's unlock (a pool's worker left over from an earlier sim may hold its queue's lock for as long as it waits for its next job). A wake made from outside the schedule (another sim's thread, a thread no sim manages) on a condition variable, futex, semaphore or address a participant waits on reaches it in the schedule, however busy the schedule keeps itself. With the same seed, a run replays exactly. What it cannot see it cannot order: a thread spinning on an atomic without ever yielding keeps the baton, and a call that blocks in the OS outside snare's hooks blocks every thread. A yield with no other thread able to run time-skips to the next pending timer, as a free-running yield does once every other thread is parked, so a yield loop waiting on a sleeper (crossbeam's Backoff::snooze, rayon's idle workers) gets there.

The sim's own code often runs on a participant from inside its hooked wait — the deterministic schedule's dispatch and the time skips in it, or an idle skip as a native wait begins — where std may already have parked the thread on its thread parker. std's parker tracks one park at a time, so a std wait there that parks the same thread again (a channel; on macOS also a contended OnceLock, Once or RwLock) would leave neither park able to be woken. snare's own process-wide singletons therefore never wait to be initialised, and the parker's blocking calls (dispatch_semaphore_wait, Linux futex, WaitOnAddress) treat a wait nested in another on the same object as a 1 ms real-time slice that returns as a spurious wakeup, and wake the outer park to re-check, so a layer that blocks in std there still makes progress.

Clock spins

Reading the clock never moves a discrete clock, so a thread waiting on time by polling it — while Instant::now() < deadline {}, with or without yield_now, spin_sleep's final spin, a loop that checks an unhooked channel between reads — would wait forever. snare catches such a clock spin and moves time for it:

  • A participant's clock reads and sched_yield/SwitchToThread calls count toward a spin; any other hooked call (a socket call, a sleep, a wait, a thread spawn, a lock that contends, a signal) ends it — a wait as it begins, whether it then blocks or finds what it waits for already there, while one whose timeout is zero or already past is a poll and leaves the spin running — and so does each of the sim's own waits, such as a tester waiting for its next message or tick: the clock reads of a tester's loop and handlers never add up to a spin. After 64 in a row the spin is caught: isolated reads, a few back-to-back reads, or reads with other hooked calls between them never get there and keep holding still. A loop polling a socket is not a clock spin: each non-blocking call is charged its microsecond instead.
  • Work the sim cannot see — computing, uncontended locks and atomics, allocation — can read the clock 64 times in a row too: an Instant::now() per item of a loop, a cache stamping each insert, an allocator reading its decay clock every so often. That is caught as a spin as well, so a spin's first steps cost what calls that do not block cost: each of its first 65,536 reads moves time 1 µs (a loop of 10,000 timestamped items takes 10 ms of virtual time, and 20,000 allocations under jemalloc about 18 ms). A spin that yields between its reads is waiting outright and skips this stretch.
  • After those, each read moves time 1/64 of the virtual time since the spin was caught, up to one second. The steps grow geometrically, so a spin on a deadline a second or an hour away takes about 66,000 or 70,000 reads (about 1,000 or 4,500 when it yields between them), a fraction of a second of real time, and a spin overshoots its own deadline by at most 1/64 of its length or 1 µs (one more read before any other hooked call moves it one more step). A step never jumps a pending timer: one that would reach the earliest timer lands 1 ns short of it, so a spinner whose deadline comes first sees it first, and the next lands just past it, as a time skip does, waking its waiter.
  • Time moves only while every other participant is parked or spinning itself, no woken waiter has yet to run and no lease is held, so other threads' work at the current instant comes first and timers fire in order. A free-running spin waits for a waiter its step reached to take its deadline (up to 5 ms of real time) before stepping on. Under deterministic() each step is a scheduling point: the spinner hands the baton to any runnable thread that is not spinning, and after a step lets the threads it released run before the next. Runs replay exactly.
  • A yield loop that does not read the clock waits on another thread, not on a deadline of its own, and keeps the full time skip to the next timer; it gets none while a clock spinner is about, since that would jump past the spinner's deadline. A thread that read the clock since its last yield is polling, and its yields only hand others a turn.
  • A paused clock, or one an executive drives, is never moved by a spin: the spinner keeps reading the held instant, yielding the CPU, until something outside the sim moves or resumes the clock; stuck_after reports it. A spin that moves time is progress to the watchdog. A scaled or as-fast-as-possible clock moves on its own and a wall_clock() sim reads real time, so neither is touched. Only participants' spins count; a background or driver thread spinning on the clock is left alone.

Thread classes and leases

Every thread created inside Sim::run is simulated — its sockets, clock, files and environment are the sim's — and by default it is a participant: time skips only once every participant is blocked, a blocked wait gives up only when they all are, and deterministic() runs participants one at a time. A thread the run should not wait on changes its class with snare::sched:

Class Set with Effect
Participant the default; participate(name) for a scope Counts toward quiescence and runs in the deterministic schedule.
Background / Helper mark_background(label) / mark_helper() Still simulated, but never holds up quiescence or the schedule. Its sleeps and timeouts are satisfied as the clock passes them rather than steering a time skip, and its non-blocking calls cost no time.
Driver mark_driver_thread() As Background, and its clock reads, sleeps and timeouts (socket, poll and condition-variable timeouts alike) are real; sched::now() reads the sim's clock.

spawn_as(class, label) makes the threads the caller creates while the guard lives start in class (for a pool or library that spawns its own threads); later children, and a background thread's own children, are participants again. Sim waits (a receive, a poll, an accept) still give up when every participant is blocked with no participant timer left, however many background timers are pending. Once no participant can give up — each is in a join, a lock, a condition variable or a channel receive, waiting on what only a background thread can do, as when the test stops a sampler and joins it — time skips to the background threads' timers instead, so the join returns. Under deterministic() a thread that leaves the schedule (mark_background on a running participant, or a participate guard dropping) hands the baton on as it goes, and a thread of another class runs beside the schedule: whatever it wakes — a channel send, a condvar signal, a socket write — still reaches the schedule, but when it runs relative to the participants depends on the OS, so its effects on them only replay if they wait for it. The same holds for a background thread entering the schedule through participate: the point where it joins depends on real time, so the run replays only if the participants wait for it there (for a message it sends from inside the guard, say).

busy(label) (Send; also Sim::busy) and setup_scope(label) are leases: while one is held the sim is never quiescent, so time does not skip and no blocked wait gives up, even on the discrete clock; a deterministic schedule idles until the lease is dropped. Taking one waits out a time skip already in flight, so from any thread, inside the sim or not, no skip lands once busy returns. A lease has a kind: busy takes a LeaseKind::Busy one, for work the sim cannot see that lasts about as long as a reaction, and setup_scope a LeaseKind::Setup one, which may last as long as setup takes without anything being stuck. held_leases() lists them as LeaseInfo { label, kind, holder }, the holder being the name of the thread that took it. Off a sim both return an inert lease (is_held() is false) rather than panicking, so code shared with a real deployment can take them unconditionally; sched::in_sim() says whether the calling thread runs in a sim at all, and sched::current_sim() which one (a SimId, also Sim::id). A thread's name is whatever it set through pthread_setname_np or SetThreadDescription (std's Builder::name), which go to the OS unchanged — a Linux name over 15 bytes still fails with ERANGE — or thread-<lineage> otherwise.

Stuck runs

A participant that runs without making a hooked call — spinning on an atomic, or computing — keeps the sim from going quiescent, so the discrete clock cannot move and every sleeper it may be waiting for waits for good: the test hangs. SimBuilder::stuck_after(d) (off by default) turns that into a failure. A watchdog thread outside the sim samples it every eighth of d (between 1 ms and 250 ms) and fires once, for d of real time, a participant has been running while none of the sim's progress has happened: no participant blocked or woke, no lease was taken or given back, virtual time did not move (a time skip, a clock write, a scaled or granted clock flowing, the microsecond each non-blocking call is charged, a clock spin's step), and under deterministic() no participant began a wait in the schedule. It never fires while every participant is blocked — on a paused clock, on an executive, on locks only a thread outside the sim can release — since the design waits for those, nor while a lease is held or an executive's timestamp is open, nor on a wall_clock() sim, whose time moves on its own. A participant that legitimately runs longer than d with no hooked call (a long computation, a blocking call snare does not hook, work inside real) holds busy() across it. Off by default because that work looks exactly like a spin.

A stuck thread cannot be made to panic from outside, and the test would never finish, so the watchdog attempts its report on the process's stderr — past the test harness's output capture — with a 100 ms reporting budget, then aborts the test process. Locked or blocked stderr may leave the report incomplete, and OS scheduling may delay detection and termination. The report names every participant: running (and, under deterministic(), which one holds the baton), or what it waits in and until when, what it last waited in, and the leases it holds.

The same listing explains a deterministic schedule that has idled for a second with every participant blocked — on a held clock, on a lease, or on locks only something outside the sim can release: that is waiting rather than stuck, so it is a one-time warning on stderr and the schedule carries on.

Sim lifecycle

A Sim is running while at least one thread is inside Sim::run, and dormant between runs and after it is dropped. Threads spawned inside a run that are still alive when it returns — a background loop the test never stopped, a pool's idle workers — stay in the sim: their clock, sockets and files are still its own, and the sim's state lives as long as any of them does, even past the Sim's drop. But nothing waits on them any more, so while the sim is dormant:

  • Time moves for them at real time's pace. A time skip lands no further past the reading the run ended at than real time has moved since, so a leftover thread sleeping in a loop sleeps in real time (a near-idle CPU) instead of racing the virtual clock ahead at full speed. An as-fast-as-possible clock holds still between such skips too. With no leftover thread waiting on a timer the clock does not move at all, so a sim without leftovers starts its next run where the last one ended. A paused clock, or one an executive still drives, stays held.
  • A wait with nothing left to wait for blocks instead of giving up as a deadlock: a receive no thread will ever answer stays blocked (polling a few times a second) rather than failing with EAGAIN, so a leftover loop that retries on errors does not spin.
  • They no longer count anywhere else: not toward another sim's quiescence (each sim's woken waiters are its own, however many sims run in parallel), not toward the stuck_after watchdog (a leftover thread spinning on an atomic is the test's own business once the run has ended), and a deterministic schedule has already let them go to run freely.
  • They can still be stopped and joined from outside: set the flag they poll and join them; their sleeps end in real time.

The next Sim::run on the same sim takes its leftovers back: they are its participants again, and time skips go back to virtual time at once (monotonic time never goes backwards across the switch). A test that wants none of this stops its threads before the run returns.

Process-wide pools. A pool kept in a static — rayon's global registry, a LazyLock of a futures ThreadPool, a threadpool, a static tokio runtime's workers and blocking pool — starts its threads under whichever sim touches it first, and they park on condition variables, futexes and semaphores between jobs. A later sim that hands one of them work wakes it, and the work is that sim's: a participant of a dormant sim woken by a thread of a sim whose run is in progress follows the waker into its sim, as a thread it had spawned, before it runs another line. It runs on the later sim's clock, in its schedule, and counts toward its quiescence, so time does not skip while it computes for it. Its place there is made ready at the wake, before the OS wakes it, and counted live, so the later sim cannot read as quiescent in between; a wake the OS may give to any of several waiters (notify_one on a shared queue) makes as many places as it can wake, each for whichever dormant waiter returns first. A place nobody takes within 200 ms of real time — the OS woke a thread the census never saw — is given back. Under deterministic() the place joins the schedule where the wake was made, as a spawn does, so runs replay. To leave no worker still spinning between jobs (rayon's yield 32 times before they park), a run's end waits, a bounded real time, until its leftover participants have all come to rest in a wait. Threads of a sim whose run is in progress never move, nor do background, helper or driver threads; a pool two sims share while both are running still serves both from one world (see OPEN_BUGS.md).

A new sim continues the process's time. Each plain sim's clocks start where the furthest clock of every sim built before it had got to (see Controlling time), so a process static that remembers a timestamp from an earlier sim — a static tokio runtime's timer wheel, a global memo or DNS cache — still sees its deadlines ahead and its entries age. A deterministic() or fixed_epoch() sim starts at the fixed epoch instead, where such a static stays stale: a cache entry from a sim that ran further may never expire in it.

Process-wide reactors. An epoll set, a kqueue, an eventfd or a timerfd belongs to the process, not to a network, and a library may create one once per process: async-io keeps its poller and the "async-io" thread that waits on it in statics, made under whichever sim touches it first. When a running sim reaches such a descriptor of a dormant (or dropped) sim, it takes over every one that sim holds, and the leftover threads blocked on them join it, as threads it had spawned: they run on its clock, in its schedule, and count toward its quiescence. A timerfd keeps the time it had left. One closed outside every sim is gone, and its number is the OS's to reuse for a real file. Network sockets never move: a world has no instance of another world's connection, so another sim's socket (or one gone with its sim) is a connection that is gone there, as a connection pool kept in a static (ureq's agent, reqwest's client) must find it: its first read or write reports ECONNRESET, later reads end of stream and later writes EPIPE, poll, epoll and kqueue show the hang-up and the error, and getpeername is ENOTCONN. A registration of such a socket in an epoll set or kqueue that moves to another sim stays, and reports the same. Closing it, from any sim or from a thread outside every sim, closes it in the sim that made it, whose peer sees the connection end, and gives its number back to the OS, so std's I/O-safety check never sees a close fail. A socketpair end has no network in it either, but it carries bytes, so instead of moving it is mirrored: each sim that reaches one another sim made (tokio keeps its signal self-pipe in a static and every runtime with I/O dups it) gets an instance of the pair of its own under the same numbers, both ends open, and bytes written in one world are never read in another. Its numbers are the ends of a real socketpair, the outside world's instance, so a thread outside every sim (a runtime built after the sims) uses a working pair. Every other descriptor of a sim's network (a socket, an epoll set, a kqueue, an eventfd, a timerfd) is a dup of a placeholder that fails loudly when a call no hook serves reaches it: the read end of a pipe whose writer has gone, so a write fails with EBADF and a read is at end of file. A wake of another running sim's reactor (an eventfd write, a kqueue EVFILT_USER trigger) is delivered to it as a wake from outside its world. Two sims running at the same time cannot otherwise share one such object: a sim that reaches a descriptor another running sim holds waits until that run ends, but one that only touches the library's memory (a timer queued while the reactor's notifier is already pending) gives snare nothing to wait at, so tests that share a process-wide reactor must take turns (a static Mutex held across the test, or --test-threads=1). cargo snare test --parallel <smol|async-std|rayon|all> patches in shims that give each sim its own reactor, executor and thread pool and lift this for async-io, smol, async-std and rayon's global pool (see Shims).

Controlling time

The clock every Sim runs on can be driven from the test, through Sim methods or a TimeHandle (sim.time(), or snare::time() from any thread running inside the sim):

Call Effect
set_time_rate(r) / SimBuilder::time_rate(r) r in (0, 1e6]: virtual time runs at r times real time (larger values clamp). 0.0 pauses. f64::INFINITY returns to virtual time — discrete-event, or as-fast-as-possible for a SimHost or Windows sim built with wall_clock() — keeping the reading. Negative or NaN panics.
time_rate() 0.0 while paused, the rate while scaled, f64::INFINITY on virtual time.
pause_time() / resume_time() Pausing holds the clock: reads return the same instant, and every sleep, socket timeout, poll timeout and condition-variable or futex timeout blocks until time moves, as nanosleep(2) and Sleep wait out their whole interval. Resuming restores the mode and rate in force before the pause.
advance_time(d) Moves the clock forward by d, paused or not, waking what it reaches.
set_time_value(v) / time_value() Sets sim time — how far the clock has moved since the sim was built — to v, CLOCK_MONOTONIC and the realtime clocks moving with it; forward only (setting it back panics and leaves the clock alone). time_value() reads without ticking.

Monotonic time never goes backwards, whatever moves the clock: every monotonic source is non-decreasing on each thread and across threads that synchronize, every mode switch re-anchors at the last reading handed out, and a request to move the clock back is refused. Realtime moves with monotonic time, a fixed boot time apart.

Where a sim's clock starts. A plain sim's clock continues the process's time: CLOCK_MONOTONIC (and CLOCK_BOOTTIME, Instant, mach_absolute_time, QueryPerformanceCounter, GetTickCount64) starts at the highest reading any sim's clock in the process has handed out — every sim built before it, the ones still running included — plus at least a second, on a whole second (sim time only: no real time passes), and CLOCK_REALTIME (SystemTime, FILETIME, CLOCK_TAI with it) no earlier than the highest realtime reading. So a library that keeps a timestamp in a static — tokio's timer wheel in a static runtime, a memo or cache expiry, a time-zone cache, a UUIDv7 counter, a coarse clock's "recent" value — never sees time go backwards from one sim to the next. A sim already running when another starts keeps its own timeline: only its readings up to then count. A deterministic() sim, or one built with SimBuilder::fixed_epoch(), starts at the fixed epoch instead: monotonic zero and realtime 2023-11-14T22:13:20Z (Unix time 1 700 000 000 s, a snare choice), so a replay reads the same absolute values; a static timestamp from an earlier sim may then be ahead of its clocks. Sim time — time_value, set_time_value, recorded events, the socket table, an Executive's times — counts from the clock's start either way, so it reads the same in any order. Realtime minus monotonic, the sim's boot time, never changes within a sim. A SimHost's clock is placed once, by the first sim built on it, and later sims on the host continue it. An Instant taken outside every sim, or in a wall_clock() sim, is on the real timeline and may look ahead of one taken inside. A driver thread is the one exception by design: it reads real time, and an executive's timestamp time while it acts at one.

A paused clock is moved by something outside the simulation — the test thread before or after run, or a thread it starts under snare::real — so a sim where every thread waits on a paused clock is waiting, not deadlocked; it says so on stderr after a second. Once resumed, a wait nothing can satisfy gives up as before. On an as-fast-as-possible clock, a timed wait that is still held when the clock resumes times out then, the clock jumping to its deadline as a sleep's does; one already running when the clock is paused or scaled keeps the time it had left and holds with the clock. Under deterministic() the clock can be paused, advanced and set; a write made while every simulated thread is blocked on the paused clock releases the waiters in the schedule's fixed order, but one made while a simulated thread runs lands wherever real time puts it in that thread's work, so the run only replays if outside writes wait for the sim to block; an Executive's jump_to and enter_timestamp_checked act only while the schedule is idle, so they always do. A real-time rate is refused, since nothing in a deterministic run may depend on real time. A plain unix Sim built with wall_clock() runs on the real clock and has none of these controls; time_rate(1.0) gives a controllable clock that tracks real time. With a SimHost, rate, value and pause apply to its timestamps, PHC samples and SO_TXTIME alike. Scaled time follows real time, so it is not reproducible run to run.

On Windows, Rust sleep durations that exceed its signed 100 ns timer range reach the simulated clock before conversion, including std::thread::sleep(Duration::MAX). They saturate at the clock's finite limit. Ordinary Rust sleeps retain the Windows timer grid; native Sleep(INFINITE) and non-alertable SleepEx(INFINITE) still wait indefinitely. This uses a typed hook on the linked Rust sleep implementation, following transparent entry tail jumps. Entirely inlined implementations and separately linked std copies need their own instrumentation. Hook installation errors are reported immediately.

Executive

A simulation that drives the code under test from outside — sham inside theater — owns the sim's clock through an Executive, in the same process: sim.executive(ExecutiveConfig::default()) from any thread, or snare::sched::attach(cfg) from a thread inside the run. Attaching fails with AlreadyAttached, NotInSim (the free function off a sim's thread) or WallClock (a plain unix sim on the real clock, which has no clock to own). From then until the executive drops, only it moves time: there is no time skip on quiescence and no deadlock give-up — a wait nothing can satisfy waits for the executive, and a deterministic schedule idles — and pause_time, resume_time, advance_time, set_time_rate, set_time_value and their TimeHandle forms panic with "this Sim's clock is owned by an Executive". The clock is frozen at its reading until the first grant.

Call Effect
grant(Grant { anchor_v, anchor_wall, rate, horizon }) Time runs from anchor_v at the real instant anchor_wall, rate virtual seconds per real second (in [0, 1e6], NaN as 0), never past horizon. An anchor behind the clock anchors at the clock's reading, now. Timers the clock flows past fire as real time reaches them.
freeze() Holds the clock where it is (horizon at the reading, rate 0).
next_deadline() The earliest participant timer after the reading. Background timers are never jump targets; they fire as the clock passes them.
jump_to(t) If the sim is quiescent — checked under the sim's locks in the same step as the move, so no participant starts running in between — moves the clock to t or to the earliest participant timer before it, exactly onto that deadline, and returns how many waits and events were due there; Err(NotQuiescent) changes nothing. The horizon rises to reach just past the landing — 1 ns, or on Windows 100 ns or one QueryPerformanceCounter tick where that is longer — so a wait that recomputes a zero timeout at its deadline can creep over it on the latency its call is charged; the rate stays the grant's. Code that runs after the landing may therefore read up to that much past it.
enter_timestamp(t) / leave_timestamp(t) The calling thread becomes the driver and acts at t: it reads exactly t from every clock and the code under test reads at least t, while the clock itself stands just before t, so timers at t are still pending. Participants released meanwhile — by what the driver sends, or by timers before t — wait at a gate, still counted blocked, until leave_timestamp(t) moves the clock to t (its horizon just past t, as for jump_to), fires what is due there (returning the count) and opens the gate. It wakes only waiters something reached: a timestamp at which nothing was sent and nothing came due leaves a quiescent sim quiescent.
enter_timestamp_checked(t) As enter_timestamp, only if the sim is quiescent and no participant timer falls before t, checked in the same step; on Err nothing changes but the calling thread becoming the driver.
with_driver_time(t, f) On a driver thread (a pool worker doing the executive's work), runs f reading t as the sim's time; its sends are held at the gate like the executive's own.
quiescence() quiescent, the epoch, runnable/blocked participants, leases, next_deadline, and the first blocker: Runnable (a running participant, by name) > Settling (a woken waiter, a participant a futex, condition-variable, semaphore or WaitOnAddress wake released, one waiting for a pthread mutex that is now free, a joiner whose thread exited, or a participant the gate let go at the end of a timestamp, yet to run) > Lease (a busy lease) > Setup (only setup leases) > Deferred (held at the gate) > TimerDue (a timer at or before the reading its waiter has yet to take).
arm_notify(epoch, f) Calls f once the epoch moves past epoch — a participant parking or waking, a lease taken or given back, a class change, a deterministic schedule going idle — on the thread that moved it, with no lock of the sim held.
participants() / timers(n) / held_leases() / drain_hints() Every participant (lineage, name, running or blocked, for how long in sim and real time, what it waits in, its deadline, what it last waited in, the leases it holds) then one row per lease, with its lease_kind; the earliest timers, participants' before the others', with their owners' names; the leases with their kinds; the hint_starving hints since the last drain.
outside_wakes() Unmatched returns from participants' native waits since the executive attached: nothing snare observed since quiescence explains the return. Always counted, with or without an audit. Spurious OS returns, such as those allowed by WaitOnAddress, can contribute; this is not proof that an unmanaged thread released the waiter. A run requiring every wake to be accounted for asserts it stays 0.
audit() With ExecutiveConfig { audit: true } (or SNARE_SCHED_AUDIT set to anything but empty or 0): background and helper threads' sim-visible effects (sends, connects, closes, futex/condvar/semaphore/address wakes, sleeps, busy), counted apart inside a setup_scope; the outside wakes; and every call the sim cannot model, as a violation.
thread_census() / sim_id() Every OS thread of the process and whose it is (see Thread census); the sim's id.

Dropping the executive (or detach) returns the clock to its base mode at its reading — never backwards, and past an open timestamp's time — opens the gate, and time skips and deadlock give-ups resume with the OS's own errors.

The gate holds participants leaving a sim wait (a socket, poll or sleep), a futex, a semaphore, a WaitOnAddress, a condition variable or a join. A condition-variable wait woken inside a timestamp lets go of its mutex while it waits at the gate and takes it back after, which the caller sees as an ordinary wakeup. A contended mutex returns holding the code under test's lock, so it passes the gate rather than risk blocking the executive's own threads on that lock. A participant already running when an unchecked enter_timestamp begins can see what the driver does at t early, as can background threads, which the gate never holds. Under deterministic() a timestamp also holds the schedule's baton: enter_timestamp waits for every participant to block, enter_timestamp_checked takes it only if the schedule is idle, and leave_timestamp hands it to the participants in the schedule's own order, whatever order the data arrived in. A grant's rate runs as 0 there, with a one-time warning, so a deterministic run driven by jumps and checked timestamps replays exactly; a flowing grant follows real time by design.

Outside deterministic(), a participant released by another thread's wake counts as Settling from the wake until it runs: the futex, condition-variable, semaphore and WaitOnAddress hooks count, before the OS wakes anyone, how many participants were parked on that address (up to the wake's count), and an exit counts its joiners. A pthread mutex is judged by who holds it instead: the lock, trylock and unlock hooks track the holder of every mutex a participant waits for, so a participant in a contended lock is Settling only while that mutex is free or held outside the sim, and one woken from a condition variable only while its mutex is free or its own. A holder that re-takes the lock before the woken waiter runs, or that notifies and then sleeps holding the lock, leaves the waiter blocked and the domain quiescent. A wake the OS hands to a thread that is not a participant (a background thread waiting on the same address) leaves the parked participant counted Settling until it next wakes, and a wake or unlock from a thread outside the sim is not seen at all.

Whatever makes a participant runnable keeps the sim from being quiescent until that participant has run: a datagram, bytes or a connection delivered to a socket it waits on (and any other readiness change) marks every waiter woken as Settling until it has taken the lock again and re-checked — a waiter that let go of the lock for a moment on its way back to sleep counts as asleep meanwhile, so a wake landing then marks it too; an Unparker::unpark is such a readiness change; a futex, condition-variable, semaphore or address wake counts as above; and a participant the gate held stays counted until it has left its wait, so the next enter_timestamp_checked cannot slip in between the gate opening and the participant running. A datagram still in flight is a timer at its arrival, not a pending wake. A readiness change wakes only the waiters of the sim it belongs to (and threads outside every sim): sims driven side by side by executives of their own never wake each other's waiters, so none reads Settling because of a neighbour, and a jump made right after reading quiescent goes through however busy the sims beside it are.

A driver time (enter_timestamp, with_driver_time) lapses on every thread when the executive is dropped: the driver reads real time again.

Migrating sham and theater: sham and theater are the simulator and are never interposed. Theater builds the Sim (seed from SimBuilder::seed) and hands it to sham's SnareDomain::new(&sim, cfg), which owns the clock through sim.executive(cfg) in place of the old attach_driver, before the code under test starts. The code under test (conductor) runs inside sim.run; everything theater does with sham, and every thread sham starts, runs under snare::real, so none of it joins the sim and nothing marks thread classes or uses driver time. Theater talks to the code under test only through the Sim API: its network (testers, or emulator threads woken from outside), faults and time. There is no global slot or register_test, and no environment handshake; the thread census is sched::thread_census() (below).

Thread census

sched::thread_census() (also Sim::thread_census, Executive::thread_census) lists every OS thread of the process — read from /proc/self/task on Linux, task_threads on macOS and a Toolhelp thread snapshot on Windows — as a CensusThread { os_id, name, owner }, where owner is ThisSim(class) for the calling thread's (or that) sim, OtherSim(id, class) for another sim's, Snare for snare's own service threads (the link-delay and timer wakers, an executive's flow thread, a stuck watchdog, the signal forwarder) and Unmanaged for everything else: the test harness, a thread started outside every sim or under snare::real, one a runtime started for itself. ThreadCensus::unmanaged(), this_sim() and other_sims() filter it. A thread counts as a sim's from its own first line in the sim until it leaves, so one being created or exiting can read as unmanaged for that moment; a run that fails on unknown threads should want one found unmanaged in two censuses in a row, as theater's tracker did with snare v1.

Cooperative primitives

snare::sched has the blocking and future primitives a simulation's own runtime (sham's participants) is written against. In a sim they run on its clock and its schedule; off one, on real time.

Call Effect
park(deadline) / current_unparker() Blocks until the thread's Unparker fires or the deadline passes (ParkResult::Unparked / TimedOut). An unpark that came first is consumed and returns at once. In a sim the park is a sim wait: it counts toward quiescence, its deadline (an Instant, read on the sim's clock) is a timer a time skip or an executive's jump lands on, a wake during an executive's timestamp is held at the gate, and under deterministic() it waits in the schedule. A participant parked with no deadline while nothing in the sim can move keeps waiting, as a join does. Unparker::unpark and Unparker::waker() work from any thread.
block_on(f) / block_on_until(f, deadline) / block_on_timeout(f, d) Polls f, parking between polls with the thread's Unparker as the waker. Past the deadline f is polled once more, so an output ready at the deadline is not lost; a timeout past the end of the clock is no deadline.
sleep_until(instant) → Sleep A future that completes when the clock reaches the instant: on its first pending poll its waker becomes a timer of the calling thread's class (a participant's steers time skips, a background thread's fires as the clock passes it or, once every participant waits on what only such a thread can do, steers them too), which the discrete time skip, Executive::jump_to, a flowing grant, a scaled clock reaching it (a real-time waker set for the real arrival) or a clock writer fires. Later polls replace the waker; dropping it, or completing, removes the timer. is_elapsed() checks the clock. On an as-fast-as-possible clock waiting on it jumps the clock to the instant, as a sleep does. Any waker works, not only block_on's: a hand-rolled executor blocking in a channel receive or a condition variable between polls is woken at the instant, under deterministic() too. The clock runs the wakers it fires on a thread of its own, outside the sim.
WakerSet The wakers of every task waiting on one piece of state: register before checking the state, wake_all after changing it.
is_driven() The calling thread is a participant and the sim's clock is virtual (discrete, scaled, paused or owned by an executive).

A driver thread outside an executive's timestamp reads real Instants, so its Sleeps and park deadlines are on real time; such threads should time their waits with real-time timers, or under with_driver_time.

Tester stages

Each message the code under test sends a tester runs through the tester's stages in the order they were added:

Stage Effect
then_test(|msg, from| Option<P>) / then_stateful_test Some passes that message — rewritten or not — to the stages after it; None drops it there.
then_edit_state(|state, from| ..) Updates the tester's state, then passes the message on unchanged.
then_action(|msg, from| TesterAction) / then_stateful_action Acts on the message at once; later stages still see it.

A stage added before a dropping test has already run for that message. recording() keeps what the code under test sent, before any stage sees it, and on_connect and cyclic actions do not go through the chain. Stages added before with_state keep working without the state.

Framing

A tester's message type says how the byte stream is cut into messages (Packet::parse). Besides Bytes (whatever each read returns) and Line (\n-ended text), two framings cover most device protocols without a hand-written Packet; both keep the raw frame (frame()) and send frames as they are:

Type Frame
Delimited<D> Up to and including the first D::DELIMITER: Lf, Cr, CrLf, or any byte string on a marker type implementing Delimiter. Delimited::new(body) appends the delimiter; body() / text() drop it.
LengthPrefixed<L> L::FIELD: LengthField { offset, width, endian, adjust }: the unsigned width-byte (1, 2, 4, 8) field at offset, Endian::Big or Little, plus adjust, is the whole frame's length — adjust is the header and trailer size when the field counts the payload, 0 when it counts the whole frame.
/// [0x02][type][len: u16 LE][payload][0x03]: the length counts the payload only.
struct Gcom;
impl FrameLength for Gcom {
    const FIELD: LengthField = LengthField { offset: 2, width: 2, endian: Endian::Little, adjust: 5 };
}
let tracker = connect_tester::<LengthPrefixed<Gcom>>("10.0.0.5:820");
let robot = connect_tester::<Delimited<CrLf>>("10.0.0.6:16001");

Over TCP a frame split across reads waits for the rest and several frames in one read come out one by one. Over UDP each datagram is parsed on its own: it may carry several frames, a frame never spans datagrams, and an incomplete tail is dropped with its datagram (with Bytes, one datagram is one message). A LengthPrefixed field giving a frame shorter than the field's own end panics the tester, which fails the test.

Cyclic actions and phase

with_cyclic_action(period, ..) runs first one period into the run, then every period; with_cyclic_action_at(period, phase, ..) runs at phase, phase + period, ... — so two emitters on an 8 ms cycle with phases 4 ms and 8 ms tick exactly half a cycle apart. Every tester in one run_testers! is phased from the same start, read on the sim's clock, and the next tick is always the previous one plus a period, so ticks do not drift. with_stateful_cyclic_action[_at] adds the tester's state.

Per-connection state

with_state(S) is one value for the whole tester. with_conn_state(|peer| C) adds one value per peer, made as it appears — per TCP connection accepted (a reconnect starts fresh), per UDP source address — and the conn methods see it beside the tester's state:

Method Sees
then_conn_action(|conn, state, msg, from| ..) / then_conn_test The sender's C; stages in the same chain as the others.
on_conn_connect(|conn, state, from| ..) The new peer's C, just made.
with_conn_cyclic_action[_at](period, [phase,] |conn, state, peer| ..) Each tick, once per live peer (open connections, every UDP source) in the order they appeared, with its C; a plain Send goes to that peer only. A plain cyclic action sees no C and sends to every peer.
until_conns(|state, conns| bool) / inspect_conns(|conns| ..) Every peer's (SocketAddr, C) of the run, closed connections included, kept until the next run.

with_state and with_conn_state go in either order, and stages added before either keep working without it.

Recorded events

Every Sim keeps one log of what happened at the edge of the code under test, readable with snare::recorded_events() from any of its threads or sim.recorded_events() from anywhere, during the run or after it. Each RecordedEntry has at, sim time (time_value()) without ticking the clock (real time since the sim was built on a plain wall_clock() sim), a seq that only ever grows, and the RecordedEvent:

Event Logged when
Accepted a TCP tester accepts a connection, before on_connect runs.
Received { transport, len } a tester parses a message; len is the bytes it was parsed from.
Sent { transport, len } a tester's bytes go toward the code under test.
Closed / Reset a tester closes or resets a connection, once, including the closes at the end of its run.
PeerClosed / PeerReset the tester reads end-of-stream, or a reset it did not cause.
Quiesced { span } / Suppressed { toward, len } a tester goes silent toward a peer, and each message it then drops in either direction.
Link { from, to, len, fault } a UdpPolicy loses a datagram, drops it for its MTU (TooBig) or duplicates it. A perfect link logs nothing.
UdpPolicyChanged / TcpPolicyChanged set_udp_policy / set_tcp_policy, with the policy now in force.
Fault { addr, fault } an injected fault takes effect on the code under test: Error { errno, call } when a call reports a raised error, IcmpPortUnreachable { from } when a call reports an ICMP error, Stalled { span, direction } when quiesce holds a link, Unreachable { errno } when a send or connect finds no route or a connect finds no neighbour, LinkDown { nic } when a send leaves through an interface without carrier; Fault::Dns { name, failure } when a DnsPolicy fails a lookup; ConnectRefused (a Refusing listener), ConnectTimedOut and ConnectDelayed { until } when a connect that waited on its SYN ends.
NicChanged { nic, admin_up, carrier } an interface is added, removed, or changes admin state or carrier (a scheduled change logs when it takes effect).
Signal { signal, origin, delivery } a signal or console control event reaches the code under test, sent by the test (Sim), the code itself (Process) or forwarded from the real process (Real).
UnmodelledOption { socket, local, option, refused } the code under test first uses, on that socket, a socket option or ioctl the sim does not model; refused under strict_sockopts. See Unmodelled options.

clear_recorded_events() starts a fresh phase (sequence numbers carry on), and SimBuilder::record_events(false) keeps nothing, for a long, chatty run. Recording never moves the clock, yields or wakes a thread, so a deterministic() run logs the same entries, in the same order, with the same stamps, every time, and logs or not without changing the run; on the free-running virtual clock, events from different threads at the same instant may come out in either order. Coming from the original snare: UdpSendFromTest { dropped } is a Sent followed by a Link { fault: Lost | TooBig }.

Packet capture

Sim::builder().pcapng(path) writes every frame that crosses the sim's network to a pcapng file, created (or truncated) at build — which panics if it cannot be — and finished when the Sim drops. Sim::pcapng_path() says where. With no explicit path, setting SNARE_PCAPNG_DIR=<dir> captures each sim to <dir>/<building thread's name>.pcapng (: becomes -, other characters outside [A-Za-z0-9._-] _, and a second sim built by the same thread gets -2, then -3); if that file cannot be written the sim warns on stderr and runs without capture.

To capture chosen tests without touching their code, set SNARE_PCAPNG_TESTS to a comma-separated list of test thread names — cargo names each test's thread after its path, module::test — and only sims built on a listed thread are captured, to SNARE_PCAPNG_DIR if set and otherwise to snare-pcapng in the temporary directory, each saying where on stderr. An entry matches the whole name or its last :: segments, so my_test lists tests::my_test; a blank list counts as unset. An explicit pcapng(path) is captured either way.

$ SNARE_PCAPNG_TESTS=my_test,other_mod::another_test cargo snare test

Stamps are always the sim's clock. SimBuilder::pcapng_wall_comment(true), or SNARE_PCAPNG_WALL_COMMENT set to anything but empty or 0, also writes the real wall time each frame was sent at as its EPB's opt_comment (draft-ietf-opsawg-pcapng §3.5), in UTC as wall 2026-10-02T09:15:42.123456789Z, to line a capture up with logs kept on real time; a frame due after it was sent (an ACK after latency, a planned retransmission) carries the real time it was written, when the capture first saw the sim's clock reach it. Wireshark lists it as the packet's comment. The comments make the file differ from run to run.

The sim has no real frames, so capture fabricates them, once per frame at its sender:

Traffic Frames
TCP connect SYN, SYN|ACK, ACK when it is accepted; SYN then RST|ACK when refused; a connect that waits shows the host OS's SYN retransmissions at their plan points up to the answer, and nothing for a timeout. A refused connect still takes an ephemeral port.
TCP data Each accepted write as MSS-sized PSH|ACK segments (MSS = egress MTU − 40, − 60 for IPv6), and the receiver's ACK stamped when the bytes arrive (TcpPolicy latency, a stalled or down link). On macOS, data reaching a TCP endpoint after SHUT_RD gets a reset instead of an ACK.
TCP close FIN|ACK from shutdown(SHUT_WR), the last close or a tester's close, once per end, with the peer's ACK when the last byte in flight arrives; SHUT_RD itself sends nothing. RST|ACK from a tester's Reset, SO_LINGER 0, a closing listener's unaccepted connections or incoming TCP data after macOS read shutdown.
UDP Each datagram at its sender — fabric, Winsock fabric, SimHost (stamped with its SO_TIMESTAMPING TX stamp) and tester endpoints — before any link policy, so loss and duplication never change the count and a frame lost on a down link is still there. A send refused (EAGAIN, EACCES, no route) writes nothing.
ICMP A port unreachable (ICMPv6 type 1 code 4) from the unreachable address, stamped when the datagram got there; an injected one at once.
Raw L2 AF_PACKET writes, /dev/bpf* writes and npcap sends, verbatim on their interface.

Frames are Ethernet II with IPv4 (DF, a per-source ID, TTL 64, or 128 on Windows) or IPv6 (v4-mapped addresses are written as IPv4), real IP/TCP/UDP/ICMP checksums, an MSS option on SYNs and a window of 65535. Sequence numbers start from a per-connection ISN drawn from the sim's seed and the connection's addresses. MACs are all zero on loopback, the group address for broadcast and multicast, the owning interface's NicSpec::mac for the host's addresses, and a locally administered hash of the address otherwise. Each interface a frame crosses gets an interface block, named after it, the first time it is used; stamps are nanoseconds of the sim's CLOCK_REALTIME (real time on a plain wall_clock() sim); epb_flags marks frames the code under test sent as outbound and its testers' as inbound.

Frames due later (a retransmission, an ACK after latency) wait until the sim's time reaches them, and the file is complete after each write. A waiting connect's planned retransmissions, and every frame after them, stay unwritten until the connect has played those points of its plan, so a nonblocking connect checked late still shows only the SYNs before its answer, and its handshake at the plan point that answered it; a connect left waiting holds the rest of the capture until it is answered, closed or the sim drops. Frames are written in time order: one stamped before a frame already written (a sender that read its stamp just before another thread wrote) takes that frame's stamp. On Windows, a refused connect keeps probing for the host's refusal window; the capture shows each SYN with the RST|ACK answering it, and the handshake if a listener appears in time.

Capture only observes: it never moves the clock, yields, wakes a thread or draws from the seed, so a run behaves the same with or without it, and under deterministic() the same seed writes the same bytes (wall-time comments aside). Keep the file on a local disk — a slow write holds up the thread that sent the frame.

Interfaces and routing

Every Sim has one topology of interfaces, addresses and routes, shared by the fabric, the Winsock fabric and a SimHost's UDP sockets. It holds host facts only; what the host does with them — error codes, strong or weak host, what happens to a link that loses carrier — is the build host's.

use snare::{IpNet, NicPolicy, NicSpec, Route, Sim};
use std::time::Duration;

let sim = Sim::builder()
    .nic(
        NicSpec::new("eth0")
            .address("10.0.0.1/24".parse::<IpNet>().unwrap())
            .station("10.0.0.2".parse::<std::net::IpAddr>().unwrap())
            .policy(NicPolicy { latency: Duration::from_micros(80), ..Default::default() }),
    )
    .route(Route::new("192.168.0.0/16".parse::<IpNet>().unwrap(), "eth0").gateway([10, 0, 0, 254]))
    .build();
sim.run(|| {
    snare::schedule_link("eth0", Duration::from_secs(2), false).unwrap(); // the cable comes out
});
  • Default topology. Loopback at index 1 (lo / lo0 / Loopback Pseudo-Interface 1, with 127.0.0.0/8 and ::1, MTU 65536 / 16384 / 4294967295) and sim0 at the next free index with no addresses and the default routes (0.0.0.0/0, ::/0, metric 100). A SimHost that declares interfaces registers them instead (Nic::network, Nic::station, Nic::policy, link_stats as the counters' start). Initial carrier follows operstate == "up" unless Nic::carrier overrides it; .operstate("unknown").carrier(true) represents an unknown-but-up interface. The default routes leave through its lowest one unless HostProfile::route declares its own.
  • Open, then routed. Until an interface has an address, the sim is open: binding any address claims it for the host (on sim0), so tests that bind whatever they like keep working. Once one does, the code under test binds only the host's addresses (127/8 always), broadcast and multicast addresses, and the interfaces' stations; anything else is EADDRNOTAVAIL / WSAEADDRNOTAVAIL.
  • Who is where. An address is the host's, a station's (a tester endpoint or listener bound there, or a NicSpec::stations address an in-process socket binds — on the segment, across the link, never reached through a host wildcard bind), or nobody's.
  • Routing. Every send and connect resolves a path: a bound device first (Linux SO_BINDTODEVICE/SO_BINDTOIFINDEX use the device even with no route; macOS IP_BOUND_IF/IPV6_BOUND_IF use scoped routes only, ENETUNREACH on a miss, ENETDOWN when it is down; Windows IP_UNICAST_IF/IPV6_UNICAST_IF for unicast), then IP_MULTICAST_IF or the source's interface for multicast, local delivery through loopback for the host's own addresses, then the longest prefix through a live interface (ties: the bound address's interface, the lowest metric, the earliest route). Each interface address brings its connected subnet. No route is ENETUNREACH (macOS sendto from a specific address: EHOSTUNREACH; Windows WSAENETUNREACH) and logs Fault::Unreachable. A loopback source leaving the host is EINVAL on Linux and EADDRNOTAVAIL on macOS; Windows is a strong host, so a bound address only leaves through its own interface (WSAENETUNREACH, as measured) and only receives there.
  • Sources and delivery. An unconnected wildcard or unbound UDP socket keeps its wildcard local address and each datagram carries the route's source; a TCP client's local address is the route's source. A UDP connect from the wildcard (or unbound) takes the route's source address, v4-mapped on an IPv6 socket, and the socket then receives only there, as every host does (getsockname reports it). Connecting again keeps that address on Linux; macOS and Windows dissolve the association first and pick anew. Dissolving it (connect to an AF_UNSPEC address; on Windows also the all-zero address) follows the host: Linux returns to the wildcard unless bind named an address and gives up the port unless bind named one (the next send or connect takes a new one); macOS returns to the wildcard, keeps the port and fails with EAFNOSUPPORT (EINVAL for an address shorter than the family's); Windows returns to what bind named, keeps the port, and refuses an address shorter than the family's with WSAEFAULT. tests/udp_connect_source.rs compares this with the real host. A datagram reaches the exact bind, else a wildcard bind only for the host's own addresses; a directed broadcast reaches its subnet's stations and the host's wildcard binds; a socket bound to a device only receives what arrives on it. Recipients are served in (port, address) order.
  • Multicast. A group's datagram reaches, on its port, every socket that joined the group. On Linux a socket also receives the groups any socket of the host joined while IP_MULTICAST_ALL (IPV6_MULTICAST_ALL) is on, the default; turning it off leaves only its own joins. macOS and Windows deliver joined groups only. On Linux and macOS the socket must be bound to the wildcard or the group itself. tests/multicast_os_truth.rs compares this with the real host. Which interface a join named is not matched against the arriving one, IP_DROP_MEMBERSHIP is not modelled, and one exact address takes one socket (SO_REUSEADDR/SO_REUSEPORT sharing is not modelled).
  • Bind to device from the test. set_socket_device(id, Some(nic)) (or None; also Sim::set_socket_device) binds an open socket as the host's option would, with that option's error for an unknown name (ENODEV, ENXIO, WSAEINVAL) and no privilege check.
  • Link state. set_link, set_nic and schedule_link (a deadline on the sim's clock, so a thread blocked on the link wakes when it changes, under every clock and deterministic()). With carrier down, Linux and macOS keep routes: sends succeed and the frame is dropped (tx_dropped, tx_carrier_errors, Fault::LinkDown), inbound is lost, and a datagram in flight across a flap is discarded. Windows media sense withdraws the interface's routes and addresses. Admin down withdraws routes (Linux keeps the addresses bindable, as measured; macOS answers ENETDOWN to sockets bound there). TCP over a dead link stalls: writes succeed, bytes wait in flight and arrive in order once the link returns (or the interface is removed); a blocked read with nothing scheduled gives up like any quiescent wait. Raw L2 frames do not cross a dead link, an oversized frame is EMSGSIZE, and Linux AF_PACKET on a down interface is ENETDOWN.
  • NicPolicy. Latency, jitter, loss and duplication on traffic crossing the interface (host to station, station to station), never host-local traffic, drawn from the sim's seed in a fixed order (loss, duplication, jitter, then the address's UdpPolicy); delays add and losses compound. TCP takes the latency and jitter per write, in order, without loss. With no policy set, draws are exactly as before.
  • Counters. nic_counters(name) counts frames with their headers (UDP +42 / +62 bytes, TCP segments of at most MTU-40 bytes +54 / +74); loopback traffic counts on loopback; set_nic_counters edits them.

add_nic, remove_nic, set_nic, set_link, schedule_link, set_nic_policy, set_nic_counters, nic, nics, nic_counters, add_route, remove_route, set_default_route, routes and route_lookup work from any thread of a sim, and as Sim methods from anywhere. tests/nic_os_truth.rs compares the error codes against the real host.

Enumerating interfaces

Every way the code under test can list interfaces reads the same topology, so set_link, set_nic and the counters show up everywhere at once:

  • libc (Linux, macOS). if_nametoindex, if_indextoname, if_nameindex/if_freenameindex, and getifaddrs/freeifaddrs: one AF_PACKET (Linux, ifa_data a rtnl_link_stats) or AF_LINK (macOS, an if_data) entry per interface, then one AF_INET/AF_INET6 entry per host address with netmask and broadcast, in index order (glibc lists the link entries first). Stations are not listed. Flags follow the link: IFF_UP while admin up, IFF_RUNNING (and Linux IFF_LOWER_UP) with carrier. Unknown names fail as the host's libc does (ENODEV from glibc, ENXIO on macOS; pinned by tests/nic_enumeration.rs against the real call).
  • ioctls on any socket. Linux SIOCGIFINDEX, SIOCGIFMTU, SIOCGIFFLAGS, SIOCGIFHWADDR; macOS SIOCGIFFLAGS, SIOCGIFMTU. An interface without a configured MAC has 02:00:<index>.
  • SimHost. /sys/class/net/<if>/{mtu,ifindex,operstate,carrier,flags,address,statistics/*} (reading carrier of an admin-down interface is EINVAL, as carrier_show), rtnetlink RTM_GETLINK (flags, IFLA_OPERSTATE, IFLA_CARRIER, IFLA_MTU, IFLA_ADDRESS, IFLA_STATS64), RTM_GETADDR and RTM_GETROUTE dumps, and macOS sysctl NET_RT_IFLIST2 all render the topology, loopback included. ethtool, hardware timestamping, queues, IRQs and PTP stay the host profile's (see NIC drivers).
  • Windows IP Helper. if_nametoindex/if_indextoname (NDIS names such as ethernet_32772, loopback_0), the ConvertInterface* alias/name/LUID/index conversions, GetIfEntry2, GetIfTable2, GetUnicastIpAddressTable, FreeMibTable, GetAdaptersAddresses (with the ERROR_BUFFER_OVERFLOW size handshake) and GetBestRoute2, rendered through the windows-sys types. OperStatus/MediaConnectState follow the link, octet and packet counters follow traffic, and addresses are listed only with carrier (media sense). Tables the sim hands out are freed by FreeMibTable; any other pointer goes to the real one.

NIC drivers: ethtool, qdiscs, threaded NAPI

On Linux a SimHost interface has a driver, declared on its Nic the way old snare's NicCaps and DriverSeed did, and SIOCETHTOOL runs against it as net/ethtool/ioctl.c __dev_ethtool does (Linux 6.12): a command outside the kernel's read-only list needs CAP_NET_ADMIN (EPERM, checked first), an operation the driver lacks is EOPNOTSUPP, and the core's own checks run before the driver's. What the code under test sets persists; SimHost::ethtool(nic) returns it all as a NicEthtool.

Command Nic builder Rules
GDRVINFO driver, bus_info, firmware, expansion_rom, register_dump_len, eeprom_len, private_flags_count dump lengths and private-flag count default to zero; n_stats from driver statistics
GLINK — admin up with carrier, from the topology
[GS]RINGPARAM rings(Rings { .. }) each size within its maximum (EINVAL); the *_max given are ignored
[GS]COALESCE coalesce(CoalesceParams, Coalesce), coalesce_limits a non-zero field outside supported_coalesce_params is EOPNOTSUPP; above the driver's limits EINVAL
[GS]CHANNELS channels(Channels { .. }) no change succeeds at once; counts within maxima, at least one RX and TX queue, not below a queue a flow rule uses (EINVAL); the queue count (/sys/class/net/<if>/queues) follows
[GS]PAUSEPARAM, [GS]EEE pause, eee an EEE mode outside supported is EINVAL
[GS]FLAGS ntuple(slots) makes ETH_FLAG_NTUPLE settable unknown flags EINVAL; a change outside hw_features EOPNOTSUPP
GRXCLSRLCNT/GRXCLSRULE/GRXCLSRLALL/SRXCLSRLINS/SRXCLSRLDEL ntuple(slots) as igb: a slot past the table, a missing queue or an empty slot EINVAL, a short rule_locs EMSGSIZE
GSSET_INFO/GSTRINGS/GSTATS driver_stats([(name, value)]), SimHost::set_driver_stat none declared: EOPNOTSUPP, as a driver without statistics

rtnetlink RTM_NEWQDISC/RTM_DELQDISC change the qdiscs RTM_GETQDISC dumps (every interface, in index order). Multiqueue goes by the transmit queues the driver allocated (num_tx_queues: Nic::tx_queues_allocated, else the channel maxima), not those in use (real_num_tx_queues, the queues/ listing): a multi-queue interface starts with mq over one default qdisc per allocated queue (pfifo_fast past the queues in use) and the dump lists the queues in use and any qdisc created with a handle, as net/sched/sch_mq.c mq_attach hashes them; a single-queue one starts with the default qdisc, and an IFF_NO_QUEUE one (Nic::no_queue, veth) with noqueue. Replacing the root or an mq class's qdisc follows net/sched/sch_api.c tc_modify_qdisc (NLM_F_CREATE/NLM_F_REPLACE/NLM_F_EXCL, automatic handles from 8001:), mq needs more than one allocated queue (EOPNOTSUPP), and etf checks its parameters as etf_init does (CLOCK_TAI only, a dynamic clock ENOTSUPP (524)); TC_ETF_OFFLOAD_ON needs Nic::etf_offload(queues) (EOPNOTSUPP without, EINVAL on another queue, as igb on an I210). Changes need CAP_NET_ADMIN and are acked with NLMSG_ERROR when NLM_F_ACK asks. /sys/class/net/<if>/threaded reads and, for root, writes the threaded-NAPI switch (Nic::threaded_napi): EPERM without CAP_NET_ADMIN, EOPNOTSUPP for a value other than 0 or 1. EasyBuilder presets ship an I210-like driver with all of the above. Tests: tests/simhost_ethtool.rs.

Windows adapter properties and restarts

Windows tunes an adapter through its device node, not an ioctl: a program enumerates the network class with SetupDiGetClassDevsW/SetupDiEnumDeviceInfo, opens the driver key with SetupDiOpenDevRegKey(DIREG_DRV), matches its NetCfgInstanceId to the interface GUID IP Helper reports, writes standardized keywords (*ReceiveBuffers, *InterruptModeration, *FlowControl, *NumRssQueues, *RSS, *SoftwareTimestamp, EEE names) as REG_SZ values, reads their limits from Ndi\Params\<keyword>\max, and restarts the device with CM_Disable_DevNode + CM_Enable_DevNode (fast-talker's nic/windows.rs does exactly this). Every one of those calls is in-process, so the Windows Sim serves them: each non-loopback interface is a device node whose two keys (DIREG_DRV, and DIREG_DEV's Device Parameters, where the interrupt affinity policy lives) are an in-memory registry that keeps what the code under test writes.

  • Describe it. SimBuilder::adapter(nic, Adapter::new().property("*ReceiveBuffers", "512", Some(4096)).without_property("*RSS").driver_version("..").restart_flap(d)), or Sim::set_adapter later. The default Adapter::new() mirrors snare 1.x's NicCaps: rings 256 of 4096, interrupt moderation on, flow control Rx & Tx, 4 of 8 RSS queues, RSS on, software timestamping off, no EEE keyword, 2 s flap.
  • Read it back. Sim::adapter_property(nic, keyword), Sim::adapter_value(nic, AdapterKey::{Driver, Device}, path, name) -> Option<RegValue>, Sim::adapter_restarts(nic).
  • Restart flap. CM_Disable_DevNode drops the interface's carrier; CM_Enable_DevNode schedules it back up restart_flap later on the sim's clock (schedule_link), so GetIfEntry2's OperStatus, media sense and the sockets all see the outage, under the virtual clock and deterministic(). An interface that had no carrier stays down.
  • Elevation is Privileges::root. A standard user may read the keys but gets ERROR_ACCESS_DENIED opening one for writing or creating a subkey, and CR_ACCESS_DENIED from the CM_* calls. A write through a handle opened KEY_READ is ERROR_ACCESS_DENIED for anyone.
  • Codes. Missing value or key ERROR_FILE_NOT_FOUND, short buffer ERROR_MORE_DATA with the size, a buffer without a length ERROR_INVALID_PARAMETER, enumerating past the end ERROR_NO_MORE_ITEMS, a wrong cbSize ERROR_INVALID_USER_BUFFER, a bad key type ERROR_INVALID_FLAGS; tests/nic_adapter_props_win.rs (registry_codes_os_truth, setupapi_codes_os_truth) compares them with the real registry and SetupAPI.
  • Passthrough. Only the sets, keys and device instances the sim minted (and the network-class enumeration that mints them) are simulated; every other registry key, device class or DEVINST goes to the real API. A sim handle passed to a SetupAPI call that is not hooked reaches the real one. CreateFileW(\\.\{GUID}) for NDIS statistics is not modelled (it fails, as for a virtual adapter), so driver counters come from GetIfEntry2 alone.

The WinHost also keeps the process's working set: SetProcessWorkingSetSizeEx / GetProcessWorkingSetSizeEx (fast-talker's reserve_working_set) never touch the real process. Measured on Windows 11: a new process has 204 800 / 1 413 120 bytes with both limits soft; bounds round down to 4 KiB and the minimum is raised to 20 pages; min above max, or ENABLE and DISABLE of one limit together, is ERROR_INVALID_PARAMETER; a minimum beyond the machine is ERROR_NO_SYSTEM_RESOURCES (the sim has 16 GiB and refuses above 7/8 of it); flags 0 keep the limits' state; both bounds (SIZE_T)-1 trims and succeeds. Raising either bound needs SeIncreaseWorkingSetPrivilege, whose stand-in is ipc_lock (ERROR_PRIVILEGE_NOT_HELD without it). Sim::working_set() reads it back; working_set_os_truth compares with the real OS.

Socket faults and back-pressure

Faults on the code under test's sockets, with the host OS's codes and the order its kernel reports them in. Every call works from any thread of a sim; the Sim methods of the same names also work outside run.

Call Effect
raise_socket_error(addr, io::Error) Makes the error pending on every stream whose local or peer address is addr and every datagram socket bound at or connected to it. A raw OS error passes as given; a kind maps to the host's code (ConnectionReset ECONNRESET/WSAECONNRESET, BrokenPipe EPIPE/WSAESHUTDOWN, ...); a kind with none panics.
inject_icmp_port_unreachable(to, from) An ICMP port unreachable from from reaches the datagram sockets bound at to.
quiesce(addr, span, Direction) Holds what arrives at (Receive), leaves (Send) or crosses (Both) addr for span of sim time: TCP bytes in flight and written meanwhile, and a FIN, land in order once it ends (counting against the window); bytes already received stay readable; datagrams arrive at its end.
SO_SNDBUF / SO_RCVBUF Bound every stream of the code under test: what is in flight plus unread is capped at the reader's receive room plus the writer's send buffer (see Socket limits and privileges).
TcpPolicy::recv_window The receive window of the endpoint at the policy's address, capping its receive room below its SO_RCVBUF (a tester's endpoint, unlimited otherwise, takes the window alone).
SO_LINGER A zero timeout makes the last close abort with a reset (the peer reads ECONNRESET, a tester logs PeerReset); another makes it wait, up to that long, until what was written has crossed. macOS also takes SO_LINGER_SEC (std and socket2 use it; its SO_LINGER counts 1/100 s ticks); Windows SO_DONTLINGER, and a nonblocking close with data still crossing fails with WSAEWOULDBLOCK.
TesterAction::RaiseSocketError(e) / IcmpPortUnreachable / QuiesceLink(span, dir) / SetRecvWindow(w) The same for the code under test's end the event came from (every peer from a cyclic action); QuiesceLink's direction is seen from the code under test.
OS Raised error reported ICMP port unreachable Writable on a bounded stream
Linux first by a send; a stream's receive returns queued bytes first, a datagram socket's the error first ECONNREFUSED to the next send or receive of a socket connected to the sender; an unconnected one only with IP_RECVERR, which also queues a sock_extended_err (origin ICMP, type 3, code 3, the offender's address) for recvmsg(MSG_ERRQUEUE) free space at least half of what is queued
macOS first by a send; a receive returns queued bytes first (MSG_PEEK leaves the error pending) ECONNREFUSED to a socket connected to the sender; an unconnected one ignores it 2048 bytes free (SO_SNDLOWAT); EVFILT_WRITE reports the free space
Windows first by any call WSAECONNRESET in receive arrival order alongside queued datagrams, for a connected or unconnected socket unless WSAIoctl(SIO_UDP_CONNRESET, FALSE); MSG_PEEK preserves the status; SO_ERROR does not return or consume this status any free space

Reporting clears the error, as getsockopt(SO_ERROR) does; while it is pending a blocked call returns at once, epoll and poll report EPOLLERR/POLLERR (macOS poll and kqueue: readable), WSAPoll POLLERR and select the read set. Windows ICMP receive status instead reports WSAPoll readability and is consumed by a receive without MSG_PEEK, leaving SO_ERROR zero. A datagram sent to an address the host or a station holds that reaches no socket or tester draws an ICMP error one round trip later (the policies' fixed latency each way, no jitter, so seeds replay); broadcast, multicast and absent addresses draw none. A full window makes a blocking send wait for room, up to SO_SNDTIMEO (then the bytes sent so far, else EAGAIN/WSAETIMEDOUT), and a nonblocking one return what fit or EAGAIN/WSAEWOULDBLOCK: on Linux what fits, on macOS nothing until 2048 bytes (SO_SNDLOWAT) are free unless the whole write fits, on Windows the whole write while anything is free. A tester's writes queue until the window takes them, and its close waits for them.

Connect faults

A connect asks the destination how it answers a SYN, as the host OS would:

  • a listener there (the code under test's or a tester's) accepts at once;
  • an address of the host or a station with no listener resets the SYN: ECONNREFUSED at once on unix; Windows retries on a reset and reports WSAECONNREFUSED 2 s later;
  • an address nobody holds stays silent, and the connect follows the OS's SYN plan — Linux retransmits at 1, 2, 3, 4, 5, 7, 11, … s and gives up at 131 s with the modern default SysLimits::tcp_syn_linear_timeouts of 4. Setting it to 0 selects the older exponential schedule and 127 s timeout. SysLimits::tcp_syn_retries or the socket's TCP_SYNCNT sets the retry limit; both sysctls are exposed by SimHost under /proc/sys/net/ipv4. SysLimits::from_real_host reads the running kernel's settings. macOS retransmits at 1, 2, 3, 4, 5, 7, 11, … s and gives up at 75 s (TCP_CONNECTIONTIMEOUT); Windows retransmits at 3 and 9 s and gives up at 21 s by default. Extending TCP_MAXRT continues with exponentially increasing intervals capped at 60 s; setting it to -1 keeps retrying until the connection completes or closes, each with ETIMEDOUT/WSAETIMEDOUT. On Linux an absent address on a connected subnet fails for want of a neighbour instead, with EHOSTUNREACH after 3 s; macOS (measured) and Windows follow their SYN plans there too;
  • a listener that appears meanwhile is reached at the next retransmission.

While an open sim (no interface given an address) still lets the code under test bind any address, only addresses it has bound, loopback and stations answer; a connect to any other address times out — at once in virtual time, but for real under wall_clock().

use std::net::{TcpListener, TcpStream};
use std::time::{Duration, Instant};
use snare::{ListenerBehavior, Sim, set_listener_behavior};

Sim::new().run(|| {
    let _listener = TcpListener::bind("127.0.0.7:7000").unwrap();
    let until = Instant::now() + Duration::from_millis(1500);
    set_listener_behavior("127.0.0.7:7000", ListenerBehavior::DelayingUntil(until));
    TcpStream::connect("127.0.0.7:7000").unwrap(); // at the next SYN retransmission
    let err = TcpStream::connect_timeout(&"10.255.0.1:80".parse().unwrap(), Duration::from_secs(5));
    assert_eq!(err.unwrap_err().kind(), std::io::ErrorKind::TimedOut); // 5 s of virtual time
});
Call Effect
set_listener_behavior(addr, b) / Sim::set_listener_behavior How addr answers from then on, kept per address (the wildcard covers its port), before or after anything listens there and across listener replacement. DelayingUntil reads its Instant on the calling thread's clock; the Sim method, called off the sim, takes it as that far ahead of the sim's clock.
TesterAction::SetListenerBehavior(b) The same for the tester's own address.

A nonblocking connect that cannot finish at once fails with EINPROGRESS (WSAEWOULDBLOCK); a second connect meanwhile gives EALREADY and one after it succeeded EISCONN. The socket reports nothing to poll/epoll/kqueue/select until it is settled: writable once connected, or POLLOUT | POLLERR | POLLHUP (EPOLLERR, EVFILT_WRITE with EV_EOF and the error in fflags, Windows exceptfds and WSAPoll POLLERR | POLLHUP) once it failed, the error waiting in SO_ERROR. Each point of the plan is a timer the sim can skip to, dropped when the connect settles or the socket closes.

Socket table

Every socket the code under test opens — fabric TCP and UDP, raw L2, Winsock, a SimHost's UDP and netlink sockets — gets one record for its whole life. Supported duplicate operations share that record; accept makes a new one. Unix socket aliases created by dup, F_DUPFD, F_DUPFD_CLOEXEC, dup2, Linux dup3 or try_clone share socket state; close-on-exec flags belong to each descriptor. Replacement updates ownership across socket and virtual-file backends. Linux eventfd/epoll aliases also share their counter and interest state; closing the last alias retires the description. Remaining descriptor boundaries are listed in open bugs. snare::socket_id(&sock) names any std or mio socket of the calling thread's sim (None for a real OS socket), and socket_entry(id), socket_table() (open, oldest first), closed_sockets() (in close order) and sockets_bound(addr) read snapshots, as do the Sim methods of the same names from any thread.

Field Meaning
id Numbered from 1 per sim in creation order and never reused. Ids are per sim: two sims both start at 1.
kind Udp, TcpStream (unconnected, connected or accepted), TcpListener, Packet, Netlink, Unix (a socketpair end).
local / peer / listener Where it is bound and connected, and for an accepted stream its listener's id.
memberships Multicast groups joined, with the interface address or index the join named.
queued / queued_bytes Datagrams (UDP) or written chunks (TCP) that have arrived and wait to be read.
delivered / delivered_bytes / sent What arrived at it, and its successful sends.
pending_error The pending socket error or receive status. SO_ERROR returns and clears socket errors; Windows ICMP receive status is returned only by a receive.
created_at / closed_at On the sim's timeline, like recorded events.
interface / bound_device / multicast_if The interface its latest traffic crossed, the one it is bound to, and the one its multicast leaves through.
last_tx_nic / last_rx_nic The interface its latest datagram or connection left by and arrived on (a connection notes both when it is made).
unmodelled_options Each setsockopt/getsockopt/ioctl the sim does not model that was used on it, once, in first-use order. See Unmodelled options.

Per-socket options (SO_RCVTIMEO, SO_SNDTIMEO, SO_LINGER, SO_REUSEADDR) live in the record, so every descriptor of a socket sees the same values. Where the hosts differ, the sim follows the one it runs on, as measured: a blocking accept honours SO_RCVTIMEO on Linux only; after SHUT_RD Linux still delivers what arrives while macOS reads end of stream at once and Windows fails reads with WSAESHUTDOWN; and a bind overlapping a wildcard or equal address follows each OS's SO_REUSEADDR rule (tests/bind_os_truth.rs compares the sim against the host).

On macOS, TCP read shutdown discards queued receive bytes. Subsequent incoming data closes that endpoint at its arrival time and returns a reset after the reverse link delay. The local endpoint writes with EPIPE; the peer retains previously received bytes, then reports ECONNRESET once before EOF. MSG_PEEK preserves that error, SO_ERROR consumes it, and kqueue EOF events carry the pending error in fflags. The native comparisons and exact link-delay tests are in tests/macos_tcp_read_shutdown*.rs; Unix socket pairs do not send TCP resets.

The data/reset exchange samples both link delays when it is scheduled. Directed stalls and carrier outages can delay either delivery, including when the connection is inspected later.

Packet timestamps

A socket that asks for kernel timestamps gets them from the sim's clock, on the plain Sim's fabric and under a SimHost alike, through one implementation (tstamp.rs) that keeps each socket's timestamping state in its socket record. A packet is stamped at the instant it reaches the receiving socket — its send time plus the link delay it crossed — so code that reads it late still sees when it arrived, as with a real kernel's software receive stamp. Every stamp reports CLOCK_REALTIME except where the option says otherwise; a clock that jumps is moved up to a stamp it hands out, so no later clock read is earlier. What each host does was measured against it (tests/timestamps.rs, the *_os_truth tests); Windows follows Microsoft Learn: Winsock timestamping (tests/timestamps_win.rs).

Host Options What the code under test reads
Linux SO_TIMESTAMP, SO_TIMESTAMPNS, SO_TIMESTAMPING (flags or struct so_timestamping) recvmsg on UDP and TCP: SCM_TIMESTAMP (timeval) or SCM_TIMESTAMPNS (timespec), then SCM_TIMESTAMPING with the software stamp in ts[0] when SOF_TIMESTAMPING_SOFTWARE reports it (OPT_RX_FILTER honoured); a TCP read gets the arrival of the last byte it took. Transmit stamps on MSG_ERRQUEUE: TX_SCHED/TX_SOFTWARE at the send time, TCP TX_ACK a round trip later, each with IP_RECVERR/IPV6_RECVERR sock_extended_err (ENOMSG, SO_EE_ORIGIN_TIMESTAMPING, ee_info the stage, ee_data the OPT_ID key, counted per datagram or by byte for TCP), the looped-back packet (a fabricated Ethernet/IP/UDP or TCP header and the payload) or nothing under OPT_TSONLY, and POLLERR while one waits. SO_TIMESTAMPING validates as the kernel does (unknown flags, OPT_ID_TCP without OPT_ID, OPT_STATS without OPT_TSONLY, OPT_ID on an unconnected TCP socket: EINVAL; BIND_PHC: EOPNOTSUPP/EINVAL, the interfaces having no PHC). SIOCGHWTSTAMP/SIOCSHWTSTAMP on the plain fabric's interfaces are EOPNOTSUPP (EPERM to set without CAP_NET_ADMIN, ENODEV for an unknown name). SO_TXTIME validates as sk_setsockopt does (exactly a struct sock_txtime, known flags and clocks: EINVAL; a clock other than CLOCK_MONOTONIC without CAP_NET_ADMIN: EPERM) and reads back all 8 bytes.
macOS SO_TIMESTAMP, SO_TIMESTAMP_MONOTONIC, SO_TIMESTAMP_CONTINUOUS recvmsg on UDP: SCM_TIMESTAMP (timeval), SCM_TIMESTAMP_MONOTONIC and SCM_TIMESTAMP_CONTINUOUS (Mach ticks, on the clock mach_absolute_time reads). Nothing on TCP, as XNU. getsockopt reports the option's bit.
Windows WSAIoctl(SIO_TIMESTAMPING) with TIMESTAMPING_FLAG_RX/TX (UDP only; WSAEINVAL on TCP) WSARecvMsg on UDP: one SO_TIMESTAMP control message, the arrival as a QueryPerformanceCounter value on the counter the code under test reads. A WSASendMsg carrying SO_TIMESTAMP_ID buffers its send time while TxTimestampsBuffered has room (a stamp generated with the buffer full is dropped); SIO_GET_TX_TIMESTAMP takes it by id, WSAEWOULDBLOCK when none waits, and WSAEOPNOTSUPP while transmit stamps are off (measured).

On Linux, Sim::builder().rx_timestamp_startup_delay(Duration::from_millis(5)) starts software receive timestamp generation cold. The first successful SO_TIMESTAMP, SO_TIMESTAMPNS or SO_TIMESTAMPING request for RX_SOFTWARE starts a single sim-wide activation window on the sim's clock. Packets arriving before its end have no software receive stamp, even if read after activation. Later option changes and other sockets do not restart the window; hardware stamps and transmit reports are independent. The default delay is zero, representing generation already active, so existing tests retain immediate timestamps. SO_TIMESTAMP and SO_TIMESTAMPNS on a datagram supply a read-time fallback for an unstamped packet, including the same fallback across repeated MSG_PEEK reads; TCP has no such fallback. This configures an initial cold-to-warm profile; it does not replay Linux's host-wide static-key reference counts or workqueue scheduling.

A Linux SimHost adds hardware stamps (tests/simhost_hwtstamp.rs). A NIC with Nic::hardware_timestamping starts with the struct hwtstamp_config of Nic::hwtstamp_config (off by default), and SIOCSHWTSTAMP changes the emulated configuration: EPERM without CAP_NET_ADMIN, an unknown flag EINVAL, an unknown transmit type or filter ERANGE, a NIC without hardware timestamping EOPNOTSUPP (and SIOCGHWTSTAMP too); then a transmit type the driver's ETHTOOL_GET_TS_INFO lacks is ERANGE and a filter it lacks is widened to the narrowest advertised filter that covers it, with the applied configuration written back. This is a synthetic policy: capability bitmaps do not specify a driver's requested-to-applied mappings. The I210 default turns PTP filters into HWTSTAMP_FILTER_ALL. With transmit stamping on, a send asking TX_HARDWARE gets a SCM_TSTAMP_SND report carrying only ts[2] (the NIC's PHC at the send), and its software report is dropped unless SOF_TIMESTAMPING_OPT_TX_SWHW asks for both (net/core/skbuff.c __skb_tstamp_tx); a datagram the NIC's filter covers arrives with ts[2] for a socket reporting RAW_HARDWARE. UDP PTP filters check a payload of at least 34 bytes and selected version/message fields on port 319; NTP filtering selects destination port 123. These checks are a synthetic classifier, without complete packet or declared-length validation. Hardware/software timestamp points have no independent NIC latency. Hardware TX reports with legacy SO_TIMESTAMP(NS) enabled receive a software read-time fallback, also visible in ts[0] when software reporting is enabled. SCM_TXTIME on a SimHost socket without SO_TXTIME, or not 8 bytes long, fails the sendmsg with EINVAL (__sock_cmsg_send). A /dev/ptp<N> answers ptp_ioctl (drivers/ptp/ptp_chardev.c) from its PtpCaps (HostProfile::ptp_clock_caps; by default nothing to adjust and no pins, channels or cross-timestamping): PTP_CLOCK_GETCAPS, PTP_SYS_OFFSET (EINVAL past PTP_MAX_SAMPLES), PTP_SYS_OFFSET_EXTENDED (system clock, PHC, system clock per sample, on CLOCK_REALTIME, CLOCK_MONOTONIC or CLOCK_MONOTONIC_RAW), PTP_SYS_OFFSET_PRECISE only with cross_timestamping (else EOPNOTSUPP), and EACCES for the changing requests on a read-only fd (tests/simhost_ptp.rs).

A control buffer too short for the messages is cut as the host cuts it: Linux writes the part that fits with cmsg_len cut to it and sets MSG_CTRUNC (even with no control buffer); macOS copies the bytes that fit, sets MSG_CTRUNC only for a message it had to cut, and drops a message that finds the buffer full. The kernel's deferred enabling of receive stamping (a static key that can leave the first packets after setsockopt unstamped) is not modelled: every packet is stamped.

Unmodelled options and strict_sockopts

Every setsockopt, getsockopt and ioctl on a sim socket is modelled, ignored on purpose as harmless, or unmodelled. An unmodelled one succeeds with no effect (a getsockopt reads an int 0), as before, but is listed in the socket's SocketEntry::unmodelled_options and logged once per socket as RecordedEvent::UnmodelledOption, so a test can assert the code under test used nothing the sim ignores. With SimBuilder::strict_sockopts() it fails instead, with what the host answers for an option it does not know — ENOPROTOOPT for an option, ENOTTY (Linux) or ENXIO (macOS) for an ioctl request, all measured — and the first refusal is also written to stderr.

On Windows every option Winsock defines is known by level and number. An unmodelled one is kept and read back as well (with Windows' default until set, such as IP_MULTICAST_LOOP on and IPV6_V6ONLY on), listed, and refused under strict_sockopts() with WSAEINVAL at SOL_SOCKET and WSAENOPROTOOPT elsewhere, the codes Winsock gives an option it does not know (measured). An option Winsock does not define fails with those codes with or without strict mode, as does an ioctlsocket command other than FIONBIO, FIONREAD and SIOCATMARK (WSAEOPNOTSUPP, measured). A WSAIoctl code the sim does not carry out cannot succeed without effect, since only the OS could fill its output: it fails with WSAEOPNOTSUPP and is listed as refused. SO_SNDLOWAT, SO_RCVLOWAT and SO_USELOOPBACK fail WSAEINVAL, as Microsoft documents. A level the socket does not take fails first: IPPROTO_IPV6 on an IPv4 socket (WSAEINVAL), IPPROTO_TCP on a datagram socket and IPPROTO_UDP on a stream (WSAENOPROTOOPT); SO_KEEPALIVE is stream-only.

The harmless options are kept and read back but change nothing, because nothing a test can observe in the sim depends on them: SO_KEEPALIVE and the TCP keepalive timers (no connection dies idle), TCP_NODELAY and Linux TCP_QUICKACK (no Nagle or delayed ACKs), SO_OOBINLINE and SO_DEBUG, macOS SO_NOSIGPIPE (the sim never raises SIGPIPE; std sets it on every socket), and IP_TOS, IP_TTL, IP_MULTICAST_TTL, IPV6_TCLASS, IPV6_UNICAST_HOPS, IPV6_MULTICAST_HOPS (no QoS or hop count); FIOCLEX/FIONCLEX for ioctls. Windows keeps the same, read back from Windows' defaults until set (IP_TTL and IPV6_UNICAST_HOPS 128, the multicast hop limits 1, the TCP keepalive timers 2 hours, 1 s and 10 probes), plus SO_DEBUG and SO_DONTROUTE, which Microsoft's providers ignore, and the SIO_KEEPALIVE_VALS, SIO_LOOPBACK_FAST_PATH and SIO_UDP_NETRESET ioctls.

Some options are modelled although nothing in the sim depends on them, so code that sets or reads them sees the host's answers. Linux SO_BUSY_POLL, SO_PREFER_BUSY_POLL and SO_BUSY_POLL_BUDGET are kept per socket with the kernel's checks (Linux 7.0, measured): any non-negative SO_BUSY_POLL (negative is EINVAL), SO_PREFER_BUSY_POLL on only with CAP_NET_ADMIN, and SO_BUSY_POLL_BUDGET raised above its current value only with CAP_NET_ADMIN (EPERM) and within 0..=65535 (EINVAL); the first two read back, the budget does not (ENOPROTOOPT, as sk_getsockopt). Nothing is polled: delivery is unchanged. Linux SO_DOMAIN and SO_PROTOCOL read the socket's family and the protocol the kernel resolved (IPPROTO_UDP/IPPROTO_TCP for protocol 0, 0 for packet and unix sockets, a netlink socket's own), and fail ENOPROTOOPT to set; macOS has neither. Windows SO_PROTOCOL_INFOW and SO_PROTOCOL_INFOA return the host's Winsock catalog entry for the socket's family, type and protocol, as Winsock does, and refuse to be set. tests/busy_poll.rs and tests/socket_family.rs compare them with the real host.

Everything else the sim does not model — SO_REUSEPORT, IP_MULTICAST_LOOP, IP_DROP_MEMBERSHIP, IPV6_V6ONLY, SO_RCVLOWAT, SIOCGSTAMP and the rest — is unmodelled. A SimHost socket keeps its existing read-back of every option set, and records the same way. std's own socket calls on each host are all modelled or harmless (tests/strict_sockopts.rs runs them under strict_sockopts).

Don't-fragment

The don't-fragment options are modelled on every IP socket of the fabric and a SimHost: Linux IP_MTU_DISCOVER and IPV6_MTU_DISCOVER (IP_PMTUDISC_DONT, WANT — the default — DO, PROBE, INTERFACE, OMIT; anything else EINVAL) and IPV6_DONTFRAG, macOS IP_DONTFRAG and IPV6_DONTFRAG (flags, read back as 0 or 1). They read back as the host does, including the optlen rules (Linux reads a short IP_MTU_DISCOVER as a byte, refuses a short IPV6_MTU_DISCOVER and reads a short IPV6_DONTFRAG as 0; macOS refuses any short value) and the other family's errors (Linux ENOPROTOOPT to set and EOPNOTSUPP to read an IPv6 option on an IPv4 socket, macOS EINVAL either way). The sim never fragments: a datagram larger than its egress interface's MTU is delivered whole, unless it may not be fragmented — Linux DO, PROBE or INTERFACE for its family, or IPV6_DONTFRAG for IPv6; macOS IP_DONTFRAG, or IPV6_DONTFRAG on an IPv6 socket, v4-mapped sends included — when the send fails EMSGSIZE once payload plus 8 bytes of UDP and 20 (IPv4) or 40 (IPv6) bytes of IP header pass the MTU. The MTU is the interface's configured one; no smaller path MTU is learned. tests/dontfrag.rs dontfrag_os_truth compares all of it with the real stack on loopback (16384 on macOS lo0, 65536 on Linux lo, which only IPv6 datagrams can exceed).

Windows has IP_DONTFRAGMENT and IPV6_DONTFRAG (flags) and IP_MTU_DISCOVER/IPV6_MTU_DISCOVER (a PMTUD_STATE: NOT_SET, the default, DO, DONT or PROBE; anything else WSAEINVAL). A datagram may not be fragmented under the flag or DO/PROBE for its family, and then fails with WSAEMSGSIZE past the egress MTU, by the same header arithmetic (tests/dontfrag_win.rs).

Socket limits and privileges

A datagram socket's receive buffer is enforced as the host's kernel does: each datagram is admitted or dropped at the moment it arrives — not when the code under test next looks — against what the buffer holds then. SocketEntry reports rcvbuf/sndbuf (what getsockopt returns), rmem_alloc, overflowed (dropped for a full buffer), wire_lost (lost by a link policy, never counted as a drop) and drops (the kernel's counter: overflows plus inject_socket_drops(id, n), which SO_RXQ_OVFL and SO_MEMINFO report). Testers' endpoints are unlimited.

A TCP stream's buffers bound it too. What the code under test has written and its peer not yet read is capped at the writer's send buffer (its SO_SNDBUF as getsockopt reports it) plus the reader's receive room (its SO_RCVBUF as reported; twice it on Windows), capped further by a TcpPolicy::recv_window; a stream to a tester is bounded only by a window. An accepted stream takes its listener's sizes. Measured against the real OS with a reader that never reads (tests/tcp_buffers_os_truth.rs), this matches Windows exactly and Linux within 40%; macOS rounds buffers to whole segments and grows a loopback receive buffer by an amount that varies from run to run, and Linux autotunes a send buffer whose SO_SNDBUF was never set (measured 2.6 MB in flight); neither is modelled, so the sim usually back-pressures sooner there.

On Linux, streams with an explicitly configured receive buffer or transmit timestamps use MSS, GSO grouping and advertised-window state to release pending data as reads create room. Transmit reports quote the final transmitted segment and wait for that segment to cross the link. tests/tcp_error_queue_segments.rs compares IPv4 payloads and memory charges for writes through 128000 bytes across five receive-buffer sizes on both backends. Ordinary TCP segments share the receive-memory budget with error reports and keep their charge until fully read. Default buffer autotuning, kernel receive coalescing, congestion-window evolution and ACK pacing remain approximations; this comparison does not establish arbitrary TCP traffic parity.

OS Buffer options Admission FIONREAD
Linux SO_RCVBUF v stores max(2 * min(v, rmem_max), 2304) (-1 asks for the cap), SO_SNDBUF likewise with floor 4608; SO_*BUFFORCE skip the cap and need CAP_NET_ADMIN into an empty queue, or while rmem_alloc + truesize <= rcvbuf (6.18 on); while rmem_alloc <= rcvbuf, overshooting by one datagram, before 6.18 the next datagram's payload (a stream's unread bytes); SIOCOUTQ 0
macOS v <= 0 is EINVAL; above kern.ipc.maxsockbuf clamps, or ENOBUFS once the buffer is already at it while the record fits sb_hiwat - sb_cc and sb_mbcnt is under 8 * sb_hiwat sb_cc (16 + payload for a socket's first datagram, 32 + payload after); SO_NREAD the next payload; a datagram over SO_SNDBUF is EMSGSIZE
Windows stored as given while the queued payload is below SO_RCVBUF the queued payload capped at SO_RCVBUF

SysLimits holds the sysctls (rmem_default, rmem_max, wmem_*, tcp_*_default, unprivileged_port_start, enforce_rcvbuf) and, on Linux, the kernel facts behind admission (udp_rcvbuf_overshoot, skb_small_truesize, skb_head_overhead): SysLimits::host() is a stock install of the build host (Linux: a stock 6.18+ kernel, whatever the architecture), SysLimits::from_real_host() reads this machine, measuring the running kernel's truesize geometry over loopback with SO_MEMINFO. A datagram longer than the smallest MTU on its path arrives as IP fragments and is charged one buffer per fragment, as reassembly chains them (net/ipv4/inet_fragment.c inet_frag_reasm_finish; measured through veth by tests/hw_sockbuf_truth.rs). The Linux skb sizes depend on the kernel's build, not its architecture: Debian's 6.12 and 7.2 kernels charge the same on x86_64 and arm64 (a lone small datagram 960 bytes), OrbStack's arm64 kernel 1152. scripts/measure-sockbuf.sh boots Debian's x86_64 and arm64 kernels under QEMU and measures them (KERNEL_SUITE, and TESTS to run snare's test binaries on those kernels). Set them with SimBuilder::sys_limits, set_sys_limits or Sim::set_sys_limits; a change applies to sockets created after it.

tcp_gso_max_size selects Linux's device GSO limit for TCP grouping (65536 bytes by default). Driver-specific values need a measured profile override; from_real_host does not discover that device setting.

Privileges { root, net_admin, net_raw, net_bind_service, sys_nice, ipc_lock, sys_resource, rtprio_limit, nice_limit, memlock_limit } is per sim: Privileges::all() by default (every capability, RLIMIT_RTPRIO 99, RLIMIT_NICE 40, unlimited RLIMIT_MEMLOCK, as snare 1.x), a SimHost's profile (HostProfile::cap, HostProfile::root, with the kernel's stock limits) when it has one, and SimBuilder::privileges over both; Privileges::none() is an ordinary user with the stock limits (0, 0, 8 MiB on Linux) and Privileges::from_real_process() copies the machine's; set_privileges / Sim::set_privileges change it from any thread. The host's scheduling, mlockall, SIOCSHWTSTAMP and busy-poll gates read the same model, a SimHost answers geteuid/getuid (0 for root, else 1000 on Linux, 501 on macOS) and renders CapEff/CapPrm/CapBnd in /proc/self/status.

Process limits

The three limits are each an Rlimit { cur, max } (Rlimit::INFINITY is RLIM_INFINITY) and are what getrlimit, setrlimit and prlimit (Linux, also SYS_prlimit64 and x86_64's SYS_[gs]etrlimit through syscall) see and change inside the sim; the real process's limits are never touched, and other resources pass through. Raising a hard limit needs sys_resource (CAP_SYS_RESOURCE; root on macOS), a soft limit above the hard one is EINVAL. The calls they gate follow the build host's kernel:

OS Rule
Linux sched_setscheduler/sched_setparam/pthread_setschedparam (and the raw syscalls): EINVAL for a bad policy or priority, then without CAP_SYS_NICE EPERM to enter a real-time policy with RLIMIT_RTPRIO 0 or to raise the priority above both the current one and the limit (kernel/sched/syscalls.c). setpriority: the value is clamped to −20..19 and lowering it needs CAP_SYS_NICE or 20 − nice <= RLIMIT_NICE (EACCES). mlock: EPERM with RLIMIT_MEMLOCK 0, ENOMEM past it (locks do not nest); mlockall(MCL_CURRENT): ENOMEM when the process's mapped size exceeds it; CAP_IPC_LOCK lifts both (mm/mlock.c).
macOS mlock wires pages counted against RLIMIT_MEMLOCK whatever the privileges (EAGAIN past it); wiring nests per page, so a page is released after as many munlocks. mlockall is ENOSYS, as on the real system. A successful pthread_setschedparam, any policy, opts the thread out of QoS for good: later pthread_set_qos_class_self_np calls fail with EPERM (<pthread/qos.h>; tests/macos_qos_os_truth.rs checks it against the host).

A SimHost applies these to its own simulated threads and never reaches the real scheduler or lock memory. A plain sim gates the same calls on its privileges and then lets an allowed one through to the real kernel. tests/rlimits.rs checks the rules, and its rlimit_sequence_os_truth (Linux) and memlock_os_truth (macOS) replay one sequence against the real kernel in a forked child with Privileges::from_real_process(); run them under Docker with --ulimit rtprio=…, --ulimit nice=…, --ulimit memlock=… and --cap-add SYS_NICE/IPC_LOCK/SYS_RESOURCE to cover the other branches.

OS Gated
Linux binding a port below unprivileged_port_start (EACCES without CAP_NET_BIND_SERVICE), SO_RCVBUFFORCE/SO_SNDBUFFORCE (CAP_NET_ADMIN), SO_PRIORITY outside 0–6 and SO_MARK (CAP_NET_ADMIN or CAP_NET_RAW), changing the device of a bound socket (CAP_NET_RAW), socket(AF_PACKET) (CAP_NET_RAW); EPERM otherwise.
macOS binding a port below 1024 on a specific address, and opening /dev/bpf*, need root (EACCES).
Windows no socket call; without sys_nice a REALTIME_PRIORITY_CLASS request runs at HIGH_PRIORITY_CLASS.

tests/os_parity.rs compares buffer options, flood admission counts and reserved-port binds against the real OS, and tests/socket_limits.rs truesize_os_truth the charge of every datagram size around each step.

Protocol counters

Every sim keeps the host's UDP and TCP counters, moved by the code under test's sockets (a tester is another machine) and readable as the code under test would read them:

OS Where UDP TCP
Linux /proc/net/snmp, /proc/net/snmp6 (with or without a SimHost or VirtualFs; a snapshot at the first read, fresh after a seek to 0) InDatagrams (read), NoPorts, InErrors/RcvbufErrors (overflow), OutDatagrams, IgnoredMulti ActiveOpens, PassiveOpens, AttemptFails, EstabResets, CurrEstab, InSegs/OutSegs (data segments), OutRsts; RtoMin 200, RtoMax 120000, MaxConn -1
macOS sysctl/sysctlbyname net.inet.udp.stats (struct udpstat, 104 bytes) udps_ipackets (every arrival), udps_noport, udps_noportbcast, udps_fullsock, udps_opackets —
Windows GetUdpStatistics(Ex/Ex2), GetTcpStatistics(Ex/Ex2) per family dwInDatagrams (arrivals), dwNoPorts, dwInErrors (overflow), dwOutDatagrams, dwNumAddrs as Linux, RtoAlgorithm 4, dwRtoMin 5

proto_counters() (and Sim::proto_counters) returns them per family for the test. A unicast datagram to a port of the host nobody holds is a no-port; a broadcast, or a multicast one for a group a socket of the host joined, that no socket took is ignored-multi; a group nobody joined counts nowhere. The IP and ICMP lines count those datagrams and TCP data segments and the port unreachables sent back; checksum, memory, send-buffer and retransmission counters stay 0. tests/proto_counters_os_truth.rs compares the layouts and which counter a closed port and an unread datagram move with the real host.

Names

Inside a Sim, name resolution never leaves the process. getaddrinfo, getnameinfo and gethostbyname — and on Windows GetAddrInfoW, GetNameInfoW — answer from the sim's host table and a built-in localhost (::1, 127.0.0.1; on Windows "" too). Any other name fails as the host OS fails an unknown name: EAI_NONAME (std's "failed to lookup address information"), HOST_NOT_FOUND, WSAHOST_NOT_FOUND (11001), at once and without a packet.

let sim = Sim::builder().add_host("robot1.local", ["127.0.0.20".parse().unwrap()]).build();
sim.run(|| {
    let robot = connect_tester::<Line>("robot1.local:9000");   // testers and policies take names
    set_dns_policy("robot1.local", |p| p.latency = Duration::from_millis(30));
    // TcpStream::connect("ROBOT1.local.:9000") reaches the tester after 30 ms of virtual time
});
Call Effect
add_host(name, addrs) / remove_host(name) Resolves name (case-insensitive, one trailing dot ignored) to addrs in table order. Also on SimBuilder and Sim. A name with no address of the asked family fails with EAI_NONAME (glibc, macOS) or WSANO_DATA (Windows).
set_dns_policy(name, ..) / set_default_dns_policy(..) DnsPolicy { latency, failure, failure_rate }: the lookup waits latency on the sim's clock (virtual, so it time-skips and orders under deterministic()), then fails with failure — or, with a failure_rate, fails that often (TryAgain unless failure says otherwise), drawn from the sim's seed. Failures are logged as Fault::Dns.
SimBuilder::resolve_real(name) Looks name up on the real resolver when the sim is built and keeps the answer; panics if it fails.
SimBuilder::real_dns() Sends unknown names to the real resolver, counted as unmodelled calls. Panics in build() with deterministic(), since a real lookup blocks outside the schedule.

Numeric hosts, a null node and service names go to the real libc / ws2_32, so hints semantics and error precedence are the host's own; Windows reads a shorthand such as 127.1 as numeric only with AI_NUMERICHOST, and otherwise as a name. A known name is answered by asking the OS for each of its addresses as a numeric host with the caller's hints and joining the lists it returns; freeaddrinfo takes such a list apart again on any thread, even after the sim is gone. AI_ADDRCONFIG is ignored. getnameinfo reverses an address to the first table name holding it (and loopback to localhost on Linux and macOS), otherwise numerically, or EAI_NONAME / WSAHOST_NOT_FOUND with NI_NAMEREQD. The asynchronous resolvers (getaddrinfo_a, GetAddrInfoEx, DNSServiceGetAddrInfo, DnsQuery_W, res_query) are not modelled and are reported as unmodelled calls.

Signals

Each Sim keeps its own signal dispositions. The code under test installs its handlers with the OS's own calls, so ctrlc and signal-hook run unchanged, and the real process's handlers are never touched; two sims in one test binary each see only their own.

let sim = Sim::new();
let signals = sim.signals();                      // Clone + Send, usable from any thread
sim.run(|| {
    let (tx, rx) = std::sync::mpsc::channel();
    ctrlc::set_handler(move || tx.send(()).unwrap()).unwrap();
    assert_eq!(signals.raise(Signal::Interrupt), SignalDelivery::Handled);
    rx.recv().unwrap();
    let pending = signals.raise_after(Signal::Terminate, Duration::from_secs(30)); // a virtual timer
    pending.wait();
});
Platform What is simulated
unix sigaction, signal (BSD semantics), raise, kill of the own process (getpid(), 0, -getpgrp(); kill(-1) skips the caller, as on the host, so it goes to the OS) and pthread_kill for SIGINT, SIGTERM, SIGHUP. Every other signal, and other pids, go to the OS. A sim's table starts from the process's dispositions, so an inherited SIG_IGN reads back. Delivery is synchronous on the sending thread, as POSIX requires of a signal a process sends itself: SA_SIGINFO handlers get a siginfo_t with si_code SI_USER (Linux: SI_TKILL for raise/pthread_kill) and si_pid the parent for the test's signals; SA_RESETHAND and SA_NODEFER behave as on the host. pthread_kill to another thread of the sim runs the handler there at its next hooked call.
Windows SetConsoleCtrlHandler (a list per sim, plus the ignore-CTRL+C flag), GenerateConsoleCtrlEvent for this process (group 0 or its own id; asynchronous, as on Windows; CTRL+C to a nonzero group succeeds and reaches no one, as Windows documents) and the CRT's signal/raise for SIGINT, SIGBREAK, SIGTERM. Each event is handled on a new thread, last registered handler first, until one returns TRUE. A close event's handlers get 5 s and a logoff or shutdown's 20 s of sim time before the delivery reports HandledThenExit.
both Sim::raise_signal / SignalHandle::raise returns Handled, HandledThenExit, DefaultAction (never carried out), Ignored, or Unavailable for a signal the host OS lacks (Break/Close/Logoff/Shutdown on unix, Terminate/Hangup on Windows). From a thread outside the sim it is delivered on a new thread of the sim, joined before it returns; inside an executive's timestamp its effects wait for the timestamp to end like any other wake.

The waits those libraries' threads block on are simulated: POSIX semaphores (sem_wait, sem_timedwait, sem_clockwait, sem_trywait, sem_post; named semaphores on macOS), Windows semaphore handles in WaitForSingleObject / ReleaseSemaphore, and socketpair(AF_UNIX, SOCK_STREAM | SOCK_DGRAM), served by the fabric like any socket (poll, epoll, kqueue, EOF, EPIPE; an unnamed address as the host reports it). pipe/pipe2 stay real. Under deterministic(), once every root thread of a run has left and nothing can run, the schedule lets leftover daemon threads (the ctrlc thread) go; they wait outside it until the sim runs again.

A library that installs its dispatcher once per process and remembers that in a static (signal-hook-registry, and so tokio::signal, signal-hook, async-signal) calls sigaction only in the first sim whose code registers with it. SimBuilder::process_signal_handlers() lets a signal the sim's own code never set an action for reach the handler the code under test last installed in any sim of the process, so such a library works in every sim; tokio delivers a signal to the listeners of every runtime, as one process would.

SimBuilder::forward_real_signals() opts a sim in to the real SIGINT/SIGTERM/SIGHUP (a console control handler on Windows): while any sim forwards, a process-wide forwarder delivers each real signal into every forwarding sim, and one that no sim handled or ignored is raised for real against the process's own disposition. The last forwarding sim to drop restores the originals. Signal masks, sigwait/signalfd, EINTR on blocked calls and SIGALRM are not modelled.

Reaching the real machine: snare::real

The harness and other-side emulation sometimes need the real OS — to read a real fixture file, or a real environment value — while the code under test stays simulated. Wrap those harness operations in real:

let key = snare::real(|| std::env::var("HOME"));          // real environment
let fixture = snare::real(|| std::fs::read("tests/data/frame.bin")).unwrap();  // real file

real(f) runs f with interposition switched off on the calling thread only (it nests, and other managed threads stay simulated). Threads spawned inside real also run outside the simulation. Keep resource construction and execution of the code under test inside Sim::run, outside any real closure. A native object created by the harness must not become a resource of the code under test.

The cargo snare harness

snare only compiles with --cfg snare; without it the build stops at a compile_error!. On Linux, rustix and the raw-syscall crates reach the kernel with an inline syscall instruction unless they are built for snare (below), and such calls would silently escape the simulation.

Set a package up once:

$ cargo snare --init

It puts snare under a cfg(snare) target table (moving an existing snare dev-dependency there and combining its platform cfg), and declares the cfg to the unexpected_cfgs lint (in [workspace.lints.rust] when the package inherits its lints):

[target.'cfg(snare)'.dev-dependencies]
snare = "^3"

[lints.rust]
unexpected_cfgs = { level = "warn", check-cfg = ['cfg(snare)'] }

Guard the tests with the same cfg — #![cfg(snare)] at the top of an integration test file, #[cfg(all(test, snare))] on an in-crate test module:

#![cfg(snare)]

use snare::prelude::*;

#[test]
fn deterministic() {
    Sim::new().run(|| { /* ... */ });
}
$ cargo snare test            # builds with --cfg snare, patches in the shims, runs the tests

cargo snare sets --cfg snare (and --cfg rustix_use_libc) and injects the drop-in shims via cargo --config patch.crates-io..., from the snare release the lock file resolved (tag vX for a crates.io snare X, with snare and snare-interpose patched from it so one interposer is linked), or from the checkout or git reference a path or git snare comes from. Plain cargo test builds neither snare nor the #[cfg(snare)] tests.

The cfgs go in through RUSTFLAGS (appended to any already set), and cargo then ignores build.rustflags and target.*.rustflags from .cargo/config.toml. A crate that relies on cfgs from its own config file must put them in RUSTFLAGS as well; a CARGO_ENCODED_RUSTFLAGS in the environment overrides both and drops the snare cfgs. cargo snare test --dry-run prints the shims patched, the crates given per-sim state by --parallel (see Shims), and the assembled command.

Shims (shims/)

Some crates bypass libc in ways an import-table hook cannot see — raw io_uring rings in mmap'd memory, AF_XDP UMEM, CPU counter reads — or seed from process-wide state a sim cannot replay. For those, snare provides drop-in replacement crates injected via [patch.crates-io]. cargo snare test patches these whenever their crate is in the dependency graph:

  • io-uring — emulates the ring in-process, executing each SQE through libc read/write so the interposer sees ordinary I/O.
  • xsk-rs (AF_XDP) — routes frame TX/RX through libc send/recv.
  • sc, syscalls — route raw syscalls through libc so the syscall hook catches them.
  • quanta (0.12 and 0.13), minstant, fastant — never read the TSC or the aarch64 system counter, so their clocks are the sim's; quanta also skips its ~200 ms calibration spin.
  • fastrand — seeds each thread's generator inside a sim from the sim instead of the process-wide thread id, so deterministic sims replay whatever ran before them.
  • rayon-core — seeds each worker's steal order by its index in its pool, so pools built in a deterministic sim replay.

Others only let concurrent sims each keep their own copy of a runtime's process-wide state; sims work without them by taking turns, so they are opt-in with --parallel (or parallel = [...] under [package.metadata.snare] / [workspace.metadata.snare]):

  • async-io — one reactor and "async-io" thread per sim instead of per process, shut down when the sim drops.
  • async-global-executor, blocking — async-std's global executor and the unblock thread pool, one per sim, so async-std tasks and blocking jobs of one sim never run on another's threads.
  • smol — smol::spawn's global executor and "smol-N" threads, one set per sim.
  • rayon-core with --cfg snare_parallel_rayon — rayon's global pool, one per sim.
$ cargo snare test --parallel smol        # async-io + blocking + smol
$ cargo snare test --parallel async-std   # async-io + async-global-executor + blocking
$ cargo snare test --parallel rayon       # adds --cfg snare_parallel_rayon
$ cargo snare test --parallel all --dry-run

shims/README.md lists each shim's change and how to patch them by hand.

The shims honor snare::real: each resource is tagged with the passthrough mode it was created in, and using it in the other mode panics rather than silently mixing an emulated ring with real-OS I/O.

Platform support

Linux (x86_64, aarch64) macOS (aarch64, x86_64) Windows (x86_64, aarch64)
Interposition ELF GOT + data relocations Mach-O __got + data binds (+ variadic trampoline on arm64) PE IAT
Time / entropy / sleep ✅ ✅ ✅
TCP ✅ ✅ ✅ (Winsock)
UDP ✅ ✅ ✅ (Winsock)
Readiness ✅ (epoll, poll, eventfd) ✅ (kqueue/kevent, poll) ✅ (WSAPoll, select, IOCP for Mio)
Raw L2 AF_PACKET /dev/bpf pcap/npcap (structural)
Host scheduling ✅ ✅ (pthread/Mach) ✅ (WinHost)
Interfaces and routing ✅ (getifaddrs, SIOCGIF*, netlink) ✅ (getifaddrs, SIOCGIF*, sysctl) ✅ (IP Helper)
NIC tuning / PTP ✅ (ethtool, qdiscs, sysfs) partial (sysctl) adapter properties + restart (SetupAPI/registry/CM_*)
Process limits ✅ (RLIMIT_RTPRIO/NICE/MEMLOCK, mlock) ✅ (RLIMIT_MEMLOCK, mlock) priority class, working set (WinHost)
Names / signals ✅ ✅ ✅ (GetAddrInfoW, console control events)

Linux means glibc (target_env = "gnu"). Every other target — musl, 32-bit x86 and ARM, Android, the BSDs, illumos, Redox — stops at a single compile_error! naming the supported targets. A crate that also builds for those targets needs no gate beyond cfg(snare): only cargo snare test sets it, on the machine running the tests, so a build for any other target never pulls snare in.

[target.'cfg(snare)'.dev-dependencies]
snare = "^3"

The minimum supported Rust version is 1.88.

Repository layout

crates/snare-interpose/   the engine: patchers, hooks, Domain, backend traits, real()
crates/snare/             Sim, SimHost, HostProfile, EasyBuilder, Fabric, VirtualFs, testers
crates/cargo-snare/       the `cargo snare` subcommand
probes/interpose-probe/   test-fixture DLL for the Windows module-loading test (unpublished)
shims/                    drop-in [patch.crates-io] crates: io-uring, xsk-rs, sc, syscalls, quanta, minstant,
                          fastant, fastrand, rayon-core; with --parallel: async-io, async-global-executor,
                          blocking, smol
scripts/                  Windows VM and hardware test runners, socket-buffer and runtime measurements

Running the tests

.cargo/config.toml builds this workspace with --cfg snare; a RUSTFLAGS set in the environment replaces it, so include --cfg snare there too.

cargo test                                  # macOS / Linux
cargo clippy --all-targets
cargo xwin build --workspace --target x86_64-pc-windows-msvc   # Windows cross-build (cargo-xwin)

Linux-only surfaces are exercised in a container (docker run --rm -v "$PWD":/work -w /work rust:latest cargo test). scripts/test-windows.sh cross-builds the workspace and runs its Windows tests in a Parallels VM; scripts/test-windows-host.sh runs only the scheduling-plane target.

The Windows module-loading test needs the interpose-probe DLL in the target's profile directory, beside deps. The full Windows runner builds the workspace first to produce it; cargo test --no-run alone does not. On a native Windows checkout, run cargo build -p interpose-probe before cargo test --workspace --all-targets, using the same target and profile for both commands.

A note on the source

Behaviour that mirrors a kernel interface is cited in the code to its authority — a man-page section (man 2 sched_setscheduler), a kernel header (<linux/if_link.h> struct rtnl_link_stats64), or a Documentation/ path (Documentation/networking/timestamping.rst) — so the modeled struct layouts, magic numbers, and errno conventions can be checked against the real thing.

License

Licensed under the Apache License, Version 2.0. See LICENSE.