rdns-server 1.17.21

A high-performance, security-focused DNS server written in Rust
# rDNS Performance Benchmarks

> Read the same data with charts at https://zerosandonesllc.github.io/rDNS/benchmarks/

Benchmark results comparing rDNS against Unbound 1.19, the industry-standard recursive DNS resolver, using `dnsperf` with cached queries over UDP on the same machine. **Both servers use all cores** — earlier revisions of this document compared against a single-threaded Unbound, which is not a fair throughput test; the numbers below are all-cores-vs-all-cores.

## Test Environment

| Component | Details |
|-----------|---------|
| CPU | 24 cores (AMD64) |
| RAM | 32 GB |
| OS | Linux 6.6.87 (WSL2) |
| rDNS | v1.17.17, release build (LTO fat, codegen-units=1, `target-cpu=native`) |
| Unbound | 1.19, `num-threads: 24`, `so-reuseport: yes`, module-config: iterator |
| Workload | 100 unique queries (A, AAAA, MX, NS, TXT, NXDOMAIN), all cached |
| Tool | `dnsperf`, 12 sender threads, 10-second runs, medians of 3 |

Both servers configured as forwarders to 1.1.1.1 with DNSSEC disabled, logging at error-only level. The load generator (`dnsperf`) runs on the same 24-core box, so both servers contend with it for cores equally.

## Results Summary

### Throughput (Queries Per Second)

Both servers land in the same band — the story is parity, with rDNS ahead at
low concurrency and Unbound a touch ahead at very high concurrency.

```
                     rDNS vs Unbound — Queries Per Second
                     ════════════════════════════════════

  500 clients  ██████████████████████████████████████████▍       565,544
               █████████████████████████████████████████████▋    609,617

  200 clients  █████████████████████████████████████████████▍    605,527
               ██████████████████████████████████████████████▏   615,914

  100 clients  ██████████████████████████████████████████████▋   621,983
               ██████████████████████████████████████████████▌   621,024

   50 clients  ████████████████████████████████████████████████  639,581
               ████████████████████████████████████████████▊     596,564

               ■ rDNS    ■ Unbound (num-threads: 24)
```

### Head-to-Head Comparison

Medians of 3 runs; both servers zero packet loss.

| Clients | rDNS QPS | Unbound QPS | Ratio | Winner |
|---------|----------|-------------|-------|--------|
| 50 | 639,581 | 596,564 | 1.07x | rDNS |
| 100 | 621,983 | 621,024 | 1.00x | Parity |
| 200 | 605,527 | 615,914 | 0.98x | ~Parity |
| 500 | 565,544 | 609,617 | 0.93x | Unbound |

### Average Latency

| Clients | rDNS | Unbound |
|---------|------|---------|
| 50 | 98 us | 76 us |
| 100 | 110 us | 70 us |
| 200 | 107 us | 74 us |
| 500 | 116 us | 79 us |

Unbound keeps a latency edge here (~75 µs vs rDNS's ~110 µs under peak load); rDNS's single-client latency is lower (see Notes). Both are far below any level a client would notice.

## Optimization Journey

rDNS went through six optimization rounds. The first five (baseline → SO_REUSEPORT) took it from 29K to 437K QPS. The sixth ([#86](https://github.com/ZerosAndOnesLLC/rDNS/issues/86)) was driven by `perf` after a *fair* comparison against multi-threaded Unbound revealed rDNS was behind: profiling showed ~33% of per-query CPU going to allocation and SipHash. Eliminating it lifted the cached hot path ~22% (524K → 640K at 50 clients) and closed the gap to parity.

### Progression

```
  QPS (50 concurrent clients, cached)

  v1  ██▌                                                29,630
  v2  ████████▎                                          93,781
  v3  ███████▍                                           84,881
  v4  ███████████████████████████████▎                  309,386
  v5  █████████████████████████████████████████████████  437,434
  v6  ████████████████████████████████████████████████████████████████████  639,581

  0       100K      200K      300K      400K      500K      600K
```

| Version | QPS (50 clients) | Key Change |
|---------|------------------|------------|
| **v1** | 29,630 | Baseline — sequential recv/send loop |
| **v2** | 93,781 | Task-per-query, forwarder connection pool |
| **v3** | 84,881 | Sync fast-path for cache hits, direct wire encoding |
| **v4** | 309,386 | parking_lot sharded cache, LTO, native CPU |
| **v5** | 437,434 | SO_REUSEPORT per-worker sockets |
| **v6** | 639,581 | Zero-alloc compression probe, FxHash cache, `Arc` entries, precomputed wire responses, tuned worker count |

### v5 → v6: Kill per-query allocation and hashing (+22%)

A fair benchmark (both servers on all cores) put rDNS at 0.85–0.97× of Unbound. `perf` under cached load found the cause: **~17% of CPU in `malloc`/`free` and ~16% in SipHash**, per query. Five changes removed it, four of them byte-for-byte identical to the previous output:

- **Compression probe**`DnsName::encode_compressed` allocated a throwaway `Vec<String>` (cloning every label) on every compression-map probe, for every name in every response. Now it borrows `&[String]` — zero allocation.
- **Hashing** — the cache double-hashed each key with SipHash (once to pick a shard, once inside the shard map), and the name-compression map was SipHash-backed too. All switched to a small inline FxHash; the shard key is hashed once.
- **`Arc` cache entries**`lookup` deep-cloned the whole entry (every record `Vec`) per hit; entries are now shared behind `Arc`.
- **Precomputed responses** — cache hits re-encoded the full response every time. The wire body (compression resolved, TTL placeholders) is now built once and memoized on the entry; hits `memcpy` it and patch the TTL fields. This was the single biggest win (`encode_compressed` had grown to ~13% of CPU on its own).
- **Worker count** — retuned the UDP recv-worker default to ~¾ of cores (one-per-core measurably regressed under load), with an `RDNS_UDP_WORKERS` override.

### What Each Round Changed

#### v1 → v2: Concurrency (+216%)

The original UDP listener processed queries sequentially — `recv_from` → `await handle_query` → `send_to` in a single loop. Under concurrency, each query blocked the socket while waiting for upstream resolution.

**Fix:** Spawn a tokio task per incoming query so the recv loop is never blocked. Created a forwarder connection pool that multiplexes queries over a single connected UDP socket with async response dispatch via oneshot channels.

#### v2 → v3: Reduce Allocations (-9%, but better scaling)

Spawning a task for every query (even cache hits) added overhead from cloning resolver/cache/auth/rpz per task, plus full `Message::decode` → `Message::encode` round-trip.

**Fix:** Added a synchronous fast-path that handles cache hits, authoritative answers, and RPZ blocks inline in the recv loop without spawning a task. Built a fast query parser that extracts name/type/class from wire format without full message decode. Cache responses are encoded directly to wire format with TTL adjustment, bypassing `Message` struct construction entirely.

#### v3 → v4: Faster Cache (+264%)

DashMap was the bottleneck under contention. Its internal sharding was too coarse and `get_mut` (needed to increment hit count) took a write lock on every cache hit.

**Fix:** Replaced DashMap with a custom 256-shard cache using `parking_lot::RwLock`. Cache hits take a read lock only — multiple workers can serve cache hits simultaneously with zero contention. Write locks are only taken for inserts and expired entry cleanup. Also enabled LTO (fat), single codegen unit, and native CPU targeting in the release profile.

#### v4 → v5: Eliminate Socket Contention (+41%)

Multiple recv workers sharing a single `Arc<UdpSocket>` caused kernel-level contention on the socket receive buffer. Workers would wake up, race to `recv_from`, and only one would get the packet.

**Fix:** Use `SO_REUSEPORT` to bind a separate socket per worker on the same port. The kernel distributes incoming packets across sockets using a consistent hash, so each worker processes its packets independently with zero contention. Falls back to shared socket on platforms without `SO_REUSEPORT`. Also increased `SO_RCVBUF` to 4 MB for burst absorption.

## Architecture

```
  Client packets arrive on port 53
  ┌───────────────────────────┐
  │    Kernel (SO_REUSEPORT)  │
  │    Distributes by flow    │
  └──┬──────┬──────┬──────┬───┘
     ▼      ▼      ▼      ▼
  Worker  Worker  Worker  Worker   ← Each has its own socket
     │      │      │      │
     ▼      ▼      ▼      ▼
  ┌──────────────────────────┐
  │  parse_query_fast()      │     Wire format → name + type (no alloc)
  │  try_handle_sync()       │     Cache/Auth/RPZ check (read lock only)
  │  build_response_fast()   │     Direct wire encode (no Message struct)
  │  send_to()               │     Response back to client
  └──────────────────────────┘
           │ cache miss
     tokio::spawn()
  ┌────────▼─────────┐
  │  Resolver         │
  │  ForwarderPool    │     Single connected socket, multiplexed by query ID
  │  Cache insert     │     Write lock on one shard only
  │  send_to()        │
  └──────────────────┘
```

The hot path (cache hit) involves:
- 1 fast parse of the query wire format
- 1 read lock on one cache shard
- 1 direct wire-format encode of the response
- 0 heap allocations (except the response Vec)
- 0 task spawns
- 0 async awaits (except the socket I/O itself)

## Reproducing

```bash
# Build
RUSTFLAGS="-C target-cpu=native" cargo build --release

# Install tools
sudo apt-get install -y dnsperf unbound

# Run the benchmark suite
bash bench/run.sh

# Or manually:
./target/release/rdns -c bench/rdns-bench.toml &
dnsperf -s 127.0.0.1 -p 5553 -d bench/queryfile.txt -c 50 -l 10 -Q 500000
```

## Notes

- **Unbound runs multi-threaded here** (`num-threads: 24`, `so-reuseport: yes`) — a fair all-cores-vs-all-cores test. Earlier versions of this document benchmarked a single-threaded Unbound, which inflated rDNS's apparent lead; the current numbers supersede those.
- Single-client performance is lower than Unbound because SO_REUSEPORT distributes by flow hash — with one source, only one of N workers receives packets. This is not a realistic production scenario, but it means rDNS's *single-client* latency is very low even though its single-client throughput is not the headline number.
- Numbers are medians of 3 runs. The load generator is co-located, so run-to-run variance is real (±5–10%); treat the two servers as at parity rather than reading a precise multiplier into any single row.
- These benchmarks measure cached query throughput only. Cold-cache performance depends on upstream latency and is not measured here.
- Results will vary by hardware, kernel version, and system load.