wiretally 0.1.1

Counts a child process's network bytes per domain via an ephemeral loopback proxy
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173

# wiretally

Run any command behind an ephemeral counting proxy and get a per-domain byte report when it
exits. No root, no certificate to install, no setup.

Counts traffic from **cooperative clients** — those that honour `HTTP_PROXY`/`HTTPS_PROXY`/
`ALL_PROXY`, which covers `curl`, `git`, the AWS SDKs, `requests`, and most everything else that
speaks HTTP or TCP. Traffic a client sends without consulting the proxy is not measured; see
[cooperative clients only](#what-this-measures-cooperative-clients-only).

```bash
wiretally --domain-prefix example.com -- curl -s -o /dev/null https://example.com/
```

```text
================================================================================
                           WIRETALLY TRAFFIC SUMMARY
================================================================================
Command:          curl -s -o /dev/null https://example.com/
Target Filter:    *.example.com (Prefix Match)
Execution Time:   0.09s
Exit Code:        0

DOMAIN / ENDPOINT SUMMARY:
--------------------------------------------------------------------------------
ENDPOINT / DOMAIN                   INGRESS (Rx)      EGRESS (Tx)   CONNS
--------------------------------------------------------------------------------
example.com                              4.71 KB            585 B       1
--------------------------------------------------------------------------------
TOTAL (Matching Filter):                 4.71 KB            585 B       1
TOTAL (All Destinations):                4.71 KB            585 B       1
================================================================================
```

## Install

```bash
cargo install wiretally
```

Or download a prebuilt binary for macOS or Linux (arm64 and x86_64) from the
[latest release](https://github.com/alexjbuck/wiretally/releases/latest). Each archive holds a
single `wiretally` binary — drop it on your `PATH`; there is nothing to configure. The Linux builds
link against glibc.

Building from source needs Rust 1.88 or newer.

## Usage

```text
wiretally [OPTIONS] -- <COMMAND> [ARGS...]

  -d, --domain-prefix <STRING>  Domain suffix filter, e.g. "amazonaws.com"
  -v, --verbose                 Per-connection wire log on stderr
  -j, --json                    Emit the summary as JSON instead of a table
```

The child's stdout and stderr are inherited directly, and `wiretally` exits with the child's
exit code (`128 + signal` if it was killed), so it drops into pipelines and CI without changing
behaviour. With `--json`, the report is a single JSON document on stdout:

```bash
wiretally -j -d amazonaws.com -- mcap info s3://bucket/file.mcap \
  | jq '.endpoints[] | select(.matches_filter) | {domain, ingress_bytes}'
```

## What this measures: cooperative clients only

wiretally counts what a client **chooses** to send through it. It is not a packet capture and
not an interception layer — it is a proxy, which means it sits in the path only for clients that
read the proxy environment variables and route themselves through it. Proxy env vars are
advisory configuration, not enforcement.

```
counted:     child ──TCP──> wiretally ──TCP──> service     (service's only peer is the proxy)
invisible:   child ──UDP──────────────────────> service     (proxy never involved)
```

Two consequences worth internalising:

- **Inside a tunnel, accounting is complete.** Once a connection goes through wiretally, the
  service knows nothing but the proxy's address and replies to it, so both directions are seen
  and counted. A service cannot escape an established tunnel and talk to the child directly — it
  has no idea the child exists.
- **Outside a tunnel, there is nothing to see.** Anything the child sends without asking the
  proxy first — a raw `sendto()`, a QUIC handshake, a DNS query, a library that only honours its
  own config keys, a host matched by `NO_PROXY` — leaves the machine directly. The service then
  replies to the child, because the child is the only peer it ever had. Those bytes are not
  under-counted; they are absent, and wiretally cannot detect that they happened.

So the report is exact for cooperative traffic and silent about the rest. For clients whose
transports you know (the AWS SDKs, `curl`, `git`, `requests`, most gRPC-over-TCP stacks) that
covers everything. If you are unsure for a given command, verify from outside — `lsof -i UDP -p
<pid>` while it runs, or a one-off `sudo tcpdump -i any 'udp and port 443'`. Structurally
bypass-proof accounting needs OS-level capture (a TUN device, eBPF, or pcap), which costs root
or Linux-only support; wiretally trades that for being rootless, cross-platform, and
zero-setup.

## How it works

1. A proxy binds an OS-assigned port on `127.0.0.1` and speaks three protocols on it, chosen by
   the first byte of each connection: SOCKS5 (`0x05`), HTTP `CONNECT`, and plain HTTP.
2. The child's environment — and only the child's — gets `HTTP_PROXY`/`HTTPS_PROXY` (plus
   lowercase) pointing at `http://127.0.0.1:PORT`, and `ALL_PROXY`/`all_proxy` pointing at
   `socks5h://127.0.0.1:PORT`. Nothing global is touched.
3. `CONNECT` and SOCKS5 connections are spliced as raw TCP. TLS is never terminated: no
   certificate to install, no crypto on the data path, and the counters see exactly the
   ciphertext that crossed the wire. Because the splice is protocol-agnostic, anything the
   client tunnels is measured — HTTP/2, gRPC, WebSockets, Postgres, plain TLS. Plain HTTP is
   forwarded by hyper, with the counters on the *upstream* socket so the numbers reflect what
   left the machine rather than what the child handed to the proxy.
4. Counting is one relaxed `fetch_add` per read or write on an otherwise untouched buffer, so
   the per-chunk overhead is a few nanoseconds and nothing is buffered in memory.
5. After the child exits, in-flight connections are given up to two seconds to drain, endpoints
   that were only ever seen as bare IPs get a reverse-DNS name, and the report is printed.

`socks5h` rather than `socks5` matters: it makes the client send hostnames to the proxy instead
of resolving them first, so endpoints stay named in the report instead of collapsing to IPs.

### What gets counted

| Traffic | Counted | Via |
|---|---|---|
| HTTPS, HTTP/2 over TLS | yes | `CONNECT` tunnel |
| Plain HTTP | yes | forwarded by hyper |
| gRPC, Postgres, Redis, arbitrary TCP | yes, if the client honours `ALL_PROXY` | SOCKS5 `CONNECT` |
| UDP a client routes through the proxy | yes | SOCKS5 `UDP ASSOCIATE` relay |
| QUIC, HTTP/3, DNS sent straight to the destination | **no** | never reaches the proxy |
| Anything from a client that ignores the proxy env vars | **no** | not cooperative, so not in the path |

Endpoint names come from the `CONNECT` target or request URI when available, because that is
what the child asked for; a PTR record for an S3 address (`s3-1-w.amazonaws.com`) says much less
than the regional endpoint the SDK used. Reverse DNS is therefore a fallback, not the default.

`--domain-prefix` matches on label boundaries: `amazonaws.com` covers `amazonaws.com` and
`s3.us-east-1.amazonaws.com`, but not `notamazonaws.com`. Non-matching endpoints are dropped
from the table but still counted in the `TOTAL (All Destinations)` row, so a filter can never
hide traffic completely.

## Limitations

All of these follow from being a proxy rather than a packet capture — see
[cooperative clients only](#what-this-measures-cooperative-clients-only) for why:

- **Only cooperative clients are measured.** Anything that ignores the proxy environment
  variables is invisible: raw sockets, hosts matched by an inherited `NO_PROXY`, and SDKs that
  only honour their own config keys.
- **UDP is counted only when the client routes it through the relay.** QUIC and HTTP/3 normally
  open a UDP socket straight to the destination, and DNS almost always does. wiretally cannot
  even detect that this happened: the `Alt-Svc: h3=…` header announcing the switch arrives inside
  TLS it deliberately does not decrypt. **If an application or server can shift to UDP, treat
  the totals as a floor rather than a full account.**
- The `CONNECT`/SOCKS handshake between child and proxy is excluded from the totals — it is
  local overhead that never reaches the remote host.
- Per-endpoint totals aggregate by hostname, so two hostnames on the same IP stay separate rows
  (which is usually what you want) and one hostname across many IPs collapses into one row.

## Development

```bash
cargo test          # unit, doc, and integration tests
cargo clippy --all-targets
```

The integration tests put a raw TCP origin server behind the proxy and make that server count
what crossed its own socket; the proxy's counters must match those numbers exactly. That is the
accuracy claim, checked directly rather than by comparing the proxy against itself. No network
access is required for them, and the SOCKS5 datagram codec is additionally covered by property
tests asserting that it round-trips and never panics on arbitrary input.

CI runs fmt, clippy, and the full suite on Linux and macOS, plus the minimum supported Rust
version and a dependency licence audit. Release and publishing are documented in
[docs/RELEASING.md](docs/RELEASING.md).