omni-dev 0.29.0

A powerful Git commit message analysis and amendment toolkit
Documentation
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
# Browser Bridge

`omni-dev browser bridge serve` runs a long-lived local process that lets you drive
HTTP requests **through an authenticated browser tab**. When you are
investigating internal services (Grafana/Loki, internal dashboards, SSO-gated
admin panels), the browser holds authenticated sessions — SSO, OAuth, session
cookies — that are hard to replicate programmatically. The bridge lets
`omni-dev` (or a local tool you control) issue requests inside the browser's
authenticated context **without exfiltrating cookies or tokens**.

This is a *confused deputy by design*: the bridge borrows the browser's
authority on behalf of whoever can talk to it. That makes the
[security model](#security-model) the load-bearing part of the design, not an
add-on — both planes are authenticated and default-closed. See
[ADR-0036](adrs/adr-0036.md) for the architectural rationale.

## Table of Contents

1. [How it works]#how-it-works
2. [Quick start]#quick-start
3. [Talking to the control plane]#talking-to-the-control-plane
4. [The `request` thin client]#the-request-thin-client
5. [Routing to a specific tab]#routing-to-a-specific-tab
6. [Security model]#security-model
7. [Flags]#flags
8. [Random ports]#random-ports
9. [Worked example: downloading Grafana / Loki logs]#worked-example-downloading-grafana--loki-logs
10. [Harvesting your own data (best-effort)]#harvesting-your-own-data-best-effort
11. [WebSocket wire protocol]#websocket-wire-protocol
12. [Caveats]#caveats

## How it works

```
   operator CLI / authorized script             browser tab (DevTools console)
              │                                          │
   HTTP + bearer token + X-Omni-Bridge      WebSocket + token subprotocol
              │                                          │
              ▼                                          ▼
  ┌───────────────────────────── omni-dev browser bridge ─────────────────────┐
  │   HTTP control plane            id-correlator           WebSocket plane    │
  │   127.0.0.1:9998      ───►   pending: id → oneshot   ───►   127.0.0.1:9999 │
  │   (axum)              ◄───   resolve on WS reply     ◄───   (tungstenite)  │
  └────────────────────────────────────────────────────────────────────────-─┘
```

A request flows: **control plane (authenticated) → assign `id` + register a
`oneshot` waiter → serialize a command frame → WebSocket → browser `fetch()` →
response frame → correlator resolves the waiter by `id` → control plane returns
the HTTP response.**

## Quick start

1. Start the bridge:

   ```bash
   omni-dev browser bridge serve
   ```

   It prints the bound ports, the generated session token, and a ready-to-paste
   JS snippet.

2. Open the DevTools console on the authenticated tab (e.g.
   `https://grafana.internal/...`) and paste the snippet. It connects and
   reconnects with backoff, presenting the token via the WebSocket subprotocol.

3. Drive requests from your shell (the token is in the bridge's stdout):

   ```bash
   export OMNI_BRIDGE_TOKEN=<token printed by the bridge>
   omni-dev browser bridge request --url /loki/api/v1/labels
   ```

The bridge works only while the tab is open and the snippet is running.

## Talking to the control plane

The control plane is a local HTTP server (default `127.0.0.1:9998`). Every
request must include `Authorization: Bearer <token>` **and** `X-Omni-Bridge: 1`.

### 1. Transparent proxy (the easy path)

Any request whose path is **not** under `/__bridge/` is forwarded (method,
headers, body) to the browser, with `url` set to the request path so it resolves
against the **page's origin**:

```bash
T="$OMNI_BRIDGE_TOKEN"
curl -H "Authorization: Bearer $T" -H "X-Omni-Bridge: 1" \
  http://localhost:9998/loki/api/v1/labels
```

### 2. Control / full-fidelity endpoints (reserved `/__bridge/` prefix)

| Method & path            | Purpose                                                                 |
|--------------------------|-------------------------------------------------------------------------|
| `GET  /__bridge/status`  | `{ "connected": bool, "browser_origin": str?, "tabs": [{id, origin?}], "pending": int }`. `tabs` lists every connected tab; `browser_origin` is the lone tab's origin (present only when exactly one is connected, for back-compat). |
| `POST /__bridge/request` | Full control. Body `{url, method, headers, body, stream?, target?, allow_origin?, credentials?}`; returns a structured response envelope, or — when `"stream": true` — an NDJSON chunk stream (see [Streaming]#streaming-responses). Cross-origin `url` rejected unless `serve --allow-origin` or the per-request `allow_origin` permits it (see [Outbound request scope]#outbound-request-scope-default-closed). See [Routing to a tab]#routing-to-a-specific-tab for `target`. |

```bash
curl -s -H "Authorization: Bearer $T" -H "X-Omni-Bridge: 1" \
  http://localhost:9998/__bridge/request -d '{
  "url": "/loki/api/v1/labels", "method": "GET",
  "headers": {"Accept": "application/json"}, "body": null
}'
# → {"id": 7, "status": 200, "headers": {...}, "body": "..."}
```

## The `request` thin client

`omni-dev browser bridge request` reads the token (from `OMNI_BRIDGE_TOKEN` or
`--token-file`), adds the required headers, POSTs to `/__bridge/request` on a
*running* bridge, and prints the response envelope:

```bash
omni-dev browser bridge request --url /loki/api/v1/labels --method GET
omni-dev browser bridge request --url /api/foo --method POST --body @payload.json
omni-dev browser bridge request --control-port 9998 --url /api/foo   # custom port
omni-dev browser bridge request --url /api/foo --header "Accept: application/json"
omni-dev browser bridge request --url https://cdn.example.com/x.js --credentials omit
```

`--body @file` reads the body from a file; otherwise the value is sent verbatim.

`--credentials <include|omit|same-origin>` sets the browser `fetch()` credentials
mode (default `include`, unchanged). Use `omit` to read a wildcard-CORS
(`Access-Control-Allow-Origin: *`) cross-origin response — e.g. a public CDN
asset — which the browser refuses to expose to a credentialed request. **`omit`
sends no cookies, so never use it for a cross-origin endpoint that needs the
user's session: the request would be unauthenticated.** Pair with `--allow-origin`
to permit the cross-origin target.

Binary responses (images, gzipped blobs, file downloads) come back in the
envelope as a base64 string with `"encoding": "base64"`; the caller decodes it.
The transparent proxy (below) decodes automatically, so `curl` receives the raw
bytes — pipe straight to a file with `--output`.

Pass `--stream` to consume a streaming / chunked / long-lived endpoint (SSE,
log-tail) instead of buffering the whole response: the decoded body bytes are
written to stdout as they arrive, and the head status is printed to stderr.

```bash
omni-dev browser bridge request --stream --url /events | your-consumer   # SSE / chunked
```

## Routing to a specific tab

Several authenticated tabs can be connected at once — paste the snippet into each
(e.g. a Grafana tab *and* an internal admin tab). Each connection is independent:
a new tab never evicts an existing one, and each authenticates on its own via the
token subprotocol.

`GET /__bridge/status` lists them, each with a server-assigned **connection id**
and its `Origin`:

```bash
curl -s "${H[@]}" http://localhost:9998/__bridge/status
# → {"connected":true,
#    "tabs":[{"id":1,"origin":"https://grafana.internal"},
#            {"id":2,"origin":"https://admin.internal"}],
#    "pending":0}
```

Select which tab a request targets with either:

- the **`X-Omni-Bridge-Target`** header (works on the transparent proxy *and*
  `POST /__bridge/request`), or
- a **`target`** field in the `POST /__bridge/request` body (or `--target` on the
  thin client).

The header takes precedence over the body field. A target is either a **connection
id** (canonical, always unambiguous) or an **`Origin`** that uniquely matches one
tab:

```bash
omni-dev browser bridge request --target 2 --url /api/foo                # by id
omni-dev browser bridge request --target https://admin.internal --url /x # by origin
curl "${H[@]}" -H "X-Omni-Bridge-Target: 1" http://localhost:9998/api/foo
```

Resolution rules:

| Situation | Result |
|-----------|--------|
| No tab connected | `503` |
| Exactly one tab, no target | Routes to it (v1 back-compat) |
| Several tabs, no target | `409` — specify a target (the error lists the tabs) |
| Target id / origin matches one tab | Routes to it |
| Target id / origin matches none | `404` |
| Target origin matches several tabs | `409` — target by connection id instead |

> Requests are routed to **exactly one** tab; there is no fan-out to multiple
> tabs. `--allow-origin` remains a single global value applied to every
> connection (a per-origin allowlist is possible future work).

## Security model

The trust boundary, stated once: **a request is trusted only if it presents the
session token AND is not a cross-origin browser request; everything else is
denied.** A localhost bind is *necessary but not sufficient* — it stops off-host
access but not other local users/processes, nor web pages you visit while the
bridge runs.

### Session token (mandatory, both planes)
- Generated at startup and printed to stdout with the snippet. It is **never**
  accepted as a CLI argument (argv is world-readable via `ps`/`/proc`). It may
  optionally come from `OMNI_BRIDGE_TOKEN` or a `0600` `--token-file`.
- Control plane: every request must carry `Authorization: Bearer <token>`.
- WebSocket plane: the browser presents the token via the WS subprotocol; the
  upgrade is rejected without it. An unauthenticated peer can never connect or
  evict the authenticated browser connection.

### Control-plane anti-CSRF / anti-DNS-rebinding (all enforced)
- **`Host` allowlist** — only `localhost:<port>` / `127.0.0.1:<port>` /
  `[::1]:<port>` accepted (blocks DNS rebinding).
- **Reject browser-originated requests** — any request carrying an `Origin`
  header or `Sec-Fetch-Site: cross-site`/`same-site` is denied. A legitimate CLI
  client sends neither.
- **Require `X-Omni-Bridge: 1`** on every request — a custom header forces a
  CORS preflight, which the server refuses, blocking simple-request CSRF.
- **No CORS allow headers; `OPTIONS` is never answered.**

### Outbound request scope (default-closed)
- The browser snippet is **not trusted** to scope requests; the server enforces
  scope before sending the command.
- **Relative URLs only by default** (resolved against the page origin).
  Absolute / cross-origin URLs are rejected unless explicitly enabled with
  `--allow-origin <url>`.
- The `serve --allow-origin` global feeds **two** checks at different times: the
  WebSocket **upgrade** gate (the *extension's* origin, at connection time) and
  the per-request outbound-URL check (the *target's* origin). To widen the
  outbound scope for a single request **without** disturbing the connected tab,
  pass `request --allow-origin <url>` instead: it reaches only the outbound-URL
  check, takes precedence over the global for that request, and never the WS
  gate. A WARN is logged at dispatch whenever the per-request override is used.
- **Security note:** a per-request override lets a session-token holder direct
  the tab's `fetch` at an arbitrary cross-origin URL. Blast radius is bounded by
  the browser's own CORS (the response body is readable only if the target sends
  permissive CORS), and a token holder can already issue authenticated
  same-origin requests. It is **per-request and explicit** — never a default.

### Resource & robustness limits
- Per-request timeout (`--request-timeout`, default 30s → `504`), max response
  body size (`--max-body-bytes`), and a concurrency cap (`--max-concurrent`).
- **Fail-closed binding**: ports bind without `SO_REUSEADDR`; if a fixed port is
  taken, the bridge exits rather than racing a squatter.
- Caller header names/values with CR/LF are rejected; the request path is
  normalized and traversal-checked before the `/__bridge/` routing split.

### Accepted risks / relied-upon controls
- `ws://localhost` is plaintext; loopback traffic is not sniffable without root —
  accepted.
- The browser forbids JS from setting `Cookie` or reading `Set-Cookie`, so the
  snippet cannot smuggle those — a relied-upon browser control.
- No request URLs, queries, headers, or bodies are logged at the default level
  (they contain authenticated/sensitive data).

## Flags

| Flag | Default | Purpose |
|------|---------|---------|
| `--ws-port <PORT>` | `9999` | WebSocket-plane port. `0` binds a random free port. |
| `--control-port <PORT>` | `9998` | HTTP control-plane port. `0` binds a random free port. |
| `--request-timeout <SECS>` | `30` | Per-request timeout before the control plane returns `504`. |
| `--allow-origin <URL>` || Permit this exact cross-origin for the WS upgrade and outbound URLs. |
| `--max-body-bytes <N>` | `8388608` | Maximum browser response body size accepted. |
| `--max-concurrent <N>` | `64` | Maximum concurrent in-flight requests. |
| `--token-file <PATH>` || Read the session token from this `0600` file instead of generating one. |

The `request` subcommand takes `--url`, `--method` (default `GET`),
`--header` (repeatable), `--body` (`@file` supported),
`--credentials <include|omit|same-origin>` (default `include`), `--stream`
(chunk the response to stdout), `--target` (route to a connection id / origin —
see [Routing to a specific tab](#routing-to-a-specific-tab)), `--allow-origin`
(permit a cross-origin outbound URL for this request only — see
[Outbound request scope](#outbound-request-scope-default-closed)),
`--control-port` (default `9998`), and `--token-file`.

## Random ports

Pass `0` to either port flag to let the OS assign a free port; the bridge reads
back the actual port and reflects it in the printed instructions and snippet:

```bash
omni-dev browser bridge serve --ws-port 0 --control-port 0
```

This is useful when the defaults are taken, or when running several bridges.
Point the thin client at the printed control port with
`--control-port <printed>`.

## Worked example: downloading Grafana / Loki logs

The bridge issues the same same-origin requests the Grafana UI makes, carrying
the `grafana_session` cookie; CSRF passes because the `fetch()` runs in-page.

```bash
T="$OMNI_BRIDGE_TOKEN"; H=(-H "Authorization: Bearer $T" -H "X-Omni-Bridge: 1")

# 1. Discover the Loki datasource UID (same-origin GET):
curl "${H[@]}" http://localhost:9998/api/datasources       # find {"type":"loki","uid":"…"}

# 2. Pull a time window of logs (GET → relative URL, page origin):
curl "${H[@]}" 'http://localhost:9998/api/datasources/proxy/uid/<DS_UID>/loki/api/v1/query_range\
?query={app="foo"}|="error"&start=<ns>&end=<ns>&limit=5000&direction=backward'
```

Pagination is client-side: query `direction=backward`, take the oldest returned
timestamp as the next `end`, repeat until empty. Keep each buffered page modest —
the bridge buffers full bodies under `--request-timeout`. For genuinely streaming
endpoints (Grafana Live, SSE, chunked APIs), opt into streaming with `--stream` /
`?__stream=1` (see [Streaming responses](#streaming-responses)) instead of
paginating. The `POST /api/ds/query` form is also supported via
`omni-dev browser bridge request --method POST`.

A canonical drain loop using the `request` thin client and `jq` (the bridge
prints the response envelope as JSON; the upstream body is the envelope's
`body`):

```bash
DS_UID=<loki-datasource-uid>
QUERY='{app="foo"}|="error"'
END=$(date +%s)000000000   # now, in nanoseconds
LIMIT=5000

while :; do
  page=$(omni-dev browser bridge request \
    --url "/api/datasources/proxy/uid/${DS_UID}/loki/api/v1/query_range?query=${QUERY}&end=${END}&limit=${LIMIT}&direction=backward")

  # The upstream JSON is the envelope's `body` string; parse it once.
  body=$(printf '%s' "$page" | jq -r '.body')

  # Stop when the page returns no log entries.
  rows=$(printf '%s' "$body" | jq '[.data.result[].values[]] | length')
  [ "$rows" -eq 0 ] && break

  printf '%s\n' "$body"   # collect / process this page

  # Oldest timestamp on this page (ns) becomes the next `end`, minus 1ns so we
  # don't re-fetch the boundary entry.
  oldest=$(printf '%s' "$body" | jq -r '[.data.result[].values[][0] | tonumber] | min')
  END=$((oldest - 1))
done
```

Each page must still fit under `--max-body-bytes`. If a single page trips the
over-limit error, it tells you exactly what to do: lower `--limit`/narrow the
window so each page is smaller, or raise `--max-body-bytes` to accept a larger
page.

## Harvesting your own data (best-effort)

The recipes above drive APIs you assemble by hand. For some sites the sequence
is involved enough — harvest CSRF tokens from HTML, discover a persisted-query
id from a lazily-loaded cross-origin bundle, then replay a paginating GraphQL
query — that it is packaged as a built-in command tree:

```
omni-dev browser bridge harvest <platform> <object>
```

Today the only target is your **own** Facebook timeline:

```bash
# Page your whole timeline to a file, newest-first, one JSON post per line:
omni-dev browser bridge harvest facebook posts --output my-posts.jsonl

# Sample the most recent 20 posts to stdout:
omni-dev browser bridge harvest facebook posts --limit 20

# Incremental archive: stop once posts predate a cutoff (Unix seconds or ISO-8601):
omni-dev browser bridge harvest facebook posts --since 2024-01-01T00:00:00Z

# Resume a run interrupted by a 504 / token rotation from its saved cursor:
omni-dev browser bridge harvest facebook posts --output my-posts.jsonl --resume run.state
```

Each post is `{ id, creation_time, text, url, shared_link }`. `--format jsonl`
(default) streams one post per line and is append-friendly on `--resume`;
`--format json` writes a single array when the run completes. `--target`,
`--token-file`, and `--control-port` mirror `bridge request`.

The command reuses the same `bridge request` dispatch path: it needs a running
`bridge serve` with a Facebook tab connected (verify with `GET /__bridge/status`),
and its cross-origin `doc_id`-discovery step requires the bridge to permit
`https://static.xx.fbcdn.net` (it sends that per-request via the same machinery
as `request --allow-origin … --credentials omit`). The full manual recipe this
encapsulates is documented separately (issue #922).

> **Best-effort contract.** This drives **reverse-engineered, undocumented**
> Facebook internals. It re-harvests every volatile value (GraphQL `doc_id`s,
> relay-provider flags, session tokens) on each run — nothing is hardcoded — and
> fails with a staged, actionable error naming the step that drifted rather than
> panicking. Even so, it **can break whenever Facebook changes** its query ids,
> page structure, or response shape. It only ever uses the session already
> logged into the connected tab (**your own account only**; mind Facebook's ToS).
> For a stable archive, prefer Facebook's official **"Download Your Information"**
> export.

## WebSocket wire protocol

Newline-free JSON frames, correlated by a monotonic integer `id`.

**Server → browser (command):**
```json
{"id": 7, "url": "/loki/api/v1/labels", "method": "GET", "headers": {}, "body": null}
```

**Browser → server (success, text):**
```json
{"id": 7, "status": 200, "headers": {"content-type": "application/json"}, "body": "..."}
```

**Browser → server (success, binary):**
```json
{"id": 7, "status": 200, "headers": {"content-type": "image/png"}, "body": "iVBORw0K…", "encoding": "base64"}
```

The snippet reads non-text bodies (anything whose `Content-Type` is not
`text/*`, JSON, XML, or JavaScript) via `arrayBuffer()` and base64-encodes them,
tagging the frame with `"encoding": "base64"`. Text bodies omit `encoding`
(back-compat). `--max-body-bytes` is enforced against the **decoded** size.

**Browser → server (error):**
```json
{"id": 7, "error": "Failed to fetch"}
```

### Streaming responses

When a request opts in (`stream: true` on `POST /__bridge/request`, `--stream` on
the thin client, or `?__stream=1` on the transparent proxy), the server sends
`{… "stream": true}` in the command and the snippet reads
`response.body.getReader()`, emitting **a head frame, then one base64 chunk frame
per read, then a terminator** instead of a single buffered reply:

```json
{"id": 7, "status": 200, "headers": {"content-type": "text/event-stream"}, "stream": true}
{"id": 7, "seq": 0, "chunk": "ZGF0YTogMQ=="}
{"id": 7, "seq": 1, "chunk": "ZGF0YTogMg=="}
{"id": 7, "done": true}
```

The server streams the body out as it arrives:

- **Transparent proxy** (`?__stream=1`) → the decoded bytes as a native chunked
  HTTP body (curl-friendly: `curl -N … '/events?__stream=1'`).
- **`POST /__bridge/request`** (`"stream": true`) → an NDJSON body
  (`application/x-ndjson`): a `{status,headers}` head line, `{seq,chunk}` lines,
  then a `{done:true}` line.

For a stream, **`--request-timeout` is an inter-chunk idle timeout** (reset on
each chunk, not a total deadline) and **`--max-body-bytes` is a cumulative
ceiling** across all chunks; either limit ending the stream sends the browser a
`{"id": 7, "cancel": true}` frame so it stops its reader. The browser also
receives a cancel when the control-plane consumer disconnects mid-stream.

**Server → browser (cancel):**
```json
{"id": 7, "cancel": true}
```

Rules:
- `id` is assigned by the server; the browser echoes it back unchanged.
- Multiple commands may be in flight concurrently over one socket (id-keyed).
- A command with no reply within `--request-timeout` resolves the control-plane
  request with `504 Gateway Timeout` and is dropped from `pending`.
- **Several authenticated** tabs may be connected at once, keyed by connection
  id; a request routes to one of them (see
  [Routing to a specific tab]#routing-to-a-specific-tab). A new connection
  never evicts an existing one, and an unauthenticated peer can neither connect
  nor evict any of them.

## Caveats

- A strict CSP `connect-src` (fallback `default-src`) can block the WebSocket.
- HTTPS page → `ws://localhost` is normally allowed (localhost is a
  "potentially trustworthy" origin); confirm per page.
- CORS constrains the in-page `fetch()`, not the bridge: same-origin (relative
  URLs) is unaffected; cross-origin targets depend on their
  `Access-Control-Allow-Origin`.
- The bridge works only while the tab is open and the snippet is running.
- Streaming reads the body ahead of the consumer: memory is bounded by
  `--max-body-bytes` (cumulative), but there is no socket-level backpressure to
  the browser's `getReader()` — a fast producer with a slow consumer is capped,
  not throttled.
- Remaining limitations: no stdin piping, and a request fans out to at most one
  tab. Multiple concurrent tabs (with routing), binary bodies, *and*
  streaming/chunked responses are all supported (see the wire protocol above).