nerve-ipc 0.2.0

Binary framing protocol for local IPC over Unix Domain Sockets
Documentation
# NERVE Protocol

> Version: V1  
> Status: Stable (V1 Message Contract Locked)  
> Scope: Binary IPC protocol — browser extension ↔ local AI daemon

## Overview

NERVE is a binary, low-latency protocol that connects the Anvesha browser extension to the local AI daemon over WebSocket (carrying binary frames unchanged).

The protocol prioritizes:

- Deterministic latency
- Minimal allocations
- Streaming-first semantics
- Immediate cancellation
- Safe handling of untrusted browser input

NERVE is not a general-purpose RPC protocol.

## Architecture

```
Browser Extension
      │
      │  NERVE binary frames over WebSocket
      ▼
Local AI Daemon
      │
      │  HTTPS
      ▼
Hosted Anvesha Search API
```

## Design Principles

- Binary-only framing (20-byte fixed header, little-endian everywhere)
- Single-pass parsing
- Validate header metadata **before** allocating payload buffer
- JSON payloads in V1 for debuggability and JavaScript interoperability
- No second streaming protocol inside payloads — NERVE flags carry stream semantics

## Transport

- V1 transport: WebSocket (binary frames)
- The NERVE frame format is identical regardless of transport
- Future transports (Unix Domain Sockets, shared memory) carry the same frames

## Frame Format (Wire Contract)

All messages are transmitted as frames.

### Frame Layout

```
+----------------+----------------+
| magic (u32 LE) | version (u16)  |
+--------+-------+----------------+
| type   | flags |
| (u8)   | (u8)  |
+--------+-------+----------------+
| request_id (u64 LE)             |
+---------------------------------+
| payload_length (u32 LE)         |
+---------------------------------+
| payload (0 .. N bytes)          |
+---------------------------------+
```

### Header Fields

| Field            | Offset | Size     | Description                               |
|------------------|--------|----------|-------------------------------------------|
| `magic`          | 0      | 4 bytes  | Protocol identifier `0x4E455256` ("NERV") |
| `version`        | 4      | 2 bytes  | Protocol version (`1`)                    |
| `type`           | 6      | 1 byte   | Message type (see registry below)         |
| `flags`          | 7      | 1 byte   | `STREAM` = 0x01, `FINAL` = 0x02           |
| `request_id`     | 8      | 8 bytes  | Client-generated request identifier       |
| `payload_length` | 16     | 4 bytes  | Payload length in bytes                   |

- **Header size**: 20 bytes (fixed)
- **Endianness**: little-endian throughout
- **No padding**

### Validation Order

Implementations MUST validate these header fields **before allocating the payload buffer**:

1. `magic == 0x4E455256`
2. `version == 1`
3. `type` is a known `MessageType`
4. `payload_length <= MAX_PAYLOAD_SIZE` (1 MiB)

Violation of any check → close connection immediately.

## Constants & Limits

| Constant                | Value     |
|-------------------------|-----------|
| `MAGIC`                 | 0x4E455256 |
| `PROTOCOL_VERSION`      | 1         |
| `HEADER_SIZE`           | 20 bytes  |
| `MAX_PAYLOAD_SIZE`      | 1 MiB     |
| `MAX_INFLIGHT_REQUESTS` | 1024      |
| `MAX_RESULTS`           | 100       |

## Message Type Registry

| Type  | Rust name        | Direction                      | Payload  |
|-------|-----------------|-------------------------------|----------|
| 0x01  | `Ping`           | bidirectional                  | empty    |
| 0x02  | `SearchQuery`    | browser extension → AI daemon  | JSON     |
| 0x03  | `SearchResult`   | AI daemon → browser extension  | JSON     |
| 0x04  | `AiToken`        | AI daemon → browser extension  | JSON     |
| 0x05  | `Cancel`         | browser extension → AI daemon  | empty    |
| 0x10  | `AgentTaskStart` | reserved (agentic scaffolding) | TBD      |
| 0x11  | `AgentTaskEvent` | reserved (agentic scaffolding) | TBD      |
| 0x12  | `AgentTaskDone`  | reserved (agentic scaffolding) | TBD      |

Types 0x10–0x12 are registered and decodable but their V1 payload contract is not yet defined.

## V1 Payload Schemas

All V1 payloads are UTF-8 JSON.

### Ping (0x01)

Empty payload. Used for latency measurement and connection health checks.

**Request (bidirectional):**
```
payload: (empty)
flags: FINAL
```

The receiver echoes back with an identical frame to the same `request_id`.

---

### SearchQuery (0x02)

**Direction**: browser extension → AI daemon

**JSON schema:**
```json
{
  "v": 1,
  "query": "rust async io",
  "context": {
    "url": "https://example.com/page",
    "title": "Example Page",
    "selection": "optional highlighted text",
    "extract": "optional bounded page excerpt"
  },
  "opts": {
    "search": true,
    "max_results": 10
  }
}
```

**Field constraints:**
- `v` — must be `1`; reject any other value
- `query` — non-empty string
- `context.url` — required
- `context.title` — required
- `context.selection` — optional (omit key if absent)
- `context.extract` — optional (omit key if absent); target < 4 KiB; browser-side extraction is responsible for bounding this
- `opts.search` — whether to perform a web search
- `opts.max_results` — integer in `1..=100`

**Frame:**
```
flags: FINAL
```

A single `SearchQuery` MAY produce zero or more `SearchResult` frames followed by `AiToken` frames.

---

### SearchResult (0x03)

**Direction**: AI daemon → browser extension

**JSON schema:**
```json
{
  "results": [
    {
      "url": "https://result.example.com",
      "title": "Result Title",
      "snippet": "A short excerpt from the page.",
      "score": 0.92
    }
  ],
  "took_ms": 34
}
```

**Frame:**
```
request_id: (same as originating SearchQuery)
flags: FINAL
```

The `request_id` in the frame header links this response to the browser's `SearchQuery`. The browser MUST use `request_id` for correlation — do not use sequence numbers.

`results` may be an empty array if no results were found.

---

### AiToken (0x04)

**Direction**: AI daemon → browser extension

**JSON schema:**
```json
{ "t": "Hello" }
```

Tokens are streamed using NERVE's STREAM and FINAL flags:

```
AiToken + STREAM     ← more tokens coming
AiToken + STREAM
AiToken + FINAL      ← last token for this request_id
```

**Frame:**
```
request_id: (same as originating SearchQuery)
flags: STREAM (intermediate) or FINAL (last)
```

- `t` is a UTF-8 string (may be empty on the FINAL sentinel frame)
- Do NOT implement a second streaming protocol inside the payload
- Tokens MUST arrive in order per `request_id`

---

### Cancel (0x05)

**Direction**: browser extension → AI daemon

**Payload**: empty

**Frame:**
```
request_id: (target request to cancel)
flags: FINAL
payload: (empty)
```

The `request_id` in the frame header identifies the request to cancel. The daemon MUST NOT look for a request ID inside the payload.

Cancellation is best-effort:
- Daemon MUST stop work as soon as possible
- No acknowledgment is required
- Frames already in flight may still arrive after Cancel

---

## Request ID Semantics

- `request_id` is a `u64` generated by the browser extension
- It MUST be unique per connection
- The daemon MUST NOT reuse request IDs
- Daemons MUST ignore unknown `request_id` values
- `Cancel` after completion is a no-op

## STREAM / FINAL Flags

| Flags           | Meaning                                  |
|-----------------|------------------------------------------|
| `STREAM` (0x01) | More frames follow for this `request_id` |
| `FINAL` (0x02)  | Last frame for this `request_id`         |
| both set        | Undefined; treat as FINAL                |
| neither set     | Single complete message                  |

## Cancellation Semantics

1. Browser sends `Cancel` frame with the target `request_id`
2. Daemon stops inference / search as soon as possible
3. No acknowledgment frame is sent
4. Frames already written to the wire may still arrive
5. `Cancel` after `FINAL` is silently ignored

## Versioning Rules

- The `version` field in the frame header is `1` for all V1 frames
- The `v` field inside `SearchQuery` JSON is also `1`
- Implementations MUST reject `v != 1` with a clean error (not silently treat as V1)
- V0.x allowed breaking changes; V1 is the first stable contract

## Maximum Payload Considerations

- Hard limit: `MAX_PAYLOAD_SIZE = 1 MiB` per frame
- `SearchQuery.context.extract` should be a bounded excerpt (target < 4 KiB), not a raw DOM dump
- Browser-side content extraction is responsible for truncation before transmission
- The daemon must validate `payload_length <= MAX_PAYLOAD_SIZE` **before** reading any payload bytes (security requirement — prevents memory exhaustion from malicious headers)

## Error Handling

Protocol errors are local events — not transmitted over the wire.

| Error                  | Cause                                    | Action        |
|------------------------|------------------------------------------|---------------|
| `InvalidMagic`         | `magic` ≠ `0x4E455256`                   | Close connection |
| `UnsupportedVersion`   | `version` ≠ `1`                          | Close connection |
| `UnknownMessageType`   | `type` not in registry                   | Close connection |
| `PayloadTooLarge`      | `payload_length > MAX_PAYLOAD_SIZE`      | Close connection |
| `MalformedFrame`       | Incomplete header or truncated payload   | Close connection |
| `InternalError`        | Daemon-side bug                          | Log + close      |

Callers MUST log the error kind and close the connection. In V1, errors are not sent over the wire.

## Security Model

- Local daemon accepts connections from the local browser extension only
- WebSocket transport: `127.0.0.1` binding only (not `0.0.0.0`)
- No authentication in V1; OS-level socket permissions enforce access
- Header fields MUST be validated before payload allocation to prevent OOM from crafted frames

## JavaScript Client Notes

To implement a V1 NERVE client in JavaScript (browser extension):

1. Connect to `ws://127.0.0.1:<port>` with `binaryType = "arraybuffer"`
2. To send a `SearchQuery`:
   - Generate a unique `request_id` (u64 as BigInt or two u32s)
   - Serialize the JSON payload (see schema above)
   - Encode header: `MAGIC` (4 LE bytes) + `VERSION=1` (2 LE bytes) + `type=0x02` (1 byte) + `flags=0x02` (FINAL, 1 byte) + `request_id` (8 LE bytes) + `payload_length` (4 LE bytes)
   - Concatenate header + UTF-8 JSON bytes and send as binary message
3. Receive `SearchResult` and `AiToken` frames by parsing the 20-byte header then reading `payload_length` bytes
4. Use `request_id` to correlate responses with requests
5. Send `Cancel` (type=0x05, payload empty, flags=FINAL) with the target `request_id` to abort

Little-endian byte order: use `DataView.setUint32(offset, value, true)` and friends.

## Performance Targets

| Metric               | Target   |
|----------------------|----------|
| p99 frame RTT        | < 100 µs |
| Cancel latency       | < 1 ms   |
| First result latency | < 50 ms  |
| Allocation per frame | ≤ 1      |

## Future Work (Non-Binding)

- Error frames (v2+)
- AgentTask payload schema (0x10–0x12)
- Backpressure signaling
- Shared-memory transport
- Capability negotiation