bgpkit-parser 0.21.0

MRT/BGP/BMP data processing library
Documentation
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
# @bgpkit/parser — WebAssembly Bindings

> **Experimental**: The WASM bindings are experimental. The API surface, output
> format, and build process may change in future releases.

This module compiles bgpkit-parser's BGP/BMP/MRT parsing code to WebAssembly
for use in JavaScript and TypeScript environments.

## Install

```sh
npm install @bgpkit/parser
```

## Use Cases and Examples

### 1. Real-time BMP stream processing (Node.js)

Parse OpenBMP messages from the RouteViews Kafka stream. Each Kafka message
is a small binary frame — no memory concerns.

**Requires Node.js** — Kafka clients need raw TCP sockets, which are not
available in browsers or Cloudflare Workers.

```js
const { Kafka } = require('kafkajs');
const { parseOpenBmpMessage } = require('@bgpkit/parser');

const kafka = new Kafka({
  brokers: ['stream.routeviews.org:9092'],
});

const consumer = kafka.consumer({ groupId: 'my-app' });
await consumer.connect();
await consumer.subscribe({ topic: /^routeviews\.amsix\..+\.bmp_raw$/ });

await consumer.run({
  eachMessage: async ({ message }) => {
    const msg = parseOpenBmpMessage(message.value);
    if (!msg) return; // non-router frame (e.g. collector heartbeat)

    switch (msg.type) {
      case 'RouteMonitoring':
        for (const elem of msg.elems) {
          console.log(elem.type, elem.prefix, elem.as_path);
        }
        break;
      case 'PeerUpNotification':
        console.log(`Peer up: ${msg.peerHeader.peerIp} AS${msg.peerHeader.peerAsn}`);
        break;
      case 'PeerDownNotification':
        console.log(`Peer down: ${msg.peerHeader.peerIp} (${msg.reason})`);
        break;
    }
  },
});
```

If you have raw BMP messages without the OpenBMP wrapper (e.g. from your own
BMP collector), use `parseBmpMessage` instead:

```js
const { parseBmpMessage } = require('@bgpkit/parser');

const msg = parseBmpMessage(bmpBytes, Date.now() / 1000);
```

### 2. MRT updates file analysis (Node.js)

Parse MRT updates files from RouteViews or RIPE RIS archives. Updates files
are typically 5–50 MB compressed (20–200 MB decompressed) and fit comfortably
in memory.

Supports gzip (`.gz`, RIPE RIS) and bzip2 (`.bz2`, RouteViews) compression.
For bz2, install an optional dependency: `npm install seek-bzip`.

**Using `streamMrtFrom`** (handles fetch + decompression):

```js
const { streamMrtFrom } = require('@bgpkit/parser');

// RIPE RIS (gzip)
for await (const { elems } of streamMrtFrom('https://data.ris.ripe.net/rrc00/2025.01/updates.20250101.0000.gz')) {
  for (const elem of elems) {
    console.log(elem.type, elem.prefix, elem.as_path);
  }
}

// RouteViews (bzip2 — requires: npm install seek-bzip)
for await (const { elems } of streamMrtFrom('https://archive.routeviews.org/route-views.amsix/bgpdata/2025.01/UPDATES/updates.20250101.0000.bz2')) {
  for (const elem of elems) {
    console.log(elem.type, elem.prefix, elem.as_path);
  }
}
```

**Using `parseMrtRecords`** with manual I/O:

```js
const fs = require('fs');
const zlib = require('zlib');
const { parseMrtRecords } = require('@bgpkit/parser');

const raw = zlib.gunzipSync(fs.readFileSync('updates.20250101.0000.gz'));

for (const { elems } of parseMrtRecords(raw)) {
  for (const elem of elems) {
    if (elem.type === 'ANNOUNCE') {
      console.log(elem.prefix, elem.next_hop, elem.as_path);
    }
  }
}
```

### 3. MRT file analysis (browser)

Parse MRT files dropped or fetched in the browser. Uses the web entry point
which requires calling `init()` before any parsing.

**Live demo**: [mrt-explorer.labs.bgpkit.com](https://mrt-explorer.labs.bgpkit.com/)

```js
import { init, parseMrtRecords } from '@bgpkit/parser/web';

await init();

// Fetch and decompress a gzip-compressed MRT file
const res = await fetch('https://data.ris.ripe.net/rrc00/2025.01/updates.20250101.0000.gz');
const stream = res.body.pipeThrough(new DecompressionStream('gzip'));
const raw = new Uint8Array(await new Response(stream).arrayBuffer());

for (const { elems } of parseMrtRecords(raw)) {
  for (const elem of elems) {
    console.log(elem.type, elem.prefix, elem.as_path);
  }
}
```

### 4. Individual BGP UPDATE parsing (all platforms)

Parse a single BGP UPDATE message extracted from a pcap capture or received
via an API. The message must include the 16-byte marker, 2-byte length, and
1-byte type header.

```js
const { parseBgpUpdate } = require('@bgpkit/parser');

const elems = parseBgpUpdate(bgpMessageBytes);
for (const elem of elems) {
  console.log(elem.type, elem.prefix, elem.next_hop, elem.as_path);
}
```

### 5. RIS Live real-time stream (all platforms)

Subscribe to the [RIPE RIS Live](https://ris-live.ripe.net/manual/) WebSocket
feed and parse messages as they arrive. Two parsers are available:

- `parseRisLiveMessageJson(message)` — uses RIS Live's JSON-projected UPDATE
  fields. No subscription options required, but the projection exposes only a
  subset of BGP path attributes (`path`, `community`, `origin`, `med`,
  `aggregator`, `announcements`, `withdrawals`).
- `parseRisLiveMessageRaw(message)` — decodes the hex `data.raw` BGP wire
  message. Requires subscribing with `socketOptions.includeRaw = true`, and
  preserves **every** path attribute (large/extended communities, OTC, local
  pref, originator ID, cluster list, AIGP, raw-retained BGPSEC_PATH/ATTR_SET,
  ...) plus RFC 7606 validation warnings. `elems` use the same `BgpElem`
  shape as the JSON parser but are derived from the wire NLRI, so their
  grouping can differ from RIS's JSON projection (e.g. a multi-prefix
  MP_REACH yields one elem per prefix here).

```js
import { parseRisLiveMessageRaw } from '@bgpkit/parser';

const socket = new WebSocket('ws://ris-live.ripe.net/v1/ws/?client=bgpkit-parser');

socket.onopen = () => {
  socket.send(JSON.stringify({
    type: 'ris_subscribe',
    data: {
      host: 'rrc21', // one RRC collector, or omit for the full firehose
      socketOptions: { includeRaw: true },
    },
  }));
};

socket.onmessage = (event) => {
  try {
    const { meta, elems, attributes, validationWarnings } =
      parseRisLiveMessageRaw(event.data);
    for (const elem of elems) {
      console.log(meta.host, elem.type, elem.prefix, elem.as_path);
    }
    // Full-fidelity attribute list, one entry per UPDATE:
    // { value: { "<Variant>": payload }, flag: "OPTIONAL | TRANSITIVE" }
    for (const attr of attributes) {
      if ('OnlyToCustomer' in attr.value) {
        console.log('OTC:', attr.value.OnlyToCustomer);
      }
    }
    if (validationWarnings.length > 0) {
      console.warn('RFC 7606 warnings:', validationWarnings);
    }
  } catch {
    // non-UPDATE or control messages — ignore
  }
};
```

Non-UPDATE messages (`KEEPALIVE`, `OPEN`, `NOTIFICATION`, `RIS_PEER_STATE`)
return an empty `elems` array from `parseRisLiveMessageJson`; the raw parser
returns empty `elems`/`attributes` for them. RIS Live control messages
(`ris_error`, `ris_rrc_list`, `pong`) throw from both parsers — wrap calls in
`try`/`catch` and skip.

Note: `includeRaw` roughly doubles message size (hex-encoded wire bytes on
top of the always-present JSON fields); use the JSON parser if you only need
the projected fields and want minimal bandwidth.

## Memory Considerations

MRT parsing requires the **entire decompressed file** in memory as a
`Uint8Array` before parsing begins. `parseMrtRecords` then iterates
record-by-record, so parsed output stays small — but the raw bytes remain in
memory throughout.

| File type | Typical decompressed size | Practical? |
|---|---|---|
| MRT updates (5-min) | 20–200 MB | Yes, all platforms |
| MRT updates (15-min) | 50–500 MB | Yes, Node.js; may exceed browser/Worker limits |
| Full RIB dump | 500 MB – 2+ GB | Not recommended — use the native Rust crate |

BMP and BGP UPDATE messages are small (KB-sized) and have no memory concerns.

## API Reference

### Core parsing functions (all platforms)

| Function | Input | Output | Use case |
|---|---|---|---|
| `parseOpenBmpMessage(data)` | `Uint8Array` | `BmpParsedMessage \| null` | Real-time BMP streams |
| `parseBmpMessage(data, timestamp)` | `Uint8Array`, `number` | `BmpParsedMessage` | Real-time BMP streams |
| `parseBgpUpdate(data)` | `Uint8Array` | `BgpElem[]` | Individual BGP messages |
| `parseBgpUpdateFull(data)` | `Uint8Array` | `BgpUpdateFull` | Individual BGP messages (full attribute fidelity) |
| `dissectBgpMessage(data, fourByteAsn?)` | `Uint8Array`, `boolean` | `DissectionNode` | Wireshark-style field tree with byte spans |
| `parseRisLiveMessageJson(message)` | `string` | `BgpElem[]` | RIS Live stream (JSON projection) |
| `parseRisLiveMessageRaw(message)` | `string` | `RisLiveRawFull` | RIS Live stream (full attribute fidelity) |
| `parseMrtRecords(data)` | `Uint8Array` | `Generator<MrtRecordResult>` | MRT file analysis |
| `parseMrtRecord(data)` | `Uint8Array` | `MrtRecordResult \| null` | MRT file analysis (low-level) |
| `dissectMrtRecord(data)` | `Uint8Array` | `DissectMrtResult \| null` | MRT field tree with byte spans (header, BGP4MP subheader, embedded BGP message) |
| `resetMrtParser()` || `void` | Clear state between MRT files |

### RIS Live result type

`parseRisLiveMessageRaw` returns:

```ts
interface RisLiveRawFull {
  meta: RisLiveMeta;                  // host, id, peer, peerAsn, timestamp
  elems: BgpElem[];                   // same BgpElem shape as the JSON parser
  attributes: Attribute[];            // full attribute list of the UPDATE
  validationWarnings: BgpValidationWarning[]; // RFC 7606 parse findings
}
```

Each `Attribute` is `{ value: { "<VariantName>": payload }, flag }` (serde
external tagging) for data-carrying variants, e.g. `{ "Origin": "IGP" }` or
`{ "LargeCommunities": [...] }`; unit variants serialize as a bare string,
e.g. `{ "value": "AtomicAggregate", "flag": "..." }`. See `index.d.ts` and
`generated/` in the package for the full typed surface.

### Dissection types

`dissectBgpMessage` returns a `DissectionNode` tree; `dissectMrtRecord`
returns `{ tree, bytesRead }` where the tree's byte offsets cover the whole
record (common header, BGP4MP subheader, embedded BGP message — all in one
coordinate space):

```ts
interface DissectionNode {
  field: string;    // stable dotted path, e.g. "bgp.attr.32" (LARGE_COMMUNITY)
  label: string;    // human-readable rendering, e.g. "AS_SEQUENCE: 65001 65002"
  offset: number;   // byte offset into the dissected input
  length: number;   // byte length of this field
  children: DissectionNode[];
}
```

Attribute values are dissected one level deep (AS_PATH segments, community
entries of all three families, MP_REACH/MP_UNREACH structure, AIGP TLVs,
fixed u32 fields). Dissection is best effort: truncated or malformed input
yields a partial tree showing how far the structure could be walked — the
basis for a hex-editor UI where editing a byte immediately shows where
parsing breaks.

### Node.js I/O helpers

| Function | Input | Output | Description |
|---|---|---|---|
| `streamMrtFrom(pathOrUrl)` | `string` | `AsyncGenerator<MrtRecordResult>` | Fetch + decompress + stream-parse |
| `openMrt(pathOrUrl)` | `string` | `Promise<Buffer>` | Fetch + decompress only |

These use Node.js `fs`, `http`, `https`, and `zlib` modules. They are **not
available** in bundler, browser, or Cloudflare Worker environments.

### BMP message types

BMP parsing functions return a discriminated union on the `type` field. All
types include `timestamp` and `openBmpHeader` (present only via
`parseOpenBmpMessage`).

| `type` | Additional fields |
|---|---|
| `RouteMonitoring` | `peerHeader`, `elems` (array of `BgpElem`) |
| `PeerUpNotification` | `peerHeader`, `localIp`, `localPort`, `remotePort` |
| `PeerDownNotification` | `peerHeader`, `reason` |
| `InitiationMessage` | `tlvs` (array of `{type, value}`) |
| `TerminationMessage` | `tlvs` (array of `{type, value}`) |
| `StatisticsReport` | `peerHeader` |
| `RouteMirroringMessage` | `peerHeader` |

## Platform Support

| Platform | Import | Parsing | I/O helpers | Kafka |
|---|---|---|---|---|
| Node.js (CJS) | `require('@bgpkit/parser')` | Yes | Yes | Yes (via kafkajs) |
| Node.js (ESM) | `import from '@bgpkit/parser'` | Yes | No | Yes (via kafkajs) |
| Bundler (webpack, vite) | `import from '@bgpkit/parser'` | Yes | No | No |
| Browser | `import from '@bgpkit/parser/web'` | Yes (after `init()`) | No | No |
| Cloudflare Workers | `import from '@bgpkit/parser/web'` | Yes (after `init()`) | No | No (no TCP sockets) |

### Web target

The web target requires calling `init()` before any parsing functions:

```js
import { init, parseOpenBmpMessage } from '@bgpkit/parser/web';

await init();
const msg = parseOpenBmpMessage(data);
```

You can pass a custom URL to the `.wasm` file if the default path doesn't work:

```js
await init(new URL('./bgpkit_parser_bg.wasm', import.meta.url));
```

### Cloudflare Workers

Use the `@bgpkit/parser/web` entry point. Workers cannot connect to Kafka
(no raw TCP sockets), so BMP/BGP data must arrive via HTTP requests.

Workers have a 128 MB memory limit on the free plan (up to 256 MB on paid),
which is sufficient for MRT updates files and individual message parsing, but
not for full RIB dumps.

## Versioning

The npm package version tracks the Rust crate's minor version. For Rust crate
version `0.X.Y`, the npm package is published as `0.X.Z` where `Z` increments
independently for JS-specific changes.

## Building from Source

### Prerequisites

- [Rust]https://rustup.rs/ (stable toolchain)
- [`wasm-pack`]https://rustwasm.github.io/wasm-pack/installer/:
  `cargo install wasm-pack`
- The `wasm32-unknown-unknown` target:
  `rustup target add wasm32-unknown-unknown`

### Build

```sh
# From the repository root — builds all targets (nodejs, bundler, web)
bash src/wasm/build.sh

# Output is in pkg/
cd pkg && npm publish
```

To build a single target:

```sh
wasm-pack build --target nodejs --no-default-features --features wasm
```

## TypeScript Types

Type definitions live in `src/wasm/js/index.d.ts`. The BGP attribute surface
(`Attribute`, `AttributeValue`, community types, `Nlri`,
`BgpValidationWarning`, ...) is **generated** from the Rust models with
[ts-rs](https://github.com/Aleph-Alpha/ts-rs) into `src/wasm/js/generated/`;
`BgpElem`, `AsPath`, and `MetaCommunity` are hand-written (their Rust types
have custom serde impls).

After changing any Rust model that affects the WASM JSON output, regenerate
and commit both the bindings and the golden fixtures:

```sh
TS_RS_EXPORT_DIR=src/wasm/js/generated cargo test --features ts-rs,rislive
```

CI (`wasm-types` job) regenerates both, fails on any diff (drift gate), and
type-checks the fixtures against the shipped `.d.ts`
(`src/wasm/test/type-check/`, run `npm install && npm run check` locally).