faucet-source-nats 1.0.0

NATS source connector for the faucet-stream ecosystem
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67

# faucet-source-nats

A [NATS](https://nats.io) source for [`faucet-stream`](https://crates.io/crates/faucet-stream).

Subscribes to a subject (core NATS, with `*`/`>` wildcards and optional queue
groups) or pulls from a durable JetStream consumer, drains until `max_messages`
or `idle_timeout_secs` fires, and yields each message payload as a JSON record —
valid JSON passes through, anything else becomes a JSON string.

Core NATS is fire-and-forget at-least-once, so runs carry **no bookmark** and
are not resumable/exactly-once. In JetStream mode each page's messages are
**acked after the page is written**, giving at-least-once delivery.

## Configuration

The shared connection surface (`servers` / `auth` / `tls` / `name` — see
[`faucet-common-nats`](https://crates.io/crates/faucet-common-nats)) is flattened
in alongside these fields:

| field               | type             | default | description                                                        |
|---------------------|------------------|---------|--------------------------------------------------------------------|
| `subject`           | `String`         | —       | subject to subscribe to (`*`/`>` wildcards). Required.             |
| `queue_group`       | `Option<String>` | —       | core-NATS queue group for load-balanced subscriptions.             |
| `jetstream_stream`  | `Option<String>` | —       | JetStream stream name (enables JetStream mode).                    |
| `jetstream_consumer`| `Option<String>` | —       | durable pull-consumer name; required with `jetstream_stream`.      |
| `max_messages`      | `Option<usize>`  | —       | stop after this many messages.                                     |
| `idle_timeout_secs` | `Option<u64>`    | —       | stop after this many seconds with no new message.                  |
| `batch_size`        | `usize`          | `1000`  | records per emitted page (`0` = one page for the whole run window).|

At least one of `max_messages` / `idle_timeout_secs` must be set so the run
terminates.

## Example (core NATS)

```yaml
version: 1
pipeline:
  source:
    kind: nats
    config:
      servers: ["nats://127.0.0.1:4222"]
      subject: "events.>"
      idle_timeout_secs: 5
      batch_size: 500
  sink:
    kind: stdout
    config: {}
```

## Example (JetStream durable consumer)

```yaml
pipeline:
  source:
    kind: nats
    config:
      servers: ["nats://127.0.0.1:4222"]
      subject: "orders.>"
      jetstream_stream: ORDERS
      jetstream_consumer: faucet-worker
      max_messages: 10000
      idle_timeout_secs: 10
```

## License

Licensed under either of Apache-2.0 or MIT at your option.