datum-mq 0.9.2

Kafka sources and sinks for Datum streams, backed by rdkafka
Documentation
1
2
3
4
5
6
7
8
9
10
11
12
13
# datum-mq

`datum-mq` is the Kafka satellite crate for Datum. It provides `KafkaSource`
and `KafkaSink` on top of `rdkafka`/`librdkafka`, with at-least-once source
offset commits and producer delivery confirmation.

The crate itself uses `#![forbid(unsafe_code)]`, but `rdkafka` is an FFI wrapper
around `librdkafka` and the native client owns its own polling threads and
queues. This connector is intentionally not marketed as a pure Tokio carrier.

Transactions and Kafka EOS are deferred. The first stable guarantee is
at-least-once: commit consumed offsets only after downstream work has completed,
and acknowledge produced records only after Kafka delivery confirmation.