# datum-mq
`datum-mq` is Datum's native Kafka satellite crate. It provides `KafkaSource`
and `KafkaSink` with at-least-once source offset commits and producer delivery
confirmation. The default feature set is empty; consumer and producer traffic
always uses the Tokio-native client.
The crate uses `#![forbid(unsafe_code)]`. Its rustls TLS path uses ring (Rust
plus assembly), so it does not compile AWS-LC or a CMake-built TLS library.
Zstd codec support still uses `zstd-sys`; removing the Kafka FFI client does not
make the crate fully C-free.
The client supports plaintext, rustls TLS, SASL PLAIN, SCRAM-SHA-256, and
SCRAM-SHA-512. It handles uncompressed, gzip, Snappy, LZ4, and Zstd Kafka
record-batch v2 payloads, so the native-only client loses no compression codec.
SASL GSSAPI/OAUTHBEARER and Kafka transactions/consume-transform-produce EOS
are not supported.
Native production uses Java-compatible murmur2 key partitioning plus sticky
round-robin batches for keyless records, and defaults to `acks=all` plus
`enable.idempotence=true`. Producer id/epoch and per-partition sequences let the
broker deduplicate retried batches. Users can explicitly opt out with
`enable.idempotence=false`. Native Produce requests default to one in flight per
broker connection; larger bounds are opt-in through
`KafkaProducerSettings::with_max_in_flight_requests_per_connection`. Requests
for one topic-partition never overlap, and retry recovery preserves partition
order.
```sh
cargo add datum-mq
```
The source guarantee is at-least-once: commit consumed offsets only after
downstream work has completed, and acknowledge produced records only after
Kafka delivery confirmation.