Expand description
§sea-streamer-kafka: Kafka / Redpanda Backend
This is the Kafka / Redpanda backend implementation for SeaStreamer. This crate provides a comprehensive type system that makes working with Kafka easier and safer.
First of all, all API (many are sync) are properly wrapped as async. Methods are also marked &mut to eliminate possible race conditions.
KafkaConsumerOptions has typed parameters.
KafkaConsumer allows you to seek to point in time, rewind to particular offset, and commit message read.
KafkaProducer allows you to await a send Receipt or discard it if you are uninterested. You can also flush the Producer.
KafkaStreamer allows you to flush all producers on disconnect.
See tests for an illustration of the stream semantics.
This crate depends on rdkafka,
which in turn depends on librdkafka-sys, which itself is a wrapper of
librdkafka.
Configuration Reference: https://kafka.apache.org/documentation/#configuration
Modules§
- export
- Re-export types from
rdkafka
Structs§
- Consumer
Group Metadata - Consumer group metadata.
- Future
Record - A record for the future producer.
- Kafka
Async Runtime - Kafka
Connect Options - Kafka
Consumer - Kafka
Consumer Options - Kafka
Message - Kafka
Producer - Kafka
Producer Options - Kafka
Streamer - Sasl
Options - Send
Future - Topic
Partition List - A structure to store and manipulate a list of topics and partitions with optional offsets.
Enums§
- Auto
Offset Reset - Compression
Type - Kafka
Consumer Option Key - Kafka
Err - Represents all possible Kafka errors.
- Kafka
Producer Option Key - Sasl
Mechanism - Security
Protocol
Constants§
- DEFAULT_
TIMEOUT - The default timeout, if needed but unspecified
- KAFKA_
PORT - The default Kafka port number
Functions§
- host_id
- Get the host id. Inside Docker, it’s the container ID. Otherwise it’s the MAC address.
You can override it via the
HOST_IDenv var.
Type Aliases§
- Kafka
Message Stream - Concrete type of Stream that will yield the next messages.
- Kafka
Result - Next
Future - Concrete type of Future that will yield the next message.
- RawConsumer
- rdkafka’s StreamConsumer type
- RawProducer
- rdkafka’s FutureProducer type