Crate sea_streamer_kafka

sea-streamer-kafka: Kafka / Redpanda Backend

This is the Kafka / Redpanda backend implementation for SeaStreamer. This crate provides a comprehensive type system that makes working with Kafka easier and safer.

First of all, all API (many are sync) are properly wrapped as async. Methods are also marked &mut to eliminate possible race conditions.

KafkaConsumerOptions has typed parameters.

KafkaConsumer allows you to seek to point in time, rewind to particular offset, and commit message read.

KafkaProducer allows you to await a send Receipt or discard it if you are uninterested. You can also flush the Producer.

KafkaStreamer allows you to flush all producers on disconnect.

See tests for an illustration of the stream semantics.

This crate depends on rdkafka, which in turn depends on librdkafka-sys, which itself is a wrapper of librdkafka.

Configuration Reference: https://kafka.apache.org/documentation/#configuration

Modules

• export
  Re-export types from rdkafka

Structs

• ConsumerGroupMetadata
  Consumer group metadata.
• FutureRecord
  A record for the future producer.
• KafkaAsyncRuntime
• KafkaConnectOptions
• KafkaConsumer
• KafkaConsumerOptions
• KafkaMessage
• KafkaProducer
• KafkaProducerOptions
• KafkaStreamer
• SaslOptions
• SendFuture
• TopicPartitionList
  A structure to store and manipulate a list of topics and partitions with optional offsets.

Enums

• AutoOffsetReset
• CompressionType
• KafkaConsumerOptionKey
• KafkaErr
  Represents all possible Kafka errors.
• KafkaProducerOptionKey
• SaslMechanism
• SecurityProtocol

Constants

• DEFAULT_TIMEOUT
  The default timeout, if needed but unspecified
• KAFKA_PORT
  The default Kafka port number

Functions

• host_id
  Get the host id. Inside Docker, it’s the container ID. Otherwise it’s the MAC address. You can override it via the HOST_ID env var.

Type Aliases

• KafkaMessageStream
  Concrete type of Stream that will yield the next messages.
• KafkaResult
• NextFuture
  Concrete type of Future that will yield the next message.
• RawConsumer
  rdkafka’s StreamConsumer type
• RawProducer
  rdkafka’s FutureProducer type