Struct rdkafka::consumer::stream_consumer::StreamConsumer

#[must_use = "Consumer polling thread will stop immediately if unused"]pub struct StreamConsumer<C = DefaultConsumerContext, R = DefaultRuntime> where
    C: ConsumerContext,  { /* fields omitted */ }

A high-level consumer with a Stream interface.

This consumer doesn’t need to be polled explicitly. Extracting an item from the stream returned by the stream will implicitly poll the underlying Kafka consumer.

If you activate the consumer group protocol by calling subscribe, the stream consumer will integrate with librdkafka’s liveness detection as described in KIP-62. You must be sure that you attempt to extract a message from the stream consumer at least every max.poll.interval.ms milliseconds, or librdkafka will assume that the processing thread is wedged and leave the consumer groups.

Implementations

impl<C, R> StreamConsumer<C, R> where
    C: ConsumerContext + 'static, 

pub fn stream(&self) -> MessageStream<'_, C, R>

Constructs a stream that yields messages from this consumer.

It is legal to have multiple live message streams for the same consumer, and to move those message streams across threads. Note, however, that the message streams share the same underlying state. A message received by the consumer will be delivered to only one of the live message streams. If you seek the underlying consumer, all message streams created from the consumer will begin to draw messages from the new position of the consumer.

If you want multiple independent views of a Kafka topic, create multiple consumers, not multiple message streams.

pub fn start(&self) -> MessageStream<'_, C, R>

👎 Deprecated:

use the more clearly named “StreamConsumer::stream” method instead

Constructs a stream that yields messages from this consumer.

pub async fn recv(&self) -> Result<BorrowedMessage<'_>, KafkaError>

Receives the next message from the stream.

This method will block until the next message is available or an error occurs. It is legal to call recv from multiple threads simultaneously.

Note that this method is exactly as efficient as constructing a single-use message stream and extracting one message from it:

use futures::stream::StreamExt;

consumer.stream().next().await.expect("MessageStream never returns None");

Trait Implementations

impl<C, R> Consumer<StreamConsumerContext<C>> for StreamConsumer<C, R> where
    C: ConsumerContext, 

fn client(&self) -> &Client<StreamConsumerContext<C>>

Returns the Client underlying this consumer.

fn group_metadata(&self) -> Option<ConsumerGroupMetadata>

Returns the current consumer group metadata associated with the consumer.

fn subscribe(&self, topics: &[&str]) -> KafkaResult<()>

Subscribes the consumer to a list of topics.

fn unsubscribe(&self)

Unsubscribes the current subscription list.

fn assign(&self, assignment: &TopicPartitionList) -> KafkaResult<()>

Manually assigns topics and partitions to the consumer. If used, automatic consumer rebalance won’t be activated.

fn seek<T: Into<Timeout>>(
    &self,
    topic: &str,
    partition: i32,
    offset: Offset,
    timeout: T
) -> KafkaResult<()>

Seeks to offset for the specified topic and partition. After a successful call to seek, the next poll of the consumer will return the message with offset.

fn commit(
    &self,
    topic_partition_list: &TopicPartitionList,
    mode: CommitMode
) -> KafkaResult<()>

Commits the offset of the specified message. The commit can be sync (blocking), or async. Notice that when a specific offset is committed, all the previous offsets are considered committed as well. Use this method only if you are processing messages in order.

fn commit_consumer_state(&self, mode: CommitMode) -> KafkaResult<()>

Commits the current consumer state. Notice that if the consumer fails after a message has been received, but before the message has been processed by the user code, this might lead to data loss. Check the “at-least-once delivery” section in the readme for more information.

fn commit_message(
    &self,
    message: &BorrowedMessage<'_>,
    mode: CommitMode
) -> KafkaResult<()>

Commit the provided message. Note that this will also automatically commit every message with lower offset within the same partition.

fn store_offset(&self, message: &BorrowedMessage<'_>) -> KafkaResult<()>

Stores offset for this message to be used on the next (auto)commit. When using this enable.auto.offset.store should be set to false in the config.

fn store_offsets(&self, tpl: &TopicPartitionList) -> KafkaResult<()>

Store offsets to be used on the next (auto)commit. When using this enable.auto.offset.store should be set to false in the config.

fn subscription(&self) -> KafkaResult<TopicPartitionList>

Returns the current topic subscription.

fn assignment(&self) -> KafkaResult<TopicPartitionList>

Returns the current partition assignment.

fn committed<T>(&self, timeout: T) -> KafkaResult<TopicPartitionList> where
    T: Into<Timeout>,
    Self: Sized, 

Retrieves the committed offsets for topics and partitions.

fn committed_offsets<T>(
    &self,
    tpl: TopicPartitionList,
    timeout: T
) -> KafkaResult<TopicPartitionList> where
    T: Into<Timeout>, 

Retrieves the committed offsets for specified topics and partitions.

fn offsets_for_timestamp<T>(
    &self,
    timestamp: i64,
    timeout: T
) -> KafkaResult<TopicPartitionList> where
    T: Into<Timeout>,
    Self: Sized, 

Looks up the offsets for this consumer’s partitions by timestamp.

fn offsets_for_times<T>(
    &self,
    timestamps: TopicPartitionList,
    timeout: T
) -> KafkaResult<TopicPartitionList> where
    T: Into<Timeout>,
    Self: Sized, 

Looks up the offsets for the specified partitions by timestamp.

fn position(&self) -> KafkaResult<TopicPartitionList>

Retrieve current positions (offsets) for topics and partitions.

fn fetch_metadata<T>(
    &self,
    topic: Option<&str>,
    timeout: T
) -> KafkaResult<Metadata> where
    T: Into<Timeout>,
    Self: Sized, 

Returns the metadata information for the specified topic, or for all topics in the cluster if no topic is specified.

fn fetch_watermarks<T>(
    &self,
    topic: &str,
    partition: i32,
    timeout: T
) -> KafkaResult<(i64, i64)> where
    T: Into<Timeout>,
    Self: Sized, 

Returns the low and high watermarks for a specific topic and partition.

fn fetch_group_list<T>(
    &self,
    group: Option<&str>,
    timeout: T
) -> KafkaResult<GroupList> where
    T: Into<Timeout>,
    Self: Sized, 

Returns the group membership information for the given group. If no group is specified, all groups will be returned.

fn pause(&self, partitions: &TopicPartitionList) -> KafkaResult<()>

Pauses consumption for the provided list of partitions.

fn resume(&self, partitions: &TopicPartitionList) -> KafkaResult<()>

Resumes consumption for the provided list of partitions.

fn context(&self) -> &Arc<C>

Returns a reference to the ConsumerContext used to create this consumer.

impl<R> FromClientConfig for StreamConsumer<DefaultConsumerContext, R> where
    R: AsyncRuntime, 

fn from_config(config: &ClientConfig) -> KafkaResult<Self>

Creates a client from a client configuration. The default client context will be used.

impl<C, R> FromClientConfigAndContext<C> for StreamConsumer<C, R> where
    C: ConsumerContext + 'static,
    R: AsyncRuntime, 

Creates a new StreamConsumer starting from a ClientConfig.

fn from_config_and_context(
    config: &ClientConfig,
    context: C
) -> KafkaResult<Self>

Creates a client from a client configuration and a client context.