Module source

Module source 

Source
Expand description

Kafka source consumer implementation

This module provides a robust Kafka consumer implementation for ingesting events from Kafka topics with support for:

  • Async message consumption with tokio
  • Automatic reconnection on errors
  • Configurable commit strategies (auto, manual, periodic)
  • Multiple partition support
  • Graceful shutdown
  • Message deserialization with error handling
  • Offset management integration with state backend
  • Comprehensive metrics tracking

Structs§

KafkaMessage
Message envelope with metadata
KafkaSource
Kafka source consumer for ingesting events
KafkaSourceConfig
Configuration for Kafka source consumer
KafkaSourceMetrics
Metrics tracked by the Kafka source

Enums§

CommitStrategy
Commit strategy for offset management