Skip to main content

Module integration

Module integration 

Source
Expand description

Kafka integration for stream processing pipeline

This module provides integration between Kafka and the stream processing pipeline, enabling Kafka-to-Kafka streaming with automatic offset management, watermarking, and error recovery.

Structsยง

KafkaStreamConfig
Configuration for Kafka stream pipeline
KafkaStreamPipeline
End-to-end Kafka stream processing pipeline
KafkaStreamSink
Kafka stream sink that receives results from the pipeline
KafkaStreamSource
Kafka stream source that integrates with StreamExecutor
KafkaStreamStats
Statistics for Kafka stream processing
OffsetManager
Offset manager for tracking and committing Kafka offsets