Module kafka_config 

Shared Kafka librdkafka configuration profiles, merge helper, and file loader.

This module is always available (no feature gate). The core profile constants and merge helper have zero external dependencies. File loading supports .properties without any feature gate; YAML and JSON require the directory-config and config features respectively.

Loading from Config Git Directory

Services store librdkafka settings in their config git directory and load them with config_from_file:

use hyperi_rustlib::kafka_config::{config_from_file, merge_with_overrides, CONSUMER_PRODUCTION};

let overrides = config_from_file("/config/kafka.properties")?;
let rdkafka_config = merge_with_overrides(CONSUMER_PRODUCTION, &overrides);

Structs

DfeSource

DFE source-aware topic naming for transform services.

Enums

KafkaConfigError

Error loading librdkafka configuration from a file.

ServiceRole

DFE service role — determines consumer group naming convention.

Constants

CONSUMER_DEVTEST

Development/test consumer baseline — fast iteration, low memory.

CONSUMER_LOW_LATENCY

Low-latency consumer — minimal fetch delay.

CONSUMER_PRODUCTION

Production consumer baseline — lean, only non-defaults.

PRODUCER_DEVTEST

DevTest producer — fast acks, no compression.

PRODUCER_EXACTLY_ONCE

Exactly-once producer — idempotence + ordering.

PRODUCER_LOW_LATENCY

Low-latency producer — minimal delay, leader-ack only.

PRODUCER_PRODUCTION

Production producer baseline — high throughput, zstd compression.

TOPIC_SUFFIX_LAND

Default topic suffix for landing zone (raw ingest).

TOPIC_SUFFIX_LOAD

Default topic suffix for load-ready data (post-transform).

Functions

config_from_file

Load librdkafka configuration from a file in the config git directory.

config_from_properties_str

Parse Java-style .properties content into a librdkafka config map.

merge_with_overrides

Merge profile defaults with user overrides.

Type Aliases

KafkaConfigResult

Result type for kafka config file operations.