Crate rdkafka [−] [src]

rust-rdkafka

Kafka client library for Rust based on librdkafka.

The library

This library aims to provide a safe interface to librdkafka. It currently exports some of the funcionalities provided by the producer and consumer of librdkafka 0.9.1.

Producers and consumers can be accessed and polled directly, or alternatively a futures-based interface can be used:

  • A consumer will return a stream of messages, as they are received from Kafka.
  • A producer will return a future that will eventually contain the delivery status of the message.

Warning: this library is still at an early development stage, the API is very likely to change and it shouldn't be considered production ready.

Installation

Add this to your Cargo.toml:

[dependencies]
rdkafka = "^0.1.0"

This crate will compile librdkafka from sources and link it statically in your executable. To compile librdkafka you'll need:

  • the GNU toolchain
  • GNU make
  • pthreads
  • zlib: optional, included by default (feature: zlib).
  • libssl-dev: optional, not included by default (feature: ssl).
  • libsasl2-dev: optional, not included by default (feature: sasl).

To enable ssl and sasl, use the features field in Cargo.toml. Example:

[dependencies.rdkafka]
version = "0.1.0"
features = ["ssl", "sasl"]

Compiling from sources

To compile from sources, you'll have to update the submodule containing librdkafka:

git submodule update --init

and then compile using cargo, selecting the features that you want. Example:

cargo build --features "ssl sasl"

Examples

You can find examples in the examples folder. To run them:

RUST_LOG="rdkafka=trace" LOG_THREAD=1 cargo run --example simple_consumer

The RUST_LOG environemnt variable will configure tracing level logging for rdkafka, and LOG_THREAD will add the name of the thread to log messages.

Modules

client

Common client funcionalities.
config

Configuration to create a Consumer or Producer.
consumer

Consumer implementations.
error

Error manipulations.
message

Store and manipulate Kafka messages.
producer

Producer implementations.
util

Utility functions.