Kafka Rust Client
Documentation
Kafka Rust Client Documentation
The documentation includes some examples too.
Installation
This crate works with Cargo and is on crates.io. I will be updating the package frequently till we move out of pre-release. So add this to your Cargo.toml
(instead of a specific version):
[]
= "*"
Usage:
Load topic metadata:
[Load Metadata] (https://spicavigo.github.io/kafka-rust/kafka/client/struct.KafkaClient.html#method.load_metadata_all)
extern crate kafka;
use KafkaClient;
Fetch Offsets:
[For one topic] (https://spicavigo.github.io/kafka-rust/kafka/client/struct.KafkaClient.html#method.fetch_topic_offset)
extern crate kafka;
use KafkaClient;
[For multiple topics] (https://spicavigo.github.io/kafka-rust/kafka/client/struct.KafkaClient.html#method.fetch_offsets)
extern crate kafka;
use KafkaClient;
Produce:
[Single Message] (https://spicavigo.github.io/kafka-rust/kafka/client/struct.KafkaClient.html#method.send_message)
extern crate kafka;
use KafkaClient;
use utils;
[Multiple Messages] (https://spicavigo.github.io/kafka-rust/kafka/client/struct.KafkaClient.html#method.send_messages)
extern crate kafka;
use KafkaClient;
use utils;
Fetch Messages:
[Single (topic, partition, offset)] (https://spicavigo.github.io/kafka-rust/kafka/client/struct.KafkaClient.html#method.fetch_messages)
extern crate kafka;
use KafkaClient;
[Multiple (topic, partition, offset)] (https://spicavigo.github.io/kafka-rust/kafka/client/struct.KafkaClient.html#method.fetch_messages_multi)
extern crate kafka;
use KafkaClient;
use utils;
Commit Offsets to a Consumer Group:
[Single (group, topic, partition, offset)] (https://spicavigo.github.io/kafka-rust/kafka/client/struct.KafkaClient.html#method.commit_offset)
extern crate kafka;
use KafkaClient;
[Single group, Multiple (topic, partition, offset)] (https://spicavigo.github.io/kafka-rust/kafka/client/struct.KafkaClient.html#method.commit_offsets)
extern crate kafka;
use KafkaClient;
use utils;
Fetch Offsets of a Consumer Group:
[Offsets for all topics/partitions in a group] (https://spicavigo.github.io/kafka-rust/kafka/client/struct.KafkaClient.html#method.fetch_group_offset)
extern crate kafka;
use KafkaClient;
[Offsets for a topic and all its partitions in a group] (https://spicavigo.github.io/kafka-rust/kafka/client/struct.KafkaClient.html#method.fetch_group_topic_offset)
extern crate kafka;
use KafkaClient;
use utils;
[Offsets for Multiple (topic, partition) in a group] (https://spicavigo.github.io/kafka-rust/kafka/client/struct.KafkaClient.html#method.fetch_group_topics_offset)
extern crate kafka;
use KafkaClient;
use utils;
[Consumer] (https://spicavigo.github.io/kafka-rust/kafka/consumer/index.html)
This is a simple Consumer for kafka. It handles offset management internally (Fetching offset for the group at the start and committing offsets at a pre-defined interval - 100 consumed messages currently) and provides an Iterator interface.
extern crate kafka;
use KafkaClient;
use utils;
[Create a topic] (https://kafka.apache.org/08/quickstart.html)
The examples above assume you will create the topic my-topic, for example with
kafka-topics.sh --topic my-topic --create --zookeeper localhost:2181 --partition 1 --replication-factor 1
TODO:
- Tests - (Added tests for gzip.rs, snappy.rs, and codecs.rs)