Module kafka::producer
[−]
[src]
Kafka Producer - A higher-level API for sending messages to Kafka topics.
This module hosts a multi-topic capable producer for a Kafka cluster providing a convenient API for sending messages synchronously.
In Kafka, each message is a key/value pair where one or the other
is optional. A Record
represents all the data necessary to
produce such a message to Kafka using the Producer
. It
specifies the target topic and the target partition the message is
supposed to be delivered to as well as the key and the value.
Example
use std::fmt::Write; use std::time::Duration; use kafka::producer::{Producer, Record, RequiredAcks}; let mut producer = Producer::from_hosts(vec!("localhost:9092".to_owned())) .with_ack_timeout(Duration::from_secs(1)) .with_required_acks(RequiredAcks::One) .create() .unwrap(); let mut buf = String::with_capacity(2); for i in 0..10 { let _ = write!(&mut buf, "{}", i); // some computation of the message data to be sent producer.send(&Record::from_value("my-topic", buf.as_bytes())).unwrap(); buf.clear(); }
In this example, when the producer.send(..)
returns
successfully, we are guaranteed the message is delivered to Kafka
and persisted by at least one Kafka broker. However, when sending
multiple messages just like in this example, it is more efficient
to send them in batches using Producer::send_all
.
Since some of the Record
s attributes are optional, convenience
methods exist to ease their creation. In this example, the call
to Record::from_value
creates a key-less, value-only record with
an unspecified partition. The Record
struct, however, is
intended to provide full control over its lifecycle to client
code, and, hence, is fully open. Its current constructor methods
are provided for convience only.
Beside the target topic, key, and the value of a Record
, client
code is allowed to specify the topic partition the message is
supposed to be delivered to. If the partition of a Record
is
not specified - more precisely speaking if it's negative -
Producer
will rely on its underlying Partitioner
to find a
suitable one. A Partitioner
implementation can be supplied by
client code at the Producer
's construction time and defaults to
DefaultPartitioner
. See that for more information for its
strategy to find a partition.
Reexports
pub use client::RequiredAcks; |
pub use client::ProduceConfirm; |
pub use client::ProducePartitionConfirm; |
Structs
Builder |
A Kafka Producer builder easing the process of setting up various configuration settings. |
DefaultPartitioner |
As its name implies |
Partitions |
Producer relevant partition information of a particular topic. |
Producer |
The Kafka Producer |
Record |
A structure representing a message to be sent to Kafka through the
|
Topics |
A description of available topics and their available partitions. |
Enums
Compression |
Compression types supported by kafka. The numeral values of this enumeration correspond to the compression encoding in the attributes of a Message in the protocol. |
Constants
DEFAULT_ACK_TIMEOUT_MILLIS |
The default value for |
DEFAULT_REQUIRED_ACKS |
The default value for |
Traits
AsBytes |
A trait used by |
Partitioner |
A partitioner is given a chance to choose/redefine a partition for
a message to be sent to Kafka. See also
|
Type Definitions
DefaultHasher |
The default hasher implementation used of |