Kafka-Protocol
Rust implementation of the Kafka wire protocol.
Unlike other Kafka protocol implementations, this project uses code generation to cover the entire Kafka API surface, including different protocol versions. See Kafka's repo for an example of protocol schema.
Versioning
Protocol messages are generated against a recent stable Kafka release, currently 3.7.0.
Although the Kafka protocol remains relatively stable and strives to be backwards compatible, new fields are occasionally
added. In order to ensure forward compatibility with the protocol, this crate marks all exported items as #[non-exhaustive]
.
Protocol messages can be constructed using either Default::default
or their provided builder.
Working with messages
Using Default::default
:
use ;
use StrBytes;
let mut header = default;
header.client_id = Some;
header.request_api_key = MetadataKey as i16;
header.request_api_version = 12;
let mut request = default;
request.topics = None;
request.allow_auto_topic_creation = true;
Using kafka_protocol::protocol::Builder
:
use ;
use ;
let header = builder
.client_id
.request_api_key
.request_api_version
.build;
!
let request = builder
.topics
.allow_auto_topic_creation
.build;
Serialization
Once a message has been created, it can be serialized using Encodable
, writing
the struct to a provided bytes::BytesMut
. The API version for the given message
matching the version specified in the request header must be provided.
use BytesMut;
use MetadataRequest;
use Encodable;
let mut bytes = new;
let request = default;
request.encode.unwrap;
Deserialization
Messages can be decoded using Decodable
and providing the matching API version from their
corresponding request.
use Bytes;
use ApiVersionsRequest;
use Decodable;
let bytes: = ;
let res = decode.unwrap;
Development
Run cargo run -p protocol_codegen
in the root path of this repo to generate/update the Rust codes via the latest Kafka
protocol schema.
Originally implemented by @Diggsey in a minimal Kafka client implementation Franz