Expand description
A Kafka client core with no IO, no runtime, and no clock.
Everything here is a state machine over bytes. Nothing opens a socket, waits
on a timer, or spawns a task — a caller feeds it received bytes and drains
the bytes it wants sent. Binding crates (barnabas-glommio, and later
barnabas-tokio) supply the sockets.
§Why
An abstraction spanning async runtimes is lowest-common-denominator, and LCD
requires Send — which forbids exactly the per-core, !Send design a
thread-per-core runtime exists for. A sans-io core sidesteps the argument by
naming no runtime at all: the binding decides Send-ness, so one state
machine serves both a per-core Rc handle and a work-stealing one.
The second reason is testing, and in a Kafka client it is the bigger one. Exactly-once bugs are silent — a wrong retry duplicates records and every status code stays green — so they have to be caught by driving the state machine adversarially rather than by watching a broker behave. A core with no IO can be driven that way in a unit test, deterministically, with no broker and no executor. Every test in this crate is one.
§Layout
frame— Kafka’s length-prefixed framing, the one place partial reads are handled.conn— request/response correlation over a single broker connection.consumer— assign-only fetch positions and READ_COMMITTED filtering.metadata— the cluster map, and knowing when it is stale.partitioner— which partition a keyed record lands on, and why the answer differs between Kafka clients.producer— idempotent sequencing and the transaction state machine.
Re-exports§
pub use conn::Connection;pub use conn::PendingResponse;pub use consumer::FetchPosition;pub use consumer::IsolationLevel;pub use group::Assignment;pub use group::Assignor;pub use group::CooperativeStickyAssignor;pub use group::RangeAssignor;pub use group::RoundRobinAssignor;pub use group::StickyAssignor;pub use group::Subscription;pub use group::TopicPartition;pub use member::GroupMember;pub use member::MemberState;pub use member::RebalanceProtocol;pub use member::Step;pub use metadata::BrokerAddr;pub use metadata::Metadata;pub use partitioner::Partitioner;pub use producer::ProducerIdentity;pub use producer::ProducerState;pub use producer::SequenceRange;pub use producer::TxnState;
Modules§
- conn
- Request/response correlation over one broker connection.
- consumer
- Assign-only consumer state: fetch positions, and READ_COMMITTED filtering.
- frame
- Kafka’s framing: a 4-byte big-endian length, then that many bytes.
- group
- Consumer group assignment: who gets which partitions.
- member
- Group membership: the classic protocol’s state machine, with no IO.
- metadata
- The cluster map: which broker leads which partition, and how to reach it.
- partitioner
- Choosing a partition for a keyed record.
- producer
- The idempotent, transactional producer — as a state machine over nothing.
- records
- A record-batch reader that does not build a record per record.
Structs§
- Error
Code - A broker error code, classified.
Enums§
- Disposition
- What a caller should do about an error code.
- Error
- Everything that can go wrong in the core.