Skip to main content

Module batch

Module batch 

Source
Available on crate features transport-grpc and transport only.
Expand description

§Native batch transport wire mapper (Task 0.6)

Serde-less rustlib<->rustlib (DFE<->DFE) transfer of a whole WorkBatch over the existing gRPC mesh. One WorkBatch maps to one proto Batch and travels in a single RouteBatch RPC – batch-at-a-time, NOT record-by-record streaming.

§Payloads are OPAQUE in transit

Each Record payload maps straight onto the proto Record.payload bytes field. prost decodes that field ZERO-COPY into bytes::Bytes (.bytes(".") in build.rs), so the receive-side payload is a refcounted view of the decode buffer. The JSON / MsgPack codec (crate::transport::codec) is NEVER invoked here – non-JSON/MsgPack bytes survive a round-trip byte-identical.

§Swappable wire

The mapping is isolated to this module: a future hand-rolled frame could replace records_to_proto / proto_batch_to_records without touching the WorkBatch types or the transport seam.

§What does NOT cross the wire

commit_tokens and dlq_entries are left off the proto Batch. Commit tokens are the SENDER’s source acks – fired locally after send (at-least-once), meaningless on the receiver. Inline-DLQ entries are a local no-silent-drop concern. So the mapper takes / returns records, not the whole WorkBatch<T> – which also keeps the CommitToken generic off the wire.

Functions§

proto_batch_to_records
Map a proto Batch back onto a Vec<Record>.
records_to_proto
Map a slice of rustlib Records onto a proto Batch.