transport-grpc and transport only.Expand description
§Native batch transport wire mapper (Task 0.6)
Serde-less rustlib<->rustlib (DFE<->DFE) transfer of a whole
WorkBatch over the existing gRPC mesh. One
WorkBatch maps to one proto Batch and travels in a
single RouteBatch RPC – batch-at-a-time, NOT record-by-record streaming.
§Payloads are OPAQUE in transit
Each Record payload is mapped straight onto the
proto Record.payload bytes field and back. prost is configured (in
build.rs, .bytes(".")) to decode that field ZERO-COPY into
bytes::Bytes, so on the receive side the payload is a refcounted view of
the decode buffer, never copied. The JSON / MsgPack codec
(crate::transport::codec) is NEVER invoked here – the bytes pass through
untouched, so a payload that is not valid JSON or MsgPack survives a
round-trip byte-identical.
§Swappable wire
The wire is protobuf BY DEFAULT but the mapping is isolated to this module:
a future hand-rolled frame could replace records_to_proto /
proto_batch_to_records without touching the WorkBatch types or the
transport seam.
§What does NOT cross the wire
commit_tokens and dlq_entries are deliberately left off the proto Batch.
Commit tokens are the SENDER’s source acks – fired locally after the block
is sent (at-least-once), they have no meaning on the receiver. Inline-DLQ
entries are a local no-silent-drop concern. So the mapper takes / returns the
records (&[Record] / Vec<Record>), not the whole WorkBatch<T>; this
also avoids dragging the CommitToken generic onto the wire path.
Functions§
- proto_
batch_ to_ records - Map a proto
Batchback onto aVec<Record>. - records_
to_ proto - Map a slice of rustlib
Records onto a protoBatch.