Skip to main content

Module chunk

Module chunk 

Source
Expand description

Message framing / chunking. A message larger than the connection’s negotiated max_message_size is split into MTU-sized Chunks on the sender and reassembled on the receiver.

NOTE: this is whole-message buffering, not MSRP-style (RFC 4975) streaming. There is no incremental delivery: the receiver yields a payload only once every chunk has arrived (or drops it on TTL). The outbound scheduler keeps each message class FIFO, so two chunked messages in the same class do not interleave; a higher-priority class may run between chunks while a delivery is pending. The “split into ordered, id-tagged pieces and reassemble” idea is borrowed from MSRP chunking; MSRP interruption semantics are not implemented.

Two halves, deliberately separated:

  • Send - ChunkList turns a bytes::Bytes into ordered Chunks, where chunk_size comes from the connection’s negotiated max_message_size. The sender uses ChunkList::stream, which yields chunks lazily as zero-copy slices so one chunk is held in flight at a time; ChunkList::split (eager Vec) remains for tests.
  • Receive - MessageReassembler collects incoming Chunks keyed by message id and yields the original payload once every position has arrived.

The receiver is robust to the realities of a multi-hop / DHT overlay: out-of-order arrival, duplicates / retransmits (first write per position wins), and partial messages (evicted by TTL). It is also bounded against a hostile peer: per-chunk and per-message byte caps, a global buffered-cost ceiling (charging a per-slot overhead so tiny-chunk floods are bounded by count too), an id-count cap, and up-front rejection of already-expired chunks. No single id and no peer-supplied total can drive memory without limit. See MessageReassembler.

  send    : Bytes -> [Chunk{ chunk=[i, n], data=data_i, meta } | i in 0..n]
  receive : a message id is complete iff received positions = 0..total (all n of them);
            then payload = concat(data_i for i in 0..total)

Structs§

Chunk
One chunk of a chunked message, as it travels on the wire.
ChunkList
Sender side: an ordered list of Chunks for one message. Build it from the payload with ChunkList::split, passing the per-message data size to cut at (the connection’s negotiated max_message_size minus the envelope reserve), then iterate (or convert to Vec<Chunk>) to put each chunk on the wire. The cut size is a runtime argument rather than a type parameter because it is decided per connection from the negotiated limit. Reassembly is the receiver’s job - see super::MessageReassembler.
ChunkMeta
Meta data of a chunk
MessageReassembler
Receiver side: whole-message reassembly for reliable data-channel MessagePayload fragments. Buffers a message’s chunks keyed by id and yields the complete Bytes once every position has arrived (then forgets it).
ReassemblyLimits
The limits a super::MessageReassembler enforces on incoming chunks, as an explicit value rather than module globals. This keeps the core admission rule independent of where the numbers come from: the shell supplies them (see ReassemblyLimits::production), the reassembler only enforces what it is given, and tests can use small limits instead of giant synthetic payloads.
WireReserves
The bytes the transport adds around a payload on the wire, per framing path. Bundled as a named value so the framing rule reads reserves.plan(len, limit) instead of a row of positional usizes, and so the production reserves live in exactly one place (WireReserves::PRODUCTION).

Enums§

Framing
How one payload should be framed for a size-limited connection: sent whole, or split.