Skip to main content

Module send

Module send 

Source
Expand description

Sender pipeline.

Shape of the thing:

  ChunkCursor ──jobs──> rayon pool (read → compress → seal) ──> N stream tasks ──> wire
       │                        (cfg.workers threads)              (cfg.streams)
   one lock per chunk                                        queue_depth in flight each

The cursor hands out one chunk at a time and holds the open reader for the file it is currently walking, so the encode workers never open a file, never share a file cursor, and never consult a cache. Each stream task keeps queue_depth encodes in flight, which is what bounds memory: nothing scales with file size, only with streams × queue_depth × chunk_size.

Functions§

is_all_zero
Is every byte zero?
merkle_root
Fold per-chunk hashes into one value for the file.
send
Send sources over transport.