Skip to main content

Crate fv_streams_exchange

Crate fv_streams_exchange 

Source
Expand description

THE INTERNAL EXCHANGE (Phase 4): Arrow-IPC-carrying frames over kernel TCP on tokio, the transport a shuffle edge takes when its destination lives in another process. Decision D3 built to the letter (design/streams-v2-plan.md §1; investigation §7.1–7.3; review-followups §3):

  • kernel TCP on tokio, TCP_NODELAY — Flight, rkyv, io_uring and QUIC each rejected with evidence; the transport sits behind a trait so a SEND_ZC experiment can swap in later (D6).
  • N flows per ordered worker pair, each with its own source port so the NIC’s RSS hash spreads them across queues and CPUs — the one thing that steps past the single-flow ceiling Flink, Arroyo and timely all sit under (§3.3).
  • one owner task per socket direction — compute tasks never touch a socket; they push frames into bounded per-channel queues, so the only real sk_lock contention (more than one writer per socket) never arises.
  • credit-based flow control in bytes per channel, granted by the receiver from its own consumption of that channel alone — never conditioned on another channel draining, which is what keeps a join diamond (Nexmark q8) deadlock-free.

The transport moves opaque Frames tagged with a ChannelId and a FrameKind; it routes on the channel, acts on Credit itself, and delivers Data/Signal/Eos to the engine, which maps them to and from its runtime Msg. That boundary keeps this crate free of the engine’s types.

Structs§

Closed
The far end went away.
Frame
One frame: a channel, a kind, and an opaque body.
Grant
A received frame’s credit, granted back to its sender when this is dropped (see LinkReceiver::recv_held). Dropping never blocks, and works from any thread: the grant goes to the link’s own task, which sends it on. Credit is never lost, even for a frame dropped unread.
Link
One worker-pair link — a LinkSender and a LinkReceiver, joined for convenience. Split it with Link::into_split to send and receive from different tasks, which the sync-runtime bridge does (send is &self, recv is &mut self, so they cannot share one handle).
LinkReceiver
The receive half of a link: owns the delivered-frame stream, and grants credit back as frames are handed on. It holds the socket-owner task handles, so the flows live as long as frames are read.
LinkSender
The send half of a link: cloneable, so many producers send on their channels concurrently. The runtime’s outbound bridge holds one.

Enums§

FrameError
A frame header the reader cannot accept: a wire-format version other than its own (a peer built from another fv-streams version), or a kind byte it does not know.
FrameKind
The transport-level category of a frame. Data/Signal/Eos are delivered to the engine (and count against the channel’s credit); Credit is the transport’s own flow-control replenishment.

Constants§

HEADER
The 12-byte frame header: body_len(4) · channel(4) · kind(1) · wire_version(1) · reserved(2).
WIRE_VERSION
The wire-format version every frame carries (header byte 9). It names the whole frame contract — this header, the Arrow IPC stream framing a Data body uses, the postcard Signal layout, the one-byte reason an Eos body carries — so a peer on another version is refused at its first frame, never misread mid-stream. Bump it when any of that changes on purpose (the engine’s pinned golden-bytes test is what says it changed). A cluster’s handshake checks this same number, so it also covers the control messages between the leader and its workers. v2: an Eos says why the stream ended. v3: the leader asks each worker whether its work is done.

Traits§

Exchange
The transport seam. Today the only implementor is the TCP Link; the point of the trait is that the engine holds impl Exchange and never learns which — so a SEND_ZC / io_uring transport on a new kernel (D6, deferred behind exactly this trait) or an in-memory loopback for a test can take its place without an engine change. Deliberately not dyn-safe: the engine is generic over its transport, not boxing a heterogeneous set, so the seam stays zero-cost.

Functions§

read_loop
The read owner for one socket direction: decode frames from the wire and forward them to inbox, in order. Ends cleanly on EOF (with no partial frame left over), or when the inbox receiver drops. A junk kind byte or a frame truncated by EOF is an error.
write_loop
The write owner for one socket direction: drain outbox and write each frame, in order. Ends (and shuts the write half, signalling EOF to the far reader) when every sender of outbox drops.

Type Aliases§

ChannelId
A logical stream between two operator tasks (the engine’s EdgeId). Many channels are multiplexed over the flows of one worker-pair link.