Crate ts_pd 

TokenSpeed’s PD transfer peer (RFC 0002): one rank’s half of moving a request’s KV from the prefill node to the decode node.

The peer owns the Mooncake transfer engine, the P/D handshake over ZMQ, the bootstrap-server HTTP calls, the room states, the transfer workers and the per-round agreement on transfer progress. It also plans everything that needs the ranks’ cache layouts (layout): the routes between unequal Prefill and Decode rank sets, the block manifests read off the plans’ block tables, and the copies one transfer makes. The data plane only builds the rank’s cache transfer contract at startup and tells the forward thread when a plan’s remote stream may go out (PrefillOutbox, DecodeOutbox). The bootstrap server the nodes rendezvous through is here too (bootstrap). The wire formats are unchanged from the Python peers this crate replaced, so nodes of either version interoperate.

Re-exports

pub use bootstrap::BootstrapServer;

pub use bootstrap::Registry;

pub use layout::registration::PrefillLayout;

pub use layout::registration::PrefillRoute;

pub use layout::LayoutError;

Modules

bootstrap

The bootstrap server: the HTTP rendezvous of a disaggregated deployment’s data-source ranks (a Prefill node’s for PD, an encode node’s for EPD) and the ranks that read from them.

epd

EPD: an encode node runs the vision tower and writes each multimodal item’s embedding into the prefill node’s receive buffers over Mooncake.

layout

Transfer planning over the ranks’ cache layouts, ported from the Python PD planner (pd/cache_protocol.py, pd/transfer_plan.py and the Mooncake prefill planning).

Structs

BatchRows

A batch’s rows (the scheduler’s TransferBatchView); extend rows first.

BootstrapInfo

Where a request’s peer node reaches it: the prefill deployment’s bootstrap server and the request’s room.

DecodeConfig

A Decode rank’s settings.

DecodeOutbox

Where the forward thread hands a Decode rank’s peer its work; never waits for a transfer.

DecodePeer

A Decode rank’s PD transfer peer.

EmbeddingChunk

EPD, an encode rank: one item’s embedding in its send slot, as the transfer engine reads it.

EmbeddingItem

EPD: one multimodal item of a request whose embedding an encode node computes, as the request names it (item.encode_handshake).

LandedItem

EPD: where one received item’s rows landed, for the data plane to publish onto the item.

LandedRequest

EPD: a request whose items’ embeddings all landed.

Layerwise

Layerwise streaming: each prefill chunk’s KV goes out an interval of producer steps at a time, as the forward writes it.

PeerConfig

What both roles start from.

PrefillConfig

A Prefill rank’s settings.

PrefillOutbox

Where the forward thread hands a Prefill rank’s peer its work; never waits for a transfer.

PrefillPeer

A Prefill rank’s PD transfer peer.

RemotePrefill

A D-role request whose KV landed.

StatusMessage

A Prefill rank’s report on one room.

Enums

Copy

What one transfer copies, in order.

NetError

A socket failure at startup.

PdError

A PD failure the engine cannot serve past.

PdEvent

A transfer whose state the replica agreed changed.

StartError

A peer that failed to start.

TransferPoll

A transfer’s state, in the order it advances. The values are the wire encoding of the P/D status messages and of the replica’s MIN agreement.

Traits

EmbeddingReceiver

EPD, a prefill node’s first stage: receiving the encode nodes’ embeddings of the requests waiting for them, as the scheduling loop drives it. Every member of the replica stages the same requests in the same order and settles the same codes.

EmbeddingSender

EPD, an encode rank: sending each item’s embedding to the prefill ranks that wait for it, as the scheduling loop drives it.

TransferPeer

The transfer peer’s control face.