Struct DecodeOutbox 

pub struct DecodeOutbox { /* private fields */ }

Where the forward thread hands a Decode rank’s peer its work; never waits for a transfer.

A plan’s remote prefill is staged when the plan goes to the data plane (DecodeOutbox::stage_remote_prefill) and handed over by the data plane once the plan’s page zeroing finished (DecodeOutbox::hand_over_remote_prefill): the Prefill ranks’ writes are not ordered with the device’s, so the pages they land in must be sanitized first.

Implementations

impl DecodeOutbox

pub fn release_registration(&self) -> Result<(), String>

A sleep releases the cache: deregister it from the transfer engine, refused while a tracked request’s KV may still land.

pub fn restore_registration(&self) -> Result<(), String>

A wake re-mapped the cache: register it under a fresh transfer session, as the Prefill ranks write into this memory with the keys of its registration, and forget the registrations made under the old one, so the next request registers the new session again.

pub fn stage_remote_prefill(&self, rows: BatchRows) -> Result<(), String>

Stage a plan’s remote prefill: the prompts whose pages it admitted.

pub fn remote_prefill_pending(&self) -> bool

Whether a staged remote prefill still waits to be handed over.

pub fn hand_over_remote_prefill(&self) -> Result<(), String>

The staged remote prefill’s pages are sanitized: send each admitted prompt’s manifest to its Prefill ranks (HostDecodePeer.execute). Every row is planned before any goes out.

Trait Implementations

impl Clone for DecodeOutbox

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.