Struct CacheSyncProtocol 

pub struct CacheSyncProtocol { /* private fields */ }

Tracks distributed cache state across all ranks.

This is a metadata-only coordinator. It does not hold actual tensors — tensors live in the inference engine’s KV cache on each device.

Implementations

impl CacheSyncProtocol

pub fn new(world_size: usize, num_layers: usize) -> Self

Create a new protocol tracker.

pub fn with_max_log_size(self, max: usize) -> Self

Set the maximum event log size (oldest events are dropped).

pub fn record_event(&mut self, event: SyncEvent)

Record a synchronisation event.

pub fn latest_position(&self, rank: usize, layer: usize) -> Option<usize>

Latest known sequence position for a (rank, layer).

pub fn min_synced_position(&self, layer: usize) -> Option<usize>

Minimum sequence position across all ranks for a given layer.

Returns None if any rank hasn’t reported for that layer. This is the “safe” position up to which all ranks are in sync.

pub fn is_prefix_confirmed(&self, prefix_id: u64) -> bool

Whether a prefix is confirmed across all ranks.

pub fn prefix_tokens(&self, prefix_id: u64) -> Option<usize>

Confirmed prefix token count—None if not confirmed.

pub fn event_count(&self) -> usize

Number of events recorded.

pub fn recent_events(&self, n: usize) -> &[SyncEvent]

Recent events (up to n).

Trait Implementations

impl Clone for CacheSyncProtocol

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for CacheSyncProtocol

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.