Struct OffloadEngine 

pub struct OffloadEngine { /* private fields */ }

Central coordinator for offload pipelines.

The engine manages multiple pipelines (G1→G2, G2→G3, G2→G4) and provides a unified interface for enqueueing blocks for offload.

Storage Tier Model

• G1→G2: BlockManager<G2> destination (host memory)
• G2→G3: BlockManager<G3> destination (disk/NVMe)
• G2→G4: ObjectBlockOps destination (object storage like S3)

Distributed G2→G4 Offloading

For distributed setups where the leader doesn’t have physical layouts (only workers do), use with_enable_remote_g4(true) instead of with_g2_to_g4_pipeline(). This enables remote G4 offloading where workers execute object storage uploads via their local ObjectBlockOps implementations.

Implementations

impl OffloadEngine

pub fn builder(leader: Arc<InstanceLeader>) -> OffloadEngineBuilder

Create a new builder for the offload engine.

pub fn enqueue_g1_to_g2( &self, blocks: impl Into<SourceBlocks<G1>>, ) -> Result<TransferHandle>

Enqueue blocks for G1→G2 offload.

Returns a TransferHandle for tracking progress and cancellation.

pub fn enqueue_g1_to_g2_with_precondition( &self, blocks: impl Into<SourceBlocks<G1>>, precondition: Option<EventHandle>, ) -> Result<TransferHandle>

Enqueue blocks for G1→G2 offload with a precondition event.

The precondition event must be satisfied before the batch is processed by the transfer executor. This enables coordination with worker forward passes.

Returns a TransferHandle for tracking progress and cancellation.

pub fn enqueue_g2_to_g3( &self, blocks: impl Into<SourceBlocks<G2>>, ) -> Result<TransferHandle>

Enqueue blocks for G2→G3 offload.

Returns a TransferHandle for tracking progress and cancellation.

pub fn enqueue_g2_to_g4( &self, blocks: impl Into<SourceBlocks<G2>>, ) -> Result<TransferHandle>

Enqueue blocks for G2→G4 offload (object storage).

Returns a TransferHandle for tracking progress and cancellation.

pub fn release_transfer(&self, transfer_id: TransferId)

Release a completed transfer’s resources.

This is optional - transfers are automatically cleaned up, but call this to release resources earlier.

pub fn active_transfer_count(&self) -> usize

Get the number of active transfers.

pub fn has_g1_to_g2(&self) -> bool

Check if G1→G2 pipeline is configured.

pub fn has_g2_to_g3(&self) -> bool

Check if G2→G3 pipeline is configured.

pub fn has_g2_to_g4(&self) -> bool

Check if G2→G4 pipeline is configured.

pub fn settlement_token(&self) -> SettlementToken

Capture a causal checkpoint for every configured transfer lane.

pub async fn settle_after( &self, token: SettlementToken, target: SettlementTarget, ) -> Result<(), SettlementError>

Wait until the targeted post-token batches are causally settled.

Settlement includes synchronous executor publication and semaphore handoff to every immediately runnable successor. It deliberately does not include auto-chained downstream topology in this implementation slice.