Module runtime 

Distributed inference runtime scaffolding.

This module provides lightweight execution planning primitives that bridge placement plans to token-step pipeline schedules across heterogeneous devices.

Structs

BridgeAccumulator

ExecutionResult

Aggregate runtime execution output.

PipelinePlan

Pipeline execution plan for one model.

StageExecutionStats

Per-stage runtime telemetry captured from real in-process execution.

StagePlacement

Placement of one contiguous layer range onto a node and device.

StageWorkerSummary

Summary a stage-worker process reports (to its own stdout, and back to any caller driving it directly) once its one coordinator connection closes.

TcpTransportConfig

Runtime controls for TCP/Unix transport execution.

TokenStep

One scheduled operation in steady-state token generation.

TransportBatch

Enums

BridgeAddr

Unified transport address for TCP or Unix domain sockets.

BridgeListener

Unified transport listener wrapping platform-specific socket types.

BridgeStream

Unified transport stream wrapping platform-specific socket types.

DeviceKind

Execution target selected for a pipeline stage.

TransportKind

Transport protocol selection for inter-stage bridges.

Functions

build_token_schedule

Generate a simple steady-state pipeline schedule preview.

execute_pipeline

Execute pipeline stages with real channel/thread wiring for token_count tokens.

execute_pipeline_distributed

Execute pipeline stages distributed across the LAN using TCP transport.

execute_pipeline_tcp_loopback

execute_pipeline_tcp_loopback_with_config

Execute pipeline stages with real TCP loopback transport bridges and explicit transport config.

execute_pipeline_with_rebalance

execute_pipeline_with_rebalance_and_measured

execute_pipeline_with_remote_stage

Coordinator side of real remote execution. plan must have at least one stage assigned to local_node_id and at least one assigned to remote_node_id — matching flow’s existing local+remote 2-node model, though each node’s stages get merged (see merge_stages_for_node) since the real assignment algorithm splits one node’s layers into several raw stages, not necessarily one. The merged local stage runs in-process as usual; the merged remote stage’s batches are sent over a real outbound connection (TcpStream::connect(remote_addr)) to a separate run_stage_worker process, so the round-trip time recorded here is genuine network + remote-compute time, not a local simulation.

run_stage_worker

Runs as a real, separate OS process (see ghost-link stage-worker): accepts exactly one coordinator connection on listener, reads the stage handshake, then loops read-batch → compute → write-result until the coordinator closes the connection. One-shot, matching how cluster-start’s child listen processes already behave.

spawn_tcp_bridge

Spawn a TCP transport bridge between pipeline stages.