Module bridge 

Bridge from engine activity dispatch to connected workers. NIF bridge dispatcher that routes run_activity calls to connected workers.

WorkerActivityDispatcher implements aion::ActivityDispatcher so the engine’s activity NIFs can synchronously dispatch to a remote worker and block until the result comes back.

Threading contract

The engine invokes aion::ActivityDispatcher::dispatch from two kinds of threads: beamr scheduler threads (concurrency combinators) and spawned tokio tasks (the two-phase dispatch_activity completion task). The task send uses try_send() (non-blocking channel push) and the response wait blocks on std::sync::mpsc::Receiver::recv.

Blocking is harmless on a beamr thread, but on a tokio runtime worker it must be wrapped in tokio::task::block_in_place: the try_send wakes the per-worker gRPC stream forwarder task, and tokio schedules a task woken from task context into the current worker’s LIFO slot, which no other runtime worker can steal. Without the block_in_place core handoff the forwarder sits trapped in that slot while this thread blocks, so the queued ActivityTask is never flushed to the worker even though the worker is healthy. block_in_place moves the worker’s scheduler core (LIFO slot included) to another thread before the wait begins, so dispatch-to-delivery stays in the millisecond range and the runtime keeps full parallelism.

Wait termination

The engine imposes no activity timeout of its own: agent-style activities legitimately run for over an hour, so the completion wait is unbounded. The blocking recv terminates on exactly one of:

• Completion — the worker reports a result and the stream handler delivers it through ActivityCompletionSink::complete_activity.
• Worker loss — the worker’s gRPC stream ends (process death, disconnect, expired token); the stream teardown sweeps the worker’s in-flight tasks through the same sink as every other TRANSPORT loss (HeartbeatTracker::fail_disconnected_worker). A liminal-delivered worker’s loss is observed by its reply router thread instead (the correlated-reply awaiter wakes the moment the connection closes) and resolves the dispatch with the same transport-loss class. Both are classified by transport_loss: lost: while the transport still has budget to deliver the activity (the engine re-dispatches the SAME attempt, recording nothing), transport-exhausted: once that budget is spent. Neither ever wears the action’s retry vocabulary — the activity never ran.
• Graceful-drain park (#207) — during a drain, a worker stream ending (or the drain-timeout backstop) PARKS the worker’s in-flight tasks through ActivityCompletionSink::park_activity: the waiter resolves with the ephemeral parked sentinel (aion::PARKED_ACTIVITY_REASON), nothing is recorded or delivered, and restart recovery re-dispatches the dangling ordinal — kill -9 convergence.
• Drain timeout at shutdown — the shutdown coordinator parks all remaining in-flight tasks through the sink (HeartbeatTracker::park_all_in_flight_workers).
• Channel teardown — every sender for the pending entry is dropped (a cleanup path removed the entry without completing it); surfaced as a channel-closed dispatch error, never a hang.

An activity’s duration is bounded only by the workflow’s own timeout_seconds and by worker liveness — never by an engine constant.

Structs

PendingActivities

Tracks in-flight activity dispatches waiting for worker results.

WorkerActivityDispatcher

Dispatcher that routes run_activity NIF calls to connected workers.

Traits

OutboxDeliveryCallback

Routes an unmatched durable-outbox completion into the live workflow.