Module compiled 

Compiled-node representation. Phase 7.5 B4 (interface only).

A CompiledNode is the output of dispatch resolution: a Node plus its resolved kernel function pointer plus its OpParams. The executor consumes these and runs them.

Design intent: pipelined compile + execute (deferred)

The original B-plan called for pipelined compilation: a compiler thread that walks the graph in topological order, resolves kernels per node, and pushes CompiledNode entries onto a crossbeam::channel. An executor thread consumes the channel, looks up inputs from the graph’s storage map (blocking when inputs aren’t yet realized), calls the kernel, and stores the output back. Time-to-first-token wins because the compiler runs ahead of the executor.

Today’s “compile” step is just a KernelBindingTable lookup — roughly nanoseconds per node. There’s no meaningful work to overlap with execution. Pipelining adds std::thread + crossbeam::channel + lifecycle handling for zero current win.

When compile grows beyond trivial — residency-aware planning, transfer-cost minimization, kernel auto-tuning, dynamic-shape specialization — pipelining lands as a follow-up. The interfaces in this module (CompiledNode + compile_node + execute_compiled) are designed so the threaded variant slots in without changing call sites.

Modules

op_profile

Opt-in per-(OpKind, BackendId) host-time accounting for kernel launches (execute_compiled_with_wait_hint). Off by default (one relaxed atomic load per launch). What it measures: wall time of the kernel call on the HOST. A kernel that only enqueues work (Vulkan batches into a command buffer, CUDA launches on a stream) is charged its enqueue cost, NOT its device time; a kernel that blocks (CPU math, a host-to-device upload, a forced flush) is charged all of it. Device time lands wherever the realize finally waits, so compare the per-op sum with the caller’s own wall clock around a step.

Structs

CompiledNode

A graph node plus its resolved kernel function pointer and op-specific parameters. Produced by compile_node; consumed by execute_compiled.

Enums

CompletionHandle

A node’s completion signal, returned by execute_compiled. Step E Phase A1 foundation: today every dispatch is synchronous, so this is always CompletionHandle::Ready (the work finished before the call returned) and callers wait it immediately — byte-identical to the prior Result<()>. Phases A2/A3 add an async GPU path that returns CompletionHandle::Pending carrying a device fence/event the executor defers the wait on.

Traits

Completion

An async-dispatch completion signal — a backend’s wrapper over a CUDA event / Vulkan fence so the executor can defer the wait past the submit point (Step E Phase A2/A3). No implementors today; all dispatch is synchronous.

Functions

compile_node

Resolve a node’s kernel from the binding table and return a CompiledNode ready for execution. Production-correct: surfaces NoBackendForOp if the table doesn’t have a binding for the requested triple.

dispatched_kernel_ident

The (kernel_source, kernel_revision_hash) identity of the kernel the executor dispatches for node, derived from the SAME graph stamp + registry the executor resolves through: op_to_op_kind + build_lookup_dtypes + graph.target_backend → the first-registered binding at (op, dtypes, backend) (the entry compile_node’s lookup_with_caps picks). None when the node has no dispatch mapping (view/structural op) or no registered binding.

execute_compiled

Run a compiled node against the given inputs/outputs. The output Storages must be pre-allocated (the executor’s responsibility, using the node’s shape + dtype).

execute_compiled_with_wait_hint

Step E A4b open question #4 — “event only where waited”. Identical to execute_compiled except the caller supplies will_be_waited: whether ANY later step in this realize will individually wait this node’s completion handle before the realize-end drain (today: the cross-device Copy/Move producer wait, wait_producer_handle in pipelined.rs).