Struct VulkanExecutable 

pub struct VulkanExecutable { /* private fields */ }

Implementations

impl VulkanExecutable

pub fn compile(graph: Graph) -> Self

pub fn compile_rng(graph: Graph, rng: RngOptions) -> Self

Prepare the graph (legalize → primitive set), plan the arena, and build the dispatch schedule. Panics with a clear message if the graph contains an op no decomposition rule can reduce to SUPPORTED_OPS.

pub fn set_param(&mut self, name: &str, data: &[f32])

pub fn set_param_bytes(&mut self, name: &str, data: &[u8])

Raw-byte param upload (packed weights). The arena is f32-uniform, so callers should normally use [set_param]; this exists for symmetry.

pub fn output_dtypes(&self) -> Vec<DType>

pub fn set_active_extent(&mut self, extent: Option<(usize, usize)>)

pub fn bind_gpu_handle(&mut self, name: &str, data: &[f32]) -> bool

Persistent input buffer for KV-cache style graphs. Writes data into the input’s arena slot once; subsequent decode steps reuse it (and, with a feed wired, update it in place on-device). Returns false if name is not a graph input. Mirrors the rlx-metal handle semantics.

pub fn has_gpu_handle(&self, name: &str) -> bool

pub fn set_gpu_handle_feed(&mut self, handle_name: &str, output_index: usize)

pub fn register_kv_row_feed(&mut self, handle_name: &str, output_index: usize)

Register a row feed (vs the generic prefix feed): after a decode run, row src_row of output output_index is folded into handle handle_name’s input slot at row dst_row. For decode graphs that emit the new K/V token at the last bucket-padded output row (llama32). Driven explicitly via [feed_kv_row]; does NOT trigger the auto-propagation in run_read_outputs.

pub fn feed_kv_row(&mut self, src_row: usize, dst_row: usize, row_elems: usize)

Fold each registered row-feed’s new-token row into its resident handle slot, in-place on the arena (no host round-trip). Call after a logits-only run_read_outputs(.., Some(&[0])). row_elems is kv_dim.

pub fn read_gpu_handle(&self, name: &str) -> Option<Vec<f32>>

Read a handle back to host: from its fed output slot if wired, else the resident arena slot, else the host mirror. Used on bucket change / sync.

pub fn read_output_row( &self, out_idx: usize, row: usize, row_inner: usize, ) -> Option<Vec<f32>>

Read one row (row_inner f32 elements at row) from graph output out_idx, directly from the arena. Used by resident KV decode to pull just the new-token K/V row to the host cache (for bucket transitions) without a full-output readback.

pub fn set_rng(&mut self, rng: RngOptions)

pub fn rng(&self) -> RngOptions

pub fn run(&mut self, inputs: &[(&str, &[f32])]) -> Vec<Vec<f32>>

pub fn run_read_outputs( &mut self, inputs: &[(&str, &[f32])], read_indices: Option<&[usize]>, ) -> Vec<Vec<f32>>

pub fn clone_for_cache(&self) -> Self

Deep copy for clone_box: fresh arena/descriptors with the same params and constants already resident.

Trait Implementations

impl Drop for VulkanExecutable

fn drop(&mut self)

Executes the destructor for this type.

fn pin_drop(self: Pin<&mut Self>)

🔬This is a nightly-only experimental API. (pin_ergonomics)

Execute the destructor for this type, but different to Drop::drop, it requires self to be pinned.

impl Send for VulkanExecutable