Skip to main content

BackendGraph

ferrum_kernels::backend

Trait BackendGraph 

Source 
pub trait BackendGraph: Backend {
    // Provided methods
    fn set_decode_state(_ctx: &mut Self::Context, _token: u32, _step: u32) { ... }
    fn set_dev_state_mode(_ctx: &mut Self::Context, _enable: bool) { ... }
    fn begin_graph_capture(_ctx: &mut Self::Context) -> Result<()> { ... }
    fn end_graph_capture(_ctx: &mut Self::Context, _key: u64) -> Result<()> { ... }
    fn replay_graph(_ctx: &mut Self::Context, _key: u64) -> Result<bool> { ... }
    fn reset_graph(_ctx: &mut Self::Context, _key: u64) { ... }
    fn reset_all_graphs(_ctx: &mut Self::Context) { ... }
}
Expand description

Capability-trait for backends that can capture and replay execution as a graph (CUDA Graph). Models that call these methods bound their generic on B: BackendGraph; backends without graph support (Metal, CPU) impl this trait with an empty body and inherit no-op / unsupported defaults.

Provided Methods§

Source

fn set_decode_state(_ctx: &mut Self::Context, _token: u32, _step: u32)

Update per-step dynamic state (token id, step/pos). Fast (3x memcpy).

Source

fn set_dev_state_mode(_ctx: &mut Self::Context, _enable: bool)

Toggle between scalar-arg kernels (normal) and _dyn kernels that read their dynamic scalar args from device memory (graph-friendly).

Source

fn begin_graph_capture(_ctx: &mut Self::Context) -> Result<()>

Begin stream capture. Subsequent kernel launches are recorded into a pending graph instead of executing eagerly.

Source

fn end_graph_capture(_ctx: &mut Self::Context, _key: u64) -> Result<()>

End stream capture and install the captured graph keyed by _key (opaque caller-chosen u64; the model uses m_padded so that different batch shapes don’t thrash a single slot).

Source

fn replay_graph(_ctx: &mut Self::Context, _key: u64) -> Result<bool>

Replay the captured graph for _key. Returns Ok(false) if no graph is cached for that key; caller should run eager.

Source

fn reset_graph(_ctx: &mut Self::Context, _key: u64)

Drop the cached graph for _key — required when its kernel-arg pointers (KV cache, scratch) might no longer be valid. Use reset_all_graphs when EVERY cached graph should be evicted (hard model reload / scratch realloc).

Source

fn reset_all_graphs(_ctx: &mut Self::Context)

Drop ALL cached graphs — used by hard reset paths.

Dyn Compatibility§

This trait is not dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementors§