Struct Context 

pub struct Context {
    pub device: Device,
    pub queue: Queue,
    pub backend: Backend,
    pub adapter_name: String,
}

A compute context bound to one GPU adapter on whatever fabric is available.

Fields

device: Device

queue: Queue

backend: Backend

adapter_name: String

Implementations

impl Context

On-GPU tensor ops — the L3 graph-executor primitives. Each dispatches and returns a Tensor whose buffer stays on the device; nothing reads back until to_vec, so a whole model runs on-GPU.

pub fn tensor(&self, data: &[f32], shape: &[usize]) -> Tensor

Upload host data to a GPU tensor.

pub fn empty(&self, shape: &[usize]) -> Tensor

Allocate an empty (device-only) tensor.

pub async fn to_vec(&self, t: &Tensor) -> Result<Vec<f32>>

Read a tensor back to host (the only sync point).

pub fn mm(&self, a: &Tensor, b: &Tensor, m: u32, k: u32, n: u32) -> Tensor

C = A(m×k)·B(k×n), on-GPU.

pub fn mm_bt( &self, a: &Tensor, b: &Tensor, m: u32, n: u32, k: u32, scale: f32, ) -> Tensor

C = scale·(A(m×k)·B(n×k)ᵀ), on-GPU.

pub fn silu_t(&self, x: &Tensor) -> Tensor

pub fn dup(&self, x: &Tensor, shape: Vec<usize>) -> Tensor

Reinterpret a tensor’s data under a new shape (contiguous row-major) — Reshape/Cast/Squeeze.

pub fn gather0(&self, data: &Tensor, idx: &[u32], d: usize) -> Tensor

Row-gather along axis 0: out[i, :] = data[idx[i], :]. Embedding lookup / index_select.

pub fn sigmoid_t(&self, x: &Tensor) -> Tensor

pub fn sqrt_t(&self, x: &Tensor) -> Tensor

pub fn gelu_t(&self, x: &Tensor) -> Tensor

pub fn sub_t(&self, a: &Tensor, b: &Tensor) -> Tensor

pub fn div_t(&self, a: &Tensor, b: &Tensor) -> Tensor

pub fn mul_t(&self, a: &Tensor, b: &Tensor) -> Tensor

Elementwise C = A ⊙ B.

pub fn mul_scalar_t(&self, a: &Tensor, s: &Tensor) -> Tensor

C = A · scalar, where s is a 1-element tensor (broadcast).

pub fn transpose2d_t(&self, x: &Tensor, rows: u32, cols: u32) -> Tensor

2D transpose [rows×cols] → [cols×rows].

pub fn add_bias_t(&self, a: &Tensor, bias: &Tensor) -> Tensor

C = A + bias broadcast per row: out[i] = a[i] + bias[i % d]. (a is [.,d], bias is [d].)

pub fn relu_t(&self, x: &Tensor) -> Tensor

pub fn add_t(&self, a: &Tensor, b: &Tensor) -> Tensor

pub fn softmax_t(&self, x: &Tensor, rows: u32, d: u32) -> Tensor

pub fn rope_t(&self, x: &Tensor, t: u32, h: u32, dh: u32, base: f32) -> Tensor

Rotary position embedding (rotate-half, NeoX/Llama style). x is [T, H·dh]; rotates each head’s dh-vector by position-dependent angles. Applied to Q and K before attention.

pub fn rope_off_t( &self, x: &Tensor, t: u32, h: u32, dh: u32, base: f32, offset: u32, ) -> Tensor

RoPE with an absolute-position offset: row i is rotated for position (i + offset). Prefill uses offset 0; incremental decode of the token at position pos uses a 1-row input with offset=pos.

pub fn mha_decode_t( &self, q: &Tensor, k: &Tensor, v: &Tensor, hq: u32, hkv: u32, dh: u32, s: u32, ) -> Tensor

Attention for one query token against a KV cache of s past keys/values (incremental decode). q is [1, H·dh]; k/v are [s, H·dh]; no mask (all cached keys precede the query). Returns [1, H·dh].

pub fn mha_causal_t( &self, q: &Tensor, k: &Tensor, v: &Tensor, t: u32, hq: u32, hkv: u32, dh: u32, ) -> Tensor

Causal multi-head attention, single flash-style pass, with grouped-query attention (GQA): Q has hq heads, K/V have hkv heads (hq % hkv == 0); query head h reads kv head h/(hq/hkv). q is [T, hq·dh]; k/v are [T, hkv·dh]; scale = 1/√dh; query i attends to keys 0..=i. Returns [T, hq·dh].

pub fn rmsnorm_t( &self, x: &Tensor, w: &Tensor, rows: u32, d: u32, eps: f32, ) -> Tensor

RMSNorm (Llama/SmolVLA norm): out = x / sqrt(mean(x²)+eps) · weight. No mean-subtraction, no bias.

pub fn layernorm_t( &self, x: &Tensor, w: &Tensor, b: &Tensor, rows: u32, d: u32, eps: f32, ) -> Tensor

pub fn attention_t( &self, q: &Tensor, k: &Tensor, v: &Tensor, rq: u32, rk: u32, d: u32, dv: u32, scale: f32, ) -> Tensor

Single-head attention on-GPU: softmax(scale·Q·Kᵀ)·V, all buffers stay on device.

impl Context

pub async fn add(&self, a: &[f32], b: &[f32]) -> Result<Vec<f32>>

Elementwise C = A + B (f32).

pub async fn silu(&self, x: &[f32]) -> Result<Vec<f32>>

SiLU / swish: x · sigmoid(x).

pub async fn layernorm( &self, x: &[f32], weight: &[f32], bias: &[f32], rows: u32, d: u32, eps: f32, ) -> Result<Vec<f32>>

LayerNorm over the last dim d for rows rows: (x−μ)/√(σ²+eps) · weight + bias.

pub async fn softmax(&self, x: &[f32], rows: u32, d: u32) -> Result<Vec<f32>>

Row-wise softmax over the last dim d for rows rows (numerically stable).

pub async fn matmul_bt( &self, a: &[f32], b: &[f32], m: u32, n: u32, k: u32, scale: f32, ) -> Result<Vec<f32>>

C = scale · (A(m×k) · B(n×k)ᵀ). Row-major; B is [n×k] (not transposed in memory). This is the Q·Kᵀ shape for attention, with the 1/√d scale folded in.

pub async fn attention( &self, q: &[f32], k: &[f32], v: &[f32], rows_q: u32, rows_k: u32, d: u32, dv: u32, scale: f32, ) -> Result<Vec<f32>>

Single-head scaled dot-product attention: softmax(scale · Q·Kᵀ) · V. Q[rows_q×d], K[rows_k×d], V[rows_k×dv] → [rows_q×dv]. Composed from matmul_bt + softmax + matmul.

impl Context

pub async fn new() -> Result<Self>

Acquire the best available compute device on this fabric (native GPU or browser WebGPU).

pub async fn enumerate() -> Vec<(String, Backend, DeviceType)>

Enumerate EVERY compute adapter present (all GPUs across all backends + software/CPU adapters), so the scheduler can use all of them, not just one. Returns (name, backend, device_type).

pub async fn for_adapter(idx: usize) -> Result<Self>

Build a Context on a specific enumerated adapter (by index into enumerate()), so each GPU in the machine can be its own device in the fabric.

pub async fn matmul( &self, a: &[f32], b: &[f32], m: u32, k: u32, n: u32, ) -> Result<Vec<f32>>

C = A(m×k) · B(k×n), row-major, f32. One kernel, any fabric.