Struct EncKernels 

pub struct EncKernels { /* private fields */ }

The compiled encoder kernel set. Built once per device; the f16 GEMM twin exists only when the adapter reports SHADER_F16.

Implementations

impl EncKernels

pub fn new(ctx: &GpuCtx) -> Self

Compile all encoder pipelines on ctx.

pub fn rope_for_tests( &self, ctx: &GpuCtx, x: &[f32], seq_starts: &[u32], n_heads: usize, hd: usize, theta: f32, ) -> Result<Vec<f32>>

One-shot rope for the kernel parity tests: rotate a [T, nh·hd] buffer in place.

pub fn glu_for_tests( &self, ctx: &GpuCtx, mid: &[f32], i_width: usize, act: Act, ) -> Result<Vec<f32>>

One-shot GLU split for the kernel parity tests.

pub fn gemm_for_tests( &self, ctx: &GpuCtx, x: &[f32], w: &[f32], bias: Option<&[f32]>, m: usize, n: usize, k: usize, act: Option<Act>, w_f16: bool, ) -> Result<Vec<f32>>

One-shot GEMM (upload → dispatch → read back) for the kernel parity tests. w arrives as f32 and is converted to f16 on upload when w_f16 — the caller’s CPU reference must consume the same round-tripped values.

pub fn gemm_resident( &self, ctx: &GpuCtx, x: &[f32], wb: &Buffer, bb: &Buffer, m: usize, n: usize, k: usize, act: Option<Act>, ) -> Result<Vec<f32>>

GEMM against weights that are ALREADY on the device.

gemm_for_tests uploads w every call; at gliner2’s span_rep shapes that is ~9 MB of weights per GEMM and it dominates — measured, the one-shot runs 0.0164 s where the kernel itself sustains 0.0021 s (tests/gemm_probe.rs). Hoisting the weights out is the first half of closing that gap; the second half (keeping the ACTIVATIONS resident too) needs the whole chain on device and is tracked separately.

wb/bb are caller-owned and expected to live across calls — see Linear::gpu_weights.

pub fn gemm_resident_into( &self, ctx: &GpuCtx, xb: &Buffer, wb: &Buffer, bb: &Buffer, yb: &Buffer, m: usize, n: usize, k: usize, act: Option<Act>, ) -> Result<()>

GEMM entirely in device buffers: x and y stay resident, nothing is read back.

gemm_resident still uploads the activations and reads the result; chaining several GEMMs through it therefore pays a full round-trip PER STEP. For gliner2’s span_rep the middle tensor (hbuf) is ~22.8 MB per call, which is exactly what capped the device port at 1.53x of the ~30x the kernel sustains. This variant is what lets the whole chain stay on-device.

yb must be at least m * n f32s.

pub fn gemm_bench( &self, ctx: &GpuCtx, m: usize, n: usize, k: usize, w_f16: bool, v2: bool, reps: usize, ) -> Result<f64>

Steady-state GFLOP/s of the GEMM at one shape — the STOP INSTRUMENT for kernel tuning.

Weights and activations upload once; reps dispatches then run in ONE submit, so the number is the kernel’s throughput and not the readback, the allocation, or the submit. Min of the timed reps (a mean on a shared box measures the contention, not the code).

pub fn gemm3_bench( &self, ctx: &GpuCtx, m: usize, n: usize, k: usize, bm: usize, bn: usize, bk: usize, reps: usize, ) -> Result<f64>

[gemm_bench]’s v3 twin — the STOP instrument for deciding whether the vec4/transposed kernel earns a place in the TEXT-encoder plan (its wins were proven at vision shapes, m≈912; the text shapes are m∈{64,192} where occupancy, not instruction count, may rule). The pipeline is created ad hoc (a lab probe, not a plan resident); weight contents are irrelevant to throughput so zeros stand in for the [k, n] transposed upload.

pub fn gemm3_sk_bench( &self, ctx: &GpuCtx, m: usize, n: usize, k: usize, bm: usize, bn: usize, bk: usize, chunks: u32, reps: usize, ) -> Result<f64>

[gemm3_bench]’s split-K twin: the partial+reduce PAIR, timed as the plan would run it (both dispatches in one pass, in-pass ordering supplying the dependency). The lab probe for whether the k-dimension can rescue thin-grid shapes OUTSIDE the wired mid-band — e.g. /score’s n=384 bucket, whose best plain tile runs only 72 workgroups.

pub fn layernorm_for_tests( &self, ctx: &GpuCtx, x: &[f32], res: Option<&[f32]>, w: &[f32], b: Option<&[f32]>, t: usize, h: usize, eps: f32, ) -> Result<Vec<f32>>

One-shot LayerNorm(+residual)(+bias) for the kernel parity tests.

pub fn attention_for_tests( &self, ctx: &GpuCtx, q: &[f32], k: &[f32], v: &[f32], seq_starts: &[u32], n_heads: usize, n_kv_heads: usize, hd: usize, mask: MaskKind, window: u32, ) -> Result<Vec<f32>>

One-shot ragged attention for the kernel parity tests.

pub fn mean_pool_l2_for_tests( &self, ctx: &GpuCtx, hidden: &[f32], h: usize, seq_starts: &[u32], ) -> Result<Vec<Vec<f32>>>

One-shot mean-pool + L2-normalize for the kernel parity tests.