Crate memra_runtime 

memra inference runtime. Correctness-first: every GPU op is validated against a CPU reference before any sm_120 fast-path replaces it.

Re-exports

pub use memra_gguf;

Structs

Gpu

GPU runtime handle: a context + stream + cuBLASLt.

StreamOverride

RAII scope: while alive, Gpu::stream() on THIS thread returns the pushed stream. Nest freely (stack). Popping on Drop keeps panic paths consistent.

Functions

cpu_linear

CPU reference matmul for a linear layer y = x @ W^T. Conventions (ggml/GGUF): a weight tensor with ne=[in, out] is stored row-major as out rows of in contiguous elements — i.e. W[oin + i]. A linear layer computes y[o] = sum_i x[i] * W[oin + i], for each of out outputs. Batched over m tokens: x: [m, in] row-major (x[tin + i]); w: [out, in] row-major (w[oin + i]); y: [m, out].

push_stream_override

Push s as the ambient stream for the current thread until the guard drops.