Module ops 

The compute operations, each implemented on BOTH the GPU and the CPU.

Every op here implements crate::ComputeOp (a GPU kernel + a pure-Rust twin) and is runnable through the crate::Executor cascade.

• vector_add — the demonstrator that proves the cascade end to end.
• matmul_nt — y = x @ w^T, the shape a transformer linear layer actually performs, and where a decoder-only model spends nearly all of its time. Read that module’s warning before assuming it makes anything faster: it re-uploads the weight on every call, and making GPU offload pay needs the weight resident on the device across calls.

Re-exports

pub use block_q8_matmul_nt::block_q8_matmul_nt_cpu;

pub use block_q8_matmul_nt::ResidentBlockQ8Weight;

pub use block_q8_matmul_nt::BLOCK;

pub use matmul_nt::matmul_nt_cpu;

pub use matmul_nt::matmul_nt_gpu;

pub use matmul_nt::MatmulNt;

pub use matmul_nt::MatmulNtInput;

pub use vector_add::vector_add_cpu;

pub use vector_add::vector_add_gpu;

pub use vector_add::VectorAdd;

pub use vector_add::VectorAddInput;

Modules

block_q8_matmul_nt

y = x @ w^T with w block-scaled int8 and resident on the device.

matmul_nt

y = x @ w^T — the matmul a transformer linear layer actually performs.

vector_add

Elementwise vector add — the demonstrator op that proves the cascade.