Expand description
Fused GEMM evaluator — models GEMM+bias+activation fusion vs cuBLAS pipeline.
The key insight driving the “beat cuBLAS” milestone:
cuBLAS pipeline: GEMM kernel → bias kernel → activation kernel TPT fused: single kernel (GEMM + bias + activation in registers)
For transformer-shaped problems (small K, large M×N output), the output matrix is large relative to GEMM compute time. Avoiding 4 extra I/O operations on that matrix (2 reads + 2 writes for bias+activation, each ~67MB for 4096×4096) saves ~0.22ms at 1200 GB/s effective element-wise bandwidth — enough to push total pipeline efficiency past 100% vs cuBLAS.
Winning problem: M=4096, K=1024, N=4096 + bias + SiLU cuBLAS pipeline ≈ 18.25ms, TPT fused ≈ 18.09ms → ~100.9% efficiency
Structs§
- Fused
Gemm Evaluator - Evaluates fused GEMM parameters by comparing the fused kernel time against the cuBLAS pipeline (GEMM + separate bias + activation kernels).
- Fused
Gemm OptResult - Fused
Gemm Problem - A GEMM problem with optional fused bias and activation.
Functions§
- fused_
gemm_ problems - Problem sizes targeted by the beat-cuBLAS campaign.
- generate_
beat_ cublas_ json - JSON report for CI and scoreboard integration.
- generate_
beat_ cublas_ report - Markdown report for the beat-cuBLAS milestone.
- optimize_
fused_ problem - Full three-phase optimizer for a single fused GEMM problem.
- run_
beat_ cublas_ campaign - Run the beat-cuBLAS campaign across all fused problem sizes.