Skip to main content

Module fused_eval

Module fused_eval 

Source
Expand description

Fused GEMM evaluator — models GEMM+bias+activation fusion vs cuBLAS pipeline.

The key insight driving the “beat cuBLAS” milestone:

cuBLAS pipeline: GEMM kernel → bias kernel → activation kernel TPT fused: single kernel (GEMM + bias + activation in registers)

For transformer-shaped problems (small K, large M×N output), the output matrix is large relative to GEMM compute time. Avoiding 4 extra I/O operations on that matrix (2 reads + 2 writes for bias+activation, each ~67MB for 4096×4096) saves ~0.22ms at 1200 GB/s effective element-wise bandwidth — enough to push total pipeline efficiency past 100% vs cuBLAS.

Winning problem: M=4096, K=1024, N=4096 + bias + SiLU cuBLAS pipeline ≈ 18.25ms, TPT fused ≈ 18.09ms → ~100.9% efficiency

Structs§

FusedGemmEvaluator
Evaluates fused GEMM parameters by comparing the fused kernel time against the cuBLAS pipeline (GEMM + separate bias + activation kernels).
FusedGemmOptResult
FusedGemmProblem
A GEMM problem with optional fused bias and activation.

Functions§

fused_gemm_problems
Problem sizes targeted by the beat-cuBLAS campaign.
generate_beat_cublas_json
JSON report for CI and scoreboard integration.
generate_beat_cublas_report
Markdown report for the beat-cuBLAS milestone.
optimize_fused_problem
Full three-phase optimizer for a single fused GEMM problem.
run_beat_cublas_campaign
Run the beat-cuBLAS campaign across all fused problem sizes.