AI Agents Beat PyTorch: Writing Faster CUDA Kernels

AI agents can generate custom CUDA kernels that outperform standard PyTorch implementations, but the primary challenge lies in designing accurate benchmarks to validate these performance gains. The author demonstrates that an AI-generated kernel for matrix multiplication can achieve a 1.57x speedup over the highly optimized `torch.compile` baseline. This experiment underscores the importance of choosing the correct baseline, as comparing against the slower "eager mode" can produce misleadingly high speedup figures. Ultimately, the process of building a trustworthy evaluation framework proved more difficult than generating the optimized code itself.