Reproduce “EvoEngineer: Mastering Automated CUDA Kernel Code Evolution with Large Language Models” on OpenResearch

About this paper

Ping Guo, Chenyu Zhu, Siyuan Chen, Fei Liu, Xi Lin, Zhichao Lu, Qingfu Zhang

EvoEngineer, a systematic framework from City University of Hong Kong, automates CUDA kernel optimization using Large Language Models by formalizing the process into distinct traverse techniques and population management. It achieved an average median speedup of 2.72x over baseline kernels and a 69.8% functional correctness rate, outperforming existing LLM-based methods.

Key insights

• Decomposing LLM-based code evolution into orthogonal traverse techniques and population management enables systematic analysis, design, and comparison of optimization strategies, moving beyond ad-hoc prompting.
• Leveraging different types of 'closed-world information' (task context, historical solutions, optimization insights) provides configurable trade-offs between LLM token usage, kernel performance, and code correctness.
• The framework demonstrates that a principled approach can effectively balance performance gains with strict correctness requirements in performance-critical domains like CUDA kernel optimization.

Summary by alphaXiv.