Reproduce “GPU Kernel Scientist: An LLM-Driven Framework for Iterative Kernel Optimization” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Martin Andrews, Sam Witteveen

The "GPU Kernel Scientist" framework, developed by Martin Andrews and Sam Witteveen, introduces an LLM-driven methodology for iteratively optimizing GPU kernels, particularly for architectures with limited documentation like the AMD MI300. This system achieved an approximate execution time of 450 μs for a target kernel, a substantial optimization from the 5000 μs of a naive implementation and about half the time of the PyTorch reference.

Key insights

  • LLMs can effectively operate under severe constraints, compensating for limited documentation and the absence of fine-grained profiling tools by inferring performance impacts from end-to-end timing results.
  • The framework demonstrates autonomous discovery and iterative refinement, allowing the system to learn about target architectures and generate optimization strategies through active experimentation without requiring deep human expertise.
  • The LLM models proved robust in generating syntactically correct and structurally significant HIP code, with minimal issues like "hallucinations" due to carefully controlled contexts.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.