Reproduce “PRAGMA: A Profiling-Reasoned Multi-Agent Framework for Automatic Kernel Optimization” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Kelun Lei, Hailong Yang, Huaitao Zhang, Xin You, Kaige Zhang, Zhongzhi Luan, Yi Liu, Depei Qian

PRAGMA, a multi-agent framework from Beihang University, optimizes computational kernels by integrating fine-grained, hardware-aware profiling feedback into large language model-based code generation. The system achieved substantial speedups, including 2.81x against Torch on CPU and 2.30x on GPU, alongside a 92.0% success rate.

Key insights

  • Integrating fine-grained, hardware-aware profiling feedback is crucial for enabling LLMs to effectively diagnose and address low-level performance bottlenecks in generated code.
  • A multi-agent architecture with specialized roles and iterative refinement allows LLMs to systematically improve kernel performance and maintain stable optimization progress.
  • Automated consolidation of profiling tool documentation helps bridge the knowledge gap, making complex hardware metrics interpretable and actionable for LLM agents across different platforms.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.