Reproduce “Large Language Models for Compiler Optimization” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Chris Cummins, Volker Seeker, Dejan Grubisic, Mostafa Elhoushi, Youwei Liang, Baptiste Roziere, Jonas Gehring, Fabian Gloeckle and 3 more
The authors train a 7B-parameter transformer from scratch to predict compiler pass lists from LLVM IR, rather than searching candidate sequences at deployment. Its training targets also include instruction counts and optimized IR. On 100,000 unseen functions, its predicted lists produce 3.01% overall instruction-count improvement over LLVM’s -Oz ordering, with no additional candidate-search compilations. This measures instruction counts, not runtime or binary bytes.
Key insights
- Large Language Models can learn to predict effective compiler optimization pass sequences and accurately emulate compiler-applied transformations.
- Training an LLM with auxiliary tasks like predicting instruction counts and generating optimized code significantly improves its understanding and performance in code optimization.
- The LLM's optimization potential increases with program size, yielding greater improvements for larger functions (>40 instructions).
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.