Reproduce “MLIRBench - A Benchmark and Dataset for Compiler and Hardware-Aware Reasoning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Pranava Sai Maganti, Meghasyam Peddireddy, Muzakker Hossain, Kaining Gao, Farhan Ahmed, Ali Jannesari
MLIRBench tests whether models can classify pairs of compiler intermediate-representation programs as equivalent, divergent, or erroneous, and predict their behavior. It executes mutated PyTorch programs lowered into torch and linalg, using fixed inputs as ground truth. Gemini 3.1 Pro reached 92.65% label accuracy but a 5.04/10 behavior score; labels apply only to the benchmark setup.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.