Reproduce “Rethinking Kernel Program Repair: Benchmarking and Enhancing LLMs with RGym” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Kareem Shehada, Yifan Wu, Wyatt D. Feng, Adithya Iyer, Gryphon Kumfert, Yangruibo Ding, Zhiyun Qian
This research introduces RGym, a lightweight and platform-agnostic framework for Linux kernel program repair, coupled with an enhanced automated repair pipeline. The system achieves a 43.36% patch success rate using GPT-5 Thinking at a cost of $0.18 per bug, demonstrating improved effectiveness and affordability in addressing complex kernel-space vulnerabilities.
Key insights
- Structured patching, specifically 'function-wise patching,' significantly reduces unappliable diffs from LLMs, improving kernel patch success rates from 2.8% to 10.49% for oracle-guided KGym.
- Practical localization techniques, such as Bug-Inducing Commits (BICs) and call stacks, enhance APR performance, with BIC integration adding a 4.19% improvement to the 'Simple Agent' pass rate.
- Modern LLMs like GPT-5 Thinking offer superior repair capabilities and cost-efficiency, achieving a 43.36% pass rate at $0.18 per bug, substantially outperforming other models and prior complex solutions.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.