Reproduce “A Quantitative and Qualitative Evaluation of LLM-Based Explainable Fault Localization” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Sungmin Kang, Gabin An, Shin Yoo
AutoFL, an LLM-based explainable fault localization technique, enables large language models to autonomously navigate codebases using function calls to generate both bug locations and human-readable explanations. The approach outperforms traditional fault localization methods and provides a confidence metric for its predictions.
Key insights
- Integrating LLM function calling is an effective strategy to overcome context window limitations, enabling LLMs to perform whole-program analysis for debugging.
- The demand for human-understandable rationales is critical for the practical adoption of automated fault localization tools by developers.
- LLM-generated explanations, even with occasional inaccuracies, can be highly valued by developers, particularly when coupled with clear structure and confidence indicators.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.