Reproduce “An Automatically Created Novel Bug Dataset and its Validation in Bug Prediction” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Rudolf Ferenc, Péter Gyimesi, Gábor Gyimesi, Zoltán Tóth, Tibor Gyimóthy
Researchers from the University of Szeged developed the BugHunter Dataset, an automatically generated bug dataset that precisely captures "before-fix" and "after-fix" code states for specific bug-affected elements. This new dataset facilitates more accurate and trustworthy bug prediction models by overcoming the labeling uncertainty of traditional datasets, demonstrating that method-level metrics are more effective for predicting faulty classes.
Key insights
- The BugHunter Dataset provides a more precise and reliable representation of buggy code characteristics by capturing exact code states around bug fixes, directly addressing the inherent uncertainty found in traditional datasets.
- Fine-grained method-level metrics, when aggregated and projected to the class level, significantly outperform direct class-level metrics for predicting faulty classes, indicating the value of detailed code analysis.
- Higher F-measure values obtained from models trained on traditional datasets can be "deceptive" due to their imprecise bug labeling, suggesting that the BugHunter Dataset, despite potentially lower raw F-measures, yields more trustworthy results.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.