Reproduce “How to Better Distinguish Security Bug Reports (using Dual Hyperparameter Optimization” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Rui Shu, Tianpei Xia, Jianfeng Chen, Laurie Williams, Tim Menzies
SWIFT jointly tunes data preprocessing and classifier settings to identify security-related bug reports, using a search strategy designed to avoid spending effort on configurations with similar performance. Across five projects, median recall increased from 21.5% with FARSEC to 66.7%, and report ranking improved in all five; these gains came with a higher median false-positive rate, rising from 8% to 24%.
Key insights
- Jointly optimizing preprocessing and the learner can improve security-report recall more consistently than optimizing either component alone.
- Higher recall entails a practical trade-off: more security reports are found, but more non-security reports are also flagged for review.
- Optimization can improve prioritization as well as classification: SWIFT’s ranking placed useful reports earlier across all five projects in the reported MAP comparisons.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.