Reproduce “Towards Characterizing Adversarial Defects of Deep Learning Software from the Lens of Uncertainty” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Xiyue Zhang, Xiaofei Xie, Lei Ma, Xiaoning Du, Qiang Hu, Yang Liu, Jianjun Zhao, Meng Sun
Researchers from Peking University, Nanyang Technological University, and Kyushu University systematically characterized deep learning software defects by analyzing the uncertainty inherent in model decisions. Their work identified that existing adversarial attacks and testing methods predominantly generate samples exhibiting common uncertainty patterns, leaving other "uncommon" patterns largely unexplored. They developed a genetic algorithm-based tool, KuK, to generate these uncommon inputs, revealing that state-of-the-art defense techniques often fail against these new types of adversarial examples, with success rates dropping from over 90% to as low as 5.5% in some cases.
Key insights
- Prediction Confidence Score (PCS) and Variation Ratio for Original Prediction (VRO) are highly effective metrics for distinguishing between adversarial and benign examples, with AUC-ROC scores often exceeding 97%.
- Current state-of-the-art adversarial attack and testing techniques primarily generate adversarial examples with low confidence and high uncertainty (LH pattern) and benign examples with high confidence and low uncertainty (HL pattern).
- A significant portion of the input space representing "uncommon" uncertainty patterns (e.g., high confidence/high uncertainty or low confidence/low uncertainty) remains largely unexplored by existing methods.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.