Reproduce “Poison Forensics: Traceback of Data Poisoning Attacks in Neural Networks” on OpenResearch

About this paper

Shawn Shan, Arjun Nitin Bhagoji, Haitao Zheng, Ben Y. Zhao

Researchers at the University of Chicago developed "Poison Forensics," a system to trace back data poisoning attacks in deep neural networks by identifying the specific malicious training data responsible for a model's compromised behavior. The system achieved consistently high precision (99.1-99.8%) and recall (97.1-99.9%) across various dirty-label attacks, maintained strong performance against clean-label attacks, and demonstrated robustness against adaptive anti-forensic countermeasures.

Key insights

• Mapping training data into a projection space based on its gradient impact with a NULL label effectively separates benign and poisoned data, enabling efficient and accurate clustering.
• Functional unlearning provides a computationally feasible alternative to full model retraining, allowing for the rapid evaluation of the causal effect of removing data clusters on a model's behavior.
• An iterative clustering and pruning approach can robustly and precisely narrow down a large training dataset to identify specific poisoned samples, even when facing sophisticated clean-label attacks and adaptive anti-forensic techniques.

Summary by alphaXiv.