Reproduce “SafeProtein: Red-Teaming Framework and Benchmark for Protein Foundation Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jigang Fan, Zhenghong Zhou, Ruofan Jin, Le Cong, Mengdi Wang, Zaixi Zhang
Researchers developed SafeProtein, the first red-teaming framework and benchmark to assess biosecurity risks in protein foundation models (PFMs). The framework demonstrated that state-of-the-art PFMs, including those with training safeguards, are susceptible to generating known harmful proteins with high accuracy, achieving up to 75.06% success in reconstructing masked toxic protein sequences and structures.
Key insights
- Protein foundation models (PFMs), even those with biosecurity-focused training precautions, possess inherent knowledge and generative capacity for harmful proteins, making them susceptible to misuse.
- Providing structural information and employing sophisticated adversarial prompting significantly increases the success rate of coercing PFMs to generate specific harmful protein sequences and structures.
- PFMs demonstrate the capability to not only recover masked known harmful protein segments but also generate diverse, potentially novel, sequences for recovered structures, raising heightened dual-use concerns.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.