Reproduce “Certifying LLM Safety against Adversarial Prompting” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Aounon Kumar, Chirag Agarwal, Suraj Srinivas, Aaron Jiaxun Li, Soheil Feizi, Himabindu Lakkaraju
Researchers from Harvard University and the University of Maryland developed "erase-and-check," the first framework to provide certifiable safety guarantees for large language models against adversarial prompting. This method achieves 100% certified accuracy on harmful prompts with a fine-tuned DistilBERT safety filter and offers efficient empirical approximations for practical deployment.
Key insights
- The framework provides mathematical guarantees that if a clean harmful prompt is detected, then any adversarial modification of it (up to a certain size) will also be detected, with the safety filter's clean accuracy serving as a lower bound for certified accuracy.
- Training the DistilBERT safety filter with erased subsequences of safe prompts improves its robustness and reduces misclassifications of benign inputs.
- Efficient empirical variants (RandEC, GreedyEC, GradEC) inspired by erase-and-check offer a practical trade-off between speed and performance for scenarios where full certification is computationally prohibitive.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.