Reproduce “Robustness of Large Language Models Against Adversarial Attacks” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yiyi Tao, Yixian Shen, Hang Zhang, Yanxin Shen, Lun Wang, Chuanqi Shi, Shaoshuai Du
This study evaluates the robustness of GPT models against character-level text attacks and prompt-based adversarial (jailbreak) attacks. It reveals that all models experience substantial performance degradation with minor character perturbations, and while newer GPT models exhibit enhanced resilience to jailbreak attempts, they are not completely immune to bypassing their safety mechanisms.
Key insights
- All evaluated GPT models are vulnerable to character-level text attacks, with even minor perturbations causing significant performance drops in sentiment classification tasks.
- Newer GPT models (GPT-4o, GPT-4-turbo, GPT-4) demonstrate considerably improved robustness against jailbreak prompts compared to older generations like GPT-3.5-turbo.
- Despite generational improvements, no GPT model achieved perfect detection of jailbreak prompts, indicating persistent vulnerabilities in current safety mechanisms.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.