Reproduce “Smoothed Embeddings for Robust Language Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Ryo Hase, Md Rafi Ur Rashid, Ashley Lewis, Jing Liu, Toshiaki Koike-Akino, Kieran Parsons, Ye Wang
Researchers at Mitsubishi Electric Corporation and academic institutions developed Randomized Embedding Smoothing and Token Aggregation (RESTA), an inference-time defense for Large Language Models against jailbreaking attacks. This method significantly reduces attack success rates (e.g., from 94% to 2% against GCG on Vicuna-13B) while maintaining higher utility on benign tasks compared to existing character-level perturbation defenses.
Key insights
- Applying noise directly to embedding vectors, rather than character sequences, improves the robustness-utility trade-off by better preserving semantic information while disrupting adversarial patterns.
- Token-level aggregation through majority voting across multiple noisy generations at each step provides a more stable and robust output during autoregressive decoding.
- The "response prefix smoothing" technique is effective in reducing computational cost by focusing the intensive defense mechanism only on the critical initial tokens of the LLM's response.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.