Reproduce “RobustKV: Defending Large Language Models against Jailbreak Attacks via KV Eviction” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Tanqiu Jiang, Zian Wang, Jiacheng Liang, Changjiang Li, Yuhui Wang, Ting Wang
RobustKV defends language models against jailbreaks by ranking prompt tokens according to attention-derived importance and evicting the lowest-ranked tokens’ key–value cache representations before generation. The method aims to weaken the concealed query while retaining the wrapper. On AutoDAN against Llama-2-Chat-7B, harmful-response rate fell to 6.3%, from 61.5% without defense and 58.3% with SnapKV; results cover selected attacks and models.
Key insights
- Jailbreak prompts must allocate attention importance to their adversarial components, which often results in the concealed harmful query tokens having relatively lower importance.
- Harmful query tokens consistently rank lower in importance compared to the jailbreak prompt tokens across various attack samples.
- By selectively removing the key-value pairs associated with these least important tokens, the defense can diminish the harmful query's influence without largely affecting legitimate LLM responses.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.