Reproduce “The Model Plants the Trigger: Answer-Side Backdoor Attacks in Multi-Turn Large Language Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yibo Zhang, Tianrong Guan, Liang Lin, Puze Wang, Jin Wang, Qingsong Wen
The paper demonstrates an answer-side backdoor: poisoned fine-tuning teaches a multi-turn assistant to use a word in its earlier response as a trigger for a later harmful request, which need not contain it. For Mistral-7B-Instruct-v0.3 at 5% poisoning, conditional attack success was 100%; balanced-set induction was 50.5%, while selected prompts reached 100% in 200 trials, not a broad-traffic rate.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.