Reproduce “Backdoor Attacks on Pre-trained Models by Layerwise Weight Poisoning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Linyang Li, Demin Song, Xiaonan Li, Jiehang Zeng, Ruotian Ma, Xipeng Qiu
Researchers from Fudan University developed a Layer Weight Poisoning attack using Combinatorial Triggers (LWP(CT)) to embed backdoors in pre-trained models, making them significantly more resilient to fine-tuning and practically undetectable. The method leverages the observation that fine-tuning primarily impacts higher layers, successfully burying malicious behavior in lower layers, and maintaining high attack effectiveness (e.g., 56.5% LFR at higher fine-tuning learning rates on SST-2) while preserving clean model performance.
Key insights
- Fine-tuning primarily affects higher layers of deep transformer models, while lower layers retain more stable, general knowledge.
- Embedding backdoor functionality within these stable lower layers makes the malicious behavior significantly more resilient to erasure during subsequent fine-tuning.
- Using combinatorial triggers vastly expands the search space for detection, rendering existing single-token trigger defense mechanisms ineffective.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.