Reproduce “Learning from the Undesirable: Robust Adaptation of Language Models without Forgetting” on OpenResearch

About this paper

Yunhun Nam, Jaehyung Kim, Jongheon Jeong

Researchers from Korea and Yonsei Universities developed Learning-from-the-Undesirable (LfU), a regularization scheme for language model fine-tuning that simulates "undesirable" updates to internal representations. LfU improves model generalization and robustness, demonstrating up to a 45.0% reduction in attack success rate against adversarial fine-tuning and 92.1% lower standard deviation to prompt variations.

Key insights

• Simulating an "undesirable" optimization step at the representation level, rather than just input or output, effectively exposes the model to diverse internal states and improves generalization.
• Enforcing consistency between normal and perturbed internal representations helps LMs adapt to specific tasks without significantly forgetting pre-trained knowledge.
• Regularizing representations across all model layers is crucial for achieving comprehensive robustness and preventing overfitting effectively.

Summary by alphaXiv.