Reproduce “Refine Large Language Model Fine-tuning via Instruction Vector” on OpenResearch

About this paper

Gangwei Jiang, Zhaoyi Li, Defu Lian, Ying Wei

An Instruction Vector (IV) framework is introduced to interpret and mitigate catastrophic forgetting in large language models during fine-tuning. The research demonstrates that performance degradation primarily stems from the suppression of instruction-following capabilities, rather than the erasure of world knowledge, and proposes an IV-guided training method to preserve general and reasoning skills.

Key insights

• Catastrophic forgetting in LLMs is primarily driven by the degradation and suppression of instruction-following capabilities, rather than the erasure of world knowledge.
• Fine-tuning often introduces specialized reasoning patterns that overshadow existing skills, meaning original knowledge is suppressed but largely recoverable, rather than permanently lost.
• Task-specific Instruction Vectors (IVs) can be causally extracted and manipulated to significantly influence an LLM's ability to follow specific instructions.

Summary by alphaXiv.