Reproduce “Perturbation-Restrained Sequential Model Editing” on OpenResearch

About this paper

Jun-Yu Ma, Hong Wang, Hao-Xiang Xu, Zhen-Hua Ling, Jia-Chen Gu

Researchers at the University of Science and Technology of China provided a theoretical analysis linking the degradation of Large Language Models' general abilities during sequential editing to the increasing condition number of the edited model matrix, and introduced PRUNE, a framework that restrains singular values to mitigate this decline while preserving new knowledge. PRUNE allows edited models to maintain general ability performance close to unedited baselines and reduces forgetting of new facts across various LLMs and tasks after numerous sequential updates.

Key insights

• The condition number of the edited MLP layer matrix in LLMs rapidly increases with sequential edits, acting as a bottleneck for preserving general abilities.
• This escalating condition number leads to increased numerical sensitivity and greater perturbation of original knowledge associations.
• Restraining the large singular values of the cumulative edit update matrix can effectively lower the condition number of the final edited model, preserving both general abilities and newly acquired knowledge.

Summary by alphaXiv.