Reproduce “Attribution-Guided Continual Learning for Large Language Models” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Yazheng Liu, Yuxuan Wan, Rui Xu, Xi Zhang, Sihong Xie, Hui Xiong

AGFT adapts models to tasks in sequence while limiting changes to parameters important for earlier tasks. It estimates parameter relevance from predictions, then scales later-task updates so highly relevant parameters change less. A Llama-3.2-3B-Instruct LoRA experiment on Concept-1K reports an average score of 16.35 across 10 tasks, versus 13.37 for whole-layer freezing; these scores do not prove how forgetting is prevented.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.