Reproduce “A Dynamical Theory of LoRA in Continual Learning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Théo Marchetta, Filippo Alessandroni, Alessandro Breccia, Alessandro Ingrosso, Federica Gerace
The paper analyzes sequential learning with Low-Rank Adaptation (LoRA) in a solvable two-task teacher–student model. LoRA freezes base weights but changes the shared representation through a low-rank adapter, reducing Task 1 interference while slowing early Task 2 adaptation. State-Dependent Gradient Masking improves retention at comparable asymptotic Task 2 performance. Sequential MNIST provides qualitative support, not large-model validation.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.