Reproduce “Formalizing Forgetting: Theory of Forgetting for Online Continual Learning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Renjie Cao, Yifei Cheng, Li Shen
Authors derive bounds on forgetting in single-pass Shuffle SGD without replay. The bounds relate data per task, distances between task optima, and task order, under convex objectives. In MLP experiments, PMNIST accuracy fell from 72.52% to 57.89% as task data rose from 1,500 to 5,000, alongside BWT falling from 2.98% to −15.56%; this does not establish bound tightness.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.