Reproduce “Formalizing Forgetting: Theory of Forgetting for Online Continual Learning” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Renjie Cao, Yifei Cheng, Li Shen

Authors derive bounds on forgetting in single-pass Shuffle SGD without replay. The bounds relate data per task, distances between task optima, and task order, under convex objectives. In MLP experiments, PMNIST accuracy fell from 72.52% to 57.89% as task data rose from 1,500 to 5,000, alongside BWT falling from 2.98% to −15.56%; this does not establish bound tightness.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.