Reproduce “Accurate Forgetting for Heterogeneous Federated Continual Learning” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Abudukelimu Wuerkaixi, Sen Cui, Jingfeng Zhang, Kunda Yan, Bo Han, Gang Niu, Lei Fang, Changshui Zhang and 1 more

Researchers from Tsinghua University, RIKEN, and other institutions developed AF-FCL, a generative-replay method for federated continual learning that selectively discards irrelevant or detrimental past knowledge. This approach leverages normalizing flows to quantify knowledge credibility, leading to improved average accuracy (e.g., 47.5% on EMNIST-LTP) and reduced forgetting (e.g., 7.9% on EMNIST-LTP) in highly heterogeneous and noisy environments.

Key insights

  • Forgetting is not universally detrimental; selectively discarding biased or irrelevant past knowledge can enhance performance in heterogeneous federated continual learning.
  • In challenging settings like "Limitless Task Pool" or with noisy data, retaining all past information can lead to memorizing spurious correlations, actively degrading model effectiveness.
  • Normalizing Flow models can effectively quantify the credibility of generated past features, providing a principled mechanism for adaptive knowledge utilization and selective forgetting.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.