Reproduce “A Bayesian Federated Learning Framework with Online Laplace Approximation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Liangxi Liu, Xi Jiang, Feng Zheng, Hong Chen, Guo-Jun Qi, Heng Huang, Ling Shao
This research from Southern University of Science and Technology and collaborators introduces FOLA, a Bayesian Federated Learning framework using online Laplace approximation to estimate and aggregate probabilistic model parameters. The framework achieved over 5% higher global accuracy on CIFAR10-CNN with extreme data heterogeneity compared to FedAvg and FedCurv, significantly improving performance in non-IID settings.
Key insights
- Aggregating local posteriors via a product of Gaussians rather than a simple average can effectively identify robust modes ('ridges') in the multimodal posterior landscape, reducing aggregation error.
- Treating the global model posterior as a prior for local training effectively balances local optimization with global knowledge retention, preventing local forgetting.
- Online evaluation of the diagonal empirical Fisher information matrix provides a more accurate and stable estimation of local model variances compared to offline methods, which tend to overestimate variances.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.