Reproduce “Think Locally, Act Globally: Federated Learning with Local and Global Representations” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Paul Pu Liang, Terrance Liu, Liu Ziyin, Nicholas B. Allen, Randy P. Auerbach, David Brent, Ruslan Salakhutdinov, Louis-Philippe Morency
LG-FEDAVG is a federated learning algorithm that jointly learns compact local data representations and a smaller global model to improve training efficiency, adaptability, and fairness on distributed datasets. It reduces communication overhead by up to 50% while robustly handling data heterogeneity and enabling the learning of fair representations.
Key insights
- Theoretical analysis demonstrates that combining local and global models optimally balances data and device variance, leading to superior generalization compared to purely local or global approaches.
- The modular architecture allows integrating various local learning paradigms (e.g., supervised, unsupervised, self-supervised, adversarial) to address diverse challenges like fairness and privacy.
- Jointly learning compact local representations with a smaller global model simultaneously enhances communication efficiency and improves robustness to diverse and non-I.I.D. data distributions.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.