Reproduce “Communication-Efficient Federated Learning via Predictive Coding” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Kai Yue, Richeng Jin, Chau-Wai Wong, Huaiyu Dai
Researchers from NC State University introduce a predictive coding-based compression scheme for federated learning, significantly enhancing communication efficiency by compressing the residual of model updates rather than the raw updates. The method achieves orders of magnitude reduction in communication cost while simultaneously improving or maintaining model performance on image classification tasks like Fashion-MNIST and CIFAR-10.
Key insights
- Compressing the residual (difference between actual and predicted model updates) rather than raw updates significantly reduces redundancy and improves communication efficiency in federated learning.
- Utilizing multiple adaptive prediction modes dynamically selected in each round empirically and theoretically (Lemma 1) reduces prediction error, leading to faster learning and better convergence.
- A strategic combination of prediction and quantization (especially sign encoding) can result in a more compact representation and a 'win-win' scenario where communication costs are lower while model performance is maintained or improved.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.