Reproduce “FedBAT: Communication-Efficient Federated Learning via Learnable Binarization” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Shiwei Li, Wenchao Xu, Haozhao Wang, Xing Tang, Yining Qi, Shijie Xu, Weihong Luo, Yuhua Li and 2 more
FedBAT enables communication-efficient federated learning by integrating learnable binarization directly into the local training process, reducing communication overhead by 32 times. This approach, developed by researchers, maintains or improves model accuracy while significantly accelerating convergence compared to prior binarization methods.
Key insights
- Integrating binarization directly into the local training process, rather than applying it as a post-processing step, allows the model to adapt better to the constraints of binary communication.
- Learning an adaptive step size (alpha) for scaling binary updates significantly improves performance compared to using fixed step sizes, enabling automatic tuning.
- The proposed learnable stochastic binarization operator, combined with the Straight-Through Estimator, facilitates end-to-end gradient-based optimization for binary model updates.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.