Reproduce “Stragglers Can Contribute More: Uncertainty-Aware Distillation for Asynchronous Federated Learning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yujia Wang, Fenglong Ma, Jinghui Chen
FedEcho, developed by researchers at Pennsylvania State University, introduces an uncertainty-aware distillation framework for asynchronous federated learning that simultaneously mitigates issues from outdated updates and client bias. This framework demonstrated superior performance, achieving at least 3 percentage points higher accuracy on CIFAR-100 and a 4% improvement on MRPC over strong baselines, while enabling robust contributions from all clients.
Key insights
- Decoupling direct model parameter aggregation from information transfer through prediction logits effectively mitigates the negative impact of stale updates.
- Uncertainty-aware dynamic weighting in knowledge distillation allows the global model to learn more robustly from diverse and potentially noisy client contributions, including those from stragglers.
- Even outdated models from stragglers contain valuable data distribution information that can be effectively utilized via distillation, preventing their contributions from being neglected.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.