Reproduce “Distributionally Robust Federated Learning with Outlier Resilience” on OpenResearch

About this paper

Zifan Wang, Xinlei Yi, Xenia Konti, Michael M. Zavlanos, Karl H. Johansson

Distributionally Outlier-Robust Federated Learning (DOR-FL) is introduced by researchers from KTH, Tongji University, and Duke University as a novel framework to address data heterogeneity, distribution shifts, and sample-level outlier contamination in federated learning. It leverages an unbalanced Wasserstein distance to achieve superior classification accuracy, reaching 95.4% on synthetic data and 84.6% on the UCI Adult Income dataset, while also enhancing fairness across demographic groups.

Key insights

• Unbalanced Wasserstein distance offers a principled method to capture geometric distribution shifts while simultaneously mitigating sample-level outlier influence by relaxing the mass conservation constraint.
• The integration of an application-dependent outlier scoring function allows flexible incorporation of domain knowledge, guiding the model to robustly learn in the presence of suspected outliers.
• Complex min-max-max DRO problems can be made computationally feasible for decentralized FL through Lagrangian relaxation, providing robustness certificates and maintaining worst-case performance guarantees.

Summary by alphaXiv.