Reproduce “Efficient Data Valuation Approximation in Federated Learning: A Sampling-based Approach” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Shuyue Wei, Yongxin Tong, Zimu Zhou, Tianran He, Yi Xu
Researchers from Beihang University and City University of Hong Kong developed IPSS (Importance-Pruned Stratified Sampling), an algorithm that provides efficient and accurate approximation of Shapley values for data valuation in Federated Learning. IPSS consistently achieves lower approximation errors and superior computational efficiency compared to existing methods, making fair assessment of data contributions practical for large-scale FL deployments.
Key insights
- The Marginal Contribution-based Shapley Value (MC-SV) scheme exhibits lower variance in approximation errors compared to the Complementary Contribution-based Shapley Value (CC-SV) scheme, making it more suitable for sampling-based methods in FL.
- A "key combinations" phenomenon exists in FL data valuation, where a small subset of dataset combinations (particularly those with fewer clients) disproportionately impacts the final Shapley value, allowing for strategic pruning of less important computations.
- Model utility (e.g., accuracy) in Federated Learning generally exhibits diminishing marginal utility as more datasets are added, which can be leveraged for efficient data valuation.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.