Reproduce “Adaptive Differential Privacy in Federated Learning: A Priority-Based Approach” on OpenResearch

About this paper

Mahtab Talaei, Iman Izadi

Researchers at Isfahan University of Technology developed an adaptive differential privacy framework for federated learning, which dynamically adjusts Gaussian noise injection based on parameter importance in deep neural networks. This approach achieves a more favorable balance between privacy guarantees and model accuracy by applying more noise to less important features, yielding improved global model accuracy compared to uniform noise injection.

Key insights

• Not all parameters in deep neural networks contribute equally to model output or possess equal sensitivity, suggesting that a non-uniform distribution of the privacy budget can optimize the privacy-accuracy trade-off.
• Parameter importance can be efficiently determined on the client side during federated training by analyzing local model sensitivity to feature perturbations or the variance of weight updates.
• Intelligently allocating differential privacy noise, particularly by targeting a limited proportion of the least important features with higher noise, can preserve model accuracy more effectively than uniform noise strategies.

Summary by alphaXiv.