Reproduce “ToFU: Transforming How Federated Learning Systems Forget User Data” on OpenResearch

About this paper

Van-Tuan Tran, Hong-Hanh Nguyen-Le, Quoc-Viet Pham

ToFU, a "learning-to-unlearn" framework for Federated Learning, integrates dynamic data transformations and a transformation-invariant regularizer during training to reduce neural network memorization. This approach enables more efficient and effective unlearning by simplifying the post-unlearning process to a lightweight fine-tuning operation, yielding improved privacy protection and faster unlearning times compared to traditional methods.

Key insights

• The repeated application of data transformations to training samples consistently reduces the model's mutual information about those specific samples, thereby improving unlearning effectiveness.
• Proactively designing FL systems with inherent forgetting capabilities ('learning-to-unlearn') during the training phase is more effective than solely relying on reactive, post-hoc unlearning methods.
• A dynamic transformation strategy, coupled with a regularizer that enforces transformation-invariant feature learning, allows models to better balance utility and privacy without severe performance degradation.

Summary by alphaXiv.