Reproduce “Mitigating Privacy Risk via Forget Set-Free Unlearning” on OpenResearch

About this paper

Aviraj Newatia, Michael Cooper, Viet Nguyen, Rahul G. Krishnan

Researchers from the University of Toronto and Vector Institute developed RELOAD, a method for Partially-Blind Unlearning (PBU) that removes data influence from trained models without direct access to the raw data marked for deletion. This approach leverages cached gradients and a three-stage process, demonstrating effective unlearning in classical, entity, and corrective scenarios, while enabling immediate data deletion and mitigating privacy risks.

Key insights

• Cached summed gradients over the entire dataset can serve as a privacy-preserving proxy for the forget set, enabling inference of its influence without direct access.
• The linearity of differentiation allows for inferring the gradient direction of the forget set by subtracting the retain set's gradient from the total training gradient.
• Combining a global gradient ascent perturbation with targeted selective re-initialization effectively addresses both broad and specific influences of the forget set, restoring model utility while achieving unlearning.

Summary by alphaXiv.