Reproduce “Approximate Data Deletion from Machine Learning Models” on OpenResearch

About this paper

Zachary Izzo, Mary Anne Smart, Kamalika Chaudhuri, James Zou

This work introduces the Projective Residual Update (PRU), an algorithm designed for efficiently deleting specific training data points from linear and logistic regression models, achieving a computational cost linear in the data dimension. The researchers also present the Feature Injection Test (FIT), a new metric that more precisely evaluates how well a model removes 'knowledge' of sensitive features, with the PRU demonstrating superior performance in sensitive attribute removal and outlier deletion compared to prior methods.

Key insights

• The PRU offers the first approximate deletion method for general parametric models (linear and logistic regression) with a computational cost linear in the data dimension (O(d)), making it scalable for high-dimensional datasets.
• The method is theoretically optimal within a specific class of gradient-based updates where parameter changes are restricted to the span of the deleted feature vectors.
• Evaluating data deletion through the Feature Injection Test (FIT) provides a more privacy-centric assessment than L2 distance, confirming that a model can retain 'knowledge' of sensitive attributes even with similar overall parameters.

Summary by alphaXiv.