Reproduce “Efficiently Attacking Memorization Scores” on OpenResearch

About this paper

Tue Do, Varun Chandrasekaran, Daniel Alabi

Researchers at the University of Illinois Urbana-Champaign demonstrated that memorization-based data valuation scores are highly susceptible to adversarial manipulation. Their Pseudoinverse Attack consistently inflated these scores for attacked samples by a significant margin across diverse tasks and models without affecting model accuracy.

Key insights

• A fundamental trade-off exists: highly accurate algorithms are inherently vulnerable to memorization score manipulation by sensitive queries unless strong stability guarantees (like differential privacy) are enforced.
• The Pseudoinverse Attack effectively exploits this theoretical vulnerability by generating highly 'sensitive' data points, leading to significantly inflated memorization scores.
• Adversarial manipulation of memorization scores can be achieved without significantly impacting the model's overall test accuracy, making these attacks stealthy.

Summary by alphaXiv.