Reproduce “Rescaled Influence Functions: Accurate Data Attribution in High Dimension” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Ittai Rubinstein, Samuel B. Hopkins
The authors introduce rescaled influence functions (RIF) to estimate how removing groups of training examples changes a classifier. They adjust each sample’s influence estimate for the curvature change caused by its removal, then add adjusted estimates. In five binary logistic-regression tasks with frozen embeddings, RIF generally tracks refitted effects better than ordinary influence functions, though predictions remain imperfect.
Key insights
- RIF successfully bridges the gap between the accuracy of Newton Step approximations and the computational efficiency and additivity of Influence Functions, making high-fidelity data attribution practical.
- The method maintains its accuracy in high-dimensional and overparameterized settings, directly addressing the core limitation of traditional IFs in modern machine learning environments.
- The difference between RIF and IF often stems from a simple per-sample scaling factor, explaining why IFs empirically correlate with ground truth but consistently underestimate its magnitude.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.