Reproduce “Data Pruning via Moving-one-Sample-out” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Haoru Tan, Sitong Wu, Fei Du, Yukang Chen, Zhibin Wang, Fan Wang, Xiaojuan Qi
A novel data pruning method, Moving-one-Sample-out (MoSo), quantitatively assesses a sample's importance by its impact on optimal empirical risk. This approach, with a linear-complexity approximator, consistently outperforms state-of-the-art methods across various datasets, particularly at high pruning ratios and in the presence of label noise.
Key insights
- Sample importance is more accurately captured by its net contribution to the overall empirical risk when removed, effectively differentiating truly informative samples from harmful noisy ones.
- Integrating training dynamics through gradient information across various epochs is essential for a robust and comprehensive assessment of a sample's influence.
- Efficient gradient-based approximation allows for practical application of complex importance metrics with theoretical error bounds, making the method feasible for large datasets.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.