Reproduce “Data Valuation Without Training of a Model” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Nohyun Ki, Hoyong Choi, Hye Won Chung
KAIST researchers developed the Complexity-Gap (CG) score, a data valuation method that quantifies individual data instance importance without requiring any model training. The CG score effectively identifies mislabeled or irregular data and predicts learning difficulty, demonstrating utility in data pruning and label noise detection with competitive performance on various datasets.
Key insights
- The Complexity-Gap (CG) score offers a truly training-free method for data valuation, addressing the computational bottleneck of prior training-dependent approaches.
- High CG-scores reliably indicate irregular, mislabeled, or highly influential data instances, aligning with their geometric interpretation as data points near class boundaries or atypical within their class.
- The CG-score correlates with a data point's 'learning difficulty' and its influence on parameter movement during training, providing insights into how individual examples drive model learning.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.