Reproduce “Dataset Distillers Are Good Label Denoisers In the Wild” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Lechao Cheng, Kaifeng Chen, Jiyang Li, Shengeng Tang, Shufei Zhang, Meng Wang
This research explores dataset distillation (DD) as a method for robust learning in the presence of noisy labels, demonstrating that DD techniques can effectively filter symmetric and high-ratio natural label noise. The approach offers enhanced training efficiency and intrinsic privacy protection by generating a compact, denoised synthetic dataset offline.
Key insights
- Dataset distillation methods effectively denoise symmetric (random) and high-ratio natural label noise by preferentially capturing common, clean data patterns and largely ignoring random perturbations.
- The 'common patterns' captured by DD can include structured asymmetric noise, causing distillation methods to struggle in distinguishing and filtering such systematic label corruption.
- In scenarios with low noise rates or for challenging clean samples (e.g., from tail classes), the lossy compression inherent in distillation may inadvertently remove valuable clean information, limiting performance.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.