Reproduce “Learning from Training Dynamics: Identifying Mislabeled Data Beyond Manually Designed Features” on OpenResearch

About this paper

Qingrui Jia, Xuhong Li, Lei Yu, Jiang Bian, Penghao Zhao, Shupeng Li, Haoyi Xiong, Dejing Dou

A learning-based approach, leveraging LSTM networks to analyze raw training dynamics, effectively identifies mislabeled samples in deep learning datasets without relying on manually designed features. The technique consistently surpasses state-of-the-art data-centric methods in precision, boosts model accuracy by up to 4% on real-world noisy benchmarks like Clothing100K, and provides additional performance gains when combined with algorithm-centric solutions.

Key insights

• Deep neural networks (specifically LSTMs) can effectively learn to extract distinguishing features for mislabel detection directly from raw training dynamics, surpassing the limitations of hand-crafted features.
• A noise detector trained on synthesized noisy data demonstrates strong transferability, capable of accurately identifying mislabels in diverse, unseen datasets, including real-world noisy benchmarks.
• Improving data quality through learning-based mislabel identification offers orthogonal performance benefits when combined with existing algorithm-centric robust learning approaches, enhancing overall model reliability.

Summary by alphaXiv.