Reproduce “Semantic Redundancies in Image-Classification Datasets: The 10% You Don't Need” on OpenResearch

About this paper

Vighnesh Birodkar, Hossein Mobahi, Samy Bengio

Researchers at Google Research demonstrated that at least 10% of images in prominent datasets like CIFAR-10 and ImageNet can be removed using semantic clustering without degrading model generalization performance, and sometimes yielding slightly higher accuracy. This challenges previous assumptions about data efficiency and large-scale data requirements in deep learning.

Key insights

• Semantic redundancy in image datasets extends beyond exact duplicates, encompassing variations in pose, color, texture, and background that models learn to be invariant to.
• The choice of semantic representation layer is crucial, with the output of the last average pooling layer proving most effective for identifying meaningful redundancies.
• The nature of semantic redundancy can vary across datasets; while readily identifiable in CIFAR-10 and ImageNet, CIFAR-100 showed greater semantic diversity within 'redundant' groups, indicating dataset-specific characteristics.

Summary by alphaXiv.