Reproduce “CIFAR10 to Compare Visual Recognition Performance between Deep Neural Networks and Humans” on OpenResearch

About this paper

Tien Ho-Phuoc

Researchers at The University of Danang – University of Science and Technology conducted an extensive comparison of visual object recognition between human subjects and deep neural networks on the CIFAR10 dataset. The study found that while state-of-the-art CNNs achieved higher overall accuracy (e.g., WRNC at 96.96% compared to human average 93.91%), they did not match human perfect reliability on images deemed 'easy' and exhibited substantially lower robustness to Gaussian noise.

Key insights

• The notion of 'human-level performance' in visual recognition needs nuance, as CNNs and humans exhibit distinct recognition behaviors and error patterns.
• While CNNs can surpass overall human average accuracy, they fail to achieve human-level reliability on images that are unambiguously easy for humans.
• Deep neural networks demonstrate less robustness to out-of-distribution inputs like Gaussian noise compared to human visual perception, despite high accuracy on clean data.

Summary by alphaXiv.