Reproduce “High-Performance Neural Networks for Visual Object Classification” on OpenResearch

About this paper

Dan C. Cireşan, Ueli Meier, Jonathan Masci, Luca M. Gambardella, Jürgen Schmidhuber

Researchers at IDSIA developed a fast and flexible GPU implementation for deep Convolutional Neural Networks, achieving new state-of-the-art error rates on MNIST (0.35%), NORB (2.53%), and CIFAR10 (19.51%). This work demonstrated that deep networks trained solely with back-propagation can learn rapidly and effectively, especially with the aid of GPU acceleration.

Key insights

• Deep Convolutional Neural Networks (CNNs) can achieve state-of-the-art performance and learn rapidly when trained end-to-end with supervised back-propagation, challenging prior beliefs about training difficulties.
• Efficient and flexible GPU implementations are crucial for overcoming computational bottlenecks in deep learning, enabling systematic exploration and development of complex network architectures.
• Architectural choices like partial connectivity for scaling deep CNNs and max-pooling, alongside appropriate data augmentation (e.g., translations), significantly contribute to improved generalization performance.

Summary by alphaXiv.