Reproduce “Evolving Deep Neural Networks” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Risto Miikkulainen, Jason Liang, Elliot Meyerson, Aditya Rawal, Dan Fink, Olivier Francon, Bala Raju, Hormoz Shahrzad and 3 more
Researchers at Sentient Technologies developed CoDeepNEAT, an evolutionary optimization method that automates the design of deep neural network architectures, encompassing topology, components, and hyperparameters. The approach achieved competitive performance on tasks like CIFAR-10 object recognition and Penn Tree Bank language modeling, and successfully deployed a real-time image captioning system for a major online magazine.
Key insights
- Cooperative coevolution (CoDeepNEAT) effectively enables the discovery of deep, modular, and repetitive network structures, addressing the limitations of evolving entire DNNs directly.
- Evolutionary pressure, even with limited training during evaluation, implicitly guides the search towards architectures that learn quickly, leading to faster convergence for final training.
- The method can automatically discover novel and effective architectural elements, such as specific skip connections, which resemble advanced human-designed features and improve performance.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.