Reproduce “Neural Architecture Search with Mixed Bio-inspired Learning Rules” on OpenResearch

About this paper

Imane Hamzaoui, Riyadh Baghdadi

Researchers from École nationale Supérieure d’Informatique Algiers and New York University Abu Dhabi introduced BioNAS, a Neural Architecture Search framework that automatically discovers neural network architectures by integrating diverse bio-inspired learning rules across layers. This approach resulted in bio-inspired models achieving a 4.84% test error on CIFAR-10, outperforming some back-propagation baselines, and demonstrating enhanced adversarial robustness against various attack types.

Key insights

• Integrating diverse bio-inspired learning rules within a NAS framework significantly narrows the performance gap with back-propagation models while improving adversarial robustness.
• The benefits of mixed learning rules stem from the inherent diversity itself, rather than a specific pattern, suggesting that combining different bio-inspired mechanisms leads to improved overall performance.
• Networks trained with mixed bio-inspired learning rules exhibit consistently lower gradient variance over epochs, indicating more stable and effective learning dynamics during training.

Summary by alphaXiv.