Reproduce “Non-negative isomorphic neural networks for photonic neuromorphic accelerators” on OpenResearch

About this paper

Manos Kirtas, Nikolaos Passalis, Nikolaos Pleros, Anastasios Tefas

A framework introduces a Non-negative Isomorphic Transformation (NNT) and a Non-negative Stochastic Gradient Descent (NNSGD) optimizer to enable deep learning models to run on neuromorphic photonic accelerators. This methodology consistently achieves identical or competitive performance to conventionally trained networks across various architectures and datasets, removing the need for complex hardware to represent negative values.

Key insights

• Conventional deep learning models can be transformed into non-negative isomorphic equivalents without any loss in computational functionality or performance, making them directly deployable on power-additive photonic hardware.
• Simple parameter clipping during training (CSGD) on pre-trained networks leads to catastrophic information loss and performance degradation due to a drastic reduction in parameter variance.
• The multiplicative update rule in NNSGD effectively maintains parameter variance, enabling stable and robust non-negative training from scratch or fine-tuning, often leading to performance on par with or exceeding conventional training methods.

Summary by alphaXiv.