Reproduce “Physical Deep Learning with Biologically Plausible Training Method” on OpenResearch

About this paper

Mitsumasa Nakajima, Katsuma Inoue, Kenji Tanaka, Yasuo Kuniyoshi, Toshikazu Hashimoto, Kohei Nakajima

This paper develops Augmented Direct Feedback Alignment, a biologically plausible training method for physical deep learning. This approach enables the efficient training of physical neural networks without requiring detailed knowledge of system dynamics or backpropagation of error signals, demonstrating competitive accuracy on standard benchmarks using an optoelectronic deep reservoir computer.

Key insights

• Effective deep learning in physical systems can be achieved without precise knowledge of the system's internal dynamics or activation function derivatives, by using an arbitrary nonlinearity in the learning rule.
• Training using random projections and direct feedback is robust to the choice of arbitrary nonlinearity, even when it is uncorrelated with the true derivative.
• Biologically plausible learning rules like Augmented DFA can offer scalability and hardware efficiency advantages over backpropagation for physical neural networks.

Summary by alphaXiv.