Reproduce “A Spiking Neural Network Structure Implementing Reinforcement Learning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Mikhail Kiselev
Mikhail Kiselev developed a Spiking Neural Network (SNN) architecture that implements reinforcement learning using a purely spiking formulation, where all signals are spike trains. The system successfully learned to track a chaotically moving light spot, demonstrating efficient adaptation with a compact 48-neuron network and a novel synaptic plasticity model suitable for neuromorphic hardware.
Key insights
- The proposed SNN structure allows for fast and efficient learning of a continuous RL task within the first 50 seconds of emulation, requiring a compact network of only 48 neurons.
- The novel synaptic plasticity mechanism, which uses synaptic resources and a constancy rule, effectively bounds synaptic weights and contributes to learning stability, potentially addressing catastrophic forgetting.
- A "purely spiking formulation" for all signals, including reward/punishment, is viable for RL and inherently compatible with neuromorphic hardware, reducing the need for rate-based approximations or complex conversions.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.