Reproduce “Deep Reinforcement Learning with Spiking Q-learning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Ding Chen, Peixi Peng, Tiejun Huang, Yonghong Tian
Researchers from Shanghai Jiao Tong University and Peking University developed the Deep Spiking Q-Network (DSQN), an end-to-end deep reinforcement learning algorithm based on spiking neural networks, that represents Q-values directly as membrane voltages of non-spiking neurons. DSQN achieved a median human-normalized performance of 193.5% on Atari games while projecting significant energy efficiency and enhanced robustness against adversarial attacks.
Key insights
- Continuous values like Q-values can be robustly represented and learned directly through the membrane voltage of non-spiking neurons within an SNN, drawing inspiration from biological sensorimotor pathways.
- Directly training SNNs for deep reinforcement learning (DSQN) provides better performance and stability compared to methods relying on ANN-to-SNN conversion.
- Spiking neural networks inherently offer improved stability in learning trajectories and enhanced robustness against adversarial attacks compared to their non-spiking counterparts.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.