Reproduce “CDRL: A Reinforcement Learning Framework Inspired by Cerebellar Circuits and Dendritic Computational Strategies” on OpenResearch

About this paper

Sibo Zhang, Rui Jing, Liangfu Lv, Jian Zhang, Yunliang Zang

A reinforcement learning framework, CDRL, was developed by Tianjin University researchers, incorporating cerebellar circuits and dendritic computational strategies into a Double Deep Q-Network. This architecture achieves superior sample efficiency, enhanced robustness against various forms of noise, and improved generalization to unseen environmental conditions with a parameter count of approximately 6 million, half that of the baseline model.

Key insights

• Architectural inductive biases, derived from biological neural circuits like the cerebellum, can inherently improve core RL performance metrics such as sample efficiency, robustness, and generalization.
• Specific cerebellar features—large expansion, sparse connectivity, and sparse activation—collectively enhance representational separability, filter irrelevant features, and suppress noise propagation.
• Dendritic modulation offers an effective, non-gradient-based mechanism for state-dependent gating, contributing to an agent's adaptability and robustness without increasing the complexity of the learning process.

Summary by alphaXiv.