Reproduce “Neuroplastic Expansion in Deep Reinforcement Learning” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Jiashun Liu, Johan Obando-Ceron, Aaron Courville, Ling Pan

Researchers from HKUST and Mila - Québec AI Institute developed Neuroplastic Expansion (NE), a biologically inspired framework that dynamically expands neural network topology to maintain learning capacity in deep reinforcement learning (DRL) agents. This approach consistently outperforms state-of-the-art baselines across diverse MuJoCo and DeepMind Control Suite tasks, fostering stable continual adaptation by preserving a higher active neuron ratio.

Key insights

  • Biological cortical expansion offers a paradigm for artificial agents to dynamically grow and reorganize neural networks, fostering continuous adaptability.
  • Actively managing network topology by identifying and pruning dormant neurons, while selectively adding high-gradient connections, can effectively mitigate plasticity loss.
  • Balancing continuous network expansion with adaptive experience review is essential for maintaining both adaptability and stability in DRL agents.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.