Reproduce “Structure Learning in Motor Control:A Deep Reinforcement Learning Model” on OpenResearch

About this paper

Ari Weinstein, Matthew M. Botvinick

The authors present a recurrent neural network (RNN) forward model with a separate planner for motor adaptation in a partially observable reaching simulation. The model infers cursor dynamics from action–outcome histories and supports replanning. Rotation-only training adapted more rapidly and completely than mixed rotation, shear, and scale training, qualitatively matching human findings, but not quantitatively or biologically validating them.

Key insights

• Recurrent Neural Networks (RNNs) can effectively serve as adaptive internal forward models within a model-based reinforcement learning framework, enabling system identification and meta-learning in partially observable environments.
• The internal structure learned by the RNN, based on its pre-training experience, naturally facilitates faster adaptation to novel but structurally related tasks without requiring explicit encoding of task structure.
• The proposed framework bridges deep learning approaches to meta-learning with model-based RL theories of motor control, offering a plausible computational account for how human motor structure learning arises.

Summary by alphaXiv.