Reproduce “Self-Supervised Policy Adaptation during Deployment” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Nicklas Hansen, Rishabh Jangir, Yu Sun, Guillem Alenyà, Pieter Abbeel, Alexei A. Efros, Lerrel Pinto, Xiaolong Wang
A method called Policy Adaptation during Deployment (PAD) enables pre-trained vision-based reinforcement learning policies to adapt to novel, unknown environments without external reward signals. This approach improves policy generalization in 31 out of 36 tested environments and boosts performance in real robotic manipulation by up to 24% by continuously updating visual representations using self-supervised learning.
Key insights
- Continuous online adaptation through self-supervision offers a robust, reward-free mechanism for pre-trained policies to generalize to unforeseen visual and subtle dynamic shifts during deployment.
- The choice of self-supervised task is crucial, with Inverse Dynamics Prediction being more effective for motor control tasks and Rotation Prediction more suitable for tasks requiring rich scene understanding.
- Adapting internal visual representations in an online fashion can be more effective than pre-training a statically robust policy via domain randomization, especially for environments with unpredictable changes.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.