Reproduce “Self-Supervised Reinforcement Learning that Transfers using Random Features” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Boyuan Chen, Chuning Zhu, Pulkit Agrawal, Kaiqing Zhang, Abhishek Gupta
This paper introduces Random Features for Model-Free Planning (RaMP), an approach to enable effective transfer learning in reinforcement learning by learning task-agnostic representations. The method learns Q-basis functions from reward-agnostic data offline and quickly adapts to new reward functions online, demonstrating strong performance across various robotic manipulation and locomotion tasks.
Key insights
- Environment dynamics can be implicitly captured by learning Q-basis functions corresponding to random features, enabling task-agnostic representation.
- Decoupling the learning of environment dynamics from specific reward functions allows for effective transfer learning in model-free RL settings.
- Utilizing multi-step Q-learning for Q-basis functions reduces policy dependence, which improves their transferability to new tasks.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.