Reproduce “Learning Generalizable Skill Policy with Data-Efficient Unsupervised RL” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jongchan Park, Seungjun Oh, Seungho Baek, Yusung Kim
The GenDa framework improves unsupervised reinforcement learning by introducing skill relabeling for data-efficient pre-training and a Complementary Information Bottleneck to foster generalizable skill policies. It achieves higher state and task coverage with significantly fewer interactions and maintains robust performance in downstream tasks with varied initial conditions and environmental contexts.
Key insights
- Dynamically relabeling past experiences in the replay buffer is crucial for overcoming non-stationarity and achieving data-efficient pre-training in off-policy URL.
- Explicitly disentangling global contextual information from the skill policy's input via a complementary encoder leads to significantly more generalizable and robust skills across varied environments.
- Combining these mechanisms enables scalable unsupervised skill discovery even in high-dimensional, complex control tasks where prior methods struggle.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.