Reproduce “Controllability-Aware Unsupervised Skill Discovery” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Seohong Park, Kimin Lee, Youngwoon Lee, Pieter Abbeel
Controllability-aware Skill Discovery (CSD) enables agents to autonomously learn complex, useful behaviors by intrinsically prioritizing transitions that are challenging to achieve with current skills. CSD achieved significantly higher object state coverage in robotic manipulation tasks and superior performance in various downstream tasks compared to state-of-the-art unsupervised methods.
Key insights
- An intrinsic motivation based on the "controllability" or "difficulty" of state transitions can guide agents to discover progressively more complex and useful skills autonomously.
- Generalizing distance-maximizing skill discovery with an adaptive, non-Euclidean distance function allows for flexible and semantically meaningful skill acquisition beyond simple state diversity.
- Using the negative log-likelihood of state transitions as a proxy for "difficulty" naturally leads to a self-challenging mechanism, where agents continuously seek out and master harder aspects of the environment.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.