Reproduce “Unsupervised Hierarchical Skill Discovery” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Damion Harvey, Geraud Nangue Tasse, Benjamin Rosman, Branden Ingram, Steven James

The Hierarchical Skill Discovery (HiSD) framework extracts reusable, multi-level skill hierarchies directly from raw observational data without supervision. It achieves superior skill segmentation and more consistent hierarchies compared to baselines, enhancing sample efficiency and stability in downstream reinforcement learning tasks in complex environments like Minecraft.

Key insights

  • Multi-level skill hierarchies can be effectively discovered from raw, unlabelled observational data by combining temporal action segmentation with grammar-based compression.
  • The framework's temporal consistency prior enables coherent skill segmentation even in partially observable environments with perceptual aliasing.
  • Discovered hierarchical structures provide a strong inductive bias that significantly accelerates and stabilizes learning for downstream reinforcement learning agents, even with limited action-labelled demonstrations.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.