Reproduce “SkillTree: Explainable Skill-Based Deep Reinforcement Learning for Long-Horizon Control Tasks” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Yongyan Wen, Siyuan Li, Rongchang Zuo, Lei Yuan, Hangyu Mao, Peng Liu

SkillTree, a hierarchical reinforcement learning framework from Harbin Institute of Technology, Nanjing University, and industry partners, integrates discrete skill embeddings with decision trees to enable explainable decision-making for complex, long-horizon control tasks. It achieves performance comparable to neural network-based approaches while providing transparent, human-understandable skill selection policies, demonstrated across multiple robotic manipulation environments.

Key insights

  • A hierarchical approach combining discrete skill embeddings with decision tree-based high-level policies allows for both high performance and interpretable decision-making in long-horizon control tasks.
  • Learning reusable, discrete skills from offline data provides a higher-level abstraction that simplifies the decision space for the explainable policy.
  • Distilling a differentiable soft decision tree into a hard decision tree maintains performance while significantly enhancing the interpretability of the skill selection process.
  • Learned skills are specialized for different subtasks and objects, enabling the high-level policy to effectively decompose and execute complex tasks sequentially through rule-based skill selection.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.