Reproduce “Graph-Assisted Stitching for Offline Hierarchical Reinforcement Learning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Seungho Baek, Taegeon Park, Jongchan Park, Seungjun Oh, Yusung Kim
Graph-Assisted Stitching (GAS) introduces an offline hierarchical reinforcement learning framework that leverages Temporal Distance Representation and graph-based planning to explicitly stitch trajectories and handle long-horizon tasks. It achieves an 88.3 score in `antmaze-giant-stitch` and 94.2 in `antmaze-large-explore`, substantially outperforming prior state-of-the-art methods.
Key insights
- Explicitly embedding states based on their optimal temporal distance (TDR) provides a robust foundation for identifying subgoals and planning in offline settings.
- Filtering suboptimal data using a Temporal Efficiency metric before graph construction is crucial for building high-quality, efficient planning structures from noisy datasets.
- Replacing learned high-level policies with a graph search algorithm for subgoal selection simplifies the system and improves stability and scalability for long-horizon tasks.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.