Reproduce “Instruction-driven history-aware policies for robotic manipulations” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Pierre-Louis Guhur, Shizhe Chen, Ricardo Garcia, Makarand Tapaswi, Ivan Laptev, Cordelia Schmid
The paper introduces Hiveformer, a unified transformer-based architecture for instruction-driven robotic manipulation. It achieves an 88.4% success rate on 10 RLBench tasks and 65.4% on 74 tasks, outperforming prior methods by effectively integrating multi-view observations, action history, and fine-grained language understanding.
Key insights
- Explicitly modeling cross-modal relationships (language, vision, history) using cross-attention mechanism is crucial for high performance in instruction-driven robotic tasks.
- Incorporating the full history of observations and actions enables robust execution of sequential and long-term robotic tasks.
- Leveraging multi-view visual and proprioceptive information provides richer spatial context, leading to improved manipulation precision and resilience to occlusions.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.