Reproduce “Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Mohit Shridhar, Lucas Manuelli, Dieter Fox
PERCEIVER-ACTOR (PERACT) is a language-conditioned behavior-cloning agent for multi-task 6-DoF robotic manipulation that reformulates the problem as detecting the next best voxel action. The approach leverages a Perceiver Transformer on a voxelized observation space, enabling robust performance across 18 simulated and 7 real-world tasks with only a few demonstrations per task.
Key insights
- Formulating robotic manipulation as a 3D voxel-based classification task for "next best voxel action" allows Transformers to operate efficiently with limited data.
- The Perceiver Transformer's global receptive field on voxelized inputs is crucial for holistic scene understanding, outperforming methods with local receptive fields like 3D ConvNets.
- Integrating strong 3D structural priors (voxelization) and extracting bottleneck keyframe actions are essential for achieving data-efficient 6-DoF multi-task learning.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.