Reproduce “Compose by Focus: Scene Graph-based Atomic Skills” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Han Qi, Changhe Chen, Heng Yang
Researchers from Harvard University and the University of Michigan introduced a method that utilizes focused, scene graph-based representations combined with diffusion-based imitation learning for robotic manipulation. This approach enables robust execution of individual skills in cluttered and varied scenes, allowing for effective composition into complex, long-horizon tasks, outperforming methods based on raw visual inputs in both simulation and real-world experiments.
Key insights
- Explicitly encoding task-relevant objects and their relationships via focused scene graphs significantly improves the robustness of visuomotor policies to visual perturbations and distractors.
- Learning from policies conditioned on these structured representations enables effective compositional generalization for long-horizon tasks, even when atomic skills are trained in isolation.
- 3D object representations and graph neural networks are crucial for handling variable object counts, encoding complex spatial relationships, and providing permutation invariance, surpassing unstructured 2D or 3D inputs.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.