Reproduce “BridgeACT: Bridging Human Demonstrations to Robot Actions via Unified Tool-Target Affordances” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yifan Han, Jianxiang Liu, Haoyu Zhang, Yuqi Gu, Yunhan Guo, Wenzhao Lian
BridgeACT introduces an embodiment-agnostic framework that enables robots to learn complex manipulation skills directly from human videos by utilizing role-conditioned 3D tool-target affordances. This system achieved high real-world task success rates across diverse manipulation tasks and demonstrated robust zero-shot generalization to unseen objects and scenes.
Key insights
- An 'executable tool-target affordance' serves as a robust, embodiment-agnostic intermediate representation, effectively bridging the semantic and kinematic gap between human demonstrations and robot actions.
- Learning relative 3D motion between tool and target entities provides a more transferable and stable signal for robot manipulation than absolute motion, enabling scalable learning from diverse human video data.
- A diffusion-based generative model can effectively learn and predict smooth, geometrically consistent 3D point-flow trajectories for complex, contact-rich manipulation tasks directly from human demonstrations.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.