Reproduce “ToolDex: An Ego-Exo Human Demonstration Dataset for Creative Physical Tool Use” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Mohamed Elmoghany, Kehan Li, Ruoqu Chen, Tarun Chiruvolu, Shutong Zhang, Ryan Wang, Stephen Tian, Yaru Niu and 6 more
ToolDex records full hand–tool–target interactions, including deformable targets, using synchronized egocentric and exocentric views with annotations for hands, tools, and targets. It pairs demonstrations with benchmarks for reasoning, reconstruction, and robot policies. Across six simulated tasks, Dex-X reinforcement learning averages 55.1% success versus 14.0% for kinematic retargeting and 4.6% for behavior cloning, not real-world performance.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.