Reproduce “GauTOAO: Gaussian-based Task-Oriented Affordance of Objects” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jiawen Wang, Dingsheng Luo
Peking University researchers developed GauTOAO, a framework that identifies task-relevant object parts for robotic manipulation by adopting a "static camera, moving object" paradigm and a two-stage conditional query using DINO features. The system demonstrated superior performance on a "Sunflower Handling" task, achieving an mIoU of 61.4% and an accuracy of 81.6%, surpassing existing baselines.
Key insights
- Effective robotic manipulation requires understanding task-specific affordances—identifying which object parts are relevant for a given task—beyond mere grasping.
- A robot-centric "static camera, moving object" paradigm is crucial for practical 3D perception during in-hand manipulation, enabling more realistic and robust robotic systems.
- Combining comprehensive object masking using DINO features with fine-grained, task-specific querying via CLIP embeddings leads to more accurate and robust task-oriented affordance extraction.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.