Reproduce “Tabero: Learning Gentle Manipulation with Closed-Loop Force Feedback from Vision, Touch, and Language” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Qiwei Wu, Rui Zhang, Xin Xiang, Tao Li, Weihua Zhang, Junjie Lai, Renjing Xu
Nvidia and Hong Kong University of Science and Technology (Guangzhou) researchers introduced Tabero, a framework that provides a scalable vision-tactile-language dataset and a VTLA model suite for gentle, language-conditioned robotic manipulation. It integrates closed-loop force feedback to enable robots to modulate interaction forces based on linguistic cues.
Key insights
- Integrating rich tactile representations (like marker motion fields) with explicit force supervision is crucial for enabling effective, gentleness-aware manipulation policies.
- A decoupled hybrid controller, which independently manages grip force and applied force, is essential for accurate real-time force tracking and compliant physical interaction.
- VTLA models can learn to semantically interpret linguistic adverbs related to force, allowing for intuitive and flexible control over interaction intensity.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.