Reproduce “TacVLA: Contact-Aware Tactile Fusion for Robust Vision-Language-Action Manipulation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Kaidi Zhang, Heng Zhang, Zhengtong Xu, Zhiyuan Zhang, Md Rakibul Islam Prince, Xiang Li, Xiaojing Han, Yuhao Zhou and 2 more
TacVLA adds tactile sensing to vision-language-action control through a contact-aware gate that activates tactile tokens during physical interaction and suppresses them otherwise. On a Franka Panda robot, it raises average disassembly success to 83.75% and in-box picking success to 70%, while improving camera-occluded disassembly from about 30% to 62.5% compared with a fine-tuned π0.5 baseline.
Key insights
- Tactile information is most effective when conditioned on the robot’s physical interaction state rather than supplied continuously throughout the trajectory.
- Contact feedback helps compensate for missing visual context, allowing the robot to verify grasps, manage constrained motions, and retry after failed or disturbed interactions.
- Ablation results indicate that tactile sensing alone is insufficient: removing contact-aware gating lowers disassembly success from 83.75% to 71.25% and in-box picking from 70% to 40%.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.