Reproduce “Feeling the Unexpected: ResTacVLA for Contact-Rich Manipulation via Residual Tactile Representation” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Pengwei Zhang, Bin Xie, Xinpan Meng, Xinyu Guo, Ce Hao, Fang Deng, Long Cheng, Tiancai Wang

ResTacVLA, a multimodal robotic policy, enables enhanced contact-rich manipulation by integrating tactile feedback as a residual representation, addressing modality collapse in vision-language-action (VLA) models. The policy achieved an average success rate of 62.8% across five complex manipulation tasks, exceeding vision-only baselines like \u03c0 0.5 (28.2%) by 34.6%.

Key insights

  • Reformulating tactile feedback as a "residual" signal, representing the difference from visual predictions, effectively prevents modality collapse by focusing on high-information-gain cues.
  • Discretizing residual tactile information into Latent Contact Primitives via Vector Quantization provides a compact, interpretable vocabulary of physically grounded contact events.
  • An adaptive gating mechanism, driven by visual prediction uncertainty, allows the policy to dynamically prioritize tactile feedback when vision is unreliable, thereby enhancing control during critical interaction phases.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.