Reproduce “VITaL Pretraining: Visuo-Tactile Pretraining for Tactile and Non-Tactile Manipulation Policies” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Abraham George, Selam Gano, Pranav Katragadda, Amir Barati Farimani

VITaL pretraining from Carnegie Mellon University enables vision-only robotic policies to achieve high performance in manipulation tasks by leveraging visuo-tactile data solely during the training phase, eliminating the need for tactile sensors at inference. This approach boosts vision-only success rates for USB plugging from 20% to 85% with ACT and consistently reduces contact forces during manipulation.

Key insights

  • Tactile data used only during pretraining can significantly enhance the performance of vision-only manipulation policies at inference, implicitly imbuing them with contact awareness.
  • Pretraining with visuo-tactile data leads to policies that exert less force during manipulation, indicating a more refined and gentle interaction strategy.
  • The effectiveness of visuo-tactile pretraining generalizes across different challenging manipulation tasks and state-of-the-art imitation learning architectures (ACT, Diffusion Policy).

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.