Reproduce “VIRDO: Visio-tactile Implicit Representations of Deformable Objects” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Youngsun Wi, Pete Florence, Andy Zeng, Nima Fazeli

Researchers at the University of Michigan and Google developed VIRDO, a visio-tactile implicit representation for deformable objects that predicts deformation fields from combined visual and tactile sensor data. The model achieved high-fidelity reconstructions and demonstrated generalization to unseen contact conditions while establishing dense correspondences without explicit supervision.

Key insights

  • Integrating both visual (point clouds representing contact locations) and tactile (reaction forces) feedback is crucial for robustly learning and inferring complex object deformation fields.
  • Implicit neural representations can effectively establish dense, point-wise correspondences between deformed and nominal object shapes without needing explicit supervision.
  • Conditioning the deformation model on both a force code and an object-specific latent code allows for learning object-specific deformation behaviors, accounting for varied material properties.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.