Reproduce “AnyTouch: Learning Unified Static-Dynamic Representation across Multiple Visuo-tactile Sensors” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Ruoxuan Feng, Jiangyu Hu, Wenke Xia, Tianci Gao, Ao Shen, Yuhao Sun, Bin Fang, Di Hu

Researchers at Renmin University of China and collaborators developed `AnyTouch`, a framework that learns unified static and dynamic representations across multiple visuo-tactile sensors. The system leverages a newly collected, aligned `TacQuad` dataset, resulting in superior performance in tactile perception tasks and robust generalization to previously unseen sensors.

Key insights

  • Explicitly aligned multi-sensor, multi-modal data is crucial for bridging the gap between diverse tactile sensors and learning truly unified representations.
  • A multi-level training approach, from pixel-level reconstruction to semantic-level multi-modal alignment and cross-sensor matching, effectively fosters a sensor-agnostic representation space.
  • The use of a universal sensor token enables models to generalize effectively to entirely new sensor types not present during training, enhancing real-world applicability.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.