Reproduce “Touch100k: A Large-Scale Touch-Language-Vision Dataset for Touch-Centric Multimodal Representation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Ning Cheng, Changhao Guan, Jing Gao, Weihao Wang, You Li, Fandong Meng, Jie Zhou, Bin Fang and 2 more
Researchers introduce Touch100k, a large-scale dataset containing 100,147 entries that pair tactile, visual, and multi-granularity language data to address the scarcity of rich data in tactile AI. They develop TLV-Link, a pre-training method that achieves state-of-the-art performance in material property identification and robot grasping prediction, improving zero-shot rough/smooth classification accuracy by 14.0% and linear probing robot grasping accuracy by 12.2% over existing baselines.
Key insights
- A large-scale dataset featuring rich, multi-granularity language descriptions (sentence and phrase-level) is crucial for developing semantically informed tactile representations.
- Curriculum learning, specifically a teacher-student paradigm transferring knowledge from a vision encoder to a touch encoder, significantly enhances the generalization capabilities of learned tactile representations.
- Rigorous data quality enhancement, including AI-driven consistency checks and manual correction, is essential for ensuring the accuracy and practicality of generated language descriptions in multimodal datasets.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.