Reproduce “TactX: Learning Shared Tactile Representations Across Diverse Sensors” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Junsung Park, Sachin Bhadang, Carmelo Sferrazza, Sha Yi, Xiaolong Wang
The TACTX framework unifies tactile observations from fundamentally different sensor modalities (vision-based, magnetic, resistive) into a shared latent space. This enables zero-shot policy transfer for robotic manipulation tasks, improving average success rates from 27.5% with vision-only transfer to 45.9% across diverse sensor types.
Key insights
- Pairwise supervision is sufficient to establish a globally consistent and transitively aligned latent space across multiple heterogeneous tactile sensors, even for non-directly observed pairs.
- The learned shared latent space effectively preserves task-relevant object-level contact information, crucial for robust manipulation, while reducing sensor-specific details.
- A combination of reconstruction, contrastive alignment, and KL divergence regularizes the latent space, forcing sensor-agnostic representations that retain core tactile content.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.