Reproduce “Tac2Pose: Tactile Object Pose Estimation from the First Touch” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Maria Bauza, Antonia Bronars, Alberto Rodriguez
Researchers at MIT developed Tac2Pose, a framework for tactile object pose estimation that learns object-specific perception models almost entirely in simulation. The system provides real-time probabilistic pose distributions for known objects from the first touch, showing superior accuracy and robustness on 20 diverse objects compared to baseline methods, even with scanned object models.
Key insights
- Learning tactile perception models primarily in simulation, using geometric contact rendering and an embedding space, effectively bridges the sim-to-real gap for tactile pose estimation with minimal real-world data.
- Providing probabilistic pose distributions is crucial for handling the local and ambiguous nature of tactile information, enabling effective integration of multiple contacts and external priors to resolve ambiguities.
- The embedding-based matching approach significantly outperforms direct pixel comparison, classification, and regression baselines, especially for larger pose grids and noisy real-world data.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.