Reproduce “SimShear: Sim-to-Real Shear-based Tactile Servoing” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Kipp McAdam Freud, Yijiong Lin, Nathan F. Lepora
The SimShear pipeline enhances tactile sim-to-real policy transfer by explicitly integrating shear information into simulated tactile images, enabling direct deployment of policies on real sensors without runtime translation. This framework achieved 1-2 mm contact error in real-world tactile tracking and collaborative co-lifting tasks, demonstrating robust zero-shot generalization to diverse objects and sensor orientations.
Key insights
- Explicitly conditioning an image-to-image translation GAN (shPix2pix) on shear information resolves the ambiguity of a one-to-many sim-to-real mapping for tactile images, enabling high-fidelity synthetic data generation.
- High-accuracy contact pose and shear estimators (ShearNet) can be effectively trained solely on synthetically generated, shear-enabled tactile images and successfully deployed in real-world scenarios with zero-shot transfer.
- A true sim-to-real pipeline, by eliminating runtime real-to-sim translation, offers a more computationally efficient and robust approach for tactile control that leverages crucial shear information.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.