Reproduce “FingerSLAM: Closed-loop Unknown Object Localization and Reconstruction from Visuo-tactile Feedback” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jialiang Zhao, Maria Bauza, Edward H. Adelson
FingerSLAM combines learned tactile and visual odometry in a factor graph to localize unknown rigid objects in six degrees of freedom while assembling tactile point clouds into a 3D reconstruction. On six unseen objects, including transparent and painted glass jars, loop-closure fusion achieved median errors of 2.8 mm and 2.4°, outperforming or matching tactile-only, vision-only, and fusion-without-loop-closure baselines.
Key insights
- Tactile measurements are especially effective for small motions: below 5 mm, tactile odometry reached 1.0 mm translation error versus 2.6 mm for vision, whereas larger motions reduce tactile overlap and weaken its advantage.
- Visual tracking supplies global context but degrades on visually challenging or out-of-distribution objects such as glass jars; tactile tracking is less affected by transparency but struggles with smooth surfaces.
- Loop closure consistently improved the multimodal system, showing that global revisitation constraints are important for limiting drift in tactile reconstruction.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.