Reproduce “Topology-Agnostic Mesh Reconstruction of Deformable Objects from Sparse Touch” on OpenResearch

About this paper

Everest Yang

A unified deep learning estimator reconstructs the full mesh of 1D, 2D, and 3D deformable objects using only sparse tactile observations, reducing reconstruction error by approximately two-thirds compared to non-learned geometric baselines. An accompanying learned active sensing strategy further improves accuracy, particularly in vision-limited and heavily occluded environments.

Key insights

• The cross-attention architecture is crucial for effectively extracting global shape information from sparse tactile data, outperforming simpler global-pool encoders, especially as more touches accumulate.
• Active tactile sensing provides real, albeit modest, average gains, but its benefits are significantly amplified in vision-deprived and highly self-occluded environments where object ambiguity is high.
• When noisy depth vision or a strong shape prior is available, the benefits of active tactile sensing largely diminish, suggesting these modalities already constrain most of the hidden object state.

Summary by alphaXiv.