Reproduce “FeelAnyForce: Estimating Contact Force Feedback from Tactile Sensation for Vision-Based Tactile Sensors” on OpenResearch

About this paper

Amir-Hossein Shahidzadeh, Gabriele Caddeo, Koushik Alapati, Lorenzo Natale, Cornelia Fermüller, Yiannis Aloimonos

FeelAnyForce, developed by researchers at the University of Maryland and Istituto Italiano di Tecnologia, introduces a 3D force estimator for vision-based tactile sensors, accurately inferring contact forces up to 15 Newtons. The method achieves a mean L1 normalized error of 4.2% on unseen objects and demonstrates robust generalization across different sensor instances and types with minimal calibration.

Key insights

• Integrating a depth-reconstruction head into the multi-head architecture conditions the Vision Transformer encoder to learn features that are relevant to both force and depth, improving generalization and robustness to sensor variations.
• Subtracting a no-contact background image from tactile inputs enhances the model's focus on active contact regions, contributing to robustness against sensor manufacturing inconsistencies and lighting variations.
• The use of a DINOv2 pre-trained Vision Transformer backbone provides robust visual feature extraction across different domains, enabling better generalization to unseen objects and diverse contact conditions.

Summary by alphaXiv.