Reproduce “Few-shot transfer of tool-use skills using human demonstrations with proximity and tactile sensing” on OpenResearch

About this paper

Marina Y. Aoyama, Sethu Vijayakumar, Tetsuya Narita

Researchers developed a few-shot learning framework for robot tool use, integrating simulation pre-training and real-world human demonstrations with multimodal tactile and proximity sensing. This approach enabled robots to manipulate diverse tools across varied surfaces, consistently reducing error rates for inclination and force control and increasing task completion on physical robots.

Key insights

• The pre-trained feature extraction module successfully captures generalizable tool-environment contact dynamics, organizing latent representations by tool and surface inclination.
• A synergistic combination of proximity and tactile sensing is crucial for robust performance in contact-rich tasks, providing complementary information about both intrinsic and extrinsic contacts.
• Fine-tuning only the decoder's fully connected layer enables rapid and data-efficient adaptation of learned policies to new tools, environments, and task parameters with minimal real-world demonstrations.

Summary by alphaXiv.