Reproduce “Poke and Strike: Learning Task-Informed Exploration Policies” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Marina Y. Aoyama, Joao Moura, Juan Del Aguila Ferrandis, Sethu Vijayakumar
A reinforcement learning framework is introduced that enables robots to autonomously learn exploratory motions and infer task-relevant physical properties of objects. This approach integrates an online property estimator and an uncertainty-based switching mechanism, leading to successful one-shot manipulation tasks in both simulation, achieving a 90.1% success rate for striking, and real-world robot experiments.
Key insights
- Task performance is highly sensitive to errors in estimating specific physical properties, and this sensitivity can be automatically leveraged to design effective, prioritized exploration rewards without manual specification.
- The simultaneous training of an exploration policy with an online property estimator allows for real-time inference and immediate task execution after exploration, circumventing the need for simulator queries or re-optimization at test time.
- Predictive uncertainty of property estimates, derived from an ensemble of neural networks, serves as a reliable proxy for actual estimation error, enabling adaptive and robust policy switching from exploration to task execution in real-world scenarios.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.