Reproduce “Manipulation by Feel: Touch-Based Control with Deep Predictive Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Stephen Tian, Frederik Ebert, Dinesh Jayaraman, Mayur Mudigonda, Chelsea Finn, Roberto Calandra, Sergey Levine
Deep tactile model-predictive control (MPC) learns from autonomous interaction to predict how actions change raw tactile images, then plans actions that make the sensor reading match a user-specified goal image. Using tactile sensing at test time, the system repositions a ball, deflects an analog stick, and rolls a 20-sided die. For the die, the desired face ended up on top in 26 of 30 trials, versus 14 of 30 for a centroid baseline; the results cover three short-horizon tasks on a three-axis machine.
Key insights
- High-resolution tactile sensors, when combined with deep predictive models, allow robots to implicitly learn and leverage complex contact dynamics for fine manipulation.
- Goal specification through desired tactile images simplifies interaction and offers a general, intuitive method for defining manipulation objectives without requiring explicit pose or force estimation.
- Unsupervised learning from diverse exploratory interactions enables the system to acquire robust tactile control policies, reducing reliance on manual supervision or explicit reward functions.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.