Reproduce “Learning to Poke by Poking: Experiential Learning of Intuitive Physics” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Pulkit Agrawal, Ashvin Nair, Pieter Abbeel, Jitendra Malik, Sergey Levine

Researchers at the Berkeley AI Research Laboratory developed an experiential learning framework where a robot autonomously gathers interaction data to acquire an internal intuitive physics model. The robot learned to displace novel objects to desired goal locations through extensive self-supervised poking, demonstrating improved generalization over inverse-only models by jointly training forward and inverse dynamics models from raw visual input.

Key insights

  • Experiential, self-supervised learning through extensive interaction allows a robot to acquire an effective intuitive physics model for multi-step object manipulation.
  • Jointly training forward and inverse dynamics models provides a significant regularization benefit, leading to more robust feature representations and improved generalization, especially under data scarcity.
  • The learned model implicitly encodes crucial object properties like location, geometry, and pose directly from raw visual input without explicit supervision for these features.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.