Reproduce “Semantic Exploration from Language Abstractions and Pretrained Representations” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Allison C. Tam, Neil C. Rabinowitz, Andrew K. Lampinen, Nicholas A. Roy, Stephanie C. Y. Chan, DJ Strouse, Jane X. Wang, Andrea Banino and 1 more

DeepMind researchers developed a method to enhance reinforcement learning exploration by leveraging semantic abstractions derived from natural language and pretrained vision-language models. This approach significantly improved sample efficiency, leading to 50-70% faster learning on object manipulation tasks and up to 3 times greater map coverage in complex 3D environments.

Key insights

  • Natural language offers a semantically compact and meaningful state abstraction that filters irrelevant visual details, enabling more efficient exploration than raw visual observations.
  • The semantic alignment between language and visual states, rather than just the statistical properties of the language space, is crucial for guiding effective exploration.
  • Pretrained vision-language models provide valuable, general-purpose knowledge that inherently structures state representations in a human-aligned, causally relevant manner, directly benefiting intrinsic reward calculation.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.