Reproduce “Visual Grounding for Object-Level Generalization in Reinforcement Learning” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Haobin Jiang, Zongqing Lu

This research from Peking University and BAAI presents CLIP-guided Object-grounded Policy Learning (COPL), which enables reinforcement learning agents to generalize zero-shot to unseen objects when following natural language instructions in Minecraft. COPL leverages a modified MineCLIP to generate object-grounded confidence maps, using these maps both as a novel intrinsic 'focal reward' to guide skill learning and as a direct visual task representation for the agent's policy, yielding significantly improved success rates in unseen object-centric tasks like hunting and harvesting.

Key insights

  • Object-grounded visual representations, like a confidence map derived from a VLM, provide a more intuitive and effective task representation for RL agents than abstract language embeddings, especially for unseen objects.
  • The traditional MineCLIP reward is insensitive to target distance; a focal reward, which combines object confidence with a Gaussian kernel, creates a dense, distance-sensitive signal crucial for learning approach and interaction skills.
  • Transferring VLM knowledge via both improved intrinsic rewards and direct policy input representations enables robust zero-shot generalization for object-centric tasks in complex environments.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.