Reproduce “Vision Language Models are In-Context Value Learners” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Yecheng Jason Ma, Joey Hejna, Ayzaan Wahid, Chuyuan Fu, Dhruv Shah, Jacky Liang, Zhuo Xu, Sean Kirmani and 10 more

Google DeepMind, University of Pennsylvania, and Stanford University researchers developed Generative Value Learning (GVL), a method that enables Vision Language Models (VLMs) to estimate task progress from video frames. GVL leverages the rich knowledge within pretrained VLMs to provide universal value estimation across diverse robotic tasks and environments. The method achieves strong zero-shot and in-context learning performance, outperforming specialized baselines and enabling applications such as dataset quality assessment and improved policy learning.

Key insights

  • Vision Language Models (VLMs) can serve as in-context value learners for robotic tasks, estimating progress directly from visual observations without task-specific training.
  • Randomly shuffling input video frames forces VLMs to evaluate progress based on visual content rather than temporal order, overcoming inherent sequential biases.
  • Providing a few in-context examples significantly improves VLM value prediction accuracy, enabling adaptation to new tasks and generalization across robot embodiments.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.