Reproduce “Think Twice, Act Once: Verifier-Guided Action Selection For Embodied Agents” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Nishad Singhi, Christian Bialas, Snehal Jauhri, Vignesh Prasad, Georgia Chalvatzaki, Marcus Rohrbach, Anna Rohrbach

VeGAS asks an embodied agent to sample candidate actions, then uses a separately finetuned verifier to score them before execution. The verifier is trained on correct actions and synthetic failures, and its judgments select the highest-scoring option. On simulated LangR tasks, Qwen2.5-VL-3B-Instruct success rises from 65% to 71%; this does not establish performance in physical homes.

Key insights

  • Specialized training for the verifier is critical, as off-the-shelf MLLMs perform poorly and do not improve performance in a zero-shot capacity.
  • Synthetic failure trajectory generation, guided by an LLM, effectively provides the necessary negative examples to train a robust verifier without requiring human annotation.
  • A smaller, finetuned verifier can significantly improve the performance of much larger, pre-trained MLLM policies, demonstrating its efficiency and practical utility.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.