Reproduce “Proposer-Agent-Evaluator(PAE): Autonomous Skill Discovery For Foundation Model Internet Agents” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yifei Zhou, Qianlan Yang, Kaixiang Lin, Min Bai, Xiong Zhou, Yu-Xiong Wang, Sergey Levine, Erran Li
This paper introduces Proposer-Agent-Evaluator (PAE), a framework that enables foundation model Internet agents to autonomously discover and practice web navigation skills without human supervision. PAE leverages the asymmetric capabilities of Vision-Language Models for proposing tasks and evaluating outcomes, achieving substantial performance gains and robust zero-shot generalization on real-world web environments like WebVoyager and WebArena.
Key insights
- Even weaker VLMs exhibit asymmetric capabilities, being proficient at proposing and evaluating tasks (understanding/judging) even when less capable at reliably executing them, which can be leveraged for skill acquisition.
- Incorporating a chain-of-thought reasoning step before action output is critical for an agent's generalization to unseen human requests.
- An image-based, sparse 0/1 outcome evaluator is more robust and effective for providing reward signals than step-based or function-based evaluators, which are prone to hallucination.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.