Reproduce “CORE: Code-based Inverse Self-Training Framework with Graph Expansion for Virtual Agents” on OpenResearch

About this paper

Keyu Wang, Bingchen Miao, Wendong Bu, Yu Wu, Juncheng Li, Shengyu Zhang, Wenqiao Zhang, Siliang Tang and 2 more

CORE, developed by researchers at Zhejiang University, introduces a code-based inverse self-training framework with graph expansion to automate reward function inference and enhance behavioral diversity for multimodal virtual agents. The framework consistently outperformed existing methods on VisualWebArena and AndroidWorld, achieving significant generalization improvements on unseen tasks while eliminating the need for manual reward engineering.

Key insights

• Process-based, step-level reward signals can be automatically synthesized into executable code (Label Functions) directly from high-level semantic descriptions of agent actions, circumventing the need for manual reward engineering.
• Behavioral diversity and generalization can be systematically fostered through an iterative graph expansion process that learns to encode and utilize multiple valid solution strategies within a task.
• Both successful completion of novel tasks and the analysis of informative failures can be leveraged as valuable training signals to expand an agent's operational task space and improve out-of-domain adaptability.

Summary by alphaXiv.