Reproduce “WorldGUI: An Interactive Benchmark for Desktop GUI Automation from Any Starting Point” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Henry Hengyuan Zhao, Kaiming Yang, Wendi Yu, Difei Gao, Mike Zheng Shou
WorldGUI introduces a benchmark of GUI tasks with varied starting contexts, while WorldGUI-Agent adds checks before planning, before steps, and after actions. Executable pre-actions represent task progress or context changes across 111 Meta tasks and 500 augmented instances. With Claude-3.5-Sonnet, Windows Usage success falls from 50.0% to 19.1%; GPT-4o Actor-Critic ablation lowers success from 26.0% to 9.7%, evidence limited to this evaluation setup.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.