Reproduce “GUI-CC: Benchmarking Contextual Consistency of GUI World Models as Agent Environments” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Lin Fu, Zheyuan Yang, Tianhui Zhang, Jinbiao Wei, Guo Gan, Boxu Liu, Yilun Zhao, Yu Rong
GUI-CC tests whether mobile GUI world models can sustain usable states across action sequences, rather than merely predict plausible next screens. Its offline track replays reference actions, while its online track lets a fixed agent act on generated screens. Offline reference-action progress reaches 16.7/100 at best; history raises online milestone progress from 51.2 to 70.9, but results do not establish general task success.
Key insights
- Generating visually plausible single-step UI predictions does not guarantee a model's reliability as a multi-step agent environment, as current models widely fail to preserve long-horizon contextual consistency.
- Models frequently exhibit high global visual similarity but struggle significantly with fine-grained UI fidelity, often misplacing or distorting critical interactive elements and layouts.
- Qualitative analysis identified missing app-specific world knowledge (42%), error accumulation due to autoregressive reuse (33%), and direct context inconsistency (25%) as the primary failure modes for maintaining coherent rollouts.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.