Reproduce “Hybrid Computer Use: A Unified Programmatic, Semantic, and Visual Interface for General Agents” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Haoqing Wang, Xiaoyi Zhang, Sicheng Fan, Zhexin Zhang, Xinjie Zhang, Jiahao Li, Bin Li, Yan Lu
HCU gives agents access to code, command-line tools, accessibility controls, and GUI input, letting models choose and combine channels. It uses compact views, batched actions, feedback, and guards against stale targets. In a matched OSWorld 2.0 test, GPT-6 Astra scores 72.37% Partial versus 68.82%, with similar cost. Results support the bundled interface, not AX alone or obsolete visual interaction.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.