Reproduce “CogArena: Benchmarking Multimodal Agents on Interactive Behavioral Experiments” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Kianté Fernandez, Caitlin Chen, Jialu Zhou, Bartek Sadowski, Anthony C. Miceli, Ian Krajbich
CogArena evaluates whether browser agents can complete online behavioral experiments and reproduce their task-specific behavioral signatures. It presents ten adapted experiments through a browser and scores completion, accuracy, and trial-level patterns. Across 400 sessions, 167 passed all completion checks; among those sessions, 40% of testable signatures were reproduced. Results apply to four model–Browser-Use combinations.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.