Reproduce “VLM-in-Sandbox: Visual Workspaces for Agentic Visual Reasoning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Hexiong Yang, Mingrui Chen, Jie Cao, Ran He
The authors present VLM-in-Sandbox, a workspace for managing visual evidence during multi-step reasoning. It records generated images with stable IDs and provenance, keeps files available, and lets the model promote selected artifacts into focus and aux slots. On 1,260 examples, it reached 66.27% pooled accuracy versus 64.44% for an automatic retain-all control, using 18.6% fewer tokens; selection alone was inconclusive.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.