Reproduce “Data World: Scaling Data Environments from Scratch” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zhanli Li, Zeyuan Xu, Chaoxu Pang, Zeyuan Liu, Anyang Ji, Te Sun, Kongcheng Zhang, Wenbo Xu and 11 more
Data World generates synthetic analysis environments by specifying mechanisms and selected findings before producing imperfect files. An auditor checks whether findings are recoverable, and rubrics score computation and insight discovery. The authors train a model with reinforcement learning. On 100 held-out DataWorld-Bench tasks across 50 worlds, DataWorld-35B-A3B scores 44.70 versus 37.64 for its base, a reported gain of 7.06 points.
Key insights
- An explicit record of how a synthetic world was generated connects its data, intended analytical findings, and evaluation criteria, making targeted tasks auditable.
- Across ten evaluated models, computation scores exceeded insight-discovery scores by 4.87–21.35 points, indicating that processing files and calculating values do not guarantee useful synthesis.
- Rubrics can verify specified findings but are not exhaustive: unlisted valid insights may go uncredited, and synthetic environments do not establish real-world representativeness.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.