Reproduce “DV-World: Benchmarking Data Visualization Agents in Real-World Scenarios” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jinxiang Meng, Shaoping Huang, Fangyu Lei, Jingyu Guo, Haoxiang Liu, Jiahao Su, Sihan Wang, Yao Wang and 12 more
DV-World introduces a comprehensive benchmark for data visualization agents, featuring 260 tasks across real-world scenarios involving spreadsheet environments, cross-platform chart evolution, and interactive intent clarification. Experiments reveal that state-of-the-art agents achieve less than 50% overall performance, indicating significant limitations in handling complex, ambiguous, and dynamic professional visualization challenges.
Key insights
- Real-world data visualization tasks, particularly those involving native spreadsheet environments, diagnostic error repair, and multi-view spatial planning, present substantial challenges that current agents struggle to address.
- Cross-paradigm chart evolution requires robust semantic transfer and design preservation capabilities across diverse plotting libraries, with performance decaying as code complexity increases and visual feedback becomes critical.
- Effective interactive reasoning and proactive intent alignment are crucial for handling ambiguous user requests, where the quality of clarification questions proves more impactful than the quantity of interactions.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.