Reproduce “PlotGen: Multi-Agent LLM-based Scientific Data Visualization via Multimodal Feedback” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Kanika Goswami, Puneet Mathur, Ryan Rossi, Franck Dernoncourt
PlotGen is a multi-agent framework that uses multimodal Large Language Models (LLMs) to automate scientific data visualization, iteratively refining LLM-generated Python plotting code through numerical, lexical, and visual feedback. This approach improves visualization quality by 4-6% over strong baselines on the MatPlotBench dataset, enhancing accuracy and reducing debugging effort.
Key insights
- A multi-agent LLM architecture combined with multimodal feedback significantly enhances the accuracy and fidelity of LLM-generated scientific data visualizations, moving beyond single-modal or code-only debugging.
- Decomposing feedback into distinct numerical, lexical, and visual modalities allows for targeted and comprehensive error correction, addressing a broader spectrum of inaccuracies (data representation, text clarity, visual aesthetics).
- Automated self-reflection on generated plots using multimodal LLMs substantially reduces the need for manual debugging, leading to improved user productivity and increased trust in AI-generated visualizations.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.