Reproduce “VisCoder: Fine-Tuning LLMs for Executable Python Visualization Code Generation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yuansheng Ni, Ping Nie, Kai Zou, Xiang Yue, Wenhu Chen
Researchers from the University of Waterloo and collaborators introduced VisCoder, a fine-tuned large language model, and the VisCode-200K dataset for generating executable Python visualization code. VisCoder achieves substantial improvements in execution pass rates for visualization tasks and incorporates a novel self-correction mechanism based on runtime feedback.
Key insights
- Execution-grounded supervision and multi-turn correction dialogues are critical for training LLMs to generate reliable and semantically accurate visualization code.
- A self-debug evaluation protocol, where models iteratively correct code based on runtime feedback, offers a more realistic and robust assessment of code generation capabilities.
- VisCoder effectively recovers from 'shallow, structural errors' such as AttributeError and TypeError with execution feedback, but persistent semantic errors like KeyError and ValueError remain challenging.
- Domain-specific, curated instruction-tuning datasets that combine executable code with iterative feedback are essential for achieving high performance in complex, specialized code generation tasks.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.