Reproduce “Vision2Code: A Multi-Domain Benchmark for Evaluating Image-to-Code Generation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Ajay Vikram Periasami, Junlin Wang, Bhuwan Dhingra
Vision2Code evaluates image-to-code generation by scoring outputs with a vision-language model using dataset-specific rubrics, without requiring reference programs. Its benchmark contains 539 examples from 15 datasets across six domains. On 180 human-rated pairs, Kimi-K2.6’s tailored rubric correlated better with mean ratings than a generic rubric (Pearson r = 0.849; Spearman ρ = 0.861). The evaluator can miss subtle errors, especially in geometry.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.