Reproduce “RefineSVG: Visual Feedback-Driven Reinforcement Learning for Image-to-SVG Generation” on OpenResearch

About this paper

Shaobo Liu, Feiqiao Mao, Shuaishuai Zhou, Yan Zhan, Weiqi Tan, Zhiqiong Lu, Zhengping Liang

RefineSVG, developed by researchers at Shenzhen University and Peking University, introduces a closed-loop visual feedback framework for image-to-SVG generation using multimodal large language models (MLLMs). The approach achieves state-of-the-art performance among MLLM-based methods, with a +0.056 DINO improvement and a 20.9% LPIPS reduction compared to InternSVG-8B, while also generating highly efficient and structured SVG code.

Key insights

• Transforming MLLMs into self-correcting visual agents through a closed-loop, ReAct-style feedback mechanism is crucial for overcoming the limitations of single-pass, open-loop code generation.
• A progressive training pipeline—comprising supervised fine-tuning, a ReAct cold-start stage using rejection sampling, and agentic reinforcement learning with multi-dimensional rewards—is effective for training MLLMs to interpret visual discrepancies and perform structured code revision.
• The use of an efficient SVG-oriented semantic tokenization and a perceptually motivated multi-dimensional Diff-Map provides actionable visual feedback while enabling efficient context window management for MLLMs.

Summary by alphaXiv.