Reproduce “Self-Evolving Visual Questioner” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yijun Liang, Hengguang Zhou, Ming Li, Lichen Li, Cho-Jui Hsieh, Tianyi Zhou
A self-evolving framework enables Vision-Language Models (VLMs) to autonomously enhance their visual question generation capabilities, constructing progressively more challenging and informative supervision data from unlabeled images. This approach consistently improves question quality across multiple dimensions without degrading question answering performance, achieving an average QG score increase from 0.25 to 0.45 for Qwen2.5-VL-3B and showing continued improvement across iterative rounds.
Key insights
- VLMs can autonomously generate high-quality, diverse, and reasoning-oriented supervision data for visual question generation, overcoming the reliance on external annotations.
- A multi-stage self-supervision process involving external rewriting and rigorous filtering is crucial for expanding question difficulty and diversity, while ensuring visual grounding and preventing degeneration.
- Simultaneously training with dual-format (QG+QA) supervision effectively improves a VLM's question generation ability without compromising its question answering performance.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.