Reproduce “SC-Captioner: Improving Image Captioning with Self-Correction by Reinforcement Learning” on OpenResearch

About this paper

Lin Zhang, Xianfang Zeng, Kangcong Li, Gang Yu, Tao Chen

SC-Captioner proposes a reinforcement learning framework that enables large vision-language models to self-correct image captions, mitigating hallucinations and omissions. It introduces a correction-based reward function, refined evaluation metrics, and a human-curated dataset, achieving superior precision and recall in generated descriptions.

Key insights

• Targeted reinforcement learning with a specific correction-based reward function is crucial for instilling effective self-correction capabilities in LVLMs; direct prompting or simple fine-tuning alone is insufficient.
• The granular reward function, particularly the 'mistake punishment' component, is essential for guiding models to make precise changes, outperforming general metric-based rewards or Direct Preference Optimization (DPO).
• High-quality, human-curated datasets like RefinedCaps, focused on fine-grained corrections, are vital for training robust self-correction models that demonstrate strong generalization across different test sets.

Summary by alphaXiv.