Reproduce “CCCaption: Dual-Reward Reinforcement Learning for Complete and Correct Image Captioning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zhijiang Tang, Linhua Wang, Jiaxin Qi, Weihao Jiang, Peng Hou, Anxiang Zeng, Jianqiang Huang
The CCCaption framework employs a dual-reward reinforcement learning strategy to generate complete and correct image captions, moving beyond the limitations of human-annotated ground truths. This approach achieves state-of-the-art performance across multiple captioning benchmarks, notably improving completeness and suppressing hallucinations.
Key insights
- Objectively quantifying and optimizing for both completeness and correctness in image captions can be achieved by employing a frozen third-party judge model to evaluate generated captions against sets of visual and sub-caption queries.
- Leveraging multiple diverse MLLMs for query generation, combined with diversity filtering, overcomes the bias and limited coverage of single-MLLM approaches, leading to more comprehensive completeness supervision.
- Dynamically weighting and sampling queries based on their 'contribution' (how often they are incorrectly answered or their rollout accuracy variance) significantly enhances the efficiency of RL training for captioning tasks.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.