Reproduce “Towards Rationale-Answer Alignment of LVLMs via Self-Rationale Calibration” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yuanchen Wu, Ke Yan, Shouhong Ding, Ziyin Zhou, Xiaoqiang Li
An iterative Self-Rationale Calibration (SRC) framework aligns the rationales and answers generated by Large Vision-Language Models (LVLMs), enhancing their logical consistency and factual grounding. This approach, leveraging a specialized R-Scorer for pairwise preference curation and Direct Preference Optimization, significantly improves LVLM performance on comprehensive benchmarks, boosting LLaVA-1.5-7B's MMStar score from 32.1 to 38.9.
Key insights
- Explicitly aligning the rationale with the final answer is crucial for fostering deeper, vision-driven thinking in LVLMs, moving beyond superficial correctness.
- An iterative self-calibration process consistently improves LVLMs' performance and semantic consistency, leading to more reliable outputs over multiple iterations.
- A specialized, lightweight R-Scorer model can efficiently and accurately curate preference data for calibration, aligning better with human judgment than larger, general-purpose LLMs.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.