Reproduce “Logit-Attention Divergence: Mitigating Position Bias in Multi-Image Retrieval via Attention-Guided Calibration” on OpenResearch

About this paper

Mingtao Xian, Yifeng Yang, Qinying Gu, Xinbing Wang, Nanyang Ye

Researchers from Shanghai Jiao Tong University developed an attention-guided debiasing framework that corrects position bias in Multimodal Large Language Models (MLLMs) for multi-image retrieval. This approach, leveraging the newly identified Logit-Attention Divergence, consistently enhances retrieval accuracy and permutation invariance without requiring model retraining.

Key insights

• Position bias in MLLMs is a 'conditional bias' that dynamically shifts based on the ground truth target's location, rather than being a static, uniform offset.
• The core finding is 'Logit-Attention Divergence,' where MLLMs' internal attention mechanisms correctly identify visual evidence, but their final output logits are corrupted by strong, spurious positional priors.
• MLLMs possess latent, unbiased visual perception capabilities, which can be effectively restored by leveraging internal attention signals to guide debiasing without costly retraining or architectural modifications.

Summary by alphaXiv.